Even Small Reasoners Should Quote Their Sources: Introducing the Pleias-RAG Model Family

Pleias introduces Pleias-RAG-350m and Pleias-RAG-1B, small reasoning models for RAG and source summarization that provide native support for citation and grounding with literal quotes. These models are designed to outperform SLMs below 4 billion parameters on standardized RAG benchmarks while maintaining consistent performance across leading European languages, making them valuable for trustworthy AI applications in constrained environments.

Model Design and Training

Pleias-RAG-350m and Pleias-RAG-1B are mid-trained variants of base models released in December 2024, trained exclusively on open data compiled under the Common Corpus, which totals about two trillion tokens. A critical feature is their Enhanced multilingual support for European languages with a new dedicated tokenizer with better fertility and word fidelity than Llama in French, Italian, Spanish, German, or Polish, alongside better familiarity with source formats like PDFs.

Grounding and Verifiability

The models incorporate generated citations directly during the inference process of LLMs, opting against post-hoc citation methods. This approach allows for higher control of source presentation and display as well as an improved integration of citation materials in the answer. The system uses a Wikipedia-inspired syntax of tags, which is more demanding on the model side than external calling, providing higher control over how citations are presented.

Structured Reasoning

The models employ an iterative reasoning sequence encompassing three main situations: trivial questions, standard questions, and refusal due to lack of source backing. This makes the model proto-agentic under the definition suggested by Anthropic, dynamically directing its own processes based on query analysis and source analysis. The workflow includes a query report with standardized outputs like answerable, trivial, reformulated, and unclear.

Tokenization Strategy

To mitigate risks associated with pre-allocated special tokens in larger models, Pleias implements a tokenizer recycling method. This involves designing a new tokenizer variant based on the selection of the last tokens repurposed as special tokens. As shown in Figure 5, this strategy re-trains each of the last 19 tokens as special tokens, which is probably responsible for the initial significant loss drop and allows the model to recognize instruction structures quickly.

Mid-training Data Generation

The models were trained on a large dataset of RAG examples drawn from Common Corpus with various synthetic augmentations during a process termed mid-training. This approach focuses on the synthetic generation of training data at scale, mitigating legal risks by using models that allow for the reuse of synthetic output. The quality and complexity are ensured through filtering steps, structured reasoning traces, adversarial examples (for diversity and complexity), and a custom synthetic pipeline that generates reasoning traces for complex answers with pre-defined steps like query analysis, source analysis, and draft.

Evaluation Metrics

Standard benchmarks used include HotPotQA, 2WikiMultiHopQA, and MuSiQue. The models are evaluated using an LLM-as-a-match (instruct version of Gemma 3 12B) with grades: “yes”, “rather yes”, “rather no”, or “no”. A key finding is that Pleias RAG models show a negligible impact on performance loss when translated to four main European languages, demonstrating strong multilingual support. Qualitative evaluation also tests scenarios like Cross-lingual scenarios with mismatched query-context language pairs to test language adherence.

Ethical and Deployment Considerations

The architecture addresses ethical imperatives through several design choices: native citation provides Auditability and Evidence-based decision support, the external memory paradigm ensures Data separation and Controlled information access, and reliance on the Common Corpus provides Legal clarity by training exclusively on appropriately licensed materials. This positions the models as a blueprint for AI integration in regulated industries, addressing the challenges of the verifiability gap, the authority problem, and the control deficit.

Future Research Directions

The current roadmap focuses on several areas: context length extension to handle longer sources, built-in support for search including generating API calls to trusted sources, personality tuning (named Pico), and reinforcement learning focused specifically on citation accuracy through structured critique. The team is also experimenting with an iterative error-informed reasoning pipeline where model failures are accumulated as training data.

The gist: Pleias introduces Pleias-RAG-350m and Pleias-RAG-1B, small reasoning models for RAG and source summarization that provide native support for citation and grounding with literal quotes. These models are designed to outperform SLMs below 4 billion parameters on standardized RAG benchmarks while maintaining consistent performance across leading European languages, making them valuable for trustworthy AI applications in constrained environments.

How it works

1. The models are mid-trained on a large synthetic dataset emulating the retrieval of a wide variety of multilingual open sources from the Common Corpus.

Here are the specific improvements that can be made to AI systems using the Pleias-RAG model family, based on the provided paper:

1. Disseminate a new generation of Small Language Models (SLMs) capable of high-accuracy RAG, search, and source summarization.

2. Enable native support for citation and grounding with literal quotes integrated directly into generated answers using a Wikipedia-inspired syntax of less tags (e.g., X">Quote).

3. Implement a dynamic, proto-agentic reasoning sequence that allows models to self-determine the course of action based on query complexity and source availability. This sequence should include steps for:

4. Query reformulation for badly phrased inputs,

5. Identification of trivial questions requiring direct answering,

6. A hierarchical source analysis (reasoning re-ranker) to identify the most relevant sources, and

7. A standardized source report (extensive, basic, incomplete, or infeasible) to guide the final answer generation path.

8. Enhance multilingual support for European languages by utilizing a dedicated tokenizer with improved fertility and word fidelity compared to existing models (like Llama in French/Italian/Spanish/German/Polish).

9. Integrate a tokenizer recycling method to optimize special tokens, potentially improving instruction following and efficiency by repurposing the least useful tokens from the base model's vocabulary.

10. Facilitate deployment on constrained infrastructure (e.g., Raspberry Pi 4) for use cases requiring local or on-device AI assistants in regulated sectors, such as legal or field support.

11. Establish a robust ethical framework for regulated industries by ensuring built-in traceability and auditability through the native citation mechanism, allowing domain experts to verify outputs against source materials.

12. Enable controlled information access via an external memory paradigm, allowing organizations to precisely define which sources the model accesses, thereby minimizing data leakage risks and maintaining clear boundaries between proprietary information and AI processing.

13. Develop tiered source reliability indicators within the citation framework, enabling governance workflows for vetting and approving sources before they are used in RAG systems.

14. Improve performance on complex, multi-hop reasoning tasks (as evidenced by 2WikiMultiHopQA and MuSiQue) by training on a synthetic mid-training dataset that incorporates:

15. Synthetic generation of training data at scale, specifically focusing on structured reasoning traces (planning, evaluation, reflection, exploration).

16. Adversarial exercises during mid-training to increase retrieval task difficulty and model resilience across source selection, shuffling, refusal design (query refusal), and language switching scenarios.

17. Produce answers in the original query language by estimating the query's language first while performing reasoning in a standardized English core sequence.

18. Implement an iterative error-informed reasoning pipeline during inference where the generated answer is critiqued by a stronger LLM or domain-specific critic to identify and correct reasoning errors before final output is presented.

We introduce a new generation of small reasoning models for RAG, search, and source summarization. Pleias-RAG-350m and Pleias-RAG-1B are mid-trained on a large synthetic dataset emulating the retrieval of a wide variety of multilingual open sources from the Common Corpus. They provide native support for citation and grounding with literal quotes and reintegrate multiple features associated with RAG workflows, such as query routing, query reformulation, and source reranking. Pleias-RAG-350m and Pleias-RAG-1B outperform SLMs below 4 billion parameters on standardized RAG benchmarks (HotPotQA, 2wiki) and are competitive with popular larger models, including Qwen-2.5-7B, Llama-3.1-8B, and Gemma-3-4B. They are the only SLMs to date maintaining consistent RAG performance across leading European languages and ensuring systematic reference grounding for statements. Due to their size and ease of deployment on constrained infrastructure and higher factuality by design, the models unlock a range of new use cases for generative AI.