Evaluating the Retrieval Robustness of Large Language Models

arXiv:2505.21870 · cs.CL, cs.AI · Submitted 2025-05-28 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Evaluating the Retrieval Robustness of Large Language Models".

Jane: Retrieval-augmented generation (RAG) generally enhances large language models’ (LLMs) ability to solve knowledge-intensive tasks,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Alright, let's look at the title and who did this work. It’s "Evaluating the Retrieval Robustness of Large Language Models." The authors are Shuyang Cao, Karthik Radhakrishnan, David Rosenberg, Steven Lu, Pengxiang Cheng, Lu Wang. It looks like a solid team from different backgrounds.

Jane: That’s right; having a group with researchers from various institutions means they’re looking at this problem from multiple angles. The title itself is quite direct about what they are measuring: how robust the retrieval process is for these large language models.

Lu: Steven Lu being on the team adds a lot of weight to this, as he’s often involved in foundational work on how these models interact with external knowledge sources, which directly relates to the core issue here.

Meng: I noticed they benchmarked eleven different LLMs across three distinct prompting strategies: vanilla prompting, OwnKnow where we augment the model's own knowledge, and S2A which filters the retrieval contexts <ref:2505.21870#pg0>. That variety is important for seeing how different architectures handle imperfect data.

Lalam: And having those three strategies—vanilla, OwnKnow, and S2A—is interesting because it shows that just tweaking the prompt isn't enough; you have to consider how the model processes that external information in different ways.

The paper's summary: Tom: So they summarized the core idea as this work evaluates whether RAG is always better than not using retrieval, whether adding more documents always improves performance, and if the order of those documents actually matters at all. It’s basically a deep dive into the practical reality of RAG setups.

Jane: Exactly; they established a benchmark with one thousand five hundred open-domain questions from Natural Questions, Hotpot QA, and ASQA datasets to make sure it wasn't just a lab exercise <ref:2505.21870#pg0>. They defined three specific metrics to measure this robustness: No-Degradation Rate, Retrieval Size Robustness, and Retrieval Order Robustness.

Lu: Those metrics are what really define the study's contribution; they move beyond simple accuracy scores and try to quantify the stability of RAG under varying conditions. They aren't just looking at a single perfect answer; they’re looking at how stable the entire process is across different retrieval settings.

Meng: I see they used both a standard sparse BM25 retriever and a dense retriever based on BGE to generate their contexts, which gives them two different kinds of retrieval mechanisms to test against each other. That comparison is a good way to check if the result holds up regardless of how the documents were found.

Lalam: I think focusing on those three specific questions—is RAG better? does more context help? does order matter?—gives us a much clearer picture than just looking at final scores, which is really helpful for understanding where our systems might be failing.

The paper's improvements: Tom: One of the main points they made about how to improve things was focusing on the ways RAG can be implemented. They looked at different methods like just putting everything in one big context window versus generating answers separately and ensembling the results from each document.

Jane: That’s a good point, Tom; it highlights that *how* you feed those retrieved documents into the LLM matters as much as *having* them there. They found that simply including them all together in one context window was the setup they used for this study, opting for the simplest configuration to keep things transparent.

Lu: The paper points out that other methods exist, like using models such as FiD or RETRO which modify the architecture itself to better handle retrieved chunks. That suggests there's a whole area of research dedicated to making the model structure inherently better at utilizing external knowledge rather than just relying on prompt engineering.

Meng: For practical application, this points toward needing more sophisticated context management protocols, maybe dynamic truncation or iterative dropping of documents based on the specific LLM's context window size, which is a very real engineering hurdle.

Lalam: I think the paper’s implication for us is that we need to stop treating RAG as a black box and start thinking about how we architect the connection between the retriever and the generator itself, because that’s where much of the robustness lies.

Conclusion: Tom: So to wrap this up, the study of "Evaluating the Retrieval Robustness of Large Language Models" really shows that while LLMs are generally quite robust when it comes to RAG, there's still a lot of room for improvement if we want consistent results every single time.

Jane: They concluded that while models score well on average across those three metrics, the reality is that imperfect robustness shows undesirable behaviors like performance trade-offs among individual samples, which prevents models from fully using the RAG benefit and can destabilize quality when things change.

Lu: The finding about retrieval order is also significant; they noted that for almost all models tested, reversing the document order led to better results, meaning placing the most relevant information right at the beginning of what we feed it seems like a reliable heuristic.

Meng: From an implementation viewpoint, this means we can’t just rely on a single retrieval step; we need to build in checks that monitor those size and order changes so we don't get those unpredictable sample-level drops in production.

Lalam: Ultimately, the paper gives us a novel perspective for benchmarking because it shows that robustness is a specific thing you have to measure when deploying these systems, reminding us that consistency is what matters most for real use.

Shuyang Cao, Karthik Radhakrishnan, David Rosenberg, Steven Lu, Pengxiang Cheng, Lu Wang

Bloomberg University of Michigan

cs.CL, cs.AI

Submitted: 2025-05-28

Updated: 2026-10-02

Importance score: 90/100

The gist: Retrieval-augmented generation (RAG) generally enhances large language models’ (LLMs) ability to solve knowledge-intensive tasks, but it may also lead to performance degradation due to imperfect

Key concepts

No-Degradation Rate (NDR)
This metric measures how often using RAG results in performance that is at least as good as the model's performance without any retrieval. A high NDR means the retrieval process consistently helps or maintains quality across most questions.
Retrieval Size Robustness (RSR)
RSR checks if adding more retrieved documents to a query improves or maintains performance. The study found that while performance generally increases with more documents, this is not perfect because models still trade off performance on individual samples.
Retrieval Order Robustness (ROR)
ROR assesses whether the model's output remains consistent regardless of the order in which retrieved documents are presented. The results indicated that most models prefer a reversed order, suggesting that placing higher-ranked documents closer to the question is beneficial.

Terminology

Summary

Retrieval-augmented generation (RAG) generally enhances large language models’ (LLMs) ability to solve knowledge-intensive tasks, but it may also lead to performance degradation due to imperfect retrieval and the model’s limited ability to leverage retrieved content. This work evaluates the robustness of LLMs in practical RAG setups by focusing on whether RAG is always better than non-RAG, whether more retrieved documents always lead to better performance, and whether document orders impact results.

The gist

All 11 LLMs exhibit surprisingly high retrieval robustness; nonetheless, different degrees of imperfect robustness hinder them from fully utilizing the benefits of RAG.

How it works

The study establishes a benchmark of 1500 open-domain questions retrieved from Wikipedia using two retrievers: a canonical sparse BM25 retriever and a dense retriever based on BGE. To assess robustness, three metrics are introduced corresponding to the research questions:

  1. No-Degradation Rate (NDR): Measures how often RAG performance is at least as good as its performance without RAG.

  2. Retrieval Size Robustness (RSR): Examines if adding more retrieved documents leads to equal or better performance.

  3. Retrieval Order Robustness (ROR): Checks if the model's performance is invariant to the order of retrieved documents.

How it works

The evaluation involves 11 LLMs from both open-source and proprietary families, tested with three prompting strategies: vanilla prompting, a strategy augmenting the model’s own knowledge (OwnKnow), and a strategy filtering relevant retrieval contexts (S2A). The performance of an LLM system is denoted as f(q, k, o), where q is the query, k is the number of documents retrieved, and o specifies the order.

How it works

The three metrics are formally defined mathematically:

((1) No-Degradation Rate (NDR):

NDR = 1/Z Σ X q∈Q X k∈K X o∈O 1 f(q, k, o) ≥ f(q, 0). A high NDR implies that for most queries, using retrieval does not degrade performance relative to the non-RAG baseline.

((2) Retrieval Size Robustness (RSR):

RSR = 1/Z Σ X q∈Q X ki∈K,i>1 X o∈O RSR(q,ki,o), where RSR(q,ki,o) checks if performance is maintained or improved when adding more retrieved documents.

((3) Retrieval Order Robustness (ROR):

ROR = 1/Z Σ X q∈Q X k∈K 1 − 2σo∈O f(q, k, o), where σo∈O[f(q, k, o)] is the standard deviation of performance over all permutations o ∈ O. A higher ROR means different permutations of the same set of documents produce more consistent performance.

How it works

The benchmark utilizes 1500 samples drawn from Natural Questions, Hotpot QA, and ASQA datasets. Retrieval contexts are sourced from Wikipedia articles split into 20 million chunks. The experiments cover retrieval sizes ranging from 5 to 100 documents and three ordering strategies: original rank, reversed rank, and random shuffle.

How it works

The results indicate that LLMs generally demonstrate strong robustness, achieving over 80% scores on the geometric mean of the three retrieval robustness metrics. However, imperfect robustness reflects undesired behaviors, notably a "performance trade-off among individual samples (i.e., decreasing performance on some examples while improving it on others), which prevents the models from fully utilizing the benefits of RAG and destabilizes response quality when changing the retrieval size or order."

How it works

The study finds that while task performance generally increases as more documents are added, this does not indicate perfect retrieval size robustness, as models keep trading off performance across individual samples. Furthermore, for retrieval order, all other models prefer the reversed order, suggesting that placing higher-ranked retrieved documents closer to the question generally optimizes RAG performance.

How it works

The analysis of prompting strategies shows that only the OwnKnow strategy (see the prompt ownknow.j2 in Appendix C) that incorporates answers generated in the non-RAG setup can consistently enhance retrieval robustness. This suggests that outputs from the non-RAG setup serve as drafts and anchors, leading to reduced variance.

How it works

The paper concludes that retrieval robustness metrics provide a novel perspective for benchmarking and understanding LLMs’ RAG performance, highlighting that while models are generally robust, the remaining imperfect robustness poses risks for realistic applications demanding consistent outcomes.

Improvements for AI systems

Here are specific improvements to AI systems based on the findings in this research, categorized by technical focus:

) Retrieval-Augmented Generation (RAG) System Enhancements

  1. (System-Level Robustness Guarantee): Implement a multi-layered validation layer for RAG outputs that explicitly checks for performance degradation across varying retrieval parameters (size and order).

  2. (Dynamic Context Selection): Integrate the S2A prompting strategy into the inference pipeline. The AI system should first perform a relevance estimation step to select only the most pertinent retrieved documents before feeding them to the main LLM, mitigating distraction from irrelevant context.

  3. (Context Anchoring for Stability): Adopt the OwnKnow prompting strategy where an initial draft answer is generated using only parametric knowledge, and this draft is then inserted into the RAG prompt as a prior answer. This acts as a stable anchor, reducing variance in final outputs when retrieval is imperfect (improving NDR).

  4. (Order-Aware Ranking): When utilizing dense retrievers (like BGE), implement a secondary reranking mechanism that prioritizes document order based on the query's semantic similarity to ensure the most relevant context appears first, addressing the finding that placing higher-ranked retrieved documents closer to the question generally optimizes RAG performance.

) Model Selection and Deployment Strategies

  1. (Robustness-Aware Model Tiering): Instead of selecting only for peak raw task performance, deploy models based on their measured retrieval robustness metrics (NDR, RSR, ROR). For high-stakes applications where output consistency is paramount (e.g., legal or medical domains), prioritize models that exhibit higher Retrieval Order Robustness and No-Degradation Rate, even if a larger model shows slightly higher peak performance.

  2. (Context Length Management Protocol): Develop an automated document truncation/iterative dropping protocol for retrieval contexts that dynamically adjusts the number of documents based on the target LLM's known maximum context window, ensuring inputs never cause hard clipping while maintaining relevance.

) Evaluation and Benchmarking System Upgrades

  1. (Sample-Level Robustness Auditing): Transition from aggregate performance metrics to sample-level auditing for production readiness. The system should flag individual query samples where the performance metric falls below a predefined threshold (e.g., if the NDR for a specific query is low), signaling that the RAG system may fail or provide inconsistent results for that specific user query.

  2. (Adaptive Prompting Strategy Selection): Implement an automated prompt selection module that chooses between vanilla prompting, OwnKnow, and S2A based on the nature of the incoming query or the confidence score of the initial retrieval step, optimizing robustness per inference call.

) Capabilities of the Improved AI System

The improved AI system will be significantly more reliable and trustworthy in real-world deployment:

  1. (Guaranteed Baseline Performance): The system can confidently guarantee that its performance in a RAG setup will not fall below the performance of a non-RAG baseline for a significant majority of queries (high NDR).

  2. (Consistent Scaling): When scaling the retrieval depth (adding more documents), the system can predict with greater confidence that performance will either remain stable or improve, rather than risking unpredictable sample-level degradation.

  3. (Order Independence): The system's output quality will be highly resistant to minor changes in how retrieved documents are ordered, ensuring that minor fluctuations in the retriever's ranking do not lead to catastrophic performance drops.

  4. (Reduced Hallucination Risk): By using context anchoring (OwnKnow) and relevance filtering (S2A), the system minimizes the likelihood of generating answers based on irrelevant or noisy retrieved passages, leading to higher factual accuracy and reduced hallucination rates in complex knowledge-intensive tasks.

Sources

Related papers