SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG

The following is a long and detailed summary of the scientific paper SCAR: Semantic Continuity-Aware Retrieval for Efficient Context Expansion in RAG, based solely on its contents.

Abstract and Motivation

Retrieval-Augmented Generation (RAG) suffers from a fundamental bottleneck in document chunking, which often leads to boundary fragmentation. This occurs when critical evidence required to answer a query is split across adjacent segments, and the retriever only identifies one fragment as relevant. This fragmentation degrades retrieval recall. While existing strategies like static windowing or parent retrieval improve recall, they are token-inefficient, introducing significant amounts of irrelevant noise that can degrade LLM performance and increase inference costs.

Related Work and Gap Identification

Existing approaches fall into two camps: indexing-time coherence methods (like TextTiling or SentGraph) which fix boundaries at indexing time, or retrieval-time expansion methods (like Static Windowing or iterative methods like FLARE). The paper identifies a gap—the need for a single-shot, training-free retrieval-time policy that adaptively expands only when query relevance and structural continuity jointly justify it.

Methodology: SCAR

The proposed solution is SCAR (Semantic Continuity-Aware Retrieval), an adaptive retrieval policy designed to recover fragmented logical context at runtime.

SCAR operates by calculating a decision score for a candidate neighbor n of a retrieved chunk c. This involves two core components:

1. Continuity Score: A boundary penalty, b c,n, which quantifies the semantic discontinuity between adjacent chunks:

b c,n = 1 - (e c, e n)

2. Expansion Decision Score (S c,n): The score is defined as:

S c,n = (e q, e n) - lambda b c,n

A neighbor n is appended to the context if its expansion score exceeds a relative threshold tied to the retrieved chunk’s own query-relevance:

S c,n > gamma times (e q, e c)

This formulation possesses two key design properties: Scale robustness (the decision is approximately robust to the absolute similarity scale of the embedding model) and Adaptive thresholding (the expansion bar adapts per retrieved chunk).

Experimental Setup

SCAR was evaluated on four diverse corpora selected for their complex structural dependencies: the TCP specification (RFC 9293), the GDPR regulation, a Microsoft 10-K annual report, and a corporate merger agreement.

• Preprocessing: The documents were prepared using a hierarchical chunking strategy with Contextual Prepending. A recursive character splitter was used with a target chunk size of 600 characters and a 60-character overlap.

• Retrieval: The text-embedding-3-large model was used to generate embeddings, and retrieval was performed using k-NN search with k=5 and cosine similarity.

• Metrics: The primary metrics were Recall (the proportion of gold chunks recovered) and Chunk Efficiency (Recall divided by the average the number of unique chunks retrieved).

• Baselines: SCAR was compared against several baselines, including Top-k retrieval, Window (plus or minus1 and plus or minus2), Parent Retrieval, and Cross-Encoder Reranking.

• Hyperparameters: The fixed parameters used were lambda = 0.1 (the boundary-penalty weight) and gamma = 0.80 (the relative-threshold ratio).

Results and Discussion

** Boundary-Fragmented Queries:**

On queries where evidence was split across segments, SCAR achieved 92.8% average recall using 7.84 unique chunks on average. This represents a 22.9% reduction compared to static windowing (10.16 chunks). The gain was particularly pronounced on the Microsoft 10-K corpus, where SCAR achieved a 29.9% chunk reduction.

** Comparison with Expansion Radii:**

When comparing SCAR (plus or minus 2) against Window (plus or minus 2), SCAR selectively expands only to semantically continuous chunks (average of 9.2 chunks) compared to Window's blind retrieval (14.4 chunks). This resulted in a 36% chunk reduction at only a minor difference in recall (0.949 vs 0.985), leading to approximately 50% higher chunk efficiency.

** Ablation Study:**

To test the necessity of the continuity penalty, an ablation study was conducted comparing the full SCAR policy (lambda = 0.1) against a Relevance-Only expansion policy (lambda = 0). This confirmed that removing the continuity penalty increased chunk volume by up to 7.2% with little recall change, confirming that the continuity penalty acts as an effective semantic filter.

** Statistical Significance:**

The chunk reduction achieved by SCAR on the boundary-fragmented set was highly significant, showing a mean difference of-2.32 chunks, with a paired bootstrap p < 10-4 and Cohen’s d = -1.49 (a large effect). The recall difference was small, with a mean of-0.039, and Cohen’s d = -0.33 (a small effect).

** Cross-Embedding-Model Transfer:**

The fixed settings (lambda = 0.1, gamma = 0.80) transferred successfully across different embedding models (BGE-large-en-v1.5 and zembed-1), confirming that the relative threshold formulation is robust to changes in similarity scales without requiring recalibration.

** Downstream Generation Quality:**

Evaluating the Microsoft 10-K corpus using GPT-4o-mini, SCAR (plus or minus 1) produced essentially identical faithfulness (4.99/5 vs. 4.99/5) and slightly higher answer relevancy (4.79 vs 4.74). Crucially, SCAR achieved a 27.1% token reduction in context size compared to Window (plus or minus 1), demonstrating that the leaner context does not degrade grounded-answer quality at generation time.

Conclusion

SCAR is presented as a training-free, retrieval-time policy that cuts chunk volume by 22.9% versus static windowing (p < 10-4, d = -1.49) while preserving faithfulness and achieving a 27.1% smaller context for production RAG applications.

(Note: As a diligent AI researcher, I will provide a highly specific technical blueprint for implementation.)

The improvements are focused on replacing or augmenting the standard retrieval policy within a Retrieval-Augmented Generation (RAG) pipeline. The core advancement is shifting from static, deterministic expansion to dynamic, semantically filtered expansion.

We replace the standard fixed-window retrieval mechanism (e.g., plus or minus 1 or plus or minus 2 chunks) with a Continuity-Aware Expansion Decision Module. This module operates after the initial Top- k retrieval but before context assembly.

Implementation Details:

1. Input: The query embedding (e q), the retrieved chunk embedding (e c), and all contiguous candidate neighbor embeddings (e n).

2. Continuity Scoring (The Penalty): For every adjacent candidate neighbor n, calculate the semantic discontinuity: b c,n = 1 - (e c, e n.

3. Relevance-Continuity Decision Score: Calculate the expansion score for each neighbor S c,n:

S c,n = (e q, e n) - lambda b c,n

4. Adaptive Thresholding: Determine the required minimum relevance threshold tau based on the query's own relevance to the primary retrieved chunk: tau = gamma times (e q, e c).

5. Selection Logic: A neighbor n is appended to the final context if and only if its score exceeds this adaptive threshold: S c,n > tau.

System Capabilities (What the Improved System Can Do):

• Achieve Selective Context Expansion: The system can dynamically determine which adjacent chunks are logically necessary for a complete answer, rather than blindly including all neighbors.

• Maintain Query-Specific Adaptability: If the initial retrieved chunk (e c) is weakly relevant to the query (low (e q, e c)), the threshold tau drops accordingly, allowing more expansion. Conversely, if e c is highly relevant (high tau), only the most semantically robust neighbors will pass.

• Reduce Token Overhead: The system minimizes neighbor noise by eliminating irrelevant chunks that a static windowing approach would capture, significantly reducing context length (up to 22.9% reduction vs. static windowing).

The SCAR policy is designed to be independent of the underlying chunking strategy, making it highly adaptable. Furthermore, its core mechanism is robust across different embedding models.

The system must include a validation mechanism that proves the value of the continuity penalty b c,n.

Fixed-length chunking in Retrieval-Augmented Generation (RAG) often leads to boundary fragmentation, where critical evidence is split across segments, degrading retrieval recall. While static windowing and parent retrieval improve recall, they introduce significant token overhead. We propose SCAR (Semantic Continuity-Aware Retrieval), an adaptive retrieval policy that selectively expands neighboring chunks by weighing query-neighbor relevance against a structural continuity penalty. SCAR uses a relative expansion threshold tied to each retrieved chunk's own query-relevance, yielding an approximately scale-invariant decision rule that transfers across embedding models without recalibration. Across four diverse corpora (RFC, GDPR, a 10-K report, and a Merger agreement; N=320 queries; 160 boundary-fragmented), SCAR achieves 92.8% recall on boundary-fragmented queries with only 7.84 chunks, a 22.9% reduction compared to static windowing (10.16 chunks). Paired bootstrap tests (B=10,000) confirm the chunk reduction is highly significant (p<0.0001, Cohen's d=-1.49, large effect), with a small recall difference (Cohen's d=-0.33). The policy transfers across three embedding models (text-embedding-3-large, BGE-large-en-v1.5, zembed-1) using the same single hyperparameter setting, and downstream RAGAS evaluation on the 10-K corpus confirms SCAR preserves generation faithfulness while reducing context tokens by 27.1%.