EnComp: Lightweight Encoder-Only Context Compression for Retrieval-Augmented Question Answering

arXiv:2603.09222 · cs.CL · Submitted 2026-03-10 · Read on arXiv

cs.CL

Submitted: 2026-03-10

Updated: 2026-09-23

Code: https://github.com/thaodod/LooComp

Terminology

• Pruning — 15× in this paper · explained in 4 episodes
• RAG — 13× in this paper · explained in 3 episodes
• Accuracy — 7× in this paper · explained in 1 episode
• Question Answering — 7× in this paper
• Retrieval-Augmented Generation — 6× in this paper
• Generalization — 4× in this paper · explained in 13 episodes
• Transformer — 4× in this paper · explained in 2 episodes
• Core — 3× in this paper · explained in 2 episodes
• Data Augmentation — 3× in this paper · explained in 7 episodes
• F1 Score — 3× in this paper · explained in 3 episodes
• Hallucination — 3× in this paper · explained in 3 episodes
• Loss Function — 3× in this paper · explained in 2 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Benchmark — 2× in this paper · explained in 2 episodes
• Fidelity — 2× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 2× in this paper · explained in 6 episodes
• Natural Questions — 2× in this paper
• Redundancy — 2× in this paper · explained in 2 episodes
• Retrieval-Augmented Generation (RAG) — 2× in this paper · explained in 16 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Ablation study — 1× in this paper · explained in 1 episode
• Adaptation — 1× in this paper · explained in 1 episode
• Computational overhead — 1× in this paper · explained in 2 episodes
• Contrastive Learning — 1× in this paper · explained in 5 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Generative Models — 1× in this paper · explained in 2 episodes
• In International Conference on Learning Representations — 1× in this paper
• Korea Advanced Institute of Science and Technology — 1× in this paper
• Query Complexity — 1× in this paper · explained in 2 episodes
• Zero-Shot Generalization — 1× in this paper · explained in 5 episodes

Sources

• Provence: efficient and robust context pruning for retrieval-augmented generation
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• Retrieval-Augmented Generation for Large Language Models: A Survey
• In-context Autoencoder for Context Compression in a Large Language Model
• Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps
• EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation
• Unsupervised Dense Information Retrieval with Contrastive Learning
• LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
• TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
• Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP
• Compressing Context to Enhance Inference Efficiency of Large Language Models
• Refiner: Restructure Retrieval Content Efficiently to Advance Question-Answering Capabilities
• Task-agnostic Prompt Compression with Context-aware Sentence Embedding and Reward-guided Task Descriptor
• Lost in the Middle: How Language Models Use Long Contexts
• Kimi K2: Open Agentic Intelligence
• Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
• Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
• Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models
• Encoder vs Decoder: Comparative Analysis of Encoder and Decoder Language Models on Multilingual NLU Tasks
• LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering