Reading is not Reasoning: Bridging the Agentic Policy Gap in Vision-Text Compression
Cheng Fan, Junyi Zhou, Tingzhang Luo, RongJian Xu, Qiyanhui Lu, Mingjian Zhu, Hanting Chen, Jianyuan Guo
cs.AI
Submitted: 2026-08-09
Updated: 2026-08-11
License: http://creativecommons.org/licenses/by/4.0/
The gist: Multi-step language-model agents repeatedly process growing interaction histories, leading to substantial context costs.
Terminology
Abstract
Multi-step language-model agents repeatedly process growing interaction histories, leading to substantial context costs. Vision--text compression reduces these costs by rendering history as images, but the resulting modality shift creates a marked capability gap. Through controlled evaluations of history recovery, matched-state decisions, and complete trajectories, we show that this gap cannot be explained by OCR quality alone. Visual-history agents exhibit systematic drift in action selection, query formulation, stopping, and evidence use, revealing an agentic policy gap. We introduce CAPS, a two-stage Cross-modal Agentic Policy Self-distillation framework that uses the same model's stronger text-history policy to supervise its visual-history counterpart. Offline trajectory self-distillation transfers successful text-policy behavior to visual-history inputs, while online policy self-distillation provides dense supervision on states visited by the visual-history policy during reinforcement learning. On SearchQA, CAPS improves over AgentOCR by 5.0% and 3.4% with 3B and 7B backbones, respectively. On full-history ALFWorld, the corresponding gains are 15.6% and 14.5%. Across settings, CAPS reduces average memory-context cost by up to 63.3% and peak cost by up to 83.4% relative to matched text-history policies. These results show that explicit cross-modal policy self-distillation can preserve agent capability under vision--text compression. Our code will be made publicly available in a future release.
Sources
- Qwen2.5-VL Technical Report
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- AgentOCR: Reimagining Agent History via Optical Self-Compression
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
- ACON: Optimizing Context Compression for Long-horizon LLM Agents
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- LongSeeker: Elastic Context Orchestration for Long-Horizon Search Agents
- SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization
- WebGPT: Browser-assisted question-answering with human feedback
- Qwen2.5 Technical Report
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- HybridFlow: A Flexible and Efficient RLHF Framework
- MemOCR: Layout-Aware Visual Memory for Efficient Long-Horizon Reasoning
- ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
- Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs
- Voyager: An Open-Ended Embodied Agent with Large Language Models
- Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Library
- DeepSeek-OCR: Contexts Optical Compression
- MiMo-V2-Flash Technical Report
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection