CSPO: Alleviating Reward Ambiguity for Structured Table-to-LaTeX Generation
cs.AI
Submitted: 2026-04-13
Updated: 2026-08-26
Comments: Accepted by ACL2026 (main conference)
Journal ref: Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2026, pp. 25374-25391
DOI: 10.18653/v1/2026.acl-long.1163
Code: https://github.com/microsoft/CSPO
License: http://creativecommons.org/licenses/by/4.0/
The gist: Tables contain rich structured information, yet when stored as images their contents remain "locked" within pixels.
Terminology
Abstract
Tables contain rich structured information, yet when stored as images their contents remain "locked" within pixels. Converting table images into LaTeX code enables faithful digitization and reuse, but current multimodal large language models (MLLMs) often fail to preserve structural, style, or content fidelity. Conventional post-training with reinforcement learning (RL) typically relies on a single aggregated reward, leading to reward ambiguity that conflates multiple behavioral aspects and hinders effective optimization. We propose Component-Specific Policy Optimization (CSPO), an RL framework that disentangles optimization across LaTeX tables components-structure, style, and content. In particular, CSPO assigns component-specific rewards and backpropagates each signal only through the tokens relevant to its component, alleviating reward ambiguity and enabling targeted component-wise optimization. To comprehensively assess performance, we introduce a set of hierarchical evaluation metrics. Extensive experiments demonstrate the effectiveness of CSPO, underscoring the importance of component-specific optimization for reliable structured generation. Our code is available at https://github.com/microsoft/CSPO.
Sources
- Qwen2.5-VL Technical Report
- Nougat: Neural Optical Understanding for Academic Documents
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- ChartReasoner: Code-Driven Modality Bridging for Long-Chain Reasoning in Chart Question Answering
- Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models
- Table2LaTeX-RL: High-Fidelity LaTeX Code Generation from Table Images via Reinforced Multimodal Language Models
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
- Beyond Prediction: Reinforcement Learning as the Defining Leap in Healthcare AI
- A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- CodeReasoner: Enhancing the Code Reasoning Ability with Reinforcement Learning
- DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Models
- Qwen3 Technical Report
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- Aligning Multimodal LLM with Human Preference: A Survey
- Reinforced MLLM: A Survey on RL-Based Reasoning in Multimodal Large Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection