SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding
cs.AI
Submitted: 2026-09-04
Updated: 2026-09-28
Comments: 52 pages, 21 figures, and 20 tables. Project page: https://github.com/InternLM/SciDocBench
Code: https://github.com/InternLM/SciDocBench
License: http://creativecommons.org/licenses/by/4.0/
The gist: Scientific papers require models to reason jointly over text, equations, figures, tables, code, and datasets while preserving the provenance of supporting evidence.
Terminology
Abstract
Scientific papers require models to reason jointly over text, equations, figures, tables, code, and datasets while preserving the provenance of supporting evidence. Existing benchmarks typically evaluate these capabilities in isolation, leaving unclear whether multimodal models can support realistic scientific-reading workflows. We introduce SciDocBench, a workflow-centered benchmark for scientific document understanding. It contains 124 expert-authored and difficulty-screened questions organized into seven research-assistant capability groups and 19 subtasks across five scientific domains. Each question is instantiated under four matched conditions combining English or Chinese questions with all-images-first or interleaved document representations, yielding 496 evaluation instances for controlled analysis. The strongest evaluated system achieves only 62.6/100, with pronounced weaknesses in document perception, evidence grounding, verification, and cross-document reasoning. To translate these diagnostics into scalable training signals, we introduce SciDocIR, a typed evidence-graph representation that preserves scientific document objects, layout and cross-reference relations, and provenance. Building on SciDocIR, we construct SciDocDataset, comprising approximately 15K supervised fine-tuning samples and 8K reinforcement-learning samples across 14 verifiable subtasks. Together, SciDocBench, SciDocIR, and SciDocDataset form an evaluation-to-training framework for diagnosing and improving scientific-document assistants. The project page is available at https://github.com/InternLM/SciDocBench.
Sources
- Computation and sampling for Schubert specializations
- New limits on the Pauli forbidden transitions in 12C nuclei obtained with the complete Borexino dataset
- SpatialDreamer: Incentivizing Spatial Reasoning via Active Mental Imagery
- Magic-angle graphene superlattices: a new platform for unconventional superconductivity
- A Unified Perspective on the Dynamics of Deep Transformers
- MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention
- Provable Failure of Language Models in Learning Majority Boolean Logic via Gradient Descent
- Eagle 2.5: Boosting Long-Context Post-Training for Frontier Vision-Language Models
- MM-IFEngine: Towards Multimodal Instruction Following
- Dynamic metastability in the self-attention model
- ACE-Brain-0: Spatial Intelligence as a Shared Scaffold for Universal Embodiments
- The Euler Stratification for $\mathbb{P}^1 \times \mathbb{P}^1 \times \mathbb{P}^n$
- Strategies for Pre-training Graph Neural Networks
- Stairway Codes: Floquetifying Bivariate Bicycle Codes and Beyond
- Spa3R: Predictive Spatial Field Modeling for 3D Visual Reasoning
- Impact-resistant, autonomous robots inspired by tensegrity architecture
- Document-as-Image Representations Fall Short for Scientific Retrieval
- Kimi K2.5: Visual Agentic Intelligence
- QFlowNet: Fast, Diverse, and Efficient Unitary Synthesis with Generative Flow Networks
- Packed-Ensembles for Efficient Uncertainty Estimation
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection