The Importance of Being Statistically Earnest: A Critical Re-evaluation of GSM-Symbolic
cs.AI, cs.CL
Submitted: 2026-05-27
Updated: 2026-09-01
Code: https://github.com/the-mysh/gsm-symbolic-benchmarking
Terminology
Sources
- Training Verifiers to Solve Math Word Problems
- FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI
- Less Is More: Cognitive Load and the Single-Prompt Ceiling in LLM Mathematical Reasoning
- Efficient numeracy in language models through single-token number embeddings
- Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting
- Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges
- Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs
- Did You Forget What I Asked? Prospective Memory Failures in Large Language Models
- A Looming Replication Crisis in Evaluating Behavior in Language Models? Evidence and Solutions
- HuggingFace's Transformers: State-of-the-art Natural Language Processing
- GPT Can Solve Mathematical Problems Without a Calculator
- FoNE: Precise Single-Token Number Embeddings via Fourier Features
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection