Magenta: Closing the Loop Between Mathematical Reasoning and Lean Verification
cs.AI
Submitted: 2026-09-10
Updated: 2026-09-21
Comments: 9 pages, preprint
License: http://creativecommons.org/licenses/by/4.0/
The gist: Most of mathematical knowledge has been communicated through so-called informal use of mathematics and natural language.
Terminology
Abstract
Most of mathematical knowledge has been communicated through so-called informal use of mathematics and natural language. With large language models (LLMs) being highly adept in using natural language, they achieve strong performance, yet not perfect, in informal mathematical reasoning. Restraining LLMs to informal reasoning misses out on the opportunity to use the discrete verification abilities that machines offer through machine-checkable proofs. In this paper, we bridge the gap between informal and formal reasoning by integrating Lean signals into the informal reasoning process. We introduce Magenta, a training-free agentic pipeline that, given only a natural-language problem, produces an answer, expresses it as a Lean 4 statement, and constructs a machine-checked proof. A statement judge verifies whether the formalisation preserves the original problem, while an error-attribution judge routes failed attempts either to mathematical re-derivation or local Lean repair. Magenta achieves 100% accuracy across all evaluated olympiad benchmarks, including AIME 2025, AIME 2026, and HMMT February 2026. When paired with the open-weight K2-Horizon-7B reasoner, it solves all six IMO 2026 problems. Our analysis shows that statement adjudication is essential for preventing false certificates and that feedback-guided correction outperforms independent resampling on difficult problems.
Sources
- Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models
- Risk-Controlled Lean-as-Judge for Natural-Language Mathematical Reasoning
- Seed-Prover 1.5: Mastering Undergraduate-Level Theorem Proving via Learning from Experience
- Moonshine: An Autonomous Mathematical Research Agent Centered on Conjecture Generation
- FaithformBench: Benchmarking Faithfulness of Mathematical Chain-of-Thought Autoformalisation
- Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipeline
- From Solvers to Research: Large Language Model-Driven Formal Mathematics at the Research Frontier
- Evaluation of LLMs for Mathematical Formalization in Lean
- Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs
- OProver: A Unified Framework for Agentic Formal Theorem Proving
- OpenAI GPT-5 System Card
- Kimi K3: Open Frontier Intelligence
- Advancing Mathematics Research with AI-Driven Formal Proof Search
- LongCat-Flash-Prover: Advancing Native Formal Reasoning via Agentic Tool-Integrated Reinforcement Learning
- DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data
- AoA: Theorem Proving Agent over Abstract Syntax Tree of Redesigned Language
- Rethinking Math Reasoning Evaluation: A Robust LLM-as-a-Judge Framework Beyond Symbolic Rigidity
- A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection