Proof2Hybrid: Automatic Mathematical Benchmark Synthesis for Proof-Centric Problems

arXiv:2508.02208 · cs.CL, cs.AI · Submitted 2025-08-04 · Read on arXiv

cs.CL, cs.AI

Submitted: 2025-08-04

Updated: 2026-08-29

Code: https://github.com/RenBing-Sumeru/Proof2Hybrid

Terminology

• Benchmark — 68× in this paper · explained in 2 episodes
• Perplexity — 6× in this paper · explained in 3 episodes
• Verification — 5× in this paper · explained in 2 episodes
• Robustness — 4× in this paper · explained in 5 episodes
• Accuracy — 3× in this paper · explained in 1 episode
• Consistency — 3× in this paper · explained in 3 episodes
• Adaptation — 2× in this paper · explained in 1 episode
• Thinking mode — 2× in this paper · explained in 2 episodes
• Agentic Task — 1× in this paper · explained in 1 episode
• Agentic Tasks — 1× in this paper · explained in 1 episode
• Agentic Workflows — 1× in this paper · explained in 1 episode
• Aggregation — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 3 episodes
• Cognitive Load — 1× in this paper · explained in 2 episodes
• Iterative Refinement — 1× in this paper · explained in 3 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Trace — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes
• Verifier — 1× in this paper · explained in 2 episodes

Sources

• Wisdom from Diversity: Bias Mitigation Through Hybrid Human-LLM Crowds
• Training Verifiers to Solve Math Word Problems
• DeepSeek-V3 Technical Report
• Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models
• FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• Measuring Mathematical Problem Solving With the MATH Dataset
• Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipeline
• Motivated proofs: What they are, why they matter and how to write them
• Humanity's Last Exam
• Challenging the Boundaries of Reasoning: An Olympiad-Level Math Benchmark for Large Language Models
• PutnamBench: Evaluating Neural Theorem-Provers on the Putnam Mathematical Competition
• Do Large Language Model Benchmarks Test Reliability?
• A Survey on Large Language Models for Mathematical Reasoning
• MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark
• Qwen3 Technical Report
• Qwen2.5 Technical Report

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering