ScienceArena: Benchmarking LLMs on Latest Scientific Olympiad Competitions

arXiv:2608.30517 · cs.AI, cs.CL · Submitted 2026-08-31 · Read on arXiv

cs.AI, cs.CL

Submitted: 2026-08-31

Updated: 2026-08-31

Terminology

• Benchmark — 51× in this paper · explained in 2 episodes
• Calibration — 11× in this paper · explained in 7 episodes
• Grounding — 9× in this paper · explained in 3 episodes
• Provenance — 9× in this paper · explained in 4 episodes
• Consistency — 7× in this paper · explained in 3 episodes
• Visual Grounding — 5× in this paper · explained in 2 episodes
• Ablation — 4× in this paper · explained in 1 episode
• Fidelity — 4× in this paper · explained in 2 episodes
• Benchmarking — 3× in this paper · explained in 4 episodes
• Accuracy — 2× in this paper · explained in 1 episode
• Aggregation — 2× in this paper · explained in 1 episode
• Alignment — 2× in this paper · explained in 1 episode
• Multimodal Reasoning — 2× in this paper · explained in 5 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• LLM-as-a-Judge — 1× in this paper · explained in 3 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Verifier — 1× in this paper · explained in 2 episodes

Sources

• MathArena: Evaluating LLMs on Uncontaminated Math Competitions
• Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
• PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensions
• FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI
• OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems
• OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
• LAB-Bench: Measuring Capabilities of Language Models for Biology Research
• From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline
• An Open Source Data Contamination Report for Large Language Models
• MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
• Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
• Humanity's Last Exam
• Autorubric: A Unifying Framework for Rubric-Based LLM Evaluation on Non-Verifiable Tasks
• GPQA: A Graduate-Level Google-Proof Q&A Benchmark
• SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models
• LiveBench: A Challenging, Contamination-Limited LLM Benchmark
• HiPhO: How Far Are (M)LLMs from Humans in the Latest High School Physics Olympiad Benchmark?
• MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI
• SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Tasks

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection