Argo-Bench: Evaluating Data Agents on Enterprise-Scale Workflows

arXiv:2610.02122 · cs.CL, cs.AI, cs.DB · Submitted 2026-10-01 · Read on arXiv

cs.CL, cs.AI, cs.DB

Submitted: 2026-10-01

Updated: 2026-10-01

Code: https://github.com/TextQLLabs/Argo-Bench

Project page: https://spider2-sql.github.io

Terminology

• Benchmark — 43× in this paper · explained in 2 episodes
• Text-to-SQL — 11× in this paper · explained in 3 episodes
• Benchmarking — 6× in this paper · explained in 3 episodes
• New York City — 6× in this paper
• Oracle E-Business Suite — 5× in this paper
• Trace — 4× in this paper · explained in 2 episodes
• Calibration — 3× in this paper · explained in 7 episodes
• NYC Department of Consumer and Worker Protection — 3× in this paper
• alpha — 3× in this paper · explained in 1 episode
• Adam — 2× in this paper · explained in 2 episodes
• Computational Biology — 2× in this paper · explained in 2 episodes
• LLM judge — 2× in this paper · explained in 2 episodes
• LLM-as-a-Judge — 2× in this paper · explained in 3 episodes
• Provenance — 2× in this paper · explained in 4 episodes
• AI Agents — 1× in this paper · explained in 2 episodes
• AUC — 1× in this paper · explained in 2 episodes
• Actor — 1× in this paper · explained in 3 episodes
• Annotation Errors — 1× in this paper · explained in 1 episode
• Causal Inference — 1× in this paper · explained in 4 episodes
• Clustering — 1× in this paper · explained in 2 episodes
• Contrastive Learning — 1× in this paper · explained in 5 episodes
• Core — 1× in this paper · explained in 2 episodes
• F1 Score — 1× in this paper · explained in 3 episodes
• International Conference on Very Large Data Bases — 1× in this paper
• Reward Hacking — 1× in this paper · explained in 8 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agents
• $\tau^2$-Bench: Evaluating Conversational Agents in a Dual-Control Environment
• ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software
• BEAVER: An Enterprise Benchmark for Text-to-SQL
• DABstep: Data Agent Benchmark for Multi-step Reasoning
• The Era by Eon Benchmark: A Generated Enterprise Estate with Exact Ground Truth for Benchmarking LLM Agents
• Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards
• AvalancheBench: Evaluating Enterprise Data Agents Through Latent World Recovery
• Business Arena: Benchmarking LLM Agents in a Realistic Marketplace
• Data Agent: A Holistic Architecture for Orchestrating Data+AI Ecosystems
• ERPBench: Evaluating LLM Agents for Enterprise Decision-Making Across Competitive Market Ecologies

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering