TraceBench: Controlled Evaluation of LLM Agents for Time-Series Root-Cause Attribution
cs.LG
Submitted: 2026-08-27
Updated: 2026-08-27
Code: https://github.com/TommasoBendinelli/TraceBench
Project page: https://tracebench.github.io/environments/BallDrop
Terminology
Sources
- TimeSeriesExam: A time series understanding exam
- TimeSeriesGym: A Scalable Benchmark for (Time Series) Machine Learning Engineering Agents
- MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
- InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks
- DSBench: How Far Are Data Science Agents from Becoming Data Science Experts?
- Multi-Modal Forecaster: Jointly Predicting Time Series and Textual Data
- Time-MQA: Time Series Multi-Task Question Answering with Context Enhancement
- Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis
- Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
- AssetOpsBench: Benchmarking AI Agents for Task Automation in Industrial Asset Operations and Maintenance
- InsightBench: Evaluating Business Analytics Agents Through Multi-Step Insight Generation
- BEDTime: A Unified Benchmark for Automatically Describing Time Series
- Are Language Models Actually Useful for Time Series Forecasting?
- ITFormer: Bridging Time Series and Natural Language for Multi-Modal QA with Large-Scale Multitask Dataset
- TemporalBench: A Benchmark for Evaluating LLM-Based Agents on Contextual and Event-Informed Time Series Tasks
- SciTS: Scientific Time Series Understanding and Generation with LLMs
- ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning
- When LLM Meets Time Series: Can LLMs Perform Multi-Step Time Series Reasoning and Inference
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks