Evaluating Accuracy and Probabilistic Reliability of Zero-Shot Time Series Foundation Models

arXiv:2609.25788 · cs.LG, cs.AI · Submitted 2026-09-22 · Read on arXiv

cs.LG, cs.AI

Submitted: 2026-09-22

Updated: 2026-09-22

Comments: Accepted for publication at the 30th European Conference on Advances in Databases and Information Systems (ADBIS 2026)

Code: https://github.com/unic-ailab/TSFM-Eval

License: http://creativecommons.org/licenses/by/4.0/

The gist: Time Series Foundation Models (TSFMs) promise a paradigm shift toward zero-shot forecasting by eliminating task-specific training.

Terminology

Abstract

Time Series Foundation Models (TSFMs) promise a paradigm shift toward zero-shot forecasting by eliminating task-specific training. However, existing works often overlook trade-offs between predictive accuracy and probabilistic calibration. This paper presents a benchmark study of six TSFMs evaluated on energy, traffic, and financial datasets. We contrast their performance against statistical baselines and a supervised DL model. The study reveals that while TSFMs outperform statistical methods and supervised models, they are subject to a fundamental trade-off between point accuracy and probabilistic reliability. Specifically, xLSTM architectures provide robust probabilistic calibration across horizons. In contrast, patch-based transformers offer competitive accuracy but face calibration issues at long horizons, while transformer-based models exhibit context saturation points for optimal zero-shot reasoning. These findings offer evidence-based guidance for balancing generalization and uncertainty quantification in real-world deployments.

Sources

Related papers