ThousandWorlds: A benchmark for climate emulation of potentially habitable exoplanets
cs.LG, astro-ph.EP, astro-ph.IM
Submitted: 2026-06-16
Updated: 2026-09-25
Comments: 10 pages main text, 26 pages references/appendix, plus NeurIPS checklist. Data at https://doi.org/10.57967/hf/8695. Code at https://github.com/edstevenson/ThousandWorlds
Code: https://github.com/edstevenson/ThousandWorlds
License: http://creativecommons.org/licenses/by/4.0/
The gist: The search for life beyond Earth will depend on detecting faint signatures in the atmospheres of potentially habitable exoplanets.
Terminology
Abstract
The search for life beyond Earth will depend on detecting faint signatures in the atmospheres of potentially habitable exoplanets. Interpreting those signatures requires understanding the host planet's climate: the same molecule may signal life on one planet and abiotic chemistry on another. Global climate models (GCMs) provide this understanding, but individual runs can require up to millions of core-hours and substantial domain expert time. Machine-learning emulators could remove this bottleneck, but progress has been limited by the absence of a curated, multi-model exoclimate dataset. We introduce ThousandWorlds, an ML-ready benchmark for exoclimate emulation and for the broader regime of low-data, multi-simulator, parameter-to-field regression. The dataset contains approximately 1800 simulations from five GCMs, mapping eight planet parameters to 3D atmospheric fields including temperature, humidity, winds, clouds, and radiation. Three nested subsets define progressively harder challenges: single-simulator regression, multi-simulator regression with complete observations, and multi-simulator regression with structured missingness. We propose two evaluation protocols: one for ranking methods, and one that measures performance relative to the disagreement between GCMs themselves. We evaluate seven baselines spanning simple methods, deep learning, and Gaussian processes. GP-based methods perform best, suggesting that ThousandWorlds exposes a regime where off-the-shelf deep learning does not yet succeed. Data: https://doi.org/10.57967/hf/8695. Code: https://github.com/edstevenson/ThousandWorlds.
Sources
- Kernels for Vector-Valued Functions: a Review
- ClimART: A Benchmark Dataset for Emulating Atmospheric Radiative Transfer in Weather and Climate Models
- RealPDEBench: A Benchmark for Complex Physical Systems with Real-World Data
- ClimateSet: A Large-Scale Climate Model Dataset for Machine Learning
- The atmospheric circulation and climate of terrestrial planets orbiting Sun-like and M-dwarf stars over a broad range of planetary parameters
- 3D simulations of TRAPPIST-1e with varying CO2, CH4 and haze profiles
- Accelerating Radiative Transfer for Planetary Atmospheres by Orders of Magnitude with a Transformer-Based Machine Learning Model
- The Well: a Large-Scale Collection of Diverse Physics Simulations for Machine Learning
- A Large Repository of 3D Climate Model Outputs for Community Analysis and Postprocessing
- Climate Diversity in the Solar-Like Habitable Zone due to Varying Background Gas Pressure
- ExoPlaSim: Extending the Planet Simulator for Exoplanets
- Accelerating exoplanet climate modelling: A machine learning approach to complement 3D GCM grid simulations
- Fixing the Double Penalty in Data-Driven Weather Forecasting Through a Modified Spherical Harmonic Loss Function
- PDEBENCH: An Extensive Benchmark for Scientific Machine Learning
- The CAMELS project: public data release
- Abiotic oxygen-dominated atmospheres on terrestrial habitable zone planets
- ClimSim-Online: A Large Multi-scale Dataset and Framework for Hybrid ML-physics Climate Emulation
- Quenching-driven equatorial depletion and limb asymmetries in hot Jupiter atmospheres: WASP-96b example
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks