BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models
cs.LG, cond-mat.mtrl-sci, cs.AI
Submitted: 2025-05-03
Updated: 2026-09-15
Code: https://github.com/FLASK-LLNL/BOOM
License: http://creativecommons.org/licenses/by/4.0/
The gist: Data-driven molecular discovery leverages artificial intelligence/machine learning (AI/ML) and generative modeling to filter and design novel molecules.
Terminology
Abstract
Data-driven molecular discovery leverages artificial intelligence/machine learning (AI/ML) and generative modeling to filter and design novel molecules. Discovering novel molecules requires accurate out-of-distribution (OOD) predictions, but ML models struggle to generalize OOD. Currently, no systematic benchmarks exist for molecular OOD prediction tasks. We present BOOM, b enchmarks for o ut- o f-distribution m olecular property predictions: a chemically-informed benchmark for OOD performance on common molecular property prediction tasks. We evaluate over 150 model-task combinations to benchmark deep learning models on OOD performance. Overall, we find that no existing model achieves strong generalization across all tasks: even the top-performing model exhibited an average OOD error 3x higher than in-distribution. Current chemical foundation models do not show strong OOD extrapolation, while models with high inductive bias can perform well on OOD tasks with simple, specific properties. We perform extensive ablation experiments, highlighting how data generation, pre-training, hyperparameter optimization, model architecture, and molecular representation impact OOD performance. Developing models with strong OOD generalization is a new frontier challenge in chemical ML. This open-source benchmark is available at https://github.com/FLASK-LLNL/BOOM
Sources
- Active Learning Enables Generation of Molecules that Advance the Known Pareto Front
- ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction
- Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
- Towards Out-Of-Distribution Generalization: A Survey
- A Unified Survey on Anomaly, Novelty, Open-Set, and Out-of-Distribution Detection: Solutions and Future Challenges
- Known Unknowns: Out-of-Distribution Property Prediction in Materials and Molecules
- xVal: A Continuous Numerical Tokenization for Scientific Language Models
- GPT-4 Technical Report
- The Llama 3 Herd of Models
- GLU Variants Improve Transformer
- MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fields
- TorchMD-NET: Equivariant Transformers for Neural Network based Molecular Potentials
- Generating $\pi$-Functional Molecules Using STGG+ with Active Learning
- Regression with Large Language Models for Materials and Molecular Property Prediction
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks