Predict before you train: Scaling Laws for particle physics foundation models

arXiv:2607.23377 · hep-ex, cs.AI · Submitted 2026-07-25 · Read on arXiv

hep-ex, cs.AI

Submitted: 2026-07-25

Updated: 2026-08-25

Code: https://github.com/Jaluus/ParticleViT

Terminology

• Fine-Tuning — 18× in this paper · explained in 5 episodes
• Benchmark — 17× in this paper · explained in 2 episodes
• Transformer — 14× in this paper · explained in 2 episodes
• Accuracy — 9× in this paper · explained in 1 episode
• Foundation Model — 9× in this paper · explained in 3 episodes
• AUC — 8× in this paper · explained in 2 episodes
• Scaling Laws — 7× in this paper · explained in 4 episodes
• Foundation Models — 5× in this paper · explained in 7 episodes
• International Conference on Machine Learning — 3× in this paper
• Core — 2× in this paper · explained in 2 episodes
• Department of Energy — 2× in this paper
• Benchmarking — 1× in this paper · explained in 3 episodes
• Degeneracy — 1× in this paper · explained in 3 episodes
• Downstream Tasks — 1× in this paper · explained in 2 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• National Energy Research Scientific Computing Center — 1× in this paper
• Particle Flow Network — 1× in this paper

Sources

• Scaling Laws for Neural Language Models
• Scaling Laws for Autoregressive Generative Modeling
• Training Compute-Optimal Large Language Models
• An AI-ready, Polarized Electron-Positron Collision Dataset
• OmniLearned: A Foundation Model Framework for All Tasks Involving Jet Physics
• Particle Transformer for Jet Tagging
• OmniJet-$\alpha$: The first cross-task foundation model for particle physics
• Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models
• Scaling Laws in Jet Classification
• Neural Scaling Laws for Boosted Jet Tagging
• Neural Scaling Laws for Jet Generation
• The Machine Learning Landscape of Top Taggers
• Energy Flow Networks: Deep Sets for Particle Jets
• Olmo 3
• Root Mean Square Layer Normalization
• Scaling Vision Transformers to 22 Billion Parameters
• GLU Variants Improve Transformer
• An Efficient Lorentz Equivariant Graph Neural Network for Jet Tagging
• Explainable Equivariant Neural Networks for Particle Physics: PELICAN
• PELICAN: Permutation Equivariant and Lorentz Invariant or Covariant Aggregator Network for Particle Physics

Related papers

• Agentic Hybrid RAG for Evidence-Grounded Muon Collider Analysis
• Towards foundation-style models for energy-frontier heterogeneous neutrino detectors via self-supervised pre-training