The Shrinking Lifespan of LLMs in Science

arXiv:2604.07530 · cs.DL, cs.AI, cs.CY, cs.SI · Submitted 2026-04-08 · Read on arXiv

cs.DL, cs.AI, cs.CY, cs.SI

Submitted: 2026-04-08

Updated: 2026-09-25

Code: https://github.com/py-econometrics/pyfixest

Terminology

• Scaling Laws — 11× in this paper · explained in 4 episodes
• Transformer — 6× in this paper · explained in 2 episodes
• Foundation Model — 5× in this paper · explained in 3 episodes
• Robustness — 3× in this paper · explained in 5 episodes
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Foundation Models — 2× in this paper · explained in 7 episodes
• AI Foundation Model — 1× in this paper · explained in 1 episode
• Adaptation — 1× in this paper · explained in 1 episode
• Aggregation — 1× in this paper · explained in 1 episode
• Attention Mechanism — 1× in this paper · explained in 2 episodes
• Benchmarking — 1× in this paper · explained in 3 episodes
• Calibration — 1× in this paper · explained in 7 episodes
• Concept Drift — 1× in this paper · explained in 4 episodes
• Decoupling — 1× in this paper · explained in 2 episodes
• First Conference on Language Modeling — 1× in this paper
• Transformer Architecture — 1× in this paper · explained in 2 episodes

Sources

• Nougat: Neural Optical Understanding for Academic Documents
• On the Opportunities and Risks of Foundation Models
• The Llama 3 Herd of Models
• Training Compute-Optimal Large Language Models
• Scaling Laws for Neural Language Models
• The Semantic Scholar Open Data Platform
• S2ORC: The Semantic Scholar Open Research Corpus
• ChatGPT-4 Outperforms Experts and Crowd Workers in Annotating Political Twitter Messages with Zero-Shot Learning
• The Rapid Growth of AI Foundation Model Usage in Science
• Emergent Abilities of Large Language Models

Related papers

• MTRACE: Multilingual Retrieval-Augmented Generation for Temporally Diverse Text Corpora
• Eleven Years of BRACIS: A Meta-Scientific Study of the Brazilian Conference on Intelligent Systems
• Evidence-Based Scientific Question Discovery: A Framework with Historical Backtesting