AGNFormer I: Reconstruction of AGN spectra using a probabilistic transformer model
Benedict L. Rouse, Franz E. Bauer, Tomasz Różański, Guillermo Cabrera-Vives, Patricia Arévalo, Cristóbal Rodrigo Donoso Oliva, Pavlos Protopapas, Gordon T. Richards, Paula Sánchez-Sáez, Ezequiel Treister, Christian Wolf
astro-ph.GA, astro-ph.IM
Submitted: 2026-07-22
Comments: Submitted to A&A, comments welcomed
Code: https://github.com/RouseBen/AGNFormer
License: http://creativecommons.org/licenses/by/4.0/
The gist: We explore how an uncertainty-aware transformer-based architecture can leverage information embedded across the entire observed optical spectra of AGN, focusing on the algorithm's ability to predict
Terminology
Abstract
We explore how an uncertainty-aware transformer-based architecture can leverage information embedded across the entire observed optical spectra of AGN, focusing on the algorithm's ability to predict unseen or masked parts of luminous AGN spectra. This provides a direct probe of the learnable correlations between AGN continua and broad lines. We introduce AGNFormer, a transformer model trained to predict the mean expected flux and variance in masked spectral regions (major broad lines to plus or minus 10 4 kms-1; missing halves), inputting rest-frame spectral fluxes and uncertainties across the entire redshift range of the SDSS DR16 Quasar Catalogue. We evaluate the performance of the model on both full (no S/N limit) and high-quality (S/N > 10) spectral samples using the negative-log likelihood, and via comparisons with existing C IV and ly-a reconstruction algorithms. The model successfully reconstructs unseen AGN broad lines to better than 10-16% (4-8%) of the flux for the full (S/N > 10) test sets, up to an error floor of about 2-6% of the flux at S/N about 40, while predictions for larger unseen halves grow to 12-25% (5-15%) of the flux the further away they are from the cut-off wavelength of the seen input spectrum. Predictions faithfully reproduce the broad AGN spectral diversity across the entire optical and UV QSO main sequence parameter spaces, including both Gaussian and Lorentzian profile regimes, Feii complexes, and narrow emission lines. Performance is similar or better compared to previous spectral reconstruction algorithms. The high precision of the broad-line region reconstruction demonstrates that the method successfully aggregates information across the spectrum and highlights how the AGN continuum and weaker lines/complexes have the potential to assist astronomers in the extraction of the entire wealth of information embedded in AGN spectra.
Sources
- Adaptive Input Representations for Neural Language Modeling
- Neural Machine Translation by Jointly Learning to Align and Translate
- Probabilistic Language-Image Pre-Training
- Vision Transformers Need Registers
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- ABC-SN: Attention Based Classifier for Supernova Spectra
- Blind QSO reconstruction challenge: Exploring methods to reconstruct the Ly$\alpha$ emission line of QSOs
- On Calibration of Modern Neural Networks
- Masked Autoencoders Are Scalable Vision Learners
- MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
- SDE-Net: Equipping Deep Neural Networks with Uncertainty Estimates
- Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks
- Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
- AstroCLIP: A Cross-Modal Foundation Model for Galaxies
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
- Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regression
- SGDR: Stochastic Gradient Descent with Warm Restarts
- Decoupled Weight Decay Regularization
- An Empirical Model of Large-Batch Training
Related papers
- Apparent Stability in Self-Gravitating Turbulence and the Evolution of Molecular Clouds
- Two sets of potential-density basis pairs for the study of radial perturbations in collisionless spherical stellar systems
- Constraining reionization-era Ly alpha escape with JELS-MUSE: a highly complete H alpha-selected sample at z about6.1
- Deriving volume density profiles of filaments from observed surface densities
- Little Red Dots and Supermassive Black Hole Seed Formation in Ultralight Dark Matter Halos
- MEGATRON: how the first stars can create an iron metallicity plateau in the smallest dwarf galaxies