Uncertainty-Aware Deep Learning for the Ly alpha Forest: CNN-Based Absorber Detection and Characterization
Paryag Sharma, Vikram Khaire, Ting-Yun Cheng, Hum Chand, Prakash Gaikwad
astro-ph.GA
Submitted: 2026-07-06
Comments: Submitted to MNRAS. (Comments are welcome)
Code: https://github.com/pyigm/pyigm
License: http://creativecommons.org/licenses/by/4.0/
The gist: The Ly alpha forest is a powerful probe of the intergalactic medium and small-scale matter distribution, but deriving absorber properties traditionally requires computationally expensive
Terminology
Abstract
The Ly alpha forest is a powerful probe of the intergalactic medium and small-scale matter distribution, but deriving absorber properties traditionally requires computationally expensive Voigt-profile fitting. We present a convolutional neural network (CNN) that identifies and characterizes H I Ly alpha absorbers directly from quasar spectra. The model is trained on synthetic spectra generated from the IllustrisTNG simulation and fitted with the VIPER Voigt-profile fitting code to provide training labels. The network simultaneously predicts absorber presence, column density (N HI), Doppler parameter (b HI), and line centroid. On simulated spectra, the CNN achieves an F1 score of about 0.8, with mean absolute errors of about 0.18 in N HI and about 0.10 in b HI. It accurately reproduces the H I column density distribution function (CDDF) and the b HI -- N HI relation, recovering CDDF slopes consistent with VIPER and a lower-envelope relation with an RMS difference of only 0.36 km s-1. Applied to high-resolution UVES spectra, performance decreases to an F1 score of about 0.5, with mean absolute errors of about 0.34 in N HI and about 0.21 in b HI. Latent-space analysis reveals a significant domain shift between the simulated and observational spectra, contributing to the reduced performance. Nevertheless, the CNN preserves the observed CDDF and b HI -- N HI distributions, yielding CDDF slopes consistent with VIPER and a lower-envelope RMS difference of 2.96 km s-1. Monte Carlo dropout is implemented during inference to quantify predictive uncertainties. Together with its computational efficiency, the method provides a scalable and uncertainty-aware framework for Ly alpha forest analysis in upcoming spectroscopic surveys.
Sources
- Deep Learning using Rectified Linear Units (ReLU)
- Data Release 1 of the Dark Energy Spectroscopic Instrument
- Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
- A Measurement of the Thermal and Ionization State of the IGM at $z < 0.5$
- Adam: A Method for Stochastic Optimization
- VoigtFit: A Python package for Voigt profile fitting
- BayesVP: a Bayesian Voigt profile fitting package
- Practical Bayesian Optimization of Machine Learning Algorithms
- Reconstruction of the Density Power Spectrum from Quasar Spectra using Machine Learning
Related papers
- Apparent Stability in Self-Gravitating Turbulence and the Evolution of Molecular Clouds
- Two sets of potential-density basis pairs for the study of radial perturbations in collisionless spherical stellar systems
- Constraining reionization-era Ly alpha escape with JELS-MUSE: a highly complete H alpha-selected sample at z about6.1
- Deriving volume density profiles of filaments from observed surface densities
- Little Red Dots and Supermassive Black Hole Seed Formation in Ultralight Dark Matter Halos
- MEGATRON: how the first stars can create an iron metallicity plateau in the smallest dwarf galaxies