Unveiling Hidden Lyman Alpha Emitters in the DESI DR1 Data
summary
The gist
The gist We present an automatic method based on machine-learning convolutional neural network (CNN) architecture to detect Lyman alpha emitters (LAE) hidden in the Data Release 1 spectroscopic
In short
A machine-learning Convolutional Neural Network (CNN) was developed to automatically detect Lyman alpha emitters (LAEs) in DESI Data Release 1 spectra, which are crucial for studying high-redshift galaxies. The model achieved 95.2% purity and 95.9% completeness, successfully finding over 19,000 LAEs at redshifts between z~2 and z~3.5.
Key concepts
- Lyman Alpha Emitters (LAEs)
- These are distant galaxies that emit light at a specific wavelength called Lyman alpha. They are important because they serve as tracers for star formation in the early universe, and current DESI pipelines struggle to find them at high redshifts.
- Convolutional Neural Network (CNN)
- A type of machine learning model designed to process images or spectra. This CNN was split into two parts: one part analyzes the spectrum to predict where Lyα might be, and another part fits the detected line profile to accurately determine the galaxy's redshift.
- Signal-to-Noise Ratio (S/N) Spectrum
- The input data used for training and testing consists of spectra where the signal (the light from the galaxy) is normalized by its noise level. This standardized format helps the CNN learn to identify emission lines reliably, regardless of how noisy the original observation was.
- Redshift Determination
- This is a process that measures how much a galaxy's light has been stretched (redshifted) due to the expansion of the universe. The model attempts to find this redshift by fitting a Gaussian shape to the detected Lyα emission line, which allows scientists to place these galaxies in cosmic time.
Terminology used across episodes
This episode discusses
- Unveiling Hidden Lyman Alpha Emitters in the DESI DR1 Data · Paper Radio
- The Spectroscopic Stage-5 Experiment
- The DESI Experiment Part I: Science,Targeting, and Survey Design
- The DESI Experiment Part II: Instrument Design
- The Hobby-Eberly Telescope Dark Energy Experiment (HETDEX): Description and Early Pilot Survey Results
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- DESI Spectroscopy of HETDEX Emission-line Candidates I: Line Discrimination Validation
- The DESI Experiment, a whitepaper for Snowmass 2013
- A System for Massively Parallel Hyperparameter Tuning
- Tune: A Research Platform for Distributed Model Selection and Training
- Enhancing Ly alpha Emitter Identification in HETDEX with a Convolutional Neural Network
- A Census of Double-Peaked Lyman-alpha Emitters in MAGPI: Classification, Global Characteristics, and Spatially Resolved Properties
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- ODIN: Spectroscopic Validation of Ly alpha-Emitting Galaxy Samples with DESI
- You Only Look Once: Unified, Real-Time Object Detection
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- A Spectroscopic Road Map for Cosmic Frontier: DESI, DESI-II, Stage-5
- The MegaMapper: A Stage-5 Spectroscopic Instrument Concept for the Study of Inflation and Dark Energy
- Effect of local environment on Ly alpha line profile in DESI/ODIN LAEs
The paper
Unveiling Hidden Lyman Alpha Emitters in the DESI DR1 Data · Read on arXiv
Jui-Kuan Chan, Ting-Wen Lan, J. Xavier Prochaska, Shun Saito, J. Aguilar, S. Ahlen, D. Bianchi
Graduate Institute of Astrophysics, National Taiwan University · Department of Physics and Center for Theoretical Physics, National Taiwan University
We present an automatic method based on machine-learning convolutional neural network (CNN) architecture to detect Lyman alpha emitters (LAE) hidden in the Data Release 1 spectroscopic dataset of the Dark Energy Spectroscopic Instrument (DESI). Those LAEs mostly have incorrect redshift estimations because the current DESI pipeline is not designed to detect and measure the redshifts of galaxies at z>2. To uncover those sources, we first visually inspect thousands of DESI spectra and construct a sample, consisting of both LAEs and non-LAEs, for training and testing the CNN-based model to (1) detect LAEs in DESI spectra and (2) determine their Ly α redshifts. The final model yields 95.2% purity and 95.9% completeness for detecting LAEs. We apply this model to approximately 2 times10 6 spectra of sources targeted as emission-line galaxies and detect 19,685 LAEs from z about2 to 3.5 within 12 minutes with a single GPU, illustrating the high efficiency of this model for identifying LAEs. The detected LAEs are mostly at the bright end of the luminosity function with Ly α luminosity L Lyα 10 43 erg/s. The high signal-to-noise composite spectrum of the detected LAEs further shows various spectral features, including P-Cygni profiles of metal lines and MgII emission lines, possible indicators of Lyman continuum escape fraction, revealing the rich astrophysical information in this LAE sample. Finally, this sample can be used to train and validate the pipelines for redshift determination of LAEs for the preparation of the DESI-II survey.
Transcript
Introduction to the show: ident: Astrophysics Radio. Generated commentary on the latest astrophysics papers.
Vera: Hi, I'm Vera.
Jocelyn: And I'm Jocelyn, here with Subrahmanyan.
Vera: We're looking at the paper: "Unveiling Hidden Lyman Alpha Emitters in the DESI DR1 Data".
Jocelyn: Here's the gist of the paper. The gist We present an automatic method based on machine-learning convolutional neural network (CNN) architecture to detect Lyman alpha emitters (LAE) hidden in the Data Release 1 spectroscopic dataset of the Dark Energy Spectroscopic Instrument (DESI), which is crucial because current DESI pipelines are not designed to detect and measure the redshifts of galaxies at z > 2.
Vera: The CNN architecture consists of two parts: 1.
Jocelyn: The first part consists of fully-convolutional layers without any fully-connected layers, where each convolution layer is followed by a ReLU activation function and a max pooling layer to downscale the size of the input spectrum.
Vera: This part takes the blue-arm S/N spectrum (2438 pixels, 1 pixel = 0.8˚A) as input and outputs a lower resolution spectrum (76 pixels, 1 pixel ≈ 25˚A), where each output pixel value is from 0 to 1, indicating the probability of the corresponding region of the original spectrum containing a Lyα emission line.
Title and authors: Vera: They explain their approach by saying they first visually inspect thousands of DESI spectra to build a training set that has both LAEs and non-LAEs. That’s the truth table they use to teach their machine learning model what a real LAE looks like.
Jocelyn: Then, the core of it is this two-part CNN architecture. The first part takes the blue-arm S/N spectrum, which is two thousand four hundred thirty-eight pixels, and it outputs a lower resolution spectrum with seventy-six pixels where each pixel is just a probability between zero and one indicating if that region has a Ly alpha emission line.
Subrahmanyan: That downscaling process sounds like they’re trying to reduce the complexity of the input before the model tries to find those faint signals, which makes sense given how noisy these spectra can be.
Vera: And then they have this second part, which is a slicer and a fitter. The slicer picks the pixel from that first output with a probability greater than zero point five and then they extract that region of the original spectrum.
Jocelyn: After extracting that specific area, it goes to another part—a fitter—which uses two convolution layers followed by three fully connected layers to fit the red peak of the LAE Ly alpha emission line as a peak-normalized Gaussian.
Subrahmanyan: So they are essentially using a neural network to scan for the signal, isolate it, and then model its shape precisely, which is much more detailed than what previous pipeline tools can do.
The paper's summary: Vera: The authors point out a few things they improved over before. They simplified the architecture by removing fully-connected layers in the first part of the CNN, and they say this made it better because that specific type of architecture generalizes better for detecting Ly alpha lines.
Jocelyn: They also talk about balancing the training data, which is important because they added supplementary spectra like quasars and low-redshift ELGs to make sure the model doesn't just learn to predict the majority class, which could be non-LAEs.
Subrahmanyan: And then they have a post-processing step where they assume some identified Ly alpha features are actually other lines and calculate new "correct" redshifts based on that assumption, which helps remove sources with incorrect line identification, specifically removing two thousand seven hundred ninety-three sources with an error rate of zero point eight two percent.
Vera: They also introduce an empirical separation criterion between star-forming LAEs and AGN by using the CIV over Ly alpha flux ratio set at zero point one five as a threshold to help distinguish them.
Jocelyn: And looking ahead, they plan to use attention-based Transformer architectures, like those from AstroCLIP, to capture correlations between multiple lines like Ly alpha and CIV, which should allow the model to recover LAEs out to redshift three point five.
The paper's improvements: Vera: So in summary of "Unveiling Hidden Lyman Alpha Emitters in the DESI DR1 Data," they found a final catalog with nineteen thousand six hundred eighty-five LAEs detected, with a purity of ninety-five point two percent and a completeness of ninety-five point nine percent <ref:2605.12503#pg3> <ref:2605.12503#pg3>. These are sources mostly at the bright end of the luminosity function with Ly alpha luminosity LLy alpha greater than one hundred thousand erg per second at redshifts from two to three point five.
Jocelyn: That catalog is pretty substantial, and it shows that even with these limitations in the current DESI pipeline, you can extract a sample of sources that is mostly at the bright end of what you expect for those early galaxies.
Subrahmanyan: From a theoretical view, this suggests that the population of these LAEs dominating the fainter end is being systematically missed, so getting this sample helps us understand how structure forms at those high redshifts.
Vera: The team also found some interesting issues with the pipeline, like thirty-seven percent of LAEs didn't even have a robust redshift detection and about thirty-four percent were misidentified as quasars with correct redshifts <ref:2605.12503#pg1> <ref:2605.12503#pg1>.
Jocelyn: They also noted that another twenty one percent were misidentified as other lines like O II doublet lines and often got assigned very low redshifts, around zero point one to zero point five.
Subrahmanyan: Overall, this work provides a tool that helps us use the DESI DR1 data in a way it wasn't originally intended for, opening up new avenues for studying galaxy formation at high redshift.
Vera: This paper "Unveiling Hidden Lyman Alpha Emitters in the DESI DR1 Data" gives us a concrete catalog and some clear ways to improve how we process this kind of data before we move on to the next big thing.
Jocelyn: It's a solid step forward for making sure we don't miss these high redshift emitters when looking at surveys like DESI.
Subrahmanyan: The work shows that this method helps us use the DESI DR1 data in a way it wasn't originally intended for, opening up new avenues for studying galaxy formation at high redshift.
Conclusion: Vera: So we've been looking at "Unveiling Hidden Lyman Alpha Emitters in the DESI DR1 Data," and they've built a machine learning system to find Lyman alpha emitters hidden in the Dark Energy Spectroscopic Instrument data, specifically because current pipelines can't handle galaxies at redshift greater than two.
Jocelyn: Right. They used a convolutional neural network architecture to scan those spectra, and it works by first downscaling the input spectrum and then using a slicer and fitter to pinpoint the emission line itself.
Subrahmanyan: It’s about taking that raw spectral information that's hard to interpret with standard tools and turning it into a detectable signal for high-redshift sources.
Vera: The results they got are pretty solid, showing ninety-five point two percent purity and ninety-five point nine percent completeness on the test data. They found nineteen thousand six hundred eighty five LAEs between redshifts two and three point five in just twelve minutes on a single GPU.
Jocelyn: That catalog is pretty substantial, and it shows that even with these limitations in the current DESI pipeline, you can extract a sample of sources that is mostly at the bright end of what you expect for those early galaxies.
Subrahmanyan: From a theoretical view, this suggests that the population of these LAEs dominating the fainter end is being systematically missed by existing software, so getting this sample helps us understand how structure forms at those high redshifts.
Vera: They did point out some issues too, like thirty-seven percent of LAEs didn't even have a robust redshift detection and about thirty-four percent were misidentified as quasars with correct redshifts.
Jocelyn: And they noted another twenty one percent were misidentified as other lines like O II doublet lines and often got assigned very low redshifts, around zero point one to zero point five.
Subrahmanyan: Overall, this work provides a tool that helps us use the DESI DR1 data in a way it wasn't originally intended for, opening up new avenues for studying galaxy formation at high redshift.
Vera: This paper "Unveiling Hidden Lyman Alpha Emitters in the DESI DR1 Data" gives us a concrete catalog and some clear ways to improve how we process this kind of data before we move on to the next big thing.
Jocelyn: It's a solid step forward for making sure we don't miss these high redshift emitters when looking at surveys like DESI.
Subrahmanyan: We need more of this kind of method applied across different survey releases to get a better census of these early galaxies, and maybe look into those Transformer architectures they mentioned for capturing multiple lines together.
More episodes
- 2411.04256-The FENIKS Survey: Stellar-Halo Mass Relationship of Central and Satellite Galaxies in UDS and COSMOS at 0.2 < z < 4.5
- 2505.02928-Redshift Assessment Infrastructure Layers (RAIL): Rubin-era photometric redshift stress-testing and at-scale production
- 2509.02254-A High Incidence of Mid-infrared Variability in Local Ultraluminous Infrared Galaxies
- 2509.03802-Structure of Turbulent Radiative Mixing Layers
- 2601.12315-The MAGPI Survey: co-evolution of baryons and dark matter in star-forming disk-like galaxies at 0.1 z 0.85
- 2602.07661-Effects of Viscosity on Sloshing Cold Fronts in Galaxy Clusters
- 2603.01772-Testing the isothermal Jeans model for self-interacting dark matter halos in the collapse phase
- 2604.26393-Tidal Heating of Stellar Clusters in Fuzzy Dark Matter Halos
- 2307.01899-Improving Slow-Roll Estimates in Starobinsky Inflation Using an Analytic Hubble Parameter
- 2312.10455-Gravitational waves from cosmic superstrings and gauge strings