Probabilistic denoising for reliable signal extraction in spectroscopy

Introduction to the show: ident: Quantum Radio. Generated commentary on the latest quantum physics and condensed matter papers.

Kai: We're looking at the paper: "Probabilistic denoising for reliable signal extraction in spectroscopy".

Mira: Here's the gist of the paper. The gist The model reliably recovers the spectral features of a cuprate superconductor from Poisson-distributed noise with an average count of only 0.02 electrons per voxel Problem and Motivation The integration of deep learning into scientific research has been slower than in other practical domains because standard models typically provide point estimates without a clear representation of the underlying uncertainty and intermediate reasoning While corrupting clean data with noise is a well-defined forward problem, the recovery of the original clean data from noisy data is an ill-posed inverse problem This inherent ambiguity in denoising hinders reliable analysis and necessitates the quantification of statistical uncertainty for successful application in scientific domains One natural realization of this idea is to adopt a probabilistic framework that formulates denoising as the estimation of the underlying probability distribution of the signal Probabilistic Framework The proposed approach formulates denoising as the estimation of the underlying probability distribution of the signal In this approach, "the model simultaneously predicts the signal (mean) and its associated uncertainty (variance), by optimizing a negative log-likelihood (NLL) objective function" The NLL objective function is defined as L = y − yˆσ + log σ where y is the ground truth, ˆy is the predicted denoised intensity, and σ2 is the predicted variance representing the uncertainty Unlike standard mean squared error (MSE) or mean absolute error (MAE) losses that apply a uniform penalty to all voxels, "the NLL dynamically weights the prediction error based on the local uncertainty" This loss function guarantees that "the network outputs converge to the approximated mean and variance of the underlying data distribution" Model Architecture and Training To achieve high performance in denoising 3D ARPES data, an autoencoder denoising architecture was adopted

11: This architecture ensures a large receptive field while maintaining a relatively compact network size The receptive field of the network used in this work is 91×91×91, which is significantly larger than the 41 × 41 field utilized in a previous study on denoising 2D ARPES data To prevent the network from trivially predicting arbitrarily large uncertainty everywhere, "the second term acts as a regularizer that penalizes unnecessarily large uncertainties" Stabilization strategies were employed because "training with the NLL loss requires careful optimization, as it often leads to unstable training and/or underfitting or overfitting" These included normalizing the predicted output to unit average before evaluating the loss, providing the total count level as an additional input channel, and employing relatively strong weight decay Validation and Results The trained network was benchmarked using experimental data from Pb-doped Bi2Sr2CaCu2O8+x (Bi-2212) Despite the extremely sparse condition where "approximately 98 % of the voxels are entirely empty, and only about 2 % capture a single electron event", the network successfully denoised the data, restoring its main features The blue-shaded region in Figure 2(f) represents "the 1σ predictive uncertainty" The predicted uncertainty is proportional to the intensity, which is naturally expected for a Poisson distribution where "the standard deviation is √n for an average value of n" Furthermore, the key novelty lies in propagating this predicted uncertainty into subsequent numerical analyses by using a weighted objective function χ2(β⃗ j) = Xn i=1 wi

yi − f(xi, β⃗ j): 2 = Xn i=1 yi − f(xi, β⃗ j)σi!2 where σi is the voxel-wise uncertainty generated by our model Applicability and Conclusion The approach was validated by successfully extending it to two-dimensional (2D) X-ray diffraction (XRD) data For XRD analysis, a modified network architecture was used to generate a dual output by changing the output channels to two and adopting the Laplacian NLL as the loss function This method confirmed that "the peak positions and widths extracted from the denoised data agree with the ground truth, and the error bars are successfully propagated" The scaling of uncertainty as a function of total count n shows a clear linear dependence on 1/√n, indicating that "the trained network extracts information with near-optimal efficiency" This probabilistic denoising enables deep learning to act not merely as a visualization tool, but as a quantitatively reliable method for extracting physical information from noisy experimental data By doing so, the time required for data acquisition can be drastically reduced, which has been a fundamental bottleneck in spectroscopic experiments The framework establishes uncertainty-aware deep learning as a practical framework for early estimation of physical parameters without compromising scientific rigor Network architecture and training—The training data set consists of 56 high-count 3D ARPES data, resized into a 128×128×128 grid with a sum-conserving resizing method Noisy inputs were generated on the fly from high-count data in real time, with average count per voxel ranging from 0.005 to 0.1 The network was optimized using AdamW

35: with a relatively high weight decay of 5 × 10−3 to prevent overfitting ARPES experiments—Spectra were acquired using a Scienta ARTOF 10k electron analyzer with p-polarized light, setting the energy resolution to better than 2 meV Data Availability—The experimental data used in this work can be found at https://doi.org/10.6084/m9.figshare.31941057 Code Availability—The code used for the training of the neural networks is available at https://doi.org/10.6084/m9.figshare.31939203 References

1: R.

Kai: Roscher, B.

Mira: Bohn, M.

Kai: F.

Paper summary: Kai: So to wrap up on "Probabilistic denoising for reliable signal extraction in spectroscopy," we’ve seen how this approach simultaneously extracts the clean signal and the element-wise predictive uncertainties from noisy data using a probabilistic loss function.

Mira: It means that instead of just getting a denoised output, you get both the predicted mean and its associated uncertainty map, which provides voxel-wise confidence estimates for every point in your data. This moves us toward quantitative parameter extraction with scientifically meaningful error bars.

Lev: And from an engineering standpoint, this framework allows you to propagate those predicted uncertainties into subsequent analyses like superconducting gap calculations, which is vital for running this kind of analysis on real hardware because it shows how the uncertainty overhead lands.

Kai: The paper also confirms that the model reliably recovers spectral features from Poisson distributed noise with an average count of only zero point two electrons per voxel in three dimensional ARPES data, and they also validated its success on two dimensional X ray diffraction data <ref:2605.07819#pg3>.

Mira: Ultimately, this approach establishes uncertainty aware deep learning as a rigorous framework for scientific data analysis, which means it’s not just a visualization tool anymore but something you can trust for extracting physical information from noisy experimental results.

Lev: The limitation they state is that pinpointing positions with low counts is highly ambiguous, which directly leads to high predictive uncertainty in the output of the network, meaning where it stops working is when you have very sparse data because the physical ambiguity is just too strong for the network to resolve reliably.

Conclusion: Kai: So we’ve looked at how they build this probabilistic denoising framework for spectroscopy, and now we’re going to talk about what it actually means for the people who wrote this paper, "Probabilistic denoising for reliable signal extraction in spectroscopy."

Mira: Yeah, it’s interesting because they aren't just throwing a deep learning model at noisy data and hoping for the best; they are making the network predict both the actual signal and its own uncertainty map simultaneously.

Lev: From an engineering side, that uncertainty map is what we need to worry about when we try to actually build this on real hardware; it tells us exactly where our measurement confidence drops off.

Kai: I guess the authors are trying to solve that problem of getting reliable physical data from incredibly sparse, high-noise environments in spectroscopy.

Mira: Exactly, and they show that you can take those predicted uncertainties and use them to perform subsequent analyses, like figuring out a superconducting gap with real error bars on the parameters.

Lev: That’s the key; it means this isn't just some pretty picture of clean data; it’s a tool for extracting quantitative physical parameters while accounting for measurement noise.

Kai: It sounds like they really focused on making the deep learning part scientifically useful rather than just a visualization trick.

Mira: Right, and they tested this on three dee ARPES data, showing it works even when the signal is extremely weak—like only about two percent of the voxels have an electron count.

Lev: And they even extended it to 2D X-ray diffraction data, which shows this approach has some broad applicability across different experimental setups <ref:2605.07819#pg2>.

Kai: So if you’re looking at experiments where you can't get perfect signal quality, this probabilistic method gives you a way to get results that have quantifiable errors attached to them.

Mira: It shifts the whole paradigm from just getting *a* picture of the data to getting a statistically rigorous set of physical parameters from it.

Lev: And that rigor is something we need when we think about scaling these methods up for more complex error correction schemes later on.

Kai: That leads us into how this uncertainty handling plays into the bigger picture of building reliable quantum measurements and modeling materials.