SoftSEEPS improves ML-based precipitation forecasting

summary

The gist

The gist The authors developed SoftSEEPS, a differentiable approximation of the SEEPS score, which enables Machine Learning models to forecast precipitation directly by using it as a training loss.

In short

Authors developed SoftSEEPS, a differentiable approximation of the SEEPS score, allowing machine learning models to forecast precipitation directly by using it as a training loss. This enables training a decoder for precipitation on latent space data from pre-trained low-resolution models. Results show that combining SoftSEEPS with standard error metrics yields marginal trade-offs while improving forecasting accuracy.

Key concepts

SEEPS Score: The original SEEPS score classifies forecasts and ground truth into dry, light, or heavy precipitation categories using historical thresholds. It is a respected metric in weather forecasting that assesses the severity of precipitation events.
Soft Classification Function: Since the original SEEPS score uses hard categories, authors created a differentiable approximation using sigmoid functions with a smoothing parameter τ. This allows gradient-based optimization to be performed on the score during machine learning training.
IMERGDecoder: This is the model architecture used to predict precipitation. It maps latent space representations from a frozen backbone model (ArchesWeather-M) into the IMERG precipitation target resolution, predicting next-day accumulated rainfall without needing explicit precipitation inputs.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 19 episodes
Federated Learning — explained in 17 episodes
Retrieval-Augmented Generation (RAG) — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 16 episodes
Flow Matching — explained in 15 episodes

This episode discusses

The paper

SoftSEEPS improves ML-based precipitation forecasting · Read on arXiv

Jost Arndt, Utku Isil, Noelia Otero, Rodrigo Almeida, Wojciech Samek, Jackie Ma

Fraunhofer Heinrich Hertz Institute · Technische University of Berlin

In this paper we have developed a differentiable approximation of the well-known SEEPS score, which we name SoftSEEPS. This allows the training of a Machine Learning model to forecast precipitation directly. We test SoftSEEPS on the IMERG dataset (0.1 degree resolution) by training a decoder for precipitation on the latent space of a pre-trained low-resolution forecasting model. Combining SoftSEEPS and RMSE in a joint objective is possible with marginal trade-offs in either metric.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "SoftSEEPS improves ML-based precipitation forecasting".

Jane: The gist The authors developed SoftSEEPS, a differentiable approximation of the SEEPS score, which enables Machine Learning models to forecast precipitation directly by using it as a training loss.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, looking at the whole paper "SoftSEEPS improves ML-based precipitation forecasting," we see they developed this SoftSEEPS score, which is a differentiable approximation of the SEEPS score. It enables Machine Learning models to forecast precipitation directly by using it as a training loss.

Jane: What does that mean for us? In simple terms, it means we can train AI models to understand and predict the dry, light, or heavy precipitation categories directly from data like IMERG.

Lu: The authors are using this technique on the IMERG dataset at zero point one resolution by training a decoder on the latent space of a pre-trained low-resolution forecasting model <ref:2610.09752#pg1,on the latent space of a pre-trained low-resolution forecasting model>. This is how they test it in practice.

Meng: It’s worth remembering that they are testing this against three different optimization objectives: MSE with log normalized data only, SoftSEEPS-only, and the linear combination of both denoted as MSE plus lambda times SoftSEEPS with different lambda values.

Lalam: They found that optimization on a linear combination of SoftSEEPS and RMSE is possible with marginal trade-offs in either metric. The lambda sweep traces out a Pareto-optimal frontier between RMSE and SEEPS, with SoftSEEPS dropping by as much as eighteen point five one percent at no resolvable cost in RMSE.

Tom: So, the implication here is that we can train precipitation models to respect those discrete categories—dry, light, heavy—which is exactly what early-warning and disaster-response systems already rely on.

Jane: This gives us a clear pathway to climate impact by enhancing weather forecasting to better mitigate and adapt to the severe precipitation events driven by climate change.

Lu: The method itself is interesting because they found that the evaluation in SEEPS improves spatially uniformly, while evaluation in RMSE both improves and degrades performance in a spatially varying pattern. That tells us something about how this type of climatological information helps models work globally.

Meng: The limitation they point out is that the SoftSEEPS score itself requires those localized historical climatological thresholds t1 and t2 to define the original categories, which means the performance is tied to those specific choices.

Lalam: That’s a fair point. So, while it works well, you still have to pick those thresholds based on what you think matters for your application before training begins.

Conclusion: Tom: So, this paper is about SoftSEEPS, which they developed as a differentiable way to get an AI model to forecast precipitation by using that score as its training loss.

Jane: It essentially takes a hard classification metric and makes it soft so the machine learning can learn from it directly.

Lu: The authors define a "soft classification" function using sigmoid relaxation, where you have probabilities for dry, light, or heavy rain based on some smoothing parameter tau.

Meng: So they're trying to make the output of an AI model align with those categories without needing perfect hard data for every single pixel.

Lalam: They anneal that smoothing parameter tau automatically during training using a plateau scheduler that watches how the validation discrete SEEPS score changes and lowers tau when it stops improving.

Tom: And the results show they can train on a mix of MSE and SoftSEEPS, finding a linear combination works with only marginal trade-offs in RMSE.

Jane: That means models trained this way can respect those dry, light, and heavy distinctions that early warning systems already use for response.

Lu: They found the spatial impact is interesting; the improvement in SEEPS looks uniform globally, but the improvement in RMSE varies spatially depending on where you are looking.

Meng: So it sounds like incorporating this climatological information helps models work better everywhere, even if it doesn't perfectly fix every single spot.

Tom: It lets us train precipitation models to directly target those categories used for disaster response, which is a practical application they’re pointing toward.

Jane: That leads into how we can use this technique to get more accurate forecasts for severe weather events driven by climate change.

More episodes

← Home