When, How Long and How Much? Interpretable Neural Networks for Time Series Regression by Learning to Mask and Aggregate

arXiv:2512.03578 · cs.LG, cs.AI · Submitted 2025-12-03 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "When, How Long and How Much? Interpretable Neural Networks for Time Series Regression by Learning to Mask and Aggregate".

Jane: Time series extrinsic regression (TSER) aims to predict a continuous target variable from an input time series, and this work introduces MAGNETS,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: Let's look at the specific title of this paper again: "When, How Long and How Much? Interpretable Neural Networks for Time Series Regression by Learning to Mask and Aggregate." It really tells you what the system is designed to deliver—answers about when things happen, how long they last, and how much impact they have.

Jane: That title perfectly captures the three main questions the paper sets out to answer for time series regression tasks. It’s not just about getting a prediction; it’s about understanding the temporal context behind that prediction.

Lu: The authors are Florent Forest, Amaury Wei, and Olga Fink, and they're clearly coming from a strong background in applying deep learning techniques to structured data problems. Their contribution is proposing MAGNETS as an inherently interpretable neural architecture for TSER tasks.

Meng: So, what does this mean practically? It means we can move past models that just give us an output and start getting insights into the underlying temporal events that caused that output. That level of insight is crucial when deploying AI in critical areas like monitoring equipment or forecasting maintenance needs.

Lalam: The authors are doing something novel by designing a system where the model learns these input-dependent masks and then aggregates them into concepts without needing any manual concept supervision to start with, which is a significant methodological step.

The paper's summary: Tom: Now for the summary of what MAGNETS actually does. Basically, they introduce an architecture that learns masks to find locally relevant regions in each time series and then combines those masked segments into a compact set of predictive concepts.

Jane: To put that simply, the process involves four main steps: first, generating binary masks using a U-Net model; second, applying those masks to the input series and aggregating them over time to get scalar features; third, mapping these aggregated features into a bottleneck of concept activations with sparsity and orthogonality regularization; and finally, using these concepts in a linear layer for the final prediction.

Lu: The key mechanism here is that the mask generation uses a 1D U-Net with the Straight-Through Gumbel-Softmax estimator to ensure we get binary masks at inference time while allowing gradients to flow during training. It’s clever design to keep it both interpretable and trainable simultaneously.

Meng: I see how that U-Net part addresses the temporal dependencies, capturing both local and global patterns across the series, which is something simpler models often miss when looking at sequences. But the aggregation function they use is critical; it lets them explicitly control whether they are capturing duration or intensity of a signal within those masked regions.

Lalam: This methodology allows MAGNETS to discover human-understandable concepts directly from raw inputs rather than relying on pre-trained foundation models, which means we don't need massive labeled datasets just to get started with this kind of concept discovery.

The paper's improvements: Tom: The paper highlights several improvements over previous methods, particularly by addressing the limitations of existing approaches that either lack interpretability or require pre-defined concepts. They specifically target the inability of prior concept-based methods to capture multivariate interactions and temporal localization simultaneously.

Jane: They propose a mechanism for concept bottleneck that includes two regularization terms: an L1 sparsity loss on the weight matrix beta and an orthogonality loss on beta, which helps ensure each discovered concept is distinct and doesn't just repeat information from another one.

Lu: This dual regularization strategy is key because it enforces compactness and non-redundancy in the set of concepts they discover. It’s designed to make sure the resulting prediction isn't just a complex mess of overlapping features but a clean, structured combination of meaningful components.

Meng: From an implementation perspective, I'm interested in how robust this regularization holds when we apply it to very high-dimensional time series data; we need to know if these penalties keep the concept set manageable as the input complexity increases.

Lalam: The authors are also presenting a way to get explanations that are more faithful and informative than what you'd typically get from post-hoc attribution methods, especially in scenarios where those methods become unstable or noisy due to complex temporal patterns.

Conclusion: Tom: So, wrapping things up on "When, How Long and How Much? Interpretable Neural Networks for Time Series Regression by Learning to Mask and Aggregate," the main implication is that we can now build TSER models that are both highly accurate and inherently transparent by learning localized, mask-based concept aggregations.

Jane: It’s a big step because it directly tackles the performance-interpretability trade-off, suggesting we don't have to sacrifice one for the other in these regression tasks. The structured reasoning process they describe lets us inspect exactly which channels matter and precisely how much they contribute over time.

Lu: The ability to discover concepts without concept supervision is very powerful; it suggests a path toward building models that learn domain-specific knowledge intrinsically, rather than just memorizing training data points. It opens up possibilities for discovering novel temporal patterns we haven't even thought to look for yet.

Meng: In terms of practical impact, this architecture could drastically reduce the time needed for validation in engineering projects because you can actually trace the prediction back to specific time windows and sensor inputs, which speeds up troubleshooting immensely.

Lalam: I really think the ability to automatically identify specific physical patterns in real-world sensor streams is what’s going to make the biggest cultural impact, helping us move toward more autonomous and trustworthy decision-making across many industries.

IMOS Laboratory, EPFL

cs.LG, cs.AI

Submitted: 2025-12-03

Updated: 2026-06-27

Comments: 31 pages, 6 figures, 6 tables

Journal ref: Data Mining and Knowledge Discovery 40, article number 103 (2026)

DOI: 10.1007/s10618-026-01267-y

Code: https://github.com/FlorentF9/MAGNETS

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 89/100

The gist: Time series extrinsic regression (TSER) aims to predict a continuous target variable from an input time series, and this work introduces MAGNETS, an inherently interpretable neural architecture that

Key concepts

Mask Generation
A neural network predicts 'msoft' masks per channel which are turned into binary masks (m). This uses a 1D U-Net to capture both local and global temporal dependencies, ensuring the model selects specific, input-dependent time segments for analysis.
Aggregation Function
Aggregated features (zc,m) are created by applying each mask element-wise to the input series. A user-defined function g (like summation) is used to capture both the duration ('how long') and intensity ('how much') of the signal within those selected regions.
Concept Bottleneck
This mechanism maps aggregated features into a compact set of K concepts using linear transformations. Sparsity loss and orthogonality loss are applied to ensure each concept is non-redundant and relies on only a small subset of input features.

Terminology

Summary

Time series extrinsic regression (TSER) aims to predict a continuous target variable from an input time series, and this work introduces MAGNETS, an inherently interpretable neural architecture that learns input-dependent masks and aggregates them into meaningful concepts without requiring concept supervision. This approach addresses the critical trade-off between predictive accuracy and interpretability in TSER by providing transparent, input-specific explanations by design.

The gist

MAGNETS learns input-dependent masks that identify locally relevant regions of each time series and aggregates these resulting masked segments into a compact set of predictive concepts, targeting regression tasks driven by discrete, localized temporal events.

How it works

The MAGNETS architecture decomposes the regression task into four simple and interpretable operations:

  1. Mask generation: A neural network predicts M masks per channel (msoft), which are binarized into binary masks (m) selecting input-dependent temporal regions. The authors adopt a 1D U-Net model for this purpose to capture both local and global temporal dependencies, using the Straight-Through Gumbel-Softmax estimator to enable gradient flow during training while ensuring binary masks at inference time.

  2. Aggregation: Each mask is applied element-wise to the input series (x˜c,m,t = xc,t · mc,m,t) and aggregated over time to produce scalar interpretable features (zc,m). The authors explicitly specify a user-defined aggregation function g—such as summation—to capture both the duration (how long) and intensity (how much) of the signal within the selected region.

  3. Concept bottleneck: A linear layer maps these aggregated features (z) into K concept activations by applying a transformation: ck = X C T X M T βc,m,k · zc,m + bk. To ensure compactness and non-redundancy of concepts, the model employs two complementary regularization terms: Sparsity loss (l1 penalty on β) to encourage each concept to rely on only a small subset of aggregated features, and Orthogonality loss (β Tβ − I2F) to reduce redundancy among concepts.

  4. Prediction layer: A final transparent linear layer maps the concept activations back to the final prediction (yˆ), constructed as a transparent and structured reasoning process that is a linear combination of these concepts.

Key Contributions and Performance

The main contributions include introducing MAGNETS, an architecture that learns localized, mask-based concept aggregations, and proposing an annotation-free concept bottleneck mechanism capable of capturing both multivariate interactions and temporal localization. Experiments on synthetic and real-world datasets show that MAGNETS achieves predictive accuracy competitive with state-of-the-art black-box models while substantially outperforming existing interpretable baselines. Specifically, it consistently outperforms existing interpretable architectures, particularly in tasks involving variable interactions or localized temporal patterns, and its explanations are more faithful and informative than those produced by post-hoc attribution methods.

Interpretability Analysis

The model is designed to answer two key interpretability questions: When do relevant temporal patterns occur? (identified by the mask generator) and How long and how much do these patterns contribute? (captured by the aggregation). The structured reasoning process allows users to inspect which channels influence the prediction, when they matter, and how they contribute. For instance, on real-world datasets like BridgeDegradation1, analysis reveals that Concept 1 assigns non-zero weights to masks on bridge displacement and train load, supporting a hypothesis that degradation depends primarily on displacement during intervals when train load exceeds a critical threshold. The evaluation confirms that MAGNETS provides robust and faithful explanations even in settings where post-hoc attribution methods fail to capture the underlying temporal logic.

Experimental Setup

The evaluation involves comparing MAGNETS against black-box baselines (such as ROCKET and CNN) and various interpretable baselines (including NATM variants, GATSM, Optimal Summaries, and linear/additive models). The model is trained using a loss function that balances predictive accuracy with interpretability: L MAGNETS(x, y) = LMSE(y, yˆ) + λsparsLspars(β) + λorthoLortho(β), where LMSE encourages accurate regression. A sensitivity analysis confirms that the performance remains stable across a wide range of concepts (K) and regularization weights (λspars and λortho), demonstrating robustness without requiring per-dataset hyperparameter tuning. The model is tested on four synthetic datasets designed to test specific temporal logic, including Trivariate-2, which evaluates the ability to capture multivariate dependencies.

Conclusion and Future Directions

MAGNETS directly addresses the performance-interpretability trade-off, narrowing the gap with black-box baselines while outperforming existing interpretable architectures.

Improvements for AI systems

As a fastidious researcher, I have analyzed the MAGNETS paper (Forest et al., 2026). The core innovation lies in replacing black-box prediction pathways with an inherently interpretable decomposition based on localized temporal events.

Here are the specific improvements and capabilities this system enables:

  1. Improve AI systems by deploying a model that provides Why for every prediction, not just What.

  2. Enable domain experts (engineers, clinicians, financial analysts) to validate model outputs directly against physical or logical rules derived from the learned concepts.

  3. Achieve high predictive accuracy comparable to state-of-the-art black-box models while maintaining a transparent computational path (addressing the performance-interpretability trade-off).

  4. Capture complex, multivariate temporal interactions by learning localized masks across multiple channels simultaneously, which existing univariate interpretable models (like NATMs) cannot do.

  5. Perform What, When, and How Much analysis for any prediction:

  • What features matter (via concept activations).
  • When they matter (via temporally localized masks).
  • How much/Longer Duration their contribution is (via the aggregation function, e.g., summation over the selected time window).

  1. Identify specific, physically meaningful temporal patterns in real-world sensor streams (e.g., identifying a peak train load above 2.0 interval for bridge degradation) that are discovered automatically without requiring manual concept annotations or pre-trained foundation models.

  2. Generate faithful explanations that outperform post-hoc methods (like Integrated Gradients/DeepLIFT), particularly in multivariate, complex scenarios where black-box attributions become noisy and unstable.

  3. Develop a robust architecture for regression tasks driven by discrete, localized temporal events (e.g., predicting battery state-of-health based on specific voltage discharge phases).

Abstract

Time series extrinsic regression (TSER) refers to the task of predicting a continuous target variable from an input time series. It appears in many domains, including healthcare, finance, environmental monitoring, and engineering. In these settings, accurate predictions and trustworthy reasoning are both essential. Although state-of-the-art TSER models achieve strong predictive performance, they typically operate as black boxes, making it difficult to understand which temporal patterns drive their decisions. Post-hoc interpretability techniques, such as feature attribution, aim to to explain how the model arrives at its predictions, but often produce coarse, noisy, or unstable explanations. Recently, inherently interpretable approaches based on concepts, additive decompositions, or symbolic regression, have emerged as promising alternatives. However, these approaches remain limited: they require explicit supervision on the concepts themselves, often cannot capture interactions between time-series features, lack expressiveness for complex temporal patterns, and struggle to scale to high-dimensional multivariate data. To address these limitations, we propose MAGNETS (Mask-and-AGgregate NEtwork for Time Series), an inherently interpretable neural architecture for TSER. MAGNETS learns a compact set of human-understandable concepts without requiring any annotations. Each concept corresponds to a learned, mask-based aggregation over selected input features, explicitly revealing both which features drive predictions and when they matter in the sequence. Predictions are formed as combinations of these learned concepts through a transparent, additive structure, enabling clear insight into the model's decision process. The code implementation and datasets are publicly available at https://github.com/FlorentF9/MAGNETS.

Sources

Related papers