DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters

summary

The gist

The gist: DiTS proposes Diffusion Transformers for Time Series, a Multimodal Diffusion Transformer that treats endogenous targets and exogenous covariates as distinct modalities to jointly condition

In short

DiTS proposes Diffusion Transformers for Time Series, a multimodal model that treats historical targets and external covariates as separate inputs. It jointly conditions future generation on both target history and covariates to forecast time series. This approach uses time-aligned modulation to link these modalities effectively, achieving strong performance in both point and distributional forecasting.

Key concepts

Multimodal Diffusion Transformer: DiTS models endogenous targets (what you want to predict) and exogenous covariates (external data) as two distinct types of input. It combines the generative power of Diffusion Transformers with a multimodal approach, allowing the model to learn from both historical time series data and external information simultaneously for better forecasting.
Time-aligned Modulation: This mechanism extends a technique called AdaLN by incorporating a temporal-alignment prior. It generates modulation parameters for the diffusion process using aligned covariates and the current diffusion time, ensuring that the conditioning applied to the model is correctly synchronized with both where in time and which external data is relevant.
Flow Matching: The model uses conditional flow matching, specifically rectified flow, to learn a continuous transformation from random noise to the actual data distribution. This training objective allows covariate-dependent distributional information to influence velocity prediction, even when the direct regression target remains unchanged.
Covariate Encoder: This component encodes exogenous covariates separately while ensuring they maintain temporal correspondence with the target patches. It preserves patch-wise external information, creating a sequence of conditions where each time step has its own specific external context to guide the generation process.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 20 episodes
Retrieval-Augmented Generation (RAG) — explained in 19 episodes
Federated Learning — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 17 episodes
Interpretability — explained in 15 episodes

This episode discusses

The paper

DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters · Read on arXiv

Haoran Zhang, Haixuan Liu, Yong Liu, Yunzhong Qiu, Yuxuan Wang, Jianmin Wang, Mingsheng Long

School of Software, BNRist, Tsinghua University

While generative modeling facilitates probabilistic time series forecasting, incorporating heterogeneous exogenous information remains challenging. Diffusion Transformers (DiT) provide a scalable framework for conditional generation, yet their adaptation to forecasting calls for conditioning mechanisms tailored to time series. Endogenous targets and exogenous covariates differ in sources, semantics, and statistical characteristics, while sharing temporal coordinates that support fine-grained conditional guidance. Covariates can describe future variability and temporal dependence beyond the conditional mean targeted by direct regression. Motivated by these considerations, we propose Diffusion Transformers for Time Series (DiTS), a Multimodal Diffusion Transformer for covariate-aware forecasting. DiTS models endogenous targets and exogenous covariates as distinct modalities, jointly conditioning future generation on target history and available covariates. Flow matching makes covariate-dependent distributional information relevant to velocity prediction conditioned on noisy future states. We introduce Time-aligned Modulation, extending AdaLN with the temporal-alignment prior to generate patch-wise modulation parameters from aligned covariates and diffusion time. Across diverse covariate-aware forecasting tasks, DiTS achieves strong performance in both deterministic and probabilistic forecasting, demonstrating the effectiveness of conditional generation for both point and distributional forecasting.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters".

Jane: The gist: DiTS proposes Diffusion Transformers for Time Series,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, looking at DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters, the paper argues that this approach is a better way to do covariate-aware forecasting than just using standard methods where you try to condition on the target history and covariates separately >

Jane: The authors suggest that they model the endogenous targets and the exogenous covariates as distinct modalities, so you get information from both sources influencing the future generation simultaneously >

Lu: They use conditional flow matching, which is a technique where they learn a velocity field conditioned on target history and observed covariates, using probability paths from noise to future targets >

Meng: That means they're not just trying to predict the mean of the future; they are learning how the distribution shifts based on what those external covariates tell them about what might happen next >

Lalam: The paper specifically mentions that this covariate-dependent distributional information becomes relevant even when you don't change the original conditional mean, which is a big point for understanding variability >

Tom: And they introduce something called Time-aligned Modulation, which extends AdaLN to create patch-wise modulation parameters from aligned covariates and diffusion time >

Jane: That sounds like the mechanism that connects these two different modalities—the target history and the covariate information—so they know exactly where to apply the conditioning >

Conclusion: Tom: So, looking at DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters by Haoran Zhang and colleagues, it seems like the main contribution is creating this framework for joint modeling of targets and covariates within a diffusion transformer setting >

Jane: The authors show that they can achieve strong performance in both deterministic and probabilistic forecasting across diverse tasks, which means you get good point predictions and good distribution predictions using this method >

Lu: It shows that organizing information at the correct granularity, like treating them as distinct modalities with temporal alignment, provides complementary opportunities beyond just refining the tokenization or attention mechanisms alone >

Meng: From an engineering standpoint, they claim favorable quality with few sampling steps, which means you don't need a ton of computation to get a good forecast >

Lalam: So what this means for us is that we can model the actual behavior of external factors—how they affect the variability—not just their average effect on the target series >

More episodes

← Home