Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates

summary

The gist

The gist: Neural surrogates can gain accuracy and stability in finely-discretized regimes by predicting the temporal derivative instead of directly predicting the next state, which allows for

In short

The paper explores an alternative method for neural surrogates to solve partial differential equations (PDEs). Instead of predicting the next state directly, this framework trains a neural model to predict the temporal derivative. This allows an external ODE integrator to advance the solution over time during inference, offering better accuracy and stability in finely-discretized regimes and enabling flexible time-stepping.

Key concepts

State Prediction Framework: The traditional method where a neural network is trained to directly predict the next state of the system, $u_{n+1} = F_ heta(u_n)$. While simple, this framework can be rigid and constrain models to specific temporal discretizations when predicting physics.
Temporal Derivative Prediction: The proposed alternative where the neural network is trained to predict the rate of change (the temporal derivative) of the system. This decouples spatial learning from time evolution, allowing an external ODE integrator to handle the actual time stepping, which improves flexibility.
ODE Integrator: A numerical method used during inference to evolve a solution forward in time based on the predicted temporal derivatives. Methods like Forward Euler or Runge-Kutta are employed. This allows for flexible time-stepping during inference, enabling arbitrary temporal resolutions.
Training Loss Function ($L_ heta$): The objective function used to train the neural model, which can be defined based on whether the model predicts the next state or the temporal derivative. Precomputing derivatives for training speeds up this process, although calculating them dynamically incurs a small overhead.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 20 episodes
Retrieval-Augmented Generation (RAG) — explained in 18 episodes
Federated Learning — explained in 17 episodes
Flow Matching — explained in 16 episodes
Quantum Fisher Information (QFI) — explained in 16 episodes

This episode discusses

The paper

Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates · Read on arXiv

Anthony Zhou, Amir Barati Farimani

Department of Mechanical Engineering, Carnegie Mellon University · Machine Learning Department, Carnegie Mellon University

Neural surrogates for partial differential equations (PDEs) have become popular due to their potential to quickly simulate physics. With a few exceptions, neural surrogates generally treat the forward evolution of time-dependent PDEs as a black box by directly predicting the next state. While this is a natural and easy framework for applying neural surrogates, it can be an over-simplified and rigid framework for predicting physics. In this work, we evaluate an alternate framework in which neural solvers predict the temporal derivative and an ODE integrator forwards the solution in time, which has little overhead and is broadly applicable across model architectures and PDEs. We find that by simply changing the training target and introducing numerical integration during inference, neural surrogates can gain accuracy and stability in finely-discretized regimes. Predicting temporal derivatives also allows models to not be constrained to a specific temporal discretization, allowing for flexible time-stepping during inference or training on higher-resolution PDE data. Lastly, we investigate why this framework can be beneficial and in what situations does it work well.

DOI: 10.1016/j.cma.2025.117990

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Predicting Change, Not States".

Jane: The gist: Neural surrogates can gain accuracy and stability in finely-discretized regimes by predicting the temporal derivative instead of directly predicting the next state,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So, looking at this paper, "Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates," it’s really about finding a different way to train these physics simulators.

Jane: The authors are suggesting that instead of directly predicting the next state u n+one we should train the AI to predict the temporal derivative d u / d t <ref:2412.13074#pg2>.

Lu: This framework offers a way to improve accuracy and flexibility because it lets you decouple the spatial dynamics from the time integration step, which is useful across different PDE setups <ref:2412.13074#pg3>.

Meng: It’s a practical trade-off where you might have a little extra inference cost with the ODE integrator, but you get better performance on certain problems <ref:2412.13074#pg3>.

Tom: The implication is that this isn't just about making one model better; it’s about creating a training and inference framework that is agnostic to the specific PDE or the surrogate architecture <ref:2412.13074#pg3>.

Jane: It suggests that for simulating complex physics, we might be better off focusing on learning how things change over time rather than trying to force the network to guess where they will end up <ref:2412.13074#pg1>.

Lu: They are opening the door for more research into this alternative training and inference structure across a wider variety of problems and models <ref:2412.13074#pg3>.

Tom: It gives us a solid alternative framework to explore when we're building these next-generation physics simulators <ref:2412.13074#pg1>.

Conclusion: Tom: So we’re wrapping up this look at "Predicting Change, Not States: An Alternate Framework for Neural PDE Surrogates." Basically, the paper argues that instead of training an AI to predict the exact next state in time, it's smarter to train it to predict the rate of change—the derivative.

Jane: That’s a big shift from just predicting u at time t+one. So what’s the core idea there?

Tom: The core idea is that you can decouple the spatial part of the problem from the time integration. Instead of forcing your neural surrogate to guess every single tiny step, it learns how fast things are changing right now.

Lu: And because we have this derivative prediction, we can then use a standard ODE integrator—like Runge-Kutta or Forward Euler—to evolve the solution forward in time. It’s like the AI does the "how fast" part and a separate tool handles the "where to next" part.

Meng: From an engineering standpoint, that sounds promising for stability. If we can decouple them, maybe we don't have to worry as much about making one massive neural network handle every single temporal discretization step at once.

Lalam: Based on the results, they found that this derivative approach actually led to lower errors compared to the direct state prediction method across different equations. It’s more stable for chaotic systems like the Kolmogorov Flow, too.

Jane: So for someone just listening, it means we can get better accuracy in those finely-discretized regimes where time steps are really tight, because the model isn't trying to be perfect at every single point in time simultaneously.

Tom: Exactly. The paper suggests this framework is agnostic to the specific PDE or even the surrogate architecture you're using right now. It’s a change in training objective that can actually make a difference when you’re simulating complex physics.

Lu: The authors are really interested in *why* it works and under what conditions it’s best. They show that while the integrator still introduces some numerical error, it’s usually much smaller than the error you get from using a bad model at all.

Meng: So, there's a trade-off where you introduce a small numerical integration cost for better accuracy on certain dynamics. That makes sense for deployment considerations.

Jane: It seems like the authors are really pushing this idea to see if it can replace the direct state prediction method in more real-world scenarios across different physics problems.

Tom: Right. So, we’ve seen how predicting change instead of states opens up a new way to think about training these AI surrogates. Next up, we're going to look at the specific training loss functions they used and how that actually looks in practice.

More episodes

← Home