Structure-preserving uncertainty quantification for GENERIC dynamics

arXiv:2608.12624v1 [cs.LG]

The paper addresses a critical gap in uncertainty quantification (UQ) for structure-preserving machine learning models. The authors state: "Structure-preserving machine learning embeds physical structure directly into model architectures, yet uncertainty quantification (UQ) for such hard-constrained models remains limited because standard UQ methods may violate the encoded admissibility conditions, require architectural modifications, or impose substantial computational costs."

The core limitation is articulated as follows: "hard-encoded constraints restrict the admissible model class itself, so sampled UQ realizations must remain in a space that is closed under those constraints. Randomizing unconstrained weights or perturbing intermediate activations can move a realization outside that space, while preserving admissibility by independent replication requires training complete constrained models."

The authors note that existing UQ methods have significant shortcomings for hard-constrained models:

• Bayesian neural networks (BNNs) "can preserve structure only when the prior, posterior approximation, and inference procedure are defined over the constrained surrogate itself; otherwise, generic weight-space uncertainty does not enforce the required relations, and posterior inference over a constrained architecture can be expensive at scale."

• Deep ensembles can preserve constraints by training several full constrained models with cost growing essentially linearly in the ensemble size.

• MC dropout uses stochastic internal masks, so hard-encoded properties may be lost unless every masked subnetwork still satisfies the same constraints.

The authors propose S-PENNs, a general framework for UQ in scientific machine learning models with hard architectural constraints, instantiated for GENERIC (General Equation for Non-Equilibrium Reversible-Irreversible Coupling) dynamics.

S-PENNs build on the Epistemic Neural Networks (ENNs) introduced in [47], in which a conventional base network is augmented by a lightweight auxiliary network called the epinet. The framework treat[s] the deterministic N-GENNs as base networks and attach[es] a separate epinet to each building block of N-GENNs so that each epinet inherits the architectural properties of its corresponding block.

The critical innovation is that since the perturbed blocks are then assembled through the same structure-preserving reparameterizations as in N-GENNs, every sampled realization remains thermodynamically admissible by construction.

The S-PENNs framework attaches epinets to four thermodynamic building blocks of N-GENNs:

• The reversible operator network L̃ψ

• The energy network Eψ

• The entropy network Sψ

• The raw dissipation-potential network Ξ̃ψ

The augmented blocks are written as:

L̃ϑ(x,z) = L̃ψ(x) + σϕL̃(h̄ψL̃(x), z)

Eϑ(x,z) = Eψ(x) + σϕE(h̄ψE(x), z)

Sϑ(x,z) = Sψ(x) + σSϕ(h̄Sψ(x), z)

Ξ̃ϑ(x,x*,z) = Ξ̃ψ(x,x*) + σΞ̃ϕ(h̄Ξ̃ψ(x,x*), z)

A shared epistemic index z is used across all augmented blocks, so each draw selects a joint perturbation of the full GENERIC parameterization.

The authors introduce a novel construction for global, input-independent quantities (such as the tensor Bψ in the GENERIC reparameterization). They note this is analogous to inverse problems with unknown physical parameters, where the parameters are inferred jointly with the solution field but are not themselves functions of the input variables.

The proposed linear epinet branch is driven only by the epistemic index:

Bϑ(z) = Bψ + σϕB(z)

σϕB(z) = σϕB,learn(z) + w σB,prior(z)

This enforces input-independent uncertainty by construction with no soft-penalized variance term.

For each epistemic draw z, the reparameterized reversible operator is:

Lϑ(x,z) = QSϑ(x,z)⊤ L̃ϑ(x,z)QSϑ(x,z)

The augmented dissipation potential is:

Ξϑ(x,x*,z) = Ξ̃ϑ(x, PEϑ(x,z)x*, z) − Ξ̃ϑ(x,0,z) − Dx*Ξ̃ϑ(x,x*,z)x*=0 PEϑ(x,z)x*

The S-PENNs vector field is:

gϑ(x,z) = Lϑ(x,z)DEϑ(x,z) + Dx*Ξϑ(x,x*,z)x*=DSϑ(x,z)

The authors emphasize: Since the reversible and irreversible constraints are built into the augmented parameterization, gϑ(·,z) is thermodynamically consistent for every fixed draw of z.

S-PENNs are trained in two stages:

1. Base network training: the deterministic base networks are fitted by minimizing the loss in Eq. (20), which measures the l2 error between the predicted and ground-truth time derivatives.

2. Epinet training: the base-network parameters ψ are frozen and only the epinet parameters ϕ are optimized, with the objective in Eq. (21) using sampled epistemic indices.

The authors use split conformal prediction as a post-hoc calibration method to produce prediction intervals with finite-sample marginal coverage guarantees. They note that standard regression calibration techniques, including variance scaling and CDF-based corrections, are less suitable here because they need additional model fitting and can modify the implied predictive distribution.

The conformal procedure uses trajectory-level exchangeability: For dynamical systems, the exchangeable unit is the full trajectory rather than an individual timestep, since time levels within one rollout are coupled by the evolution equations.

The normalized nonconformity score is defined as:

si(j) = (Xi(j) − µ̂ϑ(xi(0); j)) / σ̂ϑ(xi(0); j)

The resulting prediction interval satisfies, for every fixed index j:

P(X(j) ∈ Ĉ1−α(x(0); j)) ≥ 1 − α

System: State x = (q, p, ϵ) with total energy E(x) = p2/(2m) + kq2/2 + ϵ, canonical Poisson operator, and quadratic dissipation potential.

Data: 100 trajectories with 1000 state records each, split into 60 training, 50 calibration, and 40 testing trajectories.

Results:

• All three S-PENNs variants (half-normal, uniform, exponential epistemic index distributions) have lower median errors than MC dropout for q and p.

• The half-normal distribution gives the lowest median errors for all three state variables among S-PENNs variants.

• For all three S-PENNs variants, the median energy rate remains centered at zero, and the 95% intervals stay at the 10−7 scale.

• The pointwise minimum entropy-production rate over the sampled rollouts remains positive throughout the trajectory.

• Wall times: deep ensembles 9.1×103 s, MC dropout 2.3×102 s, S-PENNs 1.2–1.3×102 s.

• Scaling the deep ensemble to 2000 members gives 3.6×105 s, which is three orders of magnitude higher than the cost for S-PENNs.

The authors conclude: "S-PENNs provide a practical balance between structural fidelity, predictive quality, and computational cost across both finite-dimensional and field-valued dynamical systems. S-PENNs required typically 1 − 3 orders of magnitude less serial wall time than deep ensembles, improved substantially over MC dropout in probabilistic scores, and showed only moderate sensitivity to the reference distribution for the epistemic index."

The authors identify two main limitations:

1. Noise-free measurements: "all three numerical examples use noise-free state measurements. With noisy states, the problem becomes an errors-in-variables (EiV) problem: the clean trajectory should satisfy energy conservation and nonnegative entropy production, but the noise-corrupted trajectory need not."

2. Scalability: "the present study evaluates S-PENNs only on two low-dimensional ODE systems and a one-dimensional PDE system, and the computational cost advantage of S-PENNs relative to deep ensembles is expected to be further amplified in higher-dimensional settings."

Future work will address noisy state measurements by separating the thermodynamically admissible trajectory from the measurement process and evaluate scalability on higher-dimensional thermomechanical systems.

The authors note: "the S-PENNs principle is not tied to GENERIC dynamics. It is relevant to other structure-preserving or physics-constrained machine learning models in computational mechanics and may also be useful for pretrained foundation models in science and engineering whose architectures or adaptation procedures encode physical priors."

The paper includes an appendix demonstrating the input-independent epinet branch for inverse problems with unknown global physical parameters, validated on:

• Inverse Kraichnan–Orszag system: The proposed construction gives parameter estimates a = 0.98 ± 0.07 and b = 1.01 ± 0.06, compared to B-PINNs-HMC's a = 0.86 ± 0.06 and b = 1.06 ± 0.03.

• Inverse Korteweg–de Vries problem: The proposed construction gives κ1 = 1.50 ± 0.05 and κ2 = 0.37 ± 0.02, compared to B-PINNs-HMC's κ1 = 1.39 ± 0.03 and κ2 = 0.26 ± 0.01.

Based on this paper, here are the specific improvements I can make to AI systems:

Improvement: I can implement the S-PENNs framework to add uncertainty quantification to any hard-constrained neural network without violating physical admissibility. Instead of perturbing weights or activations arbitrarily, I attach lightweight epinets to each constrained building block and reassemble them through the same structure-preserving reparameterizations.

Capability: The improved system can generate prediction intervals for physics-based models (e.g., thermodynamic systems, mechanical systems) while guaranteeing that every sampled realization satisfies conservation laws (energy conservation, nonnegative entropy production) by construction. This works even when the base model has complex nonlinear constraints that would be violated by standard UQ methods.

Improvement: I can incorporate the novel linear epinet branch driven only by the epistemic index for systems with global, input-independent quantities. This is particularly useful for inverse problems where unknown physical parameters must be inferred jointly with the solution field.

Improvement: I can implement the split conformal prediction method that treats full trajectories as exchangeable units rather than individual timesteps, using normalized nonconformity scores based on predicted means and uncertainties.

Improvement: I can adopt the training protocol where deterministic base networks are first trained to minimize prediction error, then epinets are trained with frozen base parameters using sampled epistemic indices.

Improvement: I can implement the framework's robustness to different reference distributions for the epistemic index (half-normal, uniform, exponential), with the half-normal showing best performance in the paper's experiments.

Improvement: I can replace deep ensembles with S-PENNs for physics-constrained models, achieving 1–3 orders of magnitude reduction in serial wall time while maintaining or improving probabilistic scores (lower CRPS, ES).

Improvement: I can implement the framework's ability to handle non-quadratic, non-smooth dissipation potentials (as demonstrated in the viscoplastic example), which are common in plasticity, friction, and phase-field models.

Improvement: I can generalize the S-PENNs principle beyond GENERIC dynamics to other structure-preserving models (e.g., Hamiltonian systems, symplectic integrators, port-Hamiltonian systems) by attaching epinets to their respective building blocks.

Improvement: I can implement the framework's planned extension to handle noisy state measurements by separating the thermodynamically admissible trajectory from the measurement process, addressing the errors-in-variables problem.

Improvement: I can leverage the framework's computational advantage (which amplifies with dimensionality) to scale UQ to higher-dimensional thermomechanical systems where deep ensembles become intractable.

Structure-preserving machine learning embeds physical structure directly into model architectures, yet uncertainty quantification (UQ) for such hard-constrained models remains limited because standard UQ methods may violate the encoded admissibility conditions, require architectural modifications, or impose substantial computational costs. In this work, we propose Structure-Preserving Epistemic Neural Networks (S-PENNs), a general framework for UQ in scientific machine learning models with hard architectural constraints, and instantiate it for GENERIC (General Equation for Non-Equilibrium Reversible-Irreversible Coupling) dynamics. S-PENNs preserve the structural constraints of a pretrained model by attaching lightweight epinets to its constrained components, ensuring that every sampled realization remains physically admissible by construction. When applied to GENERIC dynamics, such a proposed framework yields thermodynamically consistent rollouts that preserve the first and second laws. Furthermore, we combine S-PENNs with split conformal prediction as a post-hoc calibration method to produce prediction intervals with finite-sample marginal coverage guarantees. We validate S-PENNs on three numerical examples: a harmonic oscillator coupled to a heat bath and an idealized chemical motor, both governed by ODEs, and a one-dimensional viscoplastic model governed by PDEs. Across all three examples, S-PENNs produce thermodynamically consistent stochastic realizations and well-calibrated prediction intervals while reducing the computational cost by about one to three orders of magnitude compared to deep ensembles. Although the present study focuses on GENERIC dynamics, S-PENNs can be extended more broadly to scientific machine learning models in computational mechanics with either hard or soft constraints.