Generator-Guided Inverse Sampling for L'evy-Driven Generative Models

This paper studies inverse sampling for Lévy-driven generative models from the perspective of Markov generators. Unlike conventional diffusion models, Lévy-driven dynamics involve infinite jump activities, which makes their reverse process nonlocal and difficult to characterize using score information alone. We address this challenge by analyzing the forward and reversed generators. It is derived that the reversed jump component generally becomes a state-dependent Markov jump process governed by a nonlocal density ratio. This observation motivates a structured reverse sampler that decomposes the dynamics into diffusion, small jump, and large jump components. Based on this characterization, we develop a computationally tractable sampler for a class of isotropic linear Lévy SDEs with symmetric α-stable jump components. For the jump component, the neural network is used only to amortize the rate of large jump activities, while jump amplitudes are generated from analytically derived conditional distributions, which improves interpretability and controllability. Efficient implementation techniques are further introduced under this setting to avoid expensive high-dimensional integration and sampling. The sampler is further adapted to approximate observation-guided sampling and applied to OFDM-SISO channel estimation under mixed Gaussian and impulsive noise. Simulations show robust estimation performance with a favorable tradeoff between complexity and performance.

The paper's main contributions are summarized as follows. First, the authors provide a generator-based characterization of the time-reversed dynamics of Lévy-driven Markov processes. The analysis shows that the reversed jump component is generally a state-dependent Markov jump process whose kernel involves a nonlocal density ratio. This result clarifies why reverse sampling for Lévy-driven processes cannot, in general, be reduced to score-based local diffusion updates. Second, motivated by this characterization, they develop a practical inverse sampler for isotropic linear Lévy SDEs with symmetric α-stable jump components. The reverse dynamics are decomposed into diffusion, small jump, and large jump parts. The small jump contribution is approximated through a Gaussian surrogate, while large jumps are handled explicitly through a rate-and-amplitude decomposition. The neural network is used to amortize the large jump rate, whereas jump amplitudes are sampled from analytically derived conditional distributions. Third, they adapt the proposed sampler to approximate observation-guided sampling under observation models and demonstrate its use in OFDM-SISO channel estimation with mixed Gaussian and impulsive noise. In particular, the prior-trained jump rate network is retained as a proposal mechanism and the observation likelihood is incorporated through reweighted distributions. The resulting algorithm provides a robust estimator under heavy-tailed perturbations and offers an interpretable alternative to fully neural reverse-transition parameterizations.

The paper begins by noting that conventional diffusion models are mainly built on the Wiener process, and their reverse process is characterized by local score information and infinitesimal Gaussian perturbations. As a result, long-range probability transport is usually achieved through a sequence of small reverse steps. This local update mechanism may affect sampling efficiency and modeling flexibility, especially when the target distribution has heavy tails, impulsive components, or highly multimodal structures. Lévy-driven generative models provide a natural extension by incorporating jump components into the forward dynamics. The resulting nonlocal transitions are well suited for modeling heavy-tailed perturbations. However, these advantages bring new challenges for inverse sampling. Unlike the case without jumps, the time reversal of a Lévy-driven process cannot be fully characterized by local score information alone. Its reversed jump component depends on a nonlocal density ratio. It generally becomes a Markov jump process with state-dependent kernels, which makes the inverse sampler difficult to design.

The paper considers a Lévy process with drift and diffusion components, written as dXt = b(Xt, t)dt + ΦG(t)dWt + ΦS(t)dLt, where D denotes the dimension of the SDE, b(Xt, t) is the drift coefficient, Wt and Lt denote the Wiener process and the α-stable process, respectively, and ΦG(t) and ΦS(t) denote the scaling matrices. The paper focuses on the SDE under the assumptions that the Wiener process and the Lévy process are mutually independent, and that Lt is a D-dimensional symmetric isotropic α-stable Lévy process with stability index α ∈ (0, 2).

The paper derives the backward generator in Proposition 1. For any test function f ∈ Cc2(RD), the generator of the reversed process can be written as (LB f)(xt) = [−b(xt, t) + Σ(t)∇ log p(xt)] · ∇f(xt) + 1/2 Tr(Σ(t)∇2f(xt)) + (LJ,B f)(xt), where (LJ,B f)(xt) = p.v. ∫ [f(xt + v) − f(xt)] p(xt+v)/p(xt) νt(dv). The jump integral is understood in the Cauchy principal-value sense. For α < 1, the integral is locally absolutely integrable under standard smoothness assumptions. For α ≥ 1, the uncompensated integral is generally not absolutely integrable near the origin and should be interpreted in the Cauchy principal-value sense. The paper notes that the backward kernel is related to the density ratio p(xt+v)/p(xt), which contains global information. Therefore, the jump part only belongs to a Markov-type jump process rather than a Lévy process. This process is much more general and is not as mathematically tractable as the forward process. Consequently, the total reversed SDE corresponding to the forward process cannot be expressed as the superposition of a modified drift term, a diffusion term, and a stable process.

The paper then decomposes the reverse sampling process. The original integral is decomposed by truncation with threshold ϵ: ∫(f(x+v)−f(x))K(dv) = ∫∥v∥≤ϵ(f(x+v)−f(x))K(dv) + ∫∥v∥>ϵ(f(x+v)−f(x))K(dv). Jumps with amplitude larger than ϵ are treated as large jumps. For large jumps, the occurrence frequency is finite. The number of large jumps follows a Poisson distribution with rate λ ≜ ∫∥v∥>ϵ K(dv). For a sufficiently small time interval ∆t, the occurrence of a large jump can also be approximated by a Bernoulli distribution with probability λ∆t. For the case with ∥v∥ ≤ ϵ, the Lévy measure is not integrable, and there are infinitely many small jumps. The paper adopts a more reasonable approach: after truncation, the small jumps have a finite second moment. Thus, they replace the small jump generator by a Gaussian generator matching the first two local moments. The small jump part within [t, t + ∆t] can be written as X≤ϵ t,∆t ≈ b≤ϵ t,∆t∆t + √(∆t)(Σ≤ϵ t,∆t) 1/2W, where W is the standard Gaussian RV. The drift and covariance are approximated as b≤ϵ t,∆t ≈ A νts θ1(xt, t) and Σ≤ϵ t,∆t ≈ A νt, where A νt ≜ ∫∥v∥≤ϵ vv T νt(dv) has a closed-form expression: A νt = (2π D/2C D,α,σSϵ 2−α)/(DΓ(D/2)(2−α)) I D.

The paper introduces Theorem 1 on marginally trainable functions. It states that the optimization problem arg minθ E x∼p(x)[∥f θ(x) − l(x)∥2] is equivalent to arg minθ E x,x0∼p(x,x0)[∥f θ(x) − l(xx0)∥2] if l(x) = ∫ l(xx0)p(x0x)dx0. The paper notes that this theorem is general since it includes conventional training frameworks such as score matching and flow matching.

For sampling of large jumps, the paper shows that the unnormalized jump distribution can be written as q(xt, v) ≈ (Σ N j=1 p(xt+vx0,j)) / (Σ N j=1 p(xtx0,j)) νt(dv), for ∥v∥ > ϵ. The overall jump rate is λ(xt) = ∫∥v∥>ϵ p(xt+v)/p(xt) νt(dv). However, this introduces an unacceptable computational burden during inference. To make large jump sampling more efficient, the paper uses a neural network to amortize part of the computational burden. The paper chooses to learn the large jump rate λ(xt). Proposition 2 provides the loss function for jump activities: L(θ) = E t∼U[0,1], xt∼p(xtx0), x0∼p(x0)[∥g θ2(xt, t) − λ(xtx0)∥2].

To efficiently calculate the large jump rate, the paper makes further assumptions. Assumption 5 states that b(Xt, t) is an affine function of Xt, i.e., b(Xt, t) = RXt + s. Assumption 6 restricts ΦG(t) and ΦS(t) to diagonal matrices. Proposition 3 gives the distribution of Xt: Xt = µ(t, x0) + G(t) + S(t), where G(t) ∼ N(0, ΣG) and S(t) follows an α-stable distribution. Corollary 1 considers the special case where R, ΦG(t), and ΦS(t) are all scaled identity matrices. The paper then uses an approximate PDF for the mixed noise, given by f̃ XM(x) = 1/k1 [c1g0 exp(−x2/(4γg2)) + (αγsCα)/(c2 + x α+1)], which can be extended to high-dimensional cases.

The paper derives explicit expressions for the integrals P1(ϵ) = ∫∥v∥>ϵ exp(−∥v+µ∥2)νt(dv) and P2(ϵ) = ∫∥v∥>ϵ (c2 + ∥v+µ∥2)−(α+D)/2 νt(dv). These are transformed into polar coordinates, reducing the D-dimensional integral to a one-dimensional integral. For P1(ϵ), applying the integral identity yields P1(ϵ) = 2π 2−D/2C D,α,σS∥µ∥ D/2 exp(−∥µ∥2) ∫ ϵ+∞ exp(−r2)r−α−D/2 I(D−2)/2(2r∥µ∥)dr. For P2(ϵ), a similar expression is obtained using the hypergeometric function.

For high-dimensional sampling, the paper designs a fast sampling method for large jumps. The normalized target distribution is q̃(xt, v) ≈ (Σ N j=1 p(xt+vx0,j)νt(dv)) / (NQt,ϵ), where Qt,ϵ = ∫∥v∥>ϵ p(xt+v)νt(dv). This can be rewritten as a mixture distribution: q̃(xt, v) = Σ N j=1 gQ(x0,jxt, t)q̃(xt, vx0,j), where gQ(xxt, t) is a weighted distribution. The paper first randomly chooses a data sample x0 according to the distribution gQ(xxt, t), then generates a sample from the distribution conditioned on x0. To reduce complexity, the paper truncates gQ(xxt, t) by choosing the K samples with the largest approximate mixture weights. The sampling procedure is summarized in Algorithm 2, which samples the radial variable r, the angular variable z, and the remaining direction.

The paper also discusses an approximate observation-guided sampler. Proposition 4 states that the weighted distribution gQ(xxt, t) used for generating large jump samples should be replaced by g posterior,Q(x) = Σ N j=1 1 x0,j(x)p YX0(yx0,j)Qt,ϵ,j / (Σ N j=1 p YX0(yx0,j)Qt,ϵ,j). The paper notes that the prior-trained rate network is used as an amortized proposal rate for large jump events in the posterior sampler, and the observation information is incorporated into the large jump amplitude and direction through likelihood-reweighted empirical mixture weights.

The numerical results validate the proposed inverse sampler for OFDM-SISO channel estimation under mixed noise. The channel model uses a frequency-selective SISO-OFDM channel modeled by a tapped-delay-line (TDL) response with delay-power profiles from 3GPP TR 38.901 (TDL-A, TDL-C, TDL-D). The mixed channel noise consists of WGN and IN described by the SαS distribution. The proposed approach achieves the best performance among the baselines in terms of both NMSE and BER. The paper shows that the neural network for jump rate prediction converges very fast and only requires about 60 epochs with different values of α. Ablation studies show that using the accurate PDF does not bring a considerable performance gain, while the complexity of Accurate PDF is unacceptable. The truncation error from top-K is also not significant, and using the prior large jump rate is an efficient scheme. The time consumption comparison shows that the proposed method takes 5.6s for α = 1.2 and 5.5s for α = 1.8, while the accurate PDF takes 1717.6s and 1698.4s, respectively, and the accurate rate takes 6125.3s and 6379.6s, respectively.

In conclusion, the paper proposed a generator-guided inverse sampling algorithm for a class of isotropic linear Lévy-driven SDEs with symmetric α-stable jump components. The authors first derived the expression of the backward generator and used it to decompose the full reverse sampling process into the diffusion part and the jump components. Then, by theoretically analyzing the inverse sampling process from a statistical perspective, they determined the objective that can be learned by a lightweight network. They also designed an efficient reverse sampling method by introducing several efficient approximation operations. As an application, the proposed inverse sampling algorithm outperformed the considered baselines in channel estimation under mixed channel noise across different scenarios. Moreover, numerical results showed that the proposed approach achieves a good balance between performance and complexity.

Improvements to AI Systems:

1. Heavy-Tailed Generative Modeling: Enable generative models to handle data with heavy-tailed, impulsive, or highly multimodal distributions (e.g., financial returns, network traffic, sensor noise) by incorporating Lévy-driven jump processes instead of relying solely on Gaussian diffusion. The improved system can sample from such distributions more efficiently and accurately, avoiding the need for many small reverse steps.

2. Nonlocal Reverse Sampling: Implement a reverse sampler that explicitly models state-dependent Markov jump processes for the jump component, rather than approximating all dynamics with local score-based updates. This allows the AI to capture long-range probability transport and rare-event transitions that are missed by conventional diffusion models.

3. Hybrid Neural-Analytical Sampler: Use a lightweight neural network only to predict the large-jump rate (a scalar), while generating jump amplitudes and directions from analytically derived conditional distributions. This reduces computational cost, improves interpretability, and ensures that the sampler remains controllable—useful for applications requiring calibrated uncertainty or physical constraints.

4. Efficient High-Dimensional Jump Sampling: Replace expensive high-dimensional integration with closed-form polar-coordinate reductions and top-K truncation of mixture weights. The improved system can perform fast sampling in high dimensions (e.g., image or signal spaces) without Monte Carlo integration over the full jump kernel, achieving orders-of-magnitude speedup (e.g., from 6000s to 5.6s in the paper’s channel estimation task).

5. Observation-Guided Posterior Sampling: Adapt the prior-trained generative sampler to conditional generation by reweighting the empirical mixture weights with likelihood terms. The improved AI can perform approximate posterior inference (e.g., channel estimation, denoising, or inpainting) under heavy-tailed noise, without retraining the generative model—enabling robust Bayesian inference in non-Gaussian settings.

6. Robust Channel Estimation and Communication Systems: Deploy the improved sampler for OFDM-SISO channel estimation under mixed Gaussian and impulsive noise, achieving lower normalized mean squared error (NMSE) and bit error rate (BER) than existing baselines. The system can operate in real-time due to the efficient sampling, making it suitable for practical wireless communications.

7. Interpretable Jump Dynamics: Provide a clear decomposition of reverse dynamics into diffusion, small-jump (Gaussian surrogate), and large-jump (Poisson process) components. This enables users to inspect and control the influence of each component—useful for debugging, safety, or domain-specific constraints in generative AI.

8. Fast Training for Rate Networks: Train the jump-rate network with a simple mean-squared-error loss against conditional rates, converging in 60 epochs. The improved system can be quickly adapted to new stability indices (α) or noise profiles without extensive retraining, facilitating deployment in changing environments.