ELVAE: Evidential Learning-Based Variational Autoencoder for Uncertainty-Aware Generation

ELVAE: Evidential Learning-Based Variational Autoencoder for Uncertainty-Aware Generation

Abstract: Variational autoencoders (VAEs) generate samples from probabilistic latent representations but do not explicitly distinguish uncertainty in the latent location from variability around that location. We formulate ELVAE, an evidential learning-based VAE in which each latent coordinate is governed by an input-dependent normal-inverse-gamma (NIG) posterior. The hierarchy yields an explicit latent-location uncertainty, u epi = beta/[nu(alpha − 1)], that can stratify posterior anchors and modulate generation. In a 10,000-image MNIST pilot, samples were ranked by within-class u epi and evaluated with a classifier trained only on real MNIST. Classifier error increased from 26.30% in the bottom 20% of uncertainty to 37.80% in the top 20% (1.437×, 95% bootstrap CI 1.33–1.58). A zero-displacement control, z = gamma, retained most of this contrast (1.395×), showing that the dominant effect reflects anchor re-generation reliability. Among anchors correctly re-generated at z = gamma, an uncertainty-scaled perturbation induced semantic failure in 1.97% of the low-u epi group versus 5.92% of the high-u epi group (3.01×, 95% CI 2.03–4.75). Two caveats are reported alongside the headline: under unnormalized global u epi ranking the contrast is essentially null (1.015×), so within-class normalization is not cosmetic, and across three random seeds the headline ratio ranges from 1.126 to 1.437. The formulation is also mathematically well posed: the ELVAE objective is an exact ELBO for the corresponding hierarchical generative model, and direct NIG-to-NIG regularization identifies an uncertainty decomposition that the marginalized Student-t latent law alone does not. These results support u epi as a useful variable for posterior-anchored uncertainty-aware generation, while distinguishing anchor reliability from perturbation-attributable failure.

Introduction: Generative AI is typically evaluated by realism, fidelity, diversity, or downstream usefulness. For training-data generation and robustness testing, however, a second question is equally important: how uncertain is the model about the latent state from which a particular synthetic image is generated? Two generated images can both look plausible while one lies in a well-constrained latent region and the other is produced from a latent location for which the model has weak evidence. A conventional VAE models q phi (z y) = N mu phi (y), diag sigmaphi2 (y), where mu phi and sigmaphi2 are deterministic network outputs. This provides stochastic generation, but the model has only one level of latent uncertainty. Evidential learning suggests a richer construction: place a distribution over the latent mean and variance themselves. We call the resulting model ELVAE. ELVAE turns uncertainty into a generation control variable: low uncertainty can identify more reliable synthetic samples, while high uncertainty can be deliberately retained to create difficult stress-test samples. This is particularly attractive for scientific and medical generation, where synthetic images may be used to enlarge scarce training sets or to probe failure modes of a downstream network.

Methodology:

Evidential latent hierarchy: For each latent coordinate k = 1,..., K, the encoder predicts four NIG parameters (gamma k, nu k, alpha k, beta k), with nu k > 0, alpha k > 1, beta k > 0, which define sigmak2 y ∼ InvGamma(alpha k, beta k), mu k sigmak2, y ∼ N (gamma k, sigmak2 /nu k), and z k mu k, sigmak2 ∼ N (mu k, sigmak2). Thus q phi (mu k, sigmak2 y) = NIG(gamma k, nu k, alpha k, beta k). The hierarchy separates two statistically different sources of latent spread: u var,k ≡ E[sigmak2 y] = betak/(alphak − 1) and u epi,k ≡ Var(mu k y) = betak/[nu k (alpha k − 1)], with Var(z k y) = u var,k + u epi,k. The paper calls Eq. (6) epistemic latent uncertainty because it quantifies uncertainty in the latent location itself, while Eq. (5) quantifies variability around that location. For one image, u epi (y) is summarized as the mean over the K coordinates.

Training objective: With P input pixels and K latent coordinates, the loss is L ELVAE (y) = L recon (y) + lambda NIG L NIG (y), where L recon = (1/P)E∥y − g theta (z)∥ 22 and L NIG is the coordinate average of the KL divergence between the predicted NIG and a fixed NIG prior. The NIG KL directly regularizes the higher-order quantities to which uncertainty meaning is assigned. The coordinate-wise divergence is given by Eq. (12), which is the KL between the inverse-gamma factors plus the expected KL between the conditional normal distributions of mu.

ELBO interpretation and determination of the NIG weight: The weight lambdaNIG has a likelihood interpretation rather than being an arbitrary tuning parameter. Consider the hierarchical generative model p(mu, sigma 2) = p 0, p(z mu, sigma 2) = N (mu, sigma 2), p(y z) = N (g theta (z), s2 I P), with inference model q(mu, sigma 2, z y) = q phi (mu, sigma 2 y) p(z mu, sigma 2). The scalar s2 is the homoscedastic observation variance in image space. For this model, the negative ELBO is − ELBO(y) = (P/2) log(2pis2) + (P/2s2)L recon (y) + KLNIG (y). Matching coefficients gives lambdaNIG = 2s2 K/P, or equivalently s2 = PlambdaNIG/(2K). The observation variance can itself be estimated from reconstruction residuals: if s2 is treated as an unknown scalar and minimized, then ŝ2 = L recon, where L recon denotes the reconstruction MSE averaged over the data distribution and latent sampling. Substituting yields the practical calibration lambdâNIG = 2K L recon/P. On the trained pilot model, the held-out reconstruction MSE is L recon = 0.0408, so Eq. (18) gives lambdâNIG = 8.33 × 10−4, whereas the fixed value used for training is 5 × 10−4 (equivalently s2 = 0.0245). The fixed weight is therefore roughly a factor of 1.67 smaller than its own likelihood-consistent value.

Why the full NIG hierarchy must be regularized: Marginalizing (mu, sigma 2) yields a Student-t latent distribution. However, that marginal does not identify the decomposition. Proposition 1 states that under Eq. (3), z k is Student-t with 2alpha degrees of freedom, location gamma, and squared scale beta(1 + 1/nu)/alpha. Therefore the marginal depends on (nu, beta) only through c = beta(1 + 1/nu). Along the curve beta(1 + 1/nu) = c, the marginal distribution of z is unchanged while u var = (c/(alpha−1))(nu/(1+nu)) and u epi = (c/(alpha−1))(1/(1+nu)) can trade continuously against one another. A loss written only on the marginalized p(z) cannot supply that identification.

Prior and pilot architecture: The prior is (gamma0, nu0, alpha0, beta0) = (0, 1, 3, 1), giving E[sigma 2] = 1/2, Var(mu) = 1/2, and hence Var(z) = 1. The pilot ELVAE uses an MLP encoder 784 → 128 → 64, latent dimension K = 8, and a mirrored decoder. It is trained for four epochs with Adam, learning rate 10−3, batch size 1024, and lambdaNIG = 5 × 10−4. With P = 784 and K = 8, this fixed weight corresponds to an assumed observation variance s2 = 0.0245.

Uncertainty-Aware Generation:

Uncertainty-scaled posterior-anchored generation: For a labeled anchor (y i, c i), the encoder gives (gammai, nui, alphai, betai) and the coordinate-wise uncertainty vector u epi,i = betai ⊘ [nui ⊙ (alphai − 1)]. The pilot uses the variance-matched Gaussian perturbation zi = gammai + √u epi,i ⊙ epsilon, epsilon ∼ N (0, I), followed by xi gen = g theta (zi). This is not an exact sample from the NIG/Student-t posterior but a controlled Gaussian perturbation centered at gammai whose coordinate-wise variance matches Var(mui y i) = u epi,i. For exact posterior generation one can instead draw sigma 2 ∼ InvGamma(alpha, beta), then mu ∼ N (gamma, sigma 2 /nu) and z ∼ N (mu, sigma 2). For control-oriented generation, the variance-matched rule can be generalized to z = gamma + tauepi √u epi ⊙ epsilon epi + tauvar √u var ⊙ epsilon var, with two interpretable amplitudes.

Separating anchor reliability from perturbation-attributable effects: Equation (22) makes u epi serve two distinct roles: (i) an uncertainty score computed from the NIG posterior and used to rank anchors, and (ii) the scale of the random displacement applied to gamma. The paper uses three conditions: (A) Uncertainty-scaled generation with z = gamma + √u epi ⊙ epsilon; (C) Zero-displacement control with z = gamma (equivalently tauepi = 0), where the encoder still produces u epi for each anchor and samples are still ranked by u epi, but uncertainty-scaled displacement is disabled; (I) Generation-attributable failure, which restricts attention to anchors whose condition-(C) generation is already classified correctly and measures the failure rate under condition (A).

Low-uncertainty augmentation and high-uncertainty stress testing: For a class-conditional or posterior-anchored application, generated samples can be divided by u epi: Low u epi (latent location is comparatively well determined) for higher-confidence synthetic augmentation after task-specific validity checks; Intermediate u epi for exploratory generation and data enrichment; High u epi (latent location is weakly determined) for stress testing, failure analysis, and hard-example generation. High-u epi images are not automatically bad images; they are images generated from an anchor whose latent location ELVAE estimates less precisely.

Generation Pilot on MNIST:

Study design: The 70,000 MNIST digit images were pooled and partitioned into 60,000 training images and 10,000 held-out images. ELVAE is trained without using digit labels. A separate MLP classifier 784 → 256 → 128 → 10 is trained only on the real 60,000-image training set for three epochs and then frozen. It achieves 96.85% accuracy on the held-out real images. No generated image is used to train or tune this classifier. For each held-out anchor, one image is generated using Eq. (22). The scalar u epi is the mean of the eight coordinate-wise uncertainties. A generation error occurs when the frozen classifier prediction differs from the anchor label. Uncertainty is ranked within the intended digit class before pooling. If u epi is instead ranked globally across all 10,000 held-out images, the bottom-versus-top 20% contrast is 26.70% versus 27.10%, a ratio of 1.015×—essentially no effect. All of the reported stratification lives within digit classes and none of it survives pooling across classes. The primary quantitative comparison is classifier error in the bottom and top 20% of within-class u epi. A class-stratified bootstrap with 800 replicates gives confidence intervals. The whole pilot is repeated for three random seeds.

Quantitative result with a fixed classifier: Across all 10,000 generated images, the frozen classifier error was 29.06%. In the bottom 20% of within-class u epi, classifier error was 26.30%; in the top 20%, it was 37.80%. Thus the high-u epi group had an 11.50 percentage-point higher error, or a 1.437× error rate. The class-stratified bootstrap gave a 95% CI of 8.99–14.53 percentage points for the difference and 1.33–1.58 for the ratio. A two-proportion chi-square test gave p = 6.56 × 10−15. Under condition (C), with uncertainty-scaled displacement removed entirely, the bottom-versus-top contrast is 26.30% versus 36.70%, a ratio of 1.395×. Almost the whole condition-(A) stratification is therefore already present before uncertainty-scaled perturbation is applied. Uncertainty-scaled perturbation raises overall error only from 28.28% to 29.06%. The generation-attributable statistic (I) isolates what uncertainty-scaled perturbation does: among anchors that condition (C) already classifies correctly, the perturbation causes failure in 1.97% of the low-u epi group and 5.92% of the high-u epi group, a ratio of 3.01× (95% CI 2.03–4.75). The top uncertainty decile reaches 43.00% error. The AUROC of within-class u epi percentile for classifier failure is 0.556, and Spearman correlation with the binary failure indicator is 0.088.

Discussion and Conclusion: The pilot supports a focused and testable claim. ELVAE produces a continuous uncertainty variable that can be attached to the generation process, and populations selected by that variable differ substantially in semantic reliability. The z = gamma control shows that most of the low/high difference survives when uncertainty-scaled perturbation is switched off, even though u epi itself is still measured and used for ranking. Thus this dominant component is best described as an anchor representation/re-generation reliability effect, whereas only the smaller component isolated by statistic (I) is caused by uncertainty-scaled perturbation itself. The low/high comparison should not be interpreted as a universal threshold. First, u epi is class dependent in the present unconditional model, which is why the pilot uses within-class rankings; the effect does not survive global ranking. A conditional ELVAE is therefore a prerequisite for cross-class use. Second, the association is probabilistic and modest at the single-image level. Third, classifier error is only a semantic proxy for image utility. Fourth, the current MLP generator is intentionally small and generates blurry digits, and its lossiness is exactly what makes the condition-(C) error rate as high as 28.28%. Fifth, the headline ratio varies from 1.126 to 1.437 across three seeds, so single-seed reporting of a number like 1.437× overstates the precision of the pilot. High-u epi images should not simply be discarded: if the objective is trusted augmentation, they can be down-weighted or rejected; if the objective is robustness analysis, those same samples are valuable because they are more likely to induce semantic instability. A natural next experiment is downstream retraining, and a second extension is to vary tauepi in Eq. (24) and measure whether task error increases smoothly as uncertainty-scaled perturbation is strengthened. In conclusion, the MNIST pilot establishes two separable findings. First, u epi stratifies posterior anchors by re-generation reliability: the top 20% uncertainty group has 37.80% classification error against 26.30% in the bottom 20%, a 1.437× increase, of which a 1.395× ratio remains when tauepi = 0. Second, uncertainty-scaled perturbation itself induces semantic failure 3.01× more often in high-u epi anchors than in low-u epi anchors. The first finding is the larger anchor-reliability effect; the second is the specifically sampling-attributable generation effect. Together they support ELVAE as a framework that can stratify anchors and, to a lesser but measurable degree, control generation by model uncertainty.

Improvements to AI systems:

1. Uncertainty-aware data augmentation: AI systems can now generate synthetic training data with per-sample uncertainty scores. Low-uncertainty samples are used for high-confidence augmentation, while high-uncertainty samples are deliberately retained for stress-testing downstream models. This enables safer dataset expansion in medical or scientific domains where false synthetic samples are costly.

2. Failure-mode probing: The system can identify latent anchors that are weakly constrained by training data, then generate adversarial or edge-case examples from those regions. This allows proactive discovery of model vulnerabilities before deployment, rather than relying on post-hoc error analysis.

3. Calibrated generation control: The uncertainty variable (u epi) provides a continuous dial for generation reliability. Users can set a threshold: generate only from low-uncertainty anchors for production use, or intentionally sample high-uncertainty regions for exploratory generation and robustness testing. This is more principled than simple rejection sampling.

4. Two-component uncertainty decomposition: The system separates anchor reliability (how well the latent location is determined) from perturbation-attributable effects (how sensitive generation is to noise at that location). This allows AI systems to distinguish this sample is unreliable because the model is unsure where to generate from this sample fails because small perturbations break it.

5. Within-class uncertainty normalization: The system reveals that uncertainty must be normalized within semantic classes, not globally. This prevents cross-class bias where certain classes inherently have higher uncertainty. AI systems can now apply class-conditional uncertainty calibration for fairer sample selection.

6. Exact ELBO training with principled regularization: The NIG-prior KL divergence is not an arbitrary hyperparameter but has a likelihood-consistent value derived from reconstruction error. This makes training more interpretable and reproducible, and the objective is provably an exact ELBO for the hierarchical model.

7. Identifiability of uncertainty decomposition: Unlike standard Student-t latent VAEs, the system explicitly regularizes the full NIG hierarchy, ensuring that epistemic (location) and aleatoric (variance) uncertainties are identifiable and not confounded. This enables reliable uncertainty-based decision-making.

What the improved AI system can do:

• Generate synthetic images with a per-sample reliability score, allowing automatic filtering of low-quality generations before they enter a training set.

• Create targeted stress-test suites by sampling from high-uncertainty latent regions, revealing downstream classifier failures that standard random sampling would miss.

• Provide a tunable generation knob (τ epi) that controls how much uncertainty-scaled perturbation is applied, enabling smooth trade-offs between diversity and reliability.

• Rank existing generated samples by within-class uncertainty to identify which ones are most likely to be mislabeled or semantically unstable.

• Decompose generation failures into anchor quality versus perturbation sensitivity, helping engineers decide whether to improve the encoder (better anchors) or the decoder (more robust generation).

• Operate in low-data regimes where synthetic augmentation is critical, using uncertainty to avoid polluting training sets with unreliable samples while still exploiting high-uncertainty samples for robustness evaluation.