Coarsening Latent-Class Probabilities: Directional Distortion and Coverage Loss

Author: Marcell T. Kurbucz, Institute for Global Prosperity, The Bartlett, University College London

arXiv: 2608.11784v1 [econ.EM], August 2026

The paper addresses a recurring problem in empirical work: measuring how membership in a latent group affects an outcome when the group indicator is never observed but a calibrated probability vector for it is available. Such probabilities come from trained classifiers or model-based scores—for example, an unobserved protected attribute proxied for a fairness audit, disease status from an imaging classifier, or poverty status from administrative predictors.

The paper asks: what does coarsening that vector cost the coefficient and the interval reported from it, along which directions, and can the cost be known before either is reported? The key setting is multi-class (K+1 mutually exclusive classes), where the natural object is a vector τ ∈ R K of group effects and a vector-valued calibrated score p on the probability simplex.

The framework rests on two substantive assumptions:

Assumption 1 (Structural conditional mean): E[Y G, p, X] = μ(X) + τ T g almost surely, where τ k is the conditional mean contrast between class k and the reference class K+1, holding X fixed. This has two parts: latent-class effects are constant in X, and the probability vector is mean-independent of Y given (G, X)—an exclusion restriction.

Assumption 2 (Conditional calibration): E[g p, X] = p almost surely. This is the sole link between unobserved g and observed p, and is stronger than ordinary multiclass calibration: it requires the probability vector to be unbiased for membership after conditioning on X.

Assumption 3 (Non-degeneracy): M = E[(p−r(X))(p−r(X)) T] is nonsingular.

Assumption 4 (Moments): E[Y 4] < ∞.

Together, Assumptions 1 and 2 imply that on observables the problem reduces to a partially linear regression: E[Y p, X] = μ(X) + τ T p (Robinson, 1988). The paper states: The latent class has disappeared: on observables the model is a partially linear regression of Y on the probability vector p with nonparametric component μ.

The matrix moment identity E[aR] = Mτ holds, where a = p − r(X), R = Y − m(X), and M = E[aa T]. Under Assumption 3, τ = M-1E[aR] is point-identified from the joint law of (Y, X, p). This is the matrix analogue of the scalar identity from the binary case.

The spectral decomposition M = Σ λ j v j v j T reveals the geometry of identification:

• (a) The moment equation identifies Mτ, hence the projection of τ onto R(M). Full identification holds iff λ 1 > 0.

• (b) If rank(M) = K − q < K, the moment does not point-identify τ; it determines only the projection P R(M)τ, with the solution set being an affine subspace.

• (c) λ j = 0 iff v j T p = v j T r(X) almost surely—the score combination is a deterministic function of X.

• (d) The failure is genuinely observational, not an artifact of one moment equation. The paper constructs observationally equivalent models with different coefficient vectors along null directions, showing the identified set is convex, contains a nondegenerate segment along every null direction, and is bounded under regularity conditions.

The paper notes: When K = 1 identification is all or nothing; with several classes an analyst can have sharp inference on some contrasts and none on others in the same study, and the spectrum of M says which.

The moment ψ(W; τ, m, r) = a(R − a T τ) is Neyman-orthogonal with respect to both nuisances m and r, making the estimator directly compatible with machine-learning nuisance estimators at the usual product rate.

Under standard conditions, √n(τ̂ − τ) → N(0, Ω) with Ω = M-1ΣM-1, Σ = E[u2aa T]. The asymptotic variance along eigen-direction j is v j T Ω v j = σ j2/λ j2, so precision degrades as λ j → 0, quantifying Theorem 2.

The efficiency bound is V eff = (E[σ-2(p,X) ãã T])-1, where ã = p − p̃(X) with p̃(X) the precision-weighted conditional mean of p. The unweighted estimator attains the bound iff σ2(p,X)a = Cã almost surely for some nonsingular constant C. The paper shows heteroskedasticity is intrinsic: σ2(p,X) = τ T(diag(p) − pp T)τ + Var(εp,X), so the latent-membership term varies with p even when Var(εp,X) is constant.

The central contribution begins with the coarsening operator. For any coarsening h (argmax label, confidence-thresholded label, rounded summary, top-k), the plug-in estimator converges to:

plim τ̂ h = A h τ, where A h = D h-1C h, D h = E[a h a h T], C h = E[a h a T]

Writing u = a − a h for the discarded signal gives the key form:

Corollary 8: A h = I + D h-1E[a h u T]. Hence A h = I if and only if E[a h u T] = 0.

The paper states: Coarsening is free exactly when what it throws away is uncorrelated with what it keeps. That condition is restrictive, and the second part of the corollary says that even proportional shrinkage is non-generic.

Corollary 9 (Directional retention): Coarsening acts uniformly on τ only when A h = cI; outside that case it is direction-specific, and the retention coefficients s j = v j T A h v j need neither coincide across j nor lie in [0,1].

The most consequential result concerns inference. An analyst who coarsens and forms the usual sandwich reports a Wald interval whose limiting coverage is:

Φ(z 1−α/2 − λ v) − Φ(−z 1−α/2 − λ v), with λ v = b/σ h,v

where b v = v T(A h − I)τ is the directional coarsening bias and σ h,v2 = v TΩ h v the reported variance. Key features:

• Coverage falls as the coarsened estimator becomes more precise, because λ v grows as the reported standard error shrinks.

• Both ingredients are observable: A h is identified from (p,X) alone and the reported standard error from (Y,p,X).

• The coverage can be estimated before reporting via λ̂ v = v T(Â h − I)τ̂ / √(v T Ω̂ h v/n).

Remark 2: "Greater apparent precision can worsen coverage. Coarsened labels are more extreme than the probabilities they replace, so Ω h = D h-1Σ hD h-1 can fall below Ω... the reported interval is narrower than the one the uncoarsened regression would have produced, around a displaced center."

When calibration fails within a budget δ (E[gp,X] = p + η with ∥η∥∞ ≤ δ), the induced bias along direction v j is bounded by δ∥τ∥ 1 Ev j T a/λ j ≤ δ∥τ∥ 1 λ j-1/2. The paper notes: the directions least identified, and least precisely estimated, are also the most fragile to calibration drift.

Five coarsenings compared at n = 8000, K = 3: identity, rounding to 0.20 grid, top-2 renormalized, threshold 0.60, and argmax one-hot. Results along the weakest eigen-direction:

Coarsening diag(A h) mean SE coverage

identity 1.00 0.0405 0.952

round to 0.20 grid 0.89 0.0377 0.910

top-2 renormalized 0.73 0.0336 0.675

threshold 0.60 0.47 0.0332 0.533

argmax one-hot 0.41 0.0246 0.010

The paper states: The argmax interval is 39% shorter than the one the uncoarsened regression would have produced and covers 1% of the time.

The coverage formula tracks realized coverage to within 0.02 across the entire range, from nominal down to 0.003. A sweep over dispersion and feature overlap leaves the ordering unchanged: in every configuration examined the argmax interval covers between 1% and 6%.

Componentwise and joint Wald coverage for τ̂ lie in [0.938, 0.960] and [0.946, 0.958] at nominal 95% over 1000 replicates as n grows from 500 to 5000.

The latent group is race in 4 classes (reference White; Black, Asian–Pacific, Other), outcome is high-income indicator, proxy is a multinomial classifier on richer features. The estimated spectrum of M is anisotropic with eigenvalues (0.0006, 0.0069, 0.018) and condition number 31. Retention coefficients under argmax coarsening range from s 1 = 0.46 to s 3 = 0.90: on real data, coarsening removes more than half the signal in one direction and almost none in another.

Along the best-identified direction the proxy-based estimate matches the benchmark (0.034±0.017 against 0.028); along the two weak directions it departs sharply. The calibration battery is clean (R2 ≤ 0.006) but the direct-channel check is emphatic (ΔR2 = 0.135), pointing to exclusion violation rather than calibration failure.

Using North Carolina voter records (n = 464,700) with Census surname probabilities as the proxy, the framework's diagnostics are favorable on every operating condition. The direct-channel check is essentially zero (partial R2 = 0.001), the classifier is genuinely uncertain but skilful (argmax accuracy 0.63 against White base rate 0.55), and M is well-conditioned (eigenvalues 0.015, 0.024, 0.061; condition number 4.1).

Results show hard-labeling attenuates gaps toward zero, shrinking Hispanic and Asian gaps by roughly a fifth (−0.454 and −0.428 against benchmark values −0.551 and −0.525). The soft estimator recovers them almost exactly (−0.520 and −0.524). Overall the soft estimator is twice as close to the benchmark as hard-labeling (RMSE 0.042 versus 0.082).

On UCI Forest CoverType, the retention coefficients are s = (0.59, 0.80, 0.63) ordered from strongest to weakest eigen-direction, showing a real score geometry in which the caveat after Corollary 9 binds—the retention is non-monotone in eigenvalues. The known-truth recovery shows the soft estimator recovers τ to RMSE 0.040 while hard is attenuated to RMSE 0.292. The attenuation matrix computed from (p,X) alone forecasts the realized hard-label estimate to within 0.003 in every component.

A labeled subsample allows diagnosing which assumption failed. The protocol: split labeled data in half, fit recalibration map p̃ = E[gp,X] on first half, re-run audit with p̃ on held-out half.

• On Adult: rich-basis battery finds no conditional miscalibration (R2 ≤ 0.006), recalibration changes nothing, direct-channel check is emphatic (ΔR2 = 0.135)—exclusion violation that no calibration can repair.

• On CoverType: rich basis reveals severe conditional miscalibration (R2 up to 0.32), recalibration reduces it out-of-sample to R2 ≤ 0.003, yet the audit deteriorates threefold because the recalibrated spectrum compresses (largest eigenvalue 0.122 → 0.029) and the direct channel is amplified.

The paper concludes: The protocol that emerges is sequential: run the rich-basis (p,X) calibration battery on the labeled subsample... then recalibrate and re-run out of sample.

The framework is informative when all of the following hold:

1. Membership in several mutually exclusive latent classes is unobserved, and the probability vector is conditionally calibrated.

2. Controls X are strictly coarser than the classifier's features.

3. The classifier is genuinely uncertain (when nearly deterministic, coarsened and uncoarsened labels coincide and the framework adds nothing).

4. There is no trusted forward model of the measurement process.

The paper states: "Settings that satisfy all four conditions are the natural applications: multi-group fairness audits with validation labels, hard medical subtyping against registry gold standards, verbal-autopsy cause-of-death attribution, and multi-subtype prevalence estimation with administratively missing indicators."

1. The coarsening operator A h (Theorem 7): characterizes exactly what any coarsening costs, showing coarsening is free iff discarded signal is uncorrelated with retained signal, and is otherwise anisotropic.

2. Coverage formula (Proposition 10): the Wald interval from coarsened labels has limiting coverage Φ(z − λ v) − Φ(−z − λ v), computable before reporting.

3. Spectral geometry of identification (Theorem 2): identification fails direction-by-direction according to the spectrum of M.

4. Sensitivity bound (Proposition 11): miscalibration bias is amplified by small eigenvalues.

5. Efficiency bound (Theorem 5): the semiparametric efficiency bound with optimal weighting, showing the unweighted estimator is generally inefficient under heteroskedasticity.

The paper concludes: "Two objects organize the analysis. The covariance matrix M of the partialed-out probability vector grades which contrasts the regression (5) can recover and how precisely; the coarsening operator A h grades what is lost when that vector is replaced by a coarser version of itself, and, through the noncentrality index, what the reported interval is then worth. The binary case is the one-dimensional shadow of both: there the operator is a scalar dilution factor and identification is all or nothing, while with several classes each becomes direction-specific."

Based on this paper, here are specific improvements to AI systems:

Improvement: Add a diagnostic module that computes the coarsening operator  h and noncentrality index λ̂ v before any downstream inference is reported.

Capability: The AI system can warn users when their hard-label or thresholded classifier output will produce intervals with actual coverage far below nominal (e.g., detecting that argmax labels yield 1% coverage instead of 95%). This prevents misleading conclusions in fairness audits, medical subtyping, or any latent-class regression task.

Improvement: Replace scalar uncertainty metrics with eigen-direction-specific precision estimates using the spectral decomposition of M = E[aaT].

Capability: The system can report which linear contrasts of latent-class effects are identifiable and which are not, rather than giving a single confidence interval. For example, it can say the Black-vs-White contrast is estimable with SE 0.017, but the Asian-vs-Other contrast is essentially unidentified (λ≈0.0006). This prevents overconfident claims on weakly identified directions.

Improvement: Given a candidate set of coarsenings (argmax, threshold, top-k, rounding), the system computes A h for each and selects the one minimizing directional bias while preserving precision.

Capability: The AI can recommend use top-2 renormalized labels instead of argmax—this retains 73% of signal in the weakest direction versus 41%, with only a 15% increase in SE. This optimizes the bias-variance tradeoff before any estimation.

Improvement: Implement the bound from Proposition 11: for each eigen-direction, compute δ·‖τ‖1·λⱼ−1/2 as a fragility score.

Capability: The system flags directions where small calibration drift (e.g., δ=0.01) could induce bias exceeding the reported SE. This is critical when using proxies from external classifiers with unknown calibration quality, as in the Adult dataset example where the direct-channel check revealed exclusion violation.

Improvement: Build a pipeline that: (a) tests conditional calibration on a labeled subsample using rich basis functions of (p,X); (b) if miscalibrated, fits a recalibration map p̃ = E[gp,X]; (c) re-runs the audit on held-out data and checks whether the spectrum M compresses or the direct channel amplifies.

Capability: The system can detect when recalibration is futile (as in Adult, where exclusion violation dominates) versus harmful (as in CoverType, where recalibration compressed eigenvalues from 0.122 to 0.029, tripling estimation error). This prevents blind recalibration that worsens inference.

Improvement: Modify the reporting layer to output effective coverage alongside nominal coverage, using the formula Φ(z − λ̂ v) − Φ(−z − λ̂ v).

Capability: Instead of 95% CI: [0.1, 0.3], the system reports Nominal 95% CI: [0.1, 0.3]; estimated actual coverage: 67% due to coarsening bias. Consider using soft probabilities or widening by factor 1.8. This directly addresses the paper's finding that greater apparent precision can worsen coverage.

Improvement: Implement the semiparametric efficiency bound from Theorem 5, testing whether the unweighted estimator attains it via the condition σ2(p,X)a = Cã. If not, apply precision-weighted estimation using σ̂2(p,X) = τ̂T(diag(p) − ppT)τ̂ + Var̂(εp,X).

Capability: The system achieves up to 30-40% variance reduction in heteroskedastic settings where latent-class membership probabilities vary widely, without requiring parametric assumptions.

Improvement: Add a module that computes the retention coefficients s j = v jTA hv j from (p,X) alone and compares them to the realized attenuation from a validation subsample.

Capability: The system can forecast how much hard-labeling will attenuate each coefficient before seeing any outcome data—as demonstrated in the Forest CoverType audit where predicted attenuation matched realized estimates within 0.003. This enables pre-registration of expected effect sizes and power calculations.

Improvement: When testing K contrasts simultaneously, use the eigen-structure of M to determine effective number of independent tests, rather than assuming K independent tests.

Capability: The system avoids both over-correction (when directions are highly correlated) and under-correction (when some directions are nearly unidentifiable), producing more accurate joint confidence regions.

Improvement: Deploy the framework as a monitoring layer that continuously recomputes  h, λ̂ v, and coverage estimates as new data arrives, flagging when the coarsening operator drifts from its training-time value.

Capability: The system detects when a deployed classifier's probability distribution shifts (e.g., due to population drift), automatically alerting that previously valid inferences may now be biased or overconfident—before downstream decisions are made.

Outcomes are increasingly regressed on a calibrated probability vector for unobserved class membership, and that vector is often coarsened to a hard label first. Under a constant-coefficient structural mean and conditional calibration, the observed-data problem is a partially linear regression of the outcome on the probability vector; we take this reduction as the starting point and ask what coarsening costs. For any coarsening, the plug-in estimator converges to A tau, where the coarsening operator satisfies A=I+D-1 E[a hu] with u the discarded signal. Coarsening is therefore free exactly when what is discarded is uncorrelated with what is kept, and is otherwise anisotropic: it distorts some contrasts far more than others. The same operator governs inference. The Wald interval built from coarsened labels has limiting coverage (z-lambda)- (-z-lambda), with lambda the ratio of the coarsening bias to the reported standard error; because A and that standard error depend on observables alone, the coverage implied by the estimated index can be approximated before the interval is reported. Simulations show severe coverage loss after argmax coarsening, and three real-data audits exhibit the direction-specific distortion that hard labels induce.