Regime-Gated Residual Mixture-of-Experts for Cross-Sectional Volatility Forecasting

This paper studies how regime information should be incorporated into a neural network for cross-sectional volatility forecasting. The authors propose RG-ResMoE, a regime-gated residual mixture-of-experts architecture, and evaluate it against several baselines on a U.S. equity panel of 1,027 stocks and a Japanese replication panel of 1,552 stocks, using a rolling walk-forward evaluation framework with 30 non-overlapping test windows from April 2018 to October 2025.

The central question is not whether market regimes matter, but how such information should be incorporated into the forecasting process. The paper isolates the integration pathway by holding fixed the regime variables, forecasting backbone, parameter budget, hyperparameter selection, and evaluation protocol, while varying only where regime information enters the model. In RG-ResMoE, a frozen base network predicts the volatility level from stock features, and zero-initialized experts learn residual corrections. A soft gate observes both stock features and regime variables (market volatility and idiosyncratic volatility) to route these corrections, but the forecasting networks themselves receive only stock features.

The main results show that RG-ResMoE achieves the best overall forecast performance, with the highest IC (0.5469), lowest RMSE (0.2304), highest out-of-sample R2 (0.292), highest ICIR (6.14), and lowest QLIKE (0.735) among all models. It also exhibits perfect training stability, with 0 collapsed seeds out of 30, compared to 24 collapses for standard MoE and 3 for MLP-L.

The paper finds that the location of regime information is decisive. Appending the same regime variables directly to the forecasting input (MLP-L(+z)) reduces IC by 0.0043 and causes 30/30 seeds to collapse, whereas using them only for gated residual routing improves IC by 0.0004 and eliminates all collapses. The input pathway shows pronounced instability during the COVID crisis and recovery, while the gate pathway produces a more stable response.

Soft routing consistently outperforms hard routing. All four hard-routing variants (learned top-1, volatility quantiles, GICS sectors, market×idiosyncratic split) underperform the soft gate by 0.0021–0.0034 IC, with every comparison significant at p<10−4. The gate learns continuous state-dependent reweighting rather than discrete regime assignments, with the average maximum gate weight remaining close to uniform routing.

The residual architecture is more important than the number of experts. Reducing from four to two experts lowers IC by only 0.0004, while increasing to six provides no improvement and reduces stability. Replacing zero-initialized residual layers with random initialization lowers IC and increases collapses from 0 to 13, and removing the frozen base entirely (standard MoE) collapses in 24 of 30 runs.

In practical evaluation, RG-ResMoE achieves the lowest Kupiec VaR coverage rejection rates across nearly all settings, with the largest improvements during high-volatility periods. Its IC advantage over MLP-L is amplified during market stress: +0.0207 in the highest-volatility decile (4.3× the full-sample gain) and +0.0322 during the COVID crisis (6.7× the full-sample gain).

The cross-market replication on the Japanese TSE Prime panel confirms the findings: RG-ResMoE outperforms MLP-L by +0.0043 IC, and input concatenation again causes instability with 28/30 collapses.

The paper concludes that the primary value of regime-aware MoE models in compact financial forecasting systems lies less in increasing model capacity than in controlling how nonstationary market information influences the prediction. The design principle is that regime information is most effective when it guides residual routing rather than being treated as ordinary predictive input. The study is limited to volatility forecasting in two equity markets using a compact MLP backbone, and whether the principle extends to other tasks or architectures remains future work.

Improvements to AI Systems:

1. Regime-Gated Residual Routing Architecture – Implement a dual-pathway design where a frozen base network predicts baseline values, and zero-initialized experts learn only residual corrections. A soft gate (taking both stock features and regime variables) routes corrections, but the forecasting networks receive only stock features. This prevents regime variables from contaminating the primary feature representation.

2. Zero-Initialized Expert Layers – Initialize all expert layers to zero, ensuring that at training start, the model behaves exactly like the frozen base network. This guarantees monotonic improvement over the base model and eliminates training collapse (0/30 collapsed seeds vs. 24/30 for standard MoE).

3. Soft Continuous Gating over Discrete Routing – Replace hard routing (top-1, quantile-based, sector-based) with a soft gate that learns continuous state-dependent reweighting. This avoids brittle regime assignments and maintains near-uniform routing initially, allowing the gate to gradually specialize without overfitting to noisy regime labels.

4. Regime Information as a Controller, Not a Predictor – Never concatenate regime variables (market volatility, idiosyncratic volatility) directly into the forecasting input. Instead, feed them only to the gating mechanism. This eliminates training instability (30/30 collapses when concatenated) and improves out-of-sample IC by +0.0043.

5. Residual Learning with a Frozen Base – Keep the base network frozen during training. Only the experts and gate are trainable. This reduces effective model capacity while improving generalization, and it makes the system robust to nonstationary market conditions by learning only corrections to a stable baseline.

6. Stability-Aware Expert Count – Use a small number of experts (2–4). Increasing beyond 4 provides no accuracy gain and reduces training stability. The residual architecture matters more than expert count, so prioritize residual design over scaling.

7. Stress-Adaptive Forecasting – The system automatically amplifies its advantage during high-volatility periods (IC gain +0.0207 in highest-volatility decile, +0.0322 during COVID). This is achieved by the gate learning to rely more heavily on residual corrections during turbulent regimes, without explicit regime detection.

What the Improved AI System Can Do:

• Forecast cross-sectional volatility with 0.5469 IC, 0.2304 RMSE, 0.292 out-of-sample R2, and 6.14 ICIR—outperforming all baselines (MLP, standard MoE, hard-routing variants) on U.S. and Japanese equity panels.

• Maintain perfect training stability across 30 random seeds (0 collapses), even during market crises, unlike standard MoE (24 collapses) or input-concatenation models (30 collapses).

• Provide more reliable Value-at-Risk estimates, with the lowest Kupiec coverage rejection rates across nearly all settings, especially in high-volatility periods.

• Generalize across markets (U.S. and Japan) with consistent performance gains, demonstrating that the design principle transfers without retuning.

• Operate with a compact MLP backbone, making it computationally efficient for real-time financial forecasting while achieving state-of-the-art accuracy.

Financial volatility is regime dependent, yet incorporating regime information into neural networks can also destabilize training. This paper asks where such information should enter a neural cross-sectional volatility forecasting model. We study five-day realized-volatility forecasts for 1,027 U.S. equities using a rolling walk-forward evaluation framework in which information, model capacity, hyperparameter tuning, and random seeds are matched across architectures. We propose RG-ResMoE, a regime-gated residual mixture-of-experts architecture in which regime information is used only for expert routing rather than for direct forecasting. The base predictor models volatility from stock features, while a gating network uses regime state variables to route residual corrections. RG-ResMoE consistently outperforms a capacity-matched MLP in both forecasting accuracy and training stability in the main U.S. study. Similar gains are observed on an independent Japanese panel. The integration pathway is decisive: appending the same regime variables directly to the forecasting input degrades both predictive performance and training stability, whereas restricting them to the routing gate improves accuracy and Value-at-Risk calibration. Hard routing consistently underperforms soft routing. The results suggest that, in compact neural volatility forecasting models, the primary value of mixture-of-experts models lies less in increasing model capacity than in controlling how nonstationary regime information influences prediction.