Oracle Gaps in Reliability Coverage: Sampling Noise or Policy Specialization?

arXiv:2609.32996 · cs.CV · Submitted 2026-09-26 · Read on arXiv

cs.CV

Submitted: 2026-09-26

Updated: 2026-09-26

Code: https://github.com/KurbanIntelligenceLab/oracle-gaps

Terminology

• Accuracy — 11× in this paper · explained in 1 episode
• Verifier — 6× in this paper · explained in 2 episodes
• Benchmark — 5× in this paper · explained in 2 episodes
• Consistency — 3× in this paper · explained in 3 episodes
• Empirical Coverage — 3× in this paper · explained in 2 episodes
• GRPO — 3× in this paper · explained in 6 episodes
• Self-consistency — 3× in this paper · explained in 2 episodes
• Exchangeability — 2× in this paper · explained in 2 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Active Learning — 1× in this paper · explained in 6 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Additivity — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Cosine Similarity — 1× in this paper · explained in 3 episodes
• Embeddings — 1× in this paper · explained in 2 episodes
• Ensemble Learning — 1× in this paper · explained in 2 episodes
• Low-Rank Adaptation (LoRA) — 1× in this paper · explained in 9 episodes
• Model Merging — 1× in this paper · explained in 2 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Sparsity — 1× in this paper · explained in 4 episodes
• Unified Framework — 1× in this paper · explained in 5 episodes
• Vector Representations — 1× in this paper · explained in 2 episodes

Sources

• Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
• When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models
• LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routing
• Are Diversity Metrics Measuring Diversity? A Capability-Controlled Audit of Majority-Vote Gain in LLM Ensembles
• The Capability Frontier: Benchmarks Miss 82% of Model Performance
• Deep Ensembles: A Loss Landscape Perspective
• Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
• The Path Not Taken: RLVR Provably Learns Off the Principals
• Beyond Pass@k: Breadth-Depth Metrics for Reasoning Boundaries
• Qwen2.5-VL Technical Report
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Quantifying Diversity of Thought: A Predictive Law of Weighted LLM Ensemble Lift
• RouterBench: A Benchmark for Multi-LLM Routing System
• Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models