Confidently Wrong, Silently So: Auditing Undetectable Failures of a Deployed On-Device Language Model

arXiv:2608.23663 · cs.SE, cs.AI · Submitted 2026-08-24 · Read on arXiv

cs.SE, cs.AI

Submitted: 2026-08-24

Updated: 2026-08-26

Terminology

• AUROC — 22× in this paper · explained in 2 episodes
• Calibration — 18× in this paper · explained in 7 episodes
• Consistency — 15× in this paper · explained in 3 episodes
• Abstention — 12× in this paper · explained in 1 episode
• Accuracy — 10× in this paper · explained in 1 episode
• Hallucination — 10× in this paper · explained in 3 episodes
• Trace — 8× in this paper · explained in 2 episodes
• Benchmark — 5× in this paper · explained in 2 episodes
• Foundation Model — 3× in this paper · explained in 3 episodes
• Model-Agnostic — 3× in this paper · explained in 2 episodes
• Self-consistency — 3× in this paper · explained in 2 episodes
• Accountability — 2× in this paper · explained in 2 episodes
• Reasoning Traces — 2× in this paper · explained in 3 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Core — 1× in this paper · explained in 2 episodes
• Decode — 1× in this paper · explained in 2 episodes
• Empirical Methods in Natural Language Processing — 1× in this paper
• Failure Modes — 1× in this paper · explained in 3 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Hallucination Detection — 1× in this paper · explained in 2 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• Uncertainty Quantification — 1× in this paper · explained in 8 episodes
• Verification — 1× in this paper · explained in 2 episodes
• Verifier — 1× in this paper · explained in 2 episodes

Sources

• Apple Intelligence Foundation Language Models: Tech Report 2025
• Training Verifiers to Solve Math Word Problems
• Trace Length is a Simple Uncertainty Signal in Reasoning Models
• Gemma 3 Technical Report
• TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
• Language Models (Mostly) Know What They Know
• What Single-Prompt Accuracy Misses: A Multi-Variant Reliability Audit of Language Models
• The Llama 3 Herd of Models
• SELFDOUBT: Uncertainty Quantification for Reasoning LLMs via the Hedge-to-Verify Ratio
• Language Models are Multilingual Chain-of-Thought Reasoners
• Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation
• Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
• Reasoning Models Better Express Their Confidence

Related papers

• Falsification-Based Verification of LLM-Generated Optimization Models: Sound Test Batteries and Their Detection Limits
• GitSkills: A Dataset of Agent Skills on GitHub
• SABER: Benchmarking Operational Safety of LLM Coding Agents in Stateful Project Workspaces
• PackMonitor: Enabling Zero Package Hallucinations Through Decoding-Time Monitoring
• IntentCoding: Amplifying User Intent in Code Generation
• Incentives and Outcomes in Bug Bounties