Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfaces
cs.LG, stat.ML
Submitted: 2026-09-27
Updated: 2026-10-05
Terminology
Sources
- Revealed Rationality: Label-Free Evaluation and Regularization from Representation Theorems
- Dutch Books for Language Models
- Jev for Scientific Decisions: Evaluating Semantic Choices and Their Consequences
- Just Ask Jev: Reinforcement Learning for Calibrated Decisions as a Zero-Shot Detector of AI Alignment Failures
- Can Jev Judge Radiology Reports? Evaluating a System One Model for Clinical Factuality
- Evaluating Decision Models for Text Annotation in Computational Social Science
- Same Answer, Different Confidence: Protocol Sensitivity in LLM Confidence Calibration
- When Policies Change Probabilities: Modular Decision-Making for LLM Code Review
- ForecastBench-Sim: A Simulated-World Forecasting Benchmark
- Rethinking Uncertainty Evaluation in Large Language Models
- Calibrated Decisions at Scale: Converting Police Crash Narratives into Probabilistic Crash Variables with a System One Model (Jev)
- Type-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It
- JevOut: Natural Context Can Flip Decision Models
- When Agents Say One Thing and Do Another: Validating Elicited Beliefs from LLMs
- Same Scores, Different Decisions: Evaluating JEV and Language Models for Legal Document Understanding
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks