Certified Against Which Oracle? Execution Labels Set the Reported Risk of Conformal Abstention for Text-to-SQL
cs.LG, cs.CL
Submitted: 2026-09-22
Updated: 2026-09-22
Code: https://github.com/yahiko-l/text-to-sql-oracle-certificates
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Mitigating LLM Hallucinations via Conformal Abstention
- Conformal Risk Control
- Uncertainty Quantification for Language Models: A Suite of Black-Box, White-Box, LLM Judge, and Ensemble Scorers
- Label Noise Robustness of Conformal Prediction
- Selective Classification for Deep Neural Networks
- SynSQL: Synthesizing Relational Databases for Robust Evaluation of Text-to-SQL Systems
- FLEX: Expert-level False-Less EXecution Metric for Reliable Text-to-SQL Benchmark
- SpotIt: Evaluating Text-to-SQL Evaluation with Formal Verification
- When Can Conformal Risk Control Certify LLM Outputs? Bounds, Impossibility, and Adaptation for Structured Generation
- Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
- TrustSQL: Benchmarking Text-to-SQL Reliability with Penalty-Based Scoring
- Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs
- Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models
- Language Models with Conformal Factuality Guarantees
- Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks
- Conformal Language Modeling
- Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilities
- What Predicts Correctness in Text-to-SQL? A Selective-Prediction Study
- Test-Time Verification for Text-to-SQL via Outcome Reward Models
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks