OTROPE: Optimal Transport-based Robust Off-policy Evaluation for Large Language Models
cs.AI, cs.CL, stat.ML
Submitted: 2026-09-28
Updated: 2026-09-28
Code: https://github.com/LinerXiang/OTROPE
Terminology
Sources
- PPI++: Efficient Prediction-Powered Inference
- Off-Policy Evaluation from Logged Human Feedback
- AutoEval Done Right: Using Synthetic Data for Model Evaluation
- M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation
- Multiple-Prediction-Powered Inference
- Prediction-Powered Inference with Inverse Probability Weighting
- Pluralistic Off-policy Evaluation and Alignment
- DeepSeek-V3 Technical Report
- Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge
- Wasserstein Distributionally Robust Policy Evaluation and Learning for Contextual Bandits
- OCEAN: Offline Chain-of-thought Evaluation and Alignment in Large Language Models
- G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
- Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection