Prediction-Assisted Pricing and Admission for LLM APIs with Stochastic Token Consumption

arXiv:2609.00710 · cs.DS, cs.LG · Submitted 2026-09-01 · Read on arXiv

cs.DS, cs.LG

Submitted: 2026-09-01

Updated: 2026-09-01

Terminology

• Regret — 16× in this paper · explained in 3 episodes
• Benchmark — 12× in this paper · explained in 2 episodes
• Calibration — 3× in this paper · explained in 7 episodes
• Degeneracy — 3× in this paper · explained in 3 episodes
• Robustness — 3× in this paper · explained in 5 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Accuracy — 2× in this paper · explained in 1 episode
• Generative Model — 2× in this paper · explained in 2 episodes
• Generative Models — 2× in this paper · explained in 2 episodes
• Resource Constraints — 2× in this paper · explained in 3 episodes
• Distribution Shift — 1× in this paper · explained in 5 episodes
• Exchangeability — 1× in this paper · explained in 2 episodes
• Trace — 1× in this paper · explained in 2 episodes
• Transformer — 1× in this paper · explained in 2 episodes
• Verifier — 1× in this paper · explained in 2 episodes

Sources

• FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
• RouterBench: A Benchmark for Multi-LLM Routing System
• RouteLLM: Learning to Route LLMs with Preference Data

Related papers

• Cascaded Learned Bloom Filter for Optimizing Model-Filter Size Balance and Fast Rejection
• Edge-Private Matching Kernels Through Local Decoding
• Local Node Differential Privacy
• Cheaper by the Batch: Shared Traversal for Genotype Graph Editing
• Scalable Algorithms for Approximate DNF Model Counting
• On the Approximation Relationship between Optimizing Ratio of Submodular (RS) and Difference of Submodular (DS) Functions