Large Language Model Selection with Limited Annotations

arXiv:2605.24981 · cs.CL, cs.LG · Submitted 2026-05-24 · Read on arXiv

cs.CL, cs.LG

Submitted: 2026-05-24

Updated: 2026-09-25

Code: https://github.com/InternLM/InternLM-techreport

Terminology

• Benchmark — 29× in this paper · explained in 2 episodes
• Large Language Model — 12× in this paper
• Information Gain — 8× in this paper · explained in 2 episodes
• Benchmarking — 6× in this paper · explained in 4 episodes
• Adam — 5× in this paper · explained in 2 episodes
• F1 Score — 5× in this paper · explained in 3 episodes
• Accuracy — 4× in this paper · explained in 1 episode
• Foundation Model — 4× in this paper · explained in 3 episodes
• LLM-as-a-Judge — 4× in this paper · explained in 3 episodes
• Robustness — 4× in this paper · explained in 5 episodes
• Foundation Models — 3× in this paper · explained in 7 episodes
• Transformer — 3× in this paper · explained in 2 episodes
• alpha — 3× in this paper · explained in 1 episode
• Active Learning — 2× in this paper · explained in 6 episodes
• Core — 2× in this paper · explained in 2 episodes
• Cosine Similarity — 2× in this paper · explained in 3 episodes
• Embeddings — 2× in this paper · explained in 2 episodes
• Fine-Tuning — 2× in this paper · explained in 5 episodes
• Large Language Model (LLM) — 2× in this paper · explained in 3 episodes
• Model-Agnostic — 2× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• CVF International Conference on Computer Vision — 1× in this paper
• Calibration — 1× in this paper · explained in 7 episodes
• Chain-of-Thought Prompting — 1× in this paper · explained in 2 episodes
• Contextual Bandits — 1× in this paper · explained in 3 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• EMMA — 1× in this paper · explained in 2 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Generative Models — 1× in this paper · explained in 3 episodes
• Kullback-Leibler (KL) Divergence — 1× in this paper · explained in 3 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• Pairwise Comparisons — 1× in this paper · explained in 2 episodes
• Semantic Similarity — 1× in this paper · explained in 2 episodes
• Semi-supervised Learning — 1× in this paper · explained in 4 episodes
• Theoretical Guarantees — 1× in this paper · explained in 2 episodes
• Transfer Learning — 1× in this paper · explained in 10 episodes
• Verifier — 1× in this paper · explained in 2 episodes

Sources

• Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
• The Falcon Series of Open Language Models
• PaLM 2 Technical Report
• Qwen Technical Report
• Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
• Scaling Up Active Testing to Large Language Models
• PaliGemma: A versatile 3B VLM for transfer
• Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
• GPT-NeoX-20B: An Open-Source Autoregressive Language Model
• Training Verifiers to Solve Math Word Problems
• Gemini: A Family of Highly Capable Multimodal Models
• Mistral 7B
• Mixtral of Experts
• Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attacks
• On the Necessity of Collaboration for Online Model Selection with Decentralized Data
• Online Foundation Model Selection in Robotics
• GPT-4 Technical Report
• Training language models to follow instructions with human feedback
• GPQA: A Graduate-Level Google-Proof Q&A Benchmark
• BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering