Labels have Human Values: Value Calibration of Subjective Tasks

arXiv:2601.06631 · cs.CL · Submitted 2026-01-10 · Read on arXiv

cs.CL

Submitted: 2026-01-10

Updated: 2026-09-03

Terminology

• Calibration — 37× in this paper · explained in 7 episodes
• Clustering — 24× in this paper · explained in 2 episodes
• Alignment — 20× in this paper · explained in 1 episode
• AUC — 15× in this paper · explained in 2 episodes
• Embeddings — 11× in this paper · explained in 2 episodes
• Loss Function — 6× in this paper · explained in 2 episodes
• Accountability — 5× in this paper · explained in 2 episodes
• Accuracy — 3× in this paper · explained in 1 episode
• Commentary — 3× in this paper · explained in 2 episodes
• Reinforcement Learning — 3× in this paper · explained in 4 episodes
• Semantic Similarity — 2× in this paper · explained in 2 episodes
• Transformer — 2× in this paper · explained in 2 episodes
• Aggregation — 1× in this paper · explained in 1 episode
• Alignment Methods — 1× in this paper · explained in 1 episode
• Artificial Intelligence (AI) — 1× in this paper · explained in 2 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Composite Loss Function — 1× in this paper · explained in 2 episodes
• EU Artificial Intelligence — 1× in this paper
• Gradient Flow — 1× in this paper · explained in 2 episodes
• Wasserstein Distance — 1× in this paper · explained in 4 episodes

Sources

• A General Language Assistant as a Laboratory for Alignment
• Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
• Longformer: The Long-Document Transformer
• When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks
• Alignment of Language Agents
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• Detecting Stance in Media on Global Warming
• On Releasing Annotator-Level Labels and Information in Datasets
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• Towards Data-Centric RLHF: Simple Metrics for Preference Dataset Comparison
• Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF
• A Roadmap to Pluralistic Alignment
• On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering