How Correct Is Your Answer? A Semantic Correctness Framework for Open QA Evaluation

arXiv:2609.01369 · cs.CL · Submitted 2026-09-01 · Read on arXiv

cs.CL

Submitted: 2026-09-01

Updated: 2026-09-01

Code: https://github.com/ElitsaY/EMNLP-2026-How-Correct-Is-Your-Answer-

Terminology

Accuracy — 18× in this paper · explained in 1 episode
Benchmark — 15× in this paper · explained in 2 episodes
Alignment — 7× in this paper · explained in 1 episode
Consistency — 6× in this paper · explained in 3 episodes
Failure Modes — 4× in this paper · explained in 3 episodes
Annotator Agreement — 2× in this paper · explained in 1 episode
Calibration — 2× in this paper · explained in 7 episodes
Context-Aware Precision — 2× in this paper
Empirical Methods in Natural Language Processing — 2× in this paper
Fine-Tuning — 2× in this paper · explained in 5 episodes
LLM-as-a-Judge — 2× in this paper · explained in 3 episodes
Natural Language Inference (NLI) — 2× in this paper · explained in 2 episodes
Robustness — 2× in this paper · explained in 5 episodes
AUC — 1× in this paper · explained in 2 episodes
Adam — 1× in this paper · explained in 2 episodes
AdamW Optimizer — 1× in this paper · explained in 1 episode
Commentary — 1× in this paper · explained in 2 episodes
Generalization — 1× in this paper · explained in 13 episodes
Hallucination — 1× in this paper · explained in 3 episodes
Lexical Diversity — 1× in this paper · explained in 2 episodes
Transformer — 1× in this paper · explained in 2 episodes
Verification — 1× in this paper · explained in 2 episodes

Sources

Transforming Question Answering Datasets Into Natural Language Inference Datasets

Related papers