Multi-Legal-Bench: When the Answer Is in the Input. Label Leakage in Legal Benchmarks Built from Court Registries
cs.CL, cs.AI
Submitted: 2026-05-28
Updated: 2026-09-26
Terminology
Sources
- SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects
- LegalBench.PT: A Benchmark for Portuguese Law
- LexGLUE: A Benchmark Dataset for Legal Language Understanding in English
- VLegal-Bench: Cognitively Grounded Benchmark for Vietnamese Legal Reasoning of Large Language Models
- LEXam: Benchmarking Legal Reasoning on 340 Law Exams
- LegalRikai: Open Benchmark -- Benchmark for Complex Japanese Corporate Legal Tasks
- LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
- CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review
- ArabLegalEval: A Multitask Benchmark for Assessing Arabic Legal Knowledge in Large Language Models
- Evaluating the Limits of Large Language Models in Multilingual Legal Reasoning
- LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain
- MultiLegalPile: A 689GB Multilingual Legal Corpus
- The Tokenizer Tax Across 25 European Languages: Domain Invariance, Cross-Lingual Few-Shot Effects, and the Ukrainian Penalty
- Tokenizer Fertility and Zero-Shot Performance of Foundation Models on Ukrainian Legal Text: A Comparative Study
- One Law, Many Languages: Benchmarking Multilingual Legal Reasoning for Judicial Support
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering