MemoReason: Evaluating the Effect of Parametric Memory on Contextual Reasoning in LLMs

arXiv:2609.35312 · cs.CL · Submitted 2026-09-28 · Read on arXiv

cs.CL

Submitted: 2026-09-28

Updated: 2026-09-28

Terminology

• Benchmark — 32× in this paper · explained in 2 episodes
• Accuracy — 28× in this paper · explained in 1 episode
• Robustness — 9× in this paper · explained in 5 episodes
• Calibration — 5× in this paper · explained in 7 episodes
• Consistency — 5× in this paper · explained in 3 episodes
• Ablation — 4× in this paper · explained in 1 episode
• Alignment — 3× in this paper · explained in 1 episode
• Benchmarking — 3× in this paper · explained in 3 episodes
• Judge Match — 3× in this paper
• Ontology — 3× in this paper · explained in 2 episodes
• Core — 2× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 2× in this paper · explained in 6 episodes
• Temporal Reasoning — 2× in this paper · explained in 2 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Chain-of-Thought (CoT) — 1× in this paper · explained in 4 episodes
• Exact Match — 1× in this paper
• Generative AI — 1× in this paper · explained in 2 episodes
• Human-in-the-Loop — 1× in this paper · explained in 3 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• Trace — 1× in this paper · explained in 2 episodes

Sources

• The Llama 3 Herd of Models
• LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
• Beyond Consensus: Mitigating the Agreeableness Bias in LLM Judge Evaluations
• Let's Verify Step by Step
• RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content
• Capabilities of GPT-4 on Medical Challenge Problems
• Olmo 3
• gpt-oss-120b & gpt-oss-20b Model Card
• Impact of Pretraining Term Frequencies on Few-Shot Reasoning
• Mathematical Reasoning in Large Language Models: Assessing Logical and Arithmetic Errors across Wide Numerical Ranges
• Functional Benchmarks for Robust Evaluation of Reasoning Performance, and the Reasoning Gap
• Cofca: A Step-Wise Counterfactual Multi-hop QA benchmark
• VERINA: Benchmarking Verifiable Code Generation

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering