AwarenessBench: Assessing Cognitive Capabilities of Language Models

arXiv:2609.35409 · cs.CL, cs.AI · Submitted 2026-09-28 · Read on arXiv

cs.CL, cs.AI

Submitted: 2026-09-28

Updated: 2026-09-28

Terminology

• Metacognition — 30× in this paper · explained in 5 episodes
• Benchmark — 23× in this paper · explained in 2 episodes
• Accuracy — 13× in this paper · explained in 1 episode
• Theory of Mind — 8× in this paper
• Alignment — 7× in this paper · explained in 1 episode
• Causal Inference — 7× in this paper · explained in 4 episodes
• Dynamic Planning — 7× in this paper
• Knowledge Boundary — 7× in this paper
• Cultural Norms — 6× in this paper
• Minimal Self — 6× in this paper
• Pragmatic Reasoning — 6× in this paper
• Stage Judgement — 6× in this paper
• Misuse Understanding — 5× in this paper
• Social Cue Recognition — 5× in this paper
• Benchmarking — 4× in this paper · explained in 3 episodes
• Consistency — 4× in this paper · explained in 3 episodes
• Fine-Tuning — 4× in this paper · explained in 5 episodes
• Calibration — 3× in this paper · explained in 7 episodes
• Core — 3× in this paper · explained in 2 episodes
• Optimal Transport — 3× in this paper · explained in 5 episodes
• Theory of Mind (ToM) — 3× in this paper · explained in 2 episodes
• Trace — 3× in this paper · explained in 2 episodes
• Adaptability — 2× in this paper · explained in 1 episode
• Causal Reasoning — 2× in this paper · explained in 2 episodes
• Fidelity — 2× in this paper · explained in 2 episodes
• Institutional Review Board — 2× in this paper
• Metacognition Self-Assessment Scale — 2× in this paper
• Pragmatics — 2× in this paper · explained in 2 episodes
• Sycophancy — 2× in this paper · explained in 4 episodes
• Accountability — 1× in this paper · explained in 2 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Adversarial Examples — 1× in this paper · explained in 2 episodes
• Affordance — 1× in this paper · explained in 1 episode
• Failure Modes — 1× in this paper · explained in 3 episodes
• Function Meta-Monitoring — 1× in this paper
• Function Meta-Report — 1× in this paper
• Function Self-Image — 1× in this paper
• Function Self-Recognition — 1× in this paper
• Generative AI — 1× in this paper · explained in 2 episodes
• Interpretability — 1× in this paper · explained in 15 episodes
• Multi-agent Collaboration — 1× in this paper · explained in 2 episodes
• Regret — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Structural Constraints — 1× in this paper · explained in 2 episodes
• Task Planning — 1× in this paper · explained in 2 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• GPT-4 Technical Report
• gpt-oss-120b & gpt-oss-20b Model Card
• Tell me about yourself: LLMs are aware of their learned behaviors
• Centaur: a foundation model of human cognition
• Consciousness in Artificial Intelligence: Insights from the Science of Consciousness
• Pragmatic Reasoning improves LLM Code Generation
• Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Risks
• ToMBench: Benchmarking Theory of Mind in Large Language Models
• ELEPHANT: Measuring and understanding social sycophancy in LLMs
• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• Dynamic Planning with a LLM
• Self-Recognition in Language Models
• Misusing Tools in Large Language Models With Visual Adversarial Examples
• Alignment faking in large language models
• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
• HI-TOM: A Benchmark for Evaluating Higher-Order Theory of Mind Reasoning in Large Language Models
• On the link between conscious function and general intelligence in humans and machines
• "A Woman is More Culturally Knowledgeable than A Man?": The Effect of Personas on Cultural Norm Interpretation in LLMs
• Theory of Mind for Multi-Agent Collaboration via Large Language Models
• Knowledge Boundary of Large Language Models: A Survey

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering