Using profiles of cognitive capability to assess AI suitability for workplace tasks

arXiv:2608.25623 · cs.AI, cs.CY, cs.HC · Submitted 2026-08-26 · Read on arXiv

cs.AI, cs.CY, cs.HC

Submitted: 2026-08-26

Updated: 2026-08-26

Code: https://github.com/Kinds-of-Intelligence-CFI/Task-Suitability-Profiles

Terminology

• Benchmark — 60× in this paper · explained in 2 episodes
• Core — 23× in this paper · explained in 2 episodes
• Object Permanence — 17× in this paper
• Semantic Memory — 17× in this paper
• Social Cognition — 16× in this paper
• Accuracy — 15× in this paper · explained in 1 episode
• Instrumental Reasoning — 11× in this paper
• Theory of Mind — 10× in this paper
• Episodic Memory — 9× in this paper
• Working Memory — 7× in this paper
• Metacognition — 6× in this paper · explained in 5 episodes
• Causal Reasoning — 4× in this paper · explained in 2 episodes
• Clustering — 4× in this paper · explained in 2 episodes
• Warehouse and Logistics — 4× in this paper
• AI Agents — 3× in this paper · explained in 2 episodes
• Adam — 3× in this paper · explained in 2 episodes
• Aggregation — 3× in this paper · explained in 1 episode
• Cognitive Flexibility — 3× in this paper
• Cosine Similarity — 3× in this paper · explained in 3 episodes
• Functional Perception — 3× in this paper
• Affordance — 2× in this paper · explained in 1 episode
• Introspection — 2× in this paper · explained in 2 episodes
• Robustness — 2× in this paper · explained in 5 episodes
• Agentic AI — 1× in this paper · explained in 1 episode
• Agentic AI systems — 1× in this paper · explained in 1 episode
• Consistency — 1× in this paper · explained in 3 episodes
• Failure Modes — 1× in this paper · explained in 3 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• Grounding — 1× in this paper · explained in 3 episodes
• Information Gain — 1× in this paper · explained in 2 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• LLM-as-a-Judge — 1× in this paper · explained in 3 episodes
• Redundancy — 1× in this paper · explained in 2 episodes
• Theory of Mind (ToM) — 1× in this paper · explained in 2 episodes
• UK Artificial Intelligence Safety Institute — 1× in this paper

Sources

• Towards a Science of AI Agent Reliability
• Measuring Agents in Production
• The Illusion of Readiness in Health AI
• Line Goes Up? Inherent Limitations of Benchmarks for Evaluating Large Language Models
• Artificial Intelligence Index Report 2025
• AI and the Everything in the Whole Wide World Benchmark
• Cognitive Dark Matter: Measuring What AI Misses
• Inferring Capabilities from Task Performance with Bayesian Triangulation
• Large Language Models Fail on Trivial Alterations to Theory-of-Mind Tasks
• Visuospatial Perspective Taking in Multimodal Language Models
• From Human-Level AI Tales to AI Leveling Human Scales
• Benchmark Data Contamination of Large Language Models: A Survey
• Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversations
• LLM-BabyBench: Can Language Models Plan in Worlds They Can Simulate?
• Capabilities Ain't All You Need: Measuring Propensities in AI

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection