PIVOTSBench: Evaluating Fine-Grained Interpersonal Relationship Reasoning in Multimodal Large Language Models

arXiv:2606.23092 · cs.CL · Submitted 2026-06-22 · Read on arXiv

cs.CL

Submitted: 2026-06-22

Updated: 2026-09-02

Code: https://github.com/abwilf/Social-IQ-2

Project page: https://flynnzhangsx.github.io/PIVOTSBench

Terminology

• Benchmark — 53× in this paper · explained in 2 episodes
• Core — 20× in this paper · explained in 2 episodes
• Accuracy — 11× in this paper · explained in 1 episode
• Ablation — 6× in this paper · explained in 1 episode
• Ablation Studies — 4× in this paper · explained in 1 episode
• Cooperation — 4× in this paper · explained in 2 episodes
• Annotator Agreement — 3× in this paper · explained in 1 episode
• Calibration — 3× in this paper · explained in 7 episodes
• Grounding — 3× in this paper · explained in 3 episodes
• Social Relationship Recognition — 3× in this paper
• Verification — 3× in this paper · explained in 2 episodes
• Alignment — 2× in this paper · explained in 1 episode
• Benchmarking — 2× in this paper · explained in 3 episodes
• Cognitive Load — 2× in this paper · explained in 2 episodes
• Consistency — 2× in this paper · explained in 3 episodes
• Entanglement — 2× in this paper · explained in 4 episodes
• Systematic Bias — 2× in this paper · explained in 3 episodes
• Temporal Reasoning — 2× in this paper · explained in 2 episodes
• alpha — 2× in this paper · explained in 1 episode
• Actor — 1× in this paper · explained in 3 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Adaptive Reasoning — 1× in this paper · explained in 2 episodes
• Domain Adaptation — 1× in this paper · explained in 5 episodes
• Information Asymmetry — 1× in this paper · explained in 2 episodes
• Information Leakage — 1× in this paper · explained in 2 episodes
• Latent Representation — 1× in this paper · explained in 2 episodes
• Multi-agent Collaboration — 1× in this paper · explained in 2 episodes

Sources

• Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
• GPT-4 Technical Report
• SIV-Bench: A Video Benchmark for Social Interaction Understanding and Reasoning
• Qwen3 Technical Report
• SocialIQA: Commonsense Reasoning about Social Interactions
• SocialMaze: A Benchmark for Evaluating and Enhancing Social Reasoning in Large Language Models in Complex Social Environments
• SocialEval: Evaluating Social Intelligence of Large Language Models

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering