When Empathy Misses the Goal: A Benchmark for Goal Displacement in LLM Advice

arXiv:2610.25522 · cs.AI · Submitted 2026-10-18 · Read on arXiv

cs.AI

Submitted: 2026-10-18

Updated: 2026-10-18

Comments: Accepted for publication at the 40th Annual Conference on Neural Information Processing Systems (NeurIPS 2026), Evaluations and Datasets Track

License: http://creativecommons.org/licenses/by/4.0/

The gist: Conversational large language models can respond with warmth and reassurance while failing to advance the user's stated goal.

Terminology

Abstract

Conversational large language models can respond with warmth and reassurance while failing to advance the user's stated goal. We conceptualize this failure as goal displacement, the substitution of persona-maintenance content for goal-relevant substance. To measure it, we introduce GDB (Goal Displacement Benchmark), comprising 230 scenarios across nine domains and 4, 134 human-scored responses from 18 models. A 2 times 2 taxonomy crosses hard constraints with emotional distress, distinguishing failures to confront infeasible goals or combine empathy with practical guidance from reverse displacement, where unsolicited problem-solving replaces requested emotional support. A deletion-test rubric assesses whether substantive help remains after purely supportive sentences are removed. Independent raters achieve weighted κ= 0.82 against author scores for displacement severity. Trap displacement ranges from 0.0% to 37.7% and is strongly negatively correlated with Chatbot Arena Elo (ρ= -0.706, n = 15). Reverse displacement reaches 41.9%, including substantial rates in models with negligible trap displacement. A factorial probe supports a distinction from social sycophancy. Human-scored responses in the high-complexity, low-pressure condition exhibit 35.0% sycophancy but 7.5% displacement, with little association between the constructs. Matched OLMo-2 checkpoints show that mixed empathy-with-constraints displacement remains at 52.3% after supervised fine-tuning and falls to 13.8% after direct preference optimization. GDB provides a reproducible diagnostic of whether responses advance the operationalized user goal, supporting model comparison, post-training evaluation, and regression testing. We release the dataset, annotations, and rubric under CC-BY-4.0 and the evaluation code under MIT.

Related papers