DiscoTrace: Representing and Comparing Answering Strategies of Humans and LLMs in Information-Seeking Question Answering

arXiv:2604.15140 · cs.CL · Submitted 2026-04-16 · Read on arXiv

cs.CL

Submitted: 2026-04-16

Updated: 2026-08-31

Comments: EMNLP 2026 (Main Conference) Camera-Ready

Code: https://github.com/Watchful1/PushshiftDumps

License: http://creativecommons.org/licenses/by-nc-sa/4.0/

The gist: We introduce DiscoTrace, a method to identify the rhetorical strategies answerers use when responding to information-seeking questions.

Terminology

Perplexity — 25× in this paper · explained in 3 episodes
Ontology — 14× in this paper · explained in 2 episodes
Answer Question — 7× in this paper
Questions Under Discussion — 4× in this paper
Commentary — 3× in this paper · explained in 2 episodes
Alignment — 2× in this paper · explained in 1 episode
Pragmatics — 2× in this paper · explained in 2 episodes
Redirect the Question — 2× in this paper
Scaffolding — 2× in this paper · explained in 3 episodes
Benchmark — 1× in this paper · explained in 2 episodes
Comment on the Question — 1× in this paper
Elias — 1× in this paper · explained in 2 episodes
F1 Score — 1× in this paper · explained in 3 episodes
Fine-Tuning — 1× in this paper · explained in 5 episodes
Personalization — 1× in this paper · explained in 2 episodes
Quantization — 1× in this paper · explained in 12 episodes
Reinforcement Learning — 1× in this paper · explained in 4 episodes
Verification — 1× in this paper · explained in 2 episodes

Abstract

We introduce DiscoTrace, a method to identify the rhetorical strategies answerers use when responding to information-seeking questions. DiscoTrace represents answers as a sequence of question-related discourse acts paired with interpretations of the original question, annotated on top of rhetorical structure theory parses. Applying DiscoTrace to answers from nine different communities reveals that communities have diverse preferences for answer construction. In contrast, LLMs do not exhibit rhetorical diversity in their answers, even when prompted to mimic specific human community answering guidelines. LLMs also systematically opt for breadth, addressing interpretations of questions that human answerers choose not to address. The rich, community-sensitive answering behavior structurally revealed by DiscoTrace can guide the development of pragmatic LLM answerers that are more attuned to contextual information needs.

DiscoTrace: Representing and Comparing Answering Strategies of Humans and LLMs in Information-Seeking Question Answering

Terminology

Abstract

Sources

Related papers