Detecting AI Impostors: How Do Middle Schoolers Identify LLM Agents in a Live Collaborative Setting?

arXiv:2608.30948 · cs.CL · Submitted 2026-08-31 · Read on arXiv

cs.CL

Submitted: 2026-08-31

Updated: 2026-08-31

Comments: Accepted to EMNLP 2026 Main

Code: https://github.com/danschumac1/Detecting_AI_

License: http://creativecommons.org/licenses/by/4.0/

The gist: LLMs can imitate how people write, which raises concerns about impersonation, trust, and detection in social settings.

Terminology

Abstract

LLMs can imitate how people write, which raises concerns about impersonation, trust, and detection in social settings. These concerns are especially important for adolescents, who use generative AI frequently but may struggle to recognize it. We introduce DoppelBot, a cooperative social deduction game designed to study how young people detect and respond to AI impersonation. Through studies with middle schoolers, we investigate whether a DoppelBot prompts reflection on privacy and impersonation, how repeated exposure affects AI-detection accuracy as agents become more personalized, and which strategies students use to identify AI doppelgängers. We find that students' detection accuracy improves over time, driven by a shift from relying on linguistic cues to leveraging shared social and contextual signals. Students also demonstrated an understanding of AI limitations such as embodiment and reflected on broader issues such as data privacy. To support future research, we release an anonymized dataset of game transcripts and voting behavior.

Sources

Related papers