Algorithmic Reflectivity in Large Language Models: Methodological Study of Linguistic Structure in Mentalization

arXiv:2512.08945 · cs.CL, cs.AI · Submitted 2025-11-20 · Read on arXiv

cs.CL, cs.AI

Submitted: 2025-11-20

Updated: 2026-10-20

Comments: Published version in JMIR AI 2026;5:e90750, https://ai.jmir.org/2026/1/e90750. Licensed under Creative Commons Attribution cc-by 4.0. 3 tables, 1 figure. Previous title: "The Linguistic Architecture of Reflective Thought: Evaluation of a Large Language Model as a Tool to Isolate the Formal Structure of Mentalization"

Journal ref: JMIR AI 2026;5:e90750

DOI: 10.2196/90750

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

The gist: Mentalization is a core human capacity involving the interpretation of one's own and others' behavior in terms of underlying mental states.

Terminology

Abstract

Mentalization is a core human capacity involving the interpretation of one's own and others' behavior in terms of underlying mental states. Large language models (LLMs) can generate linguistically reflective discourse, raising the question of whether the formal linguistic structure of mentalization can be reproduced without experiential and affective processes. This study assessed whether an LLM can generate mentalization profiles structurally coherent with the dimensions of mentalization-based treatment (MBT) and recognized as formally mentalizing by expert clinicians. Fifty human-model dialogues were generated with GPT-4.1 under standardized conditions; at the end of each, the model produced a narrative mentalization profile. Five MBT-trained psychiatrists independently rated each profile on 4 canonical MBT axes and 2 higher-order dimensions (regulatory functions, synthetic profile) for evaluative and argumentative coherence. Interrater reliability was assessed with ICC(3,k). Mean scores ranged from 3.63 to 3.98, indicating moderate-to-high structural coherence; ICC(3,k) in the evaluative domain ranged from 0.60 to 0.84. Evaluative and argumentative coherence were strongly associated (Pearson r 0.826-0.967). A negative control with mismatched dialogue-profile pairs yielded minimal ratings. Clinicians described outputs as linguistically organized and clinically interpretable, but with limited affective variability and contextual integration. The findings suggest that LLMs can reproduce the formal linguistic structure of reflective discourse - a phenomenon we term algorithmic reflectivity - which appears more consistent with structural language regularities than with experiential or intentional processes. LLMs may serve as research and training tools for studying the linguistic aspects of mentalization.

Related papers