Algorithmic Reflectivity in Large Language Models: Methodological Study of Linguistic Structure in Mentalization
cs.CL, cs.AI
Submitted: 2025-11-20
Updated: 2026-10-20
Comments: Published version in JMIR AI 2026;5:e90750, https://ai.jmir.org/2026/1/e90750. Licensed under Creative Commons Attribution cc-by 4.0. 3 tables, 1 figure. Previous title: "The Linguistic Architecture of Reflective Thought: Evaluation of a Large Language Model as a Tool to Isolate the Formal Structure of Mentalization"
Journal ref: JMIR AI 2026;5:e90750
DOI: 10.2196/90750
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: Mentalization is a core human capacity involving the interpretation of one's own and others' behavior in terms of underlying mental states.
Terminology
Abstract
Mentalization is a core human capacity involving the interpretation of one's own and others' behavior in terms of underlying mental states. Large language models (LLMs) can generate linguistically reflective discourse, raising the question of whether the formal linguistic structure of mentalization can be reproduced without experiential and affective processes. This study assessed whether an LLM can generate mentalization profiles structurally coherent with the dimensions of mentalization-based treatment (MBT) and recognized as formally mentalizing by expert clinicians. Fifty human-model dialogues were generated with GPT-4.1 under standardized conditions; at the end of each, the model produced a narrative mentalization profile. Five MBT-trained psychiatrists independently rated each profile on 4 canonical MBT axes and 2 higher-order dimensions (regulatory functions, synthetic profile) for evaluative and argumentative coherence. Interrater reliability was assessed with ICC(3,k). Mean scores ranged from 3.63 to 3.98, indicating moderate-to-high structural coherence; ICC(3,k) in the evaluative domain ranged from 0.60 to 0.84. Evaluative and argumentative coherence were strongly associated (Pearson r 0.826-0.967). A negative control with mismatched dialogue-profile pairs yielded minimal ratings. Clinicians described outputs as linguistically organized and clinically interpretable, but with limited affective variability and contextual integration. The findings suggest that LLMs can reproduce the formal linguistic structure of reflective discourse - a phenomenon we term algorithmic reflectivity - which appears more consistent with structural language regularities than with experiential or intentional processes. LLMs may serve as research and training tools for studying the linguistic aspects of mentalization.
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering