PatientAct: Theory-Grounded Mental Health Client Simulation

PATIENTACT: Theory-Grounded Mental Health Client Simulation

This paper introduces PATIENTACT, a framework for simulating mental health clients using large language models (LLMs), designed to address critical shortcomings in existing simulators. The authors identify that current LLM-based simulated clients are overly cooperative: they disclose too readily, accept therapeutic reframes without resistance, and resolve core issues within a single session. This behavior undermines downstream applications like therapist training, LLM-therapist evaluation, and synthetic data generation.

The paper attributes these problems to two main design flaws in prior work: profiles that lack causal depth (describing what a client thinks and feels but not why) and behavioral mechanisms that treat all content as equally accessible (applying a single control uniformly across all profile content).

To solve these issues, PATIENTACT introduces a two-stage framework:

Stage I: Profile Generation. The framework structures client profiles around the 5Ps clinical case formulation (Presenting Problem, Precipitating Factors, Predisposing Factors, Perpetuating Factors, Protective Factors), which provides causal depth without tying the design to any single therapeutic modality. This is complemented by a psychological formulation layer with cognitive patterns (intermediate beliefs, automatic thoughts, triggers, coping patterns, emotional range) and interpersonal patterns based on the Core Conflictual Relationship Themes (CCRT) framework. Profiles are generated through a multi-step pipeline that takes a clinical situation, demographic scaffold, and psychological seed (core belief theme and attachment style) as input, with iterative validation for internal coherence.

Stage II: Client Simulation. During conversation, the profile is decomposed into a static layer (always in the system prompt) and a dynamic memory layer. Each item in the dynamic layer is assigned a disclosure threshold based on its vulnerability and observability, grounded in research on therapeutic trust: surface symptoms are available early, whereas formative memories require a sustained therapeutic alliance. At each turn, PATIENTACT processes the therapist's utterance through a multi-step pipeline:

1. Trust-Gated Retrieval: Relevant memory items are retrieved only if the client's current trust level meets their threshold; otherwise, items with a discomfort flag are placed on a blocked list.

2. Reaction and Behavior Modeling: The client's emotional reaction (from Hill's taxonomy, e.g., understood, challenged, scared) and behavior (e.g., recounting, cognitive exploration, resistance) are selected before generating a response.

3. Resistance Selection: When resistance is selected, PATIENTACT determines its specific form using Otani's taxonomy, spanning response quantity (e.g., going silent), content (e.g., changing the subject), and style (e.g., direct pushback).

4. Trust Dynamics: After each exchange, the client's trust level is updated asymmetrically (positive transitions are frequent but small; negative transitions are rarer but larger), conditioned on the client's attachment style.

The authors evaluate PATIENTACT on 40 hand-crafted clinical situations (20 for depression, 20 for anxiety). For profile evaluation, expert annotators rated profiles highly on Clinical Plausibility (4.43), Internal Consistency (4.38), Case Specificity (4.32), and Clinical Depth (4.28), with moderate inter-annotator agreement. Annotators also correctly predicted attachment styles (77.5% accuracy) and core belief themes (64.2% accuracy) from the profiles, well above chance levels.

For simulation evaluation, PATIENTACT was compared against three baselines: Patient-ψ (static theory-grounded profiles), AnnaAgent (minimal profiles with dynamic emotion), and ConsistentMI (modality-specific with dynamic state tracking). Human evaluators rated PATIENTACT significantly higher (p < 0.05) than all baselines across all five dimensions: Coherence (4.37), Disclosure Pacing (4.15), Resistance Quality (3.82), Emotional Authenticity (4.15), and Behavioral Realism (4.15). The largest gains were in Resistance Quality (+0.67) and Behavioral Realism (+0.63) over the best baseline.

Ablation studies showed that removing any of the three key components (trust-gating, dynamic memory, or the reaction-behavior-resistance pipeline) reduced human ratings across all dimensions, with the largest drop occurring when dynamic memory was removed, particularly in Resistance Quality (a 1.04 drop in human ratings).

The paper also reports an important finding about evaluation methodology: LLM-based judges produced different rankings than human evaluators, with the LLM judge favoring baselines on some dimensions. The authors note that automated evaluation alone is insufficient to assess the quality of therapy simulations and that human judgment remains essential for dimensions involving clinical appropriateness.

The paper concludes that profile depth and quality yield larger improvements in simulation realism than dynamic behavioral mechanisms alone and that PATIENTACT can facilitate more effective tools for therapist training, more rigorous benchmarks for LLM therapists, and richer synthetic data for mental health research. The code and data are publicly available via github.com/Sahandfer/PatientHub.

Improvements to AI systems:

1. Trust-Gated Memory Retrieval with Asymmetric Trust Dynamics

• Implement a memory access controller where each stored item has a disclosure threshold based on vulnerability/observability.

• Maintain a dynamic trust score that updates asymmetrically (small positive increments, rare large negative drops) conditioned on user attachment style.

• Block retrieval of high-threshold items until trust exceeds threshold; flag discomfort items to prevent premature disclosure.

• Improved AI system: A conversational agent that naturally paces sensitive information sharing, avoids over-disclosure, and builds rapport over multiple sessions—critical for mental health, counseling, or any domain requiring gradual trust (e.g., legal, HR, medical intake).

2. Causal Profile Generation via 5Ps + CCRT + Cognitive Patterns

• Generate user/agent profiles using structured causal formulation: Presenting Problem, Precipitating, Predisposing, Perpetuating, Protective factors.

• Add intermediate beliefs, automatic thoughts, triggers, coping patterns, and Core Conflictual Relationship Themes (CCRT) for interpersonal dynamics.

• Validate internal coherence iteratively before deployment.

• Improved AI system: A role-playing or simulation engine that produces deeply consistent, non-stereotypical personas with explainable behavior—useful for training clinicians, sales reps, or negotiators; also enables realistic synthetic data generation for mental health NLP.

3. Reaction–Behavior–Resistance Selection Pipeline

• Before generating a response, explicitly select: (a) emotional reaction from a taxonomy (e.g., understood, challenged, scared), (b) behavioral mode (e.g., recounting, cognitive exploration, resistance), and (c) resistance type (e.g., silence, topic change, direct pushback) when applicable.

• Use these selections as conditioning signals for the language model’s output.

• Improved AI system: A more human-like conversational partner that exhibits authentic emotional and behavioral variability, including non-compliance and defensiveness—essential for stress-testing therapeutic or customer-service AI, and for training humans to handle difficult interactions.

4. Static + Dynamic Memory Decomposition with Threshold-Based Access

• Separate stable identity/profile (always in context) from episodic memory (dynamic, updated per turn).

• Assign each dynamic memory item a disclosure threshold and a discomfort flag; retrieve only if trust allows.

• Improved AI system: A long-term conversational agent that remembers past interactions but respects privacy and emotional readiness—applicable to companion bots, educational tutors, or patient monitoring systems that must avoid overwhelming users.

5. Attachment-Style-Conditioned Trust Dynamics

• Parameterize trust update rules (frequency and magnitude of positive/negative changes) based on user’s attachment style (secure, anxious, avoidant).

• Improved AI system: A personalized interaction system that adapts its pacing and trust-building strategy to different user types—improving engagement and reducing dropout in digital health, teletherapy, or coaching platforms.

6. Human-in-the-Loop Evaluation for Clinical Appropriateness

• Integrate a hybrid evaluation pipeline: use LLM judges for surface-level metrics (coherence, fluency) but require human raters for dimensions like resistance quality, emotional authenticity, and clinical plausibility.

• Flag cases where LLM and human rankings diverge for manual review.

• Improved AI system: A development framework for any sensitive-domain AI (therapy, crisis intervention, legal advice) that prevents over-reliance on automated metrics, ensuring real-world safety and effectiveness.

7. Ablation-Informed Component Prioritization

• Use the finding that dynamic memory removal causes the largest drop (especially in resistance quality) to guide system design: prioritize dynamic, thresholded memory over other components when resources are limited.

• Improved AI system: A resource-efficient conversational agent that maximizes realism by focusing on memory management and trust-gating rather than over-engineering response generation.

What the improved AI system can do specifically:

• Conduct multi-session therapeutic role-play with realistic client behavior (hesitation, resistance, gradual disclosure) for training therapists or evaluating LLM-therapists.

• Generate synthetic mental health dialogue data with causal, coherent client profiles for fine-tuning or benchmarking.

• Serve as a patient simulator in medical education, allowing trainees to practice handling non-cooperative or emotionally guarded patients.

• Act as a personalized digital companion that respects user boundaries and adapts trust-building over time based on attachment style.

• Provide a testbed for studying trust dynamics in human-AI interaction, with tunable parameters for research.

LLM-based simulated clients are increasingly used to train novice counselors, evaluate LLM therapists, and generate synthetic data. However, current simulators produce overly cooperative clients that disclose too readily, accept therapeutic reframes without resistance, and resolve core issues within a single session. We trace these issues to profiles that lack causal depth and behavioral mechanisms that treat all content as equally accessible. We present PatientAct, a framework for client simulation grounded in established clinical theories. Our profiles integrate the 5Ps clinical case formulation, providing causal depth without tying the design to any single therapeutic modality. During simulation, profiles include a dynamic memory layer in which items carry trust thresholds (e.g., symptoms are available early, whereas formative memories require a sustained therapeutic alliance). At each turn, the client's emotional reaction and behavior are modeled before generating a response. If the therapist approaches gated content, PatientAct expresses resistance in terms of quantity, content, and style rather than defaulting to cooperation or a single resistance pattern. We evaluate our framework on 40 clinical situations and demonstrate that it generates diverse profiles with high clinical plausibility. Moreover, PatientAct significantly outperforms the baselines, yielding substantial gains in resistance quality and behavioral realism. Our code and data will be publicly available via github.com/Sahandfer/PatientHub.