Generative AI Purpose-built for Social and Mental Health: A Real-World Pilot

A single-arm longitudinal naturalistic pilot study evaluated the feasibility, engagement, and acceptability of an AI foundation model designed for mental health in a real-world setting. The primary finding is that users reported sustained reductions in depression and anxiety symptoms, with positive outcomes associated with specific usage patterns and therapeutic alliance comparable to traditional care.

Study Design and Participants

This single-arm longitudinal naturalistic pilot evaluated the feasibility of a foundation model designed for mental health in a real-world setting. The study involved adults (N=305) who engaged with the chatbot between May 15, 2025 and September 15, 2025. Participants were recruited through ads on Meta and had baseline scores of at least 10 on either the PHQ-9 or GAD-7. Inclusion criteria required English-speaking users located in the United States, age between 18 and 85, reliable internet access, and at least one follow-up assessment during the study window. Exclusion criteria included self-reported risk factors indicating a need for higher care (such as schizophrenia spectrum or severe substance use disorder) or active suicidal thoughts sufficient to warrant immediate referral.

Intervention and Model Architecture

The intervention is a mental health AI model delivered through text- and voice-based interaction for mobile devices, referred to as Ash. The AI model is built on a mental health foundation architecture pre-trained on a private, large-scale dataset of mental health-relevant data, including over one hundred thousand hours of anonymized transcripts. This base model is subsequently aligned through a multi-stage training process, including supervised fine-tuning with clinician-generated annotations to become Ash. Training data includes diverse therapeutic approaches such as psychodynamic therapy, Dialectical Behavior Therapy, Acceptance and Commitment Therapy, second-wave Cognitive Behavioral Therapy, and Motivational Interviewing. To support safety and appropriateness of responses, the model employs a two-pass guardrail architecture: the first pass is a high recall embeddings-based classifier to detect potentially inappropriate queries before they are sent by the model; flagged content is then passed through an LLM safety verification layer, which either blocks the message and replaces it, or else allows it to pass.

Assessments and Data Analytic Strategy

Participants were assessed at baseline, Week 2, Week 4, Week 6 (follow-up), and again at follow-up at Week 10. Assessments included the Patient Health Questionnaire (PHQ-9) for depressive symptoms, the Generalized Anxiety Disorder Scale (GAD-7) for anxiety symptoms, the Behavioral Activation for Depression Scale Short Form (BADS28) to assess activation and avoidance behaviors, the UCLA Loneliness Scale 4-item version to measure loneliness, and the Multidimensional Scale of Perceived Social Support (MSPSS31). The Goal Attainment Scaling procedure was adapted to capture individualized progress toward participant-defined goals. Data analytic strategy included effect sizes (Cohen’s d), growth mixture modeling, and mixed-effects models to examine users’ symptoms severity, outcome trajectories, and engagement.

Key Results on Clinical Change and Trajectories

Users reported reductions in PHQ-9 and GAD-7 sustained at follow-up. Associated improvements in Behavioral Activation, Social Interaction, Loneliness, and Perceived Social Support were observed and maintained across three outcome trajectories: Improving (n=129, 42.3%), Non-responders (n=147, 48.2%), and Rapid Improving (n=29, 9.5%). Clinical change was quantified using Cohen’s d effect sizes of 0.93 for depression and 0.79 for anxiety at follow-up. Three longitudinal patterns emerged shared across depression, anxiety, social support, and behavioral activation from baseline to 10 week follow-up: Rapid improving (n=29, 9.5%), Improving (n=129, 42.3%), and Non-responders (n=147, 48.2%).

Engagement and Predictors of Improvement

Engagement metrics included active days, minutes of use, and total words exchanged throughout the six-week period. Mixed-effects models were used to examine the relationship between engagement and symptom improvement across different two-week assessment windows. For improving participants, reductions in PHQ-9 scores were associated with increased active days (β=-0.365), minutes of use (β=-0.672), and words exchanged (β=-0.402). Higher Week 2 Working Alliance Inventory (WAI) scores predicted Improving and Rapid Improving trajectories, and Week 2 alliance scores also significantly predicted improvement for PHQ-9 and GAD-7 at the Week 6 assessment. The study concluded that AI interventions may be feasible even under naturalistic conditions, with therapeutic alliance comparable to traditional psychotherapy.

Here are specific improvements to AI systems based on the findings in this study, along with what those improved systems could achieve:

The core takeaway is that a foundation model (like Ash) designed for mental health can be feasible, engaging, and acceptably effective in a real-world setting. The key mechanisms driving improvement are personalization through therapeutic modalities and sustained engagement.

Here are the specific improvements:

1. // Enhance Therapeutic Modality Personalization based on Evidence:

2. // Implement a dynamic Therapeutic Alignment mechanism that adjusts the AI's response style (e.g., tone, complexity of advice) based on real-time user feedback derived from Working Alliance Inventory (WAI) scores and Therapist Representation Inventory (TRI) internalization metrics.

3. // Integrate Social Resource Scaffolding:

4. // Develop specific prompts that are triggered when loneliness or low social support indicators are detected, proactively suggesting engagement with existing social resources or helping the user develop social skills, moving beyond mere symptom reduction to address holistic well-being (as suggested by the findings on social health).

5. // Optimize Engagement Mechanisms using Multi-Modal Feedback Loops:

6. // Design personalized feedback and scaffolding that is tailored specifically to the user's engagement profile (e.g., if a user shows high engagement in words exchanged but low in active days, the system should prioritize content density; if they show high activity across all metrics, it should reinforce momentum).

7. // Implement Robust, Context-Aware Safety Guardrails:

8. // Refine the two-pass guardrail architecture by incorporating a fine-tuned LLM safety verification layer specifically trained on clinical scenarios (like those in Moore et al., 2025) to catch subtle, contextually inappropriate responses that might evade simple embeddings classifiers, ensuring higher fidelity in handling high-risk queries.

9. // Optimize Trajectory Prediction for Targeted Intervention:

10. // Utilize the identified LGMM trajectories (Improving, Rapid Improving, Non-responders) to trigger different intervention pathways; for Non-responders, the system should pivot to alternative therapeutic approaches or escalate to a human care pathway more aggressively, rather than maintaining a static interaction pattern.

The resulting improved AI system could achieve the following:

1. // Achieve Scalable, Acceptable Mental Health Support: The system can be deployed widely in real-world settings (via app/search) without requiring expensive human practitioner staffing for initial symptom management, effectively lowering barriers to care.

2. // Deliver Clinically Relevant Therapeutic Depth: By dynamically aligning its conversational style with the user’s perceived therapeutic alliance, the AI can move beyond generic advice to offer a more personalized and effective support experience that is comparable to traditional psychotherapy outcomes.

3. // Foster Holistic Well-being: The system won't just treat symptoms (PHQ-9/GAD-7); it will actively promote social health by integrating prompts for building social skills and connecting users with tangible community resources, thereby addressing the broader mental health spectrum identified in the longitudinal analysis.

4. // Maximize User Retention and Efficacy: By using engagement metrics to tailor the scaffolding (Improvement vs. Rapid Improvement pathways), the system can proactively maintain high user engagement, ensuring that users who are showing signs of rapid improvement receive appropriately accelerated support, while non-responders are managed through a more nuanced clinical escalation strategy.

5. // Ensure High-Fidelity Safety: The enhanced guardrail system will drastically reduce the risk of inappropriate or harmful outputs in sensitive mental health contexts, providing a safer platform for vulnerable users compared to current generalized LLM deployments.

Generative AI chatbots built for mental health could extend access to care, but evidence from real-world use is limited. We report a single-arm, naturalistic pilot of a foundation model trained for mental health, among 299 US adults with at least moderate depressive or anxiety symptoms who were followed for up to 12 months. Depression and anxiety symptoms fell by 10 weeks (Cohen's d 0.93 and 0.79), with loneliness, behavioral activation, and social interaction improving. Clinicians confirmed that automated safeguards were escalated appropriately. Participants fell into non-responding (57.2%), improving (37.1%) and rapidly improving (5.7%) trajectories. Early working alliance and greater engagement were associated with better outcomes. The AI deployed ten identified intervention families whose delivery varied with baseline anxiety and depression, with the overall ratio of clinical to non-clinical content increasing according to severity without survey information access. These findings support the feasibility of purpose-built AI for mental health.