MENU
Skip to main content
All blog posts

AI and mental health workforce learning

PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals

AI-simulated patients let trainees practise cognitive behavioral therapy. In a small study, trainees reported greater skill and confidence after using the system.

  • Peer reviewed · 2024
Complete abstract from PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals.
Complete abstract from the original paper. Source: Association for Computational Linguistics and Ruiyi Wang, Stephanie Milani, Jamie C. Chiu, Jiayin Zhi, Shaun M. Eack, Travis Labrum, Samuel M. Murphy, Nev Jones, Kate Hardy, Hong Shen, Fei Fang, Zhiyu Zoey Chen. Ruiyi Wang, Stephanie Milani, Jamie C. Chiu, Jiayin Zhi, Shaun M. Eack, Travis Labrum, Samuel M. Murphy, Nev Jones, Kate Hardy, Hong Shen, Fei Fang, Zhiyu Zoey Chen. 2024. “PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals.” In *Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, 12772–12797. Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.emnlp-main.711. Reproduced under CC BY 4.0; no third-party material is included. Source paper, page 1 View full-size excerpt

Can simulated patients help trainees practice a difficult skill?

Mental health professionals often learn cognitive behavioral therapy through lectures, videos and role-play with peers. Those methods can leave a gap between classroom learning and the complexity of a real conversation. PATIENT-Ψ is a research prototype designed to let trainees practice one specific CBT skill: formulating a patient’s cognitive model from what they say during a session.

In CBT, a cognitive model describes how past experiences, core beliefs, assumptions and quick interpretations connect to emotions and behavior. PATIENT-Ψ combines such a model with a language model to create a simulated patient. The research team assembled 106 cognitive models with clinical-psychologist input and added six conversational styles, including reserved, verbose and tangential. In the companion PATIENT-Ψ-TRAINER, a trainee conducts a simulated session, writes down their interpretation of the patient’s model and then compares it with the model that shaped the simulation.

Trainees and experts see promise

The evaluation included 13 mental-health trainees and 20 experts. Experts judged the system’s simulated interactions as closer to real patients than GPT-4 on their assessments of maladaptive beliefs, conversational styles and emotional states. Trainees reported that practicing with PATIENT-Ψ-TRAINER improved their perceived skill and confidence more than existing training formats such as textbooks, videos and peer role-play (abstract and pages 12772–12773). The framework also gives learners a reference answer, making practice possible without an instructor observing every exchange.

The study tests an important premise: if the simulated patient is built from an explicit cognitive model, a trainee can practice inferring that model and receive feedback by comparing their formulation with the original. This makes the exercise more focused than free-form chat. The authors also report that automatic evaluation by other language models failed to capture the specialized judgments needed to assess whether a simulated patient behaves faithfully. Human expertise remains necessary to assess the training tool itself.

The evidence is early. The trainee group is small, and reported confidence or perceived learning is not the same as demonstrated clinical competence. The study does not show whether skills persist, transfer to real sessions or improve patient outcomes. The cases were crafted by experts from therapy transcript summaries, so they should not be interpreted as a representative sample of patients. Simulated interactions can support practice but cannot substitute for supervised clinical education. Mental-health scenarios also raise privacy, safety and emotional-suitability questions that extend beyond model performance.

A bounded role for simulation

The work suggests that language models can make structured practice more accessible when an expert-defined learning objective and a feedback reference are available. It also highlights a limit: a fluent conversation is not enough to establish clinical realism. For educators and organizations, the relevant signal is whether the simulation helps learners practice a clearly defined skill—and whether that gain can later be observed in supervised work. Further evaluation with larger trainee cohorts and behavioral outcomes would clarify how far this kind of tool can responsibly extend.

The system’s reference case is a learning aid; it is not a diagnosis of an actual patient or an assessment of a trainee’s readiness to practice independently. Simulation offers a controlled way to rehearse conversations, but it cannot reproduce the full uncertainty, diversity or duty of care involved in clinical work.

Bibliography & sources

  1. Ruiyi Wang, Stephanie Milani, Jamie C. Chiu, Jiayin Zhi, Shaun M. Eack, Travis Labrum, Samuel M. Murphy, Nev Jones, Kate Hardy, Hong Shen, Fei Fang, Zhiyu Zoey Chen. 2024. “PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals.” In *Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, 12772–12797. Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.emnlp-main.711.