What You Should Know:
– Brightside Health, a telemental health company released research demonstrating the potential of large language models (LLM) in predicting mental health crises.
– The study, published in JMIR Mental Health, compared the performance of OpenAI’s GPT-4 to human clinicians in identifying patients at risk of suicide.
Study Background and Key Findings
The research analyzed data from over 460 patients, including those who had reported suicidal ideation with a plan. Both clinicians and GPT-4 were tasked with predicting the likelihood of a mental health crisis based solely on the patient’s initial complaint.
The study resulted in the following:
- GPT-4 Achieved Similar Accuracy: The AI model demonstrated comparable overall accuracy to human clinicians in identifying patients at risk of suicide.
- Superior Sensitivity: GPT-4 exhibited higher sensitivity, meaning it was better at correctly identifying patients who would later develop suicidal ideation.
- Time Efficiency: GPT-4 completed the analysis significantly faster than human clinicians, highlighting the potential for increased efficiency in mental healthcare.
While the study demonstrates the potential of LLMs in this area, researchers emphasize the need for further development and testing before clinical implementation. The model’s performance was influenced by the specific data used for training, and it is essential to address potential biases in LLMs.
“In line with our commitment to utilize AI in a safe and controlled manner, this research highlights the potential of large language models for triage and clinical decision support in mental health,” said Dr. Mimi Winsberg, Co-Founder and Chief Medical Officer of Brightside Health. “While clinical oversight remains paramount, technologies such as these can help alleviate provider time shortage and empower providers with risk assessment tools, which is especially crucial for patients at risk of suicide.”