Researchers have analysed one million health-related Reddit threads to build an AI platform that can identify mental health concerns quickly with hope the findings will help fast-track and better personalise care.
Victoria University’s Associate Professor Khandakar Ahmed and Saima Rani spent two years building a mental health narrative database and trained two AI models – PaLM 2 and AutoML, to analyse real-world mental health narratives shared on Reddit.
Analysing the language used and linking it to root-cause classifications from Health Direct Australia, 800 threads from the one-million-post database were then manually classified by psychiatrist and Honorary Associate Professor Dr Manjula O’Connor from the University of Melbourne. This data was then fed back to the AI models. Both models then evaluated new threads, which were separately classified by human raters as well. PaLM 2 matched the human raters’ judgement in 80 per cent of cases, while AutoML matched in 68 per cent of cases.
While there are limitations, the researchers believe these AI approaches could support mental health screening and triage, patient onboarding, reviewing patient stories and digital mental health services.
“Mental health is deeply personal, and every person has a different story behind their struggles. Two people can experience similar symptoms for very different reasons. We wanted to explore whether AI could look beyond the symptoms and recognise some of the deeper themes people describe in their own words,” Associate Professor Ahmed said.
“This is an exciting health research breakthrough. We know AI is being used in some health settings already, such as to assist practitioners with note taking in consultations, but as we have shown here, the possibilities are much greater.”
The peer-reviewed study, published in the Journal of Medical Internet Research, found the AI models, once trained on expert-verified data, could meaningfully align with human judgement in identifying the root causes behind people’s mental health struggles.
“With about 40 per cent of the population experiencing a mental illness at some time in their life this methodology has the potential to detect population level changes to help prioritise public health approach to mental illness in Australia,” Dr O’Connor said.
Key findings:
- The research demonstrates how AI models can help identify patterns that may otherwise remain hidden in large volumes of unstructured text, potentially supporting earlier intervention and more personalised mental health care.
- The analysis found differences in how the models made decisions. PaLM 2 was more consistently aligned with human reviewers when interpreting emotional tone and broader context, while AutoML relied more on specific words and phrases.
- Future research should incorporate larger evaluation sets and a more diverse range of experts to analyse the AI outputs.
