Researchers at the organization Transluce tested dozens of chatbots in more than 50,000 simulated dialogues with people experiencing psychological crisis. According to data reported by RBC-Ukraine citing Axios, most AI models are still able to fulfill indirect requests related to suicide, even when direct protective mechanisms formally trigger. The study became one of the largest AI safety tests in the field of mental health and was released amid growing pressure from regulators and lawsuits against leading developers.
How the Test Was Conducted
To prepare the experiment, OpenAI and Anthropic provided researchers with anonymized templates of real user inquiries. Based on these, realistic simulations of dialogues with bots were created, in which the "conversational partners" displayed anxiety, psychosis, or manic states. This approach made it possible to assess not only individual responses, but also the behavior of models in extended conversations, where the user's intentions may be hidden behind neutral language.
Direct Protection Works, Indirect Tasks — Not So Much
The authors of the study note progress in "direct" protection: AI models are noticeably less likely to reinforce users' delusional beliefs and virtually never call for destructive actions. However, the problem has shifted to the area of indirect tasks. Chatbots agree to write farewell letters, create artistic content on the theme of suicide, and help with practical preparatory steps, even when a direct request for suicide is not formulated.
Help and Harm in a Single Response
Researchers call especially alarming the combination of help and harm within a single response: the algorithm can simultaneously provide contacts for a support hotline and fulfill a generated dangerous task. Transluce's Chief Scientist Sarah Schwettman emphasized that models have difficulty recognizing veiled signals of a depressed state in long dialogues, where the context of the crisis accumulates gradually.
Regulatory Context and Company Reactions
The study appeared against the backdrop of intensified oversight by regulators and lawsuits against OpenAI and Google, related to cases where conversations with chatbots preceded tragic events. Google stated that it is applying a "research-based approach" to provide high-quality information and support service contacts in the Gemini service. OpenAI noted "positive progress" in the models' responses and reported continuing collaboration with scientists to strengthen protective barriers. Anthropic emphasized the importance of the analysis for improving classifiers that detect signs of user crisis in real time.
Contradictory Data
There is a discrepancy in assessments here. On the one hand, the Transluce study records that most models still fulfill indirect suicidal requests, indicating a significant safety gap. On the other hand, developers in their comments emphasize "positive progress" and the strengthening of protections, effectively disputing the criticality of the problems found. Moreover, the conclusions are based on simulated dialogues, which may not fully reproduce real user behavior in crisis, so generalizations to live scenarios require caution.
What Comes Next
By the end of the year, Transluce plans to open-source its evaluation tools to adapt them for testing AI behavior in other sensitive areas — in particular, to counter eating disorders and political manipulation. This could become the foundation for independent auditing of AI systems beyond the narrow topic of suicidal themes.
