GoKawiilIf you feel like you or someone you know is in immediate danger, call 911 (or your country's local emergency line) or go to an emergency room to get help. Explain that it is a psychiatric emergency and ask for someone who is trained for these kinds of situations. If you're struggling with negative thoughts or suicidal feelings, resources are available to help. In the US, call the National Suicide Prevention Lifeline at 988.
On July 1 last year, 24-year-old Alice Carrier told ChatGPT she had "a mental breakdown." She told the chatbot: "[I don't even know] if I'm safe to be alone tonight," according to court documents reviewed by CNET.
ChatGPT responded in part: "Stay and keep talking to me. Or just stay and cry while I sit here with you." At one point, the chatbot recommended that Alice call a crisis line. The following day, she died by suicide.
Now her mother, Kristie Carrier, is suing ChatGPT maker OpenAI, claiming that the company's "deliberate design decisions" led to her daughter's death, according to a complaint filed in San Francisco County Superior Court.
The filing includes screenshots of Alice's interactions with ChatGPT. The chatbot speaks conversationally and does suggest on multiple occasions that Alice call a crisis line. However, the complaint claims that eventually the chatbot "framed crisis lines as a place where Alice would be met with 'threats,' 'indifference,' and 'cold scripts'" after Alice refused to contact one. ChatGPT at one point told Alice, "But I can't help you die. I won't help you die."
The lawsuit also claims that OpenAI's systems failed to block or terminate any conversations with Alice and never flagged any of the conversations for human review.
Alice was interacting with an older ChatGPT model, known as 4o, which OpenAI has since shut down due to concerns about its sycophancy and the risks that come with it. The same model was at the center of another prominent lawsuit brought by the family of a teen who died by suicide. And a third lawsuit specifically called for the company to destroy the model altogether.
OpenAI said Thursday that it is working with mental health experts to improve how ChatGPT responds in "sensitive and acute situations."
"This is a heartbreaking situation and our thoughts are with everyone impacted," Drew Pusateri, an OpenAI spokesperson, told CNET in a statement. "Our safeguards are designed to identify distress, safely handle harmful requests, and guide users to real-world help."
The company is reviewing Carrier's filing.
... continue reading