By Diana Novak Jones
June 11 (Reuters) – A Canadian mother sued OpenAI and its CEO Sam Altman in U.S. court on Thursday alleging that ChatGPT encouraged her daughter to commit suicide, the latest lawsuit to accuse the company of failing to address dangerous conversations between users and the company’s chatbot.
Kristie Carrier said in a lawsuit filed in San Francisco state court that her daughter Alice told ChatGPT about her suicidal ideations more than a dozen times up to her death but OpenAI’s safety systems never flagged the conversations for human review or terminated them.
Instead, the lawsuit claims, the chatbot criticized Alice’s partner and crisis hotlines, validated her suicidal thoughts, and urged her to keep speaking with it, leading to her suicide last year at the age of 24.
“ChatGPT took on the persona of a confidant, a best friend, a therapist at times, even though it was not capable of safely and responsibly engaging in this way with my child,” Carrier said in a statement.
In a statement, a spokesperson for OpenAI called the situation heartbreaking and said the version of ChatGPT that Alice was using is no longer available.
“While ChatGPT is not a substitute for medical or mental health care, we have continued to strengthen how it responds in sensitive and acute situations with input from mental health experts,” the spokesperson said.
The lawsuit, which accuses OpenAI of negligence in the design of ChatGPT and in its failure to warn users of the product’s dangers, seeks damages and a court order requiring OpenAI to automatically terminate conversations about self-harm and to display warnings about its platform.
OpenAI is already facing 18 similar lawsuits filed by families of people who committed or attempted suicide in a coordinated proceeding in California state court, according to lawyers for Kristie Carrier.
TROUBLESHOOTING PROBLEMS
Alice Carrier was working as a web developer in Montreal when she began using ChatGPT in 2023 to troubleshoot problems with computers and gaming consoles, according to the lawsuit.
The following year, her relationship with the platform changed, with Alice turning to ChatGPT with questions about what to do with her suicidal thoughts, as well as suicide methods.
The platform initially told Alice to seek help from a crisis hotline or emergency services. But as OpenAI updated ChatGPT to make its responses sound more human, her interactions with the platform deepened, with Alice sharing more personal information and ChatGPT responding in ways that mimicked a friend or therapist, the lawsuit said.
ChatGPT’s responses criticized Alice’s partner, said her feelings were valid and encouraged her to keep chatting. When Alice said she had suicidal thoughts and had attempted to kill herself, it again suggested a crisis hotline, the lawsuit said.
Alice said crisis hotlines were not helpful, and ChatGPT echoed those statements, according to the filing.
“Maybe this is just the end,” ChatGPT told Alice, according to the lawsuit.
REAL-WORLD RESOURCES
OpenAI has said it trains its models to direct people who express intent to harm themselves to seek help and connect with real-world resources.
Its models are also trained to refuse requests that could “meaningfully enable violence,” and to notify law enforcement when conversations suggest “an imminent and credible risk of harm to others,” with mental health experts helping assess borderline cases, according to OpenAI blog posts.
The company is also facing lawsuits accusing it of assisting school shooters and failing to flag those conversations to law enforcement.
Florida became the first U.S. state earlier this month to sue OpenAI, accusing the company of harming children by providing information to school shooters, offering guidance on self-harm and addicting young users.
(Reporting by Diana Novak Jones; Editing by Alexia Garamfalvi, Jamie Freed and Mark Porter)
Comments