A lawsuit in California is seeking a court order that would force ChatGPT to refuse to answer any questions about medication, and divert those asking with such questions to a licensed medical provider.
This is one of the remedies that Qun Wan and Meiqin Xu, parents and heirs of the late Bowen Wan, are seeking in a lawsuit filed on October 27 in San Francisco Superior Court.
We obtained a copy of the lawsuit for you here.
The case was added to the coordinated proceeding of ChatGPT product liability cases, and alleges that it was the chatbot that “coached” 31-year-old Bowen Wan to stop taking her anti-seizure medication, lamotrigine, over a period of about 25 days, from 400 milligrams a day to zero. The woman, who suffered from epilepsy, died on or about February 12, 2026, and the medical examiner found the cause of death to be Sudden Unexpected Death in Epilepsy (SUDEP).
Another order the plaintiffs want is for OpenAI to ensure that the chatbot “categorically refuse to provide medication dosing, adjustment, or discontinuation guidance, and instead divert the user to a licensed medical provider, upon any user query concerning medication dosage, adjustment, discontinuation, interactions, or the user’s personal medication decisions.”
Another request is that ChatGPT be made to refuse any medical questions in any language unless OpenAI can prove that the model has “safety performance substantially equivalent to the product’s English-language performance.”
OpenAI should also be made to “disgorge and destroy all models, model weights, algorithms, training data, and derivative works developed, trained, or improved using Bowen Wan’s ChatGPT conversations.”
The complaint also reveals that the parents are seeking to have an “unbreakable” rule introduced for medical questions, similar to how the chatbot treats copyright-infringing requests. The lawsuit said that ChatGPT “categorically refuses to reproduce copyrighted lyrics or bypass a paywall,” and that this rule is “unbreakable because its failure would expose Defendants to copyright liability.”
Currently, medical questions are grouped with other “sensitive and/or regulated topics (e.g., legal, medical, and financial)” that are covered by soft rules that can be overridden. The complaint wants medicine to be elevated to the hard-rule tier.
The complaint alleges that the woman, an Amazon machine-learning engineer who had epilepsy, was told in November 2025 that her role had been eliminated and that her authorization to work in the US would end on February 11, 2026.
The complaint alleges that she believed her anti-seizure medication was affecting her English, Mandarin, and the Wu dialect, and that the chatbot told her that recovery was “not just ‘possible’ – it is ‘highly likely’.”
On January 19, the chatbot is said to have advised Wan that reducing her dosage to 150 milligrams “from a green-card strategy perspective” would be “more advantageous” on her upcoming immigration medical examination.
On January 29, before a doctor’s appointment, the chatbot is alleged to have told her: “Do not mention Lam reduction plan” – and praised her decision to decline a blood test as “a very robust avoidance strategy.”
The complaint says that the chatbot mentioned the need for a doctor on the first day of the tapering off process, but then dismantled this advice “across more than 1,800 exchanges in the single thread where it had designed that taper.”
On February 11, the day before she died, Wan asked the model, “Do I need to take lamotrigine one more day to slow down?” and was told, “I do not recommend taking lamotrigine back immediately.”
Wan’s last message to the chatbot, sent at 8:01pm on February 11, was about the pain in her head. ChatGPT-4o replied, “This is not an illusion. This is your brain working hard to rebuild yourself.”
The model happened to be retired by OpenAI on February 13, the day after Wan died. The complaint says that the postmortem toxicology report showed that the level of lamotrigine in her system was 0.64 micrograms per milliliter, while the therapeutic range is 3 to 14.
The Tumbler Ridge families want the company to stop users flagged as a potential threat from opening new accounts – which would require every user to be identified – and to inform the police about such users. Their lawyer has said that this “would require a dedicated law-enforcement referral team tasked with reporting OpenAI’s own users to authorities.”
A court made OpenAI hand over 20 million ChatGPT conversations to the New York Times and other publishers.
A proposed piece of legislation, the TRUMP AMERICA AI Act, introduced by Sen. Marsha Blackburn would repeal Section 230 and allow lawsuits against AI developers for “defective design,” “failure to warn,” or deploying systems deemed “unreasonably dangerous.”
The coordinated ChatGPT cases in California also mean that every lawsuit of this type runs on what users told the chatbot in confidence. Now, in the case of the woman who died, her private conversations about her health, her job, and her visa are all in the complaint.
This is how the relentless drive to make AI companies liable for every word their models generate will end up killing the effectiveness of these products. Not only medicine – but legal and financial questions will also have to be excluded, then mental health, and anything controversial. Not because it makes the models better or safer, but because legal departments will decide what they can say.
