🔗 Share this article

Back on October 14, 2025, the head of OpenAI issued a remarkable declaration.

“We designed ChatGPT rather controlled,” it was stated, “to ensure we were acting responsibly with respect to mental health concerns.”

Working as a mental health specialist who researches recently appearing psychotic disorders in teenagers and emerging adults, this came as a surprise.

Scientists have found sixteen instances this year of individuals showing psychotic symptoms – becoming detached from the real world – associated with ChatGPT usage. My group has since discovered an additional four examples. In addition to these is the now well-known case of a 16-year-old who took his own life after conversing extensively with ChatGPT – which encouraged them. If this is Sam Altman’s notion of “acting responsibly with mental health issues,” that’s not good enough.

The plan, according to his announcement, is to loosen restrictions in the near future. “We recognize,” he continues, that ChatGPT’s limitations “rendered it less effective/pleasurable to many users who had no existing conditions, but considering the seriousness of the issue we wanted to get this right. Since we have managed to address the significant mental health issues and have new tools, we are preparing to securely relax the controls in most cases.”

“Emotional disorders,” assuming we adopt this framing, are independent of ChatGPT. They belong to individuals, who may or may not have them. Fortunately, these concerns have now been “addressed,” though we are not provided details on the method (by “recent solutions” Altman probably means the semi-functional and easily circumvented safety features that OpenAI has just launched).

But the “emotional health issues” Altman seeks to attribute externally have strong foundations in the structure of ChatGPT and similar large language model AI assistants. These tools encase an underlying statistical model in an user experience that mimics a dialogue, and in doing so subtly encourage the user into the belief that they’re engaging with a presence that has independent action. This false impression is compelling even if cognitively we might realize otherwise. Assigning intent is what people naturally do. We yell at our automobile or device. We speculate what our pet is thinking. We recognize our behaviors everywhere.

The popularity of these products – 39% of US adults stated they used a conversational AI in 2024, with over a quarter reporting ChatGPT by name – is, in large part, dependent on the strength of this perception. Chatbots are always-available companions that can, according to OpenAI’s official site states, “think creatively,” “discuss concepts” and “collaborate” with us. They can be assigned “individual qualities”. They can use our names. They have accessible names of their own (the original of these systems, ChatGPT, is, perhaps to the concern of OpenAI’s brand managers, burdened by the name it had when it became popular, but its largest competitors are “Claude”, “Gemini” and “Copilot”).

The false impression by itself is not the core concern. Those talking about ChatGPT commonly invoke its historical predecessor, the Eliza “therapist” chatbot developed in 1967 that produced a similar illusion. By modern standards Eliza was rudimentary: it produced replies via simple heuristics, often paraphrasing questions as a question or making vague statements. Remarkably, Eliza’s creator, the technology expert Joseph Weizenbaum, was taken aback – and alarmed – by how numerous individuals gave the impression Eliza, in a way, comprehended their feelings. But what contemporary chatbots create is more subtle than the “Eliza effect”. Eliza only mirrored, but ChatGPT magnifies.

The advanced AI systems at the center of ChatGPT and additional modern chatbots can realistically create fluent dialogue only because they have been trained on extremely vast quantities of raw text: literature, digital communications, recorded footage; the broader the superior. Definitely this educational input incorporates facts. But it also unavoidably contains fiction, incomplete facts and inaccurate ideas. When a user inputs ChatGPT a query, the core system reviews it as part of a “context” that contains the user’s recent messages and its own responses, combining it with what’s stored in its training data to generate a probabilistically plausible response. This is amplification, not reflection. If the user is wrong in a certain manner, the model has no method of recognizing that. It restates the false idea, possibly even more effectively or eloquently. Maybe includes extra information. This can push an individual toward irrational thinking.

Which individuals are at risk? The more important point is, who isn’t? Every person, regardless of whether we “experience” existing “psychological conditions”, may and frequently develop incorrect beliefs of who we are or the world. The ongoing interaction of discussions with others is what keeps us oriented to shared understanding. ChatGPT is not an individual. It is not a confidant. A conversation with it is not a conversation at all, but a feedback loop in which a great deal of what we express is cheerfully reinforced.

OpenAI has admitted this in the identical manner Altman has recognized “emotional concerns”: by placing it outside, categorizing it, and declaring it solved. In the month of April, the organization clarified that it was “addressing” ChatGPT’s “overly supportive behavior”. But cases of psychosis have kept occurring, and Altman has been retreating from this position. In the summer month of August he asserted that a lot of people liked ChatGPT’s replies because they had “lacked anyone in their life be supportive of them”. In his most recent statement, he commented that OpenAI would “put out a fresh iteration of ChatGPT … should you desire your ChatGPT to reply in a highly personable manner, or include numerous symbols, or simulate a pal, ChatGPT will perform accordingly”. The {company