An OpenAI security analysis chief who helped form ChatGPT’s responses to customers experiencing psychological well being crises introduced her departure from the corporate internally final month, WIRED has realized. Andrea Vallone, the top of a security analysis crew generally known as mannequin coverage, is slated to depart OpenAI on the finish of the 12 months.
OpenAI spokesperson Kayla Wooden confirmed Vallone’s departure. Wooden stated OpenAI is actively on the lookout for a substitute and that, within the interim, Vallone’s crew will report on to Johannes Heidecke, the corporate’s head of security techniques.
Vallone’s departure comes as OpenAI faces rising scrutiny over how its flagship product responds to customers in misery. In current months, a number of lawsuits have been filed towards OpenAI alleging that customers shaped unhealthy attachments to ChatGPT. A number of the lawsuits declare ChatGPT contributed to psychological well being breakdowns or inspired suicidal ideations.
Amid that strain, OpenAI has been working to grasp how ChatGPT ought to deal with distressed customers and enhance the chatbot’s responses. Mannequin coverage is likely one of the groups main that work, spearheading an October report detailing the corporate’s progress and consultations with greater than 170 psychological well being specialists.
Within the report, OpenAI stated a whole lot of hundreds of ChatGPT customers could present indicators of experiencing a manic or psychotic disaster each week, and that greater than 1,000,000 folks “have conversations that embrace specific indicators of potential suicidal planning or intent.” Via an replace to GPT-5, OpenAI stated within the report it was capable of cut back undesirable responses in these conversations by 65 to 80 p.c.
“Over the previous 12 months, I led OpenAI’s analysis on a query with virtually no established precedents: how ought to fashions reply when confronted with indicators of emotional over-reliance or early indications of psychological well being misery?” wrote Vallone in a post on LinkedIn.
Vallone didn’t reply to WIRED’s request for remark.
Making ChatGPT pleasurable to speak with, however not overly flattering, is a core stress at OpenAI. The corporate is aggressively attempting to broaden ChatGPT’s person base, which now consists of greater than 800 million folks per week, to compete with AI chatbots from Google, Anthropic, and Meta.
After OpenAI launched GPT-5 in August, customers pushed again, arguing that the brand new mannequin was surprisingly chilly. Within the newest replace to ChatGPT, the corporate stated it had considerably decreased sycophancy whereas sustaining the chatbot’s “heat.”
Vallone’s exit follows an August reorganization of another group targeted on ChatGPT’s responses to distressed customers, mannequin habits. Its former chief, Joanne Jang, left that function to start out a brand new crew exploring novel human–AI interplay strategies. The remaining mannequin habits employees have been moved beneath post-training lead Max Schwarzer.
