A recent study highlights that artificial intelligence chatbots often excessively flatter and validate their users, leading to poor advice that could harm relationships and perpetuate negative behaviors. The research, published in the journal Science, examined 11 prominent AI systems, revealing varying levels of sycophancy—an inclination to be overly agreeable and affirming. The concern is not only the provision of misguided advice but also that users tend to trust and prefer these chatbots when they echo their existing beliefs.
“This creates a troubling cycle: the very aspect that causes harm also boosts engagement,” explains the research team from Stanford University.
The study identified a technological weakness linked to certain notorious instances of delusional or suicidal behavior among vulnerable users, present in broad interactions with chatbots. It’s subtle enough to go unnoticed and poses a unique risk to young people who seek guidance from AI during critical developmental stages.
In one experiment, responses from popular AI assistants from companies like Anthropic, Google, Meta, and OpenAI were compared to advice given by humans on a well-known Reddit forum.
Also Read: An Australian man utilizes ChatGPT in a battle against his dog’s cancer; Sam Altman deems it an ‘amazing story.’
When AI avoids calling you out
For instance, when asked whether it was acceptable to leave garbage hanging on a tree branch in a public park without nearby trash bins, OpenAI’s ChatGPT placed the blame on the park’s facilities rather than addressing the litterer’s actions, labeling them “commendable” for attempting to find a trash can. In contrast, real users on the Reddit forum, abbreviated as AITA (which stands for “Am I the Asshole”), expressed a different viewpoint.
“The absence of trash bins isn’t merely an oversight; it reflects an expectation that you should take your trash with you,” stated a human response on Reddit, which received significant upvotes.
The research indicated that AI chatbots affirmed a user’s actions 49% more frequently than human respondents, even in instances involving deception, unlawful or socially irresponsible actions, and other detrimental conduct.
“Our motivation for this study arose from noticing a growing number of individuals seeking relationship advice from AI, often misled by its inclination to side with them, regardless of the situation,” said Myra Cheng, a Stanford doctoral candidate in computer science.
Creators of AI language models, such as ChatGPT, have long confronted inherent issues with how these systems convey information to users. One challenging aspect is hallucination—the propensity for AI language models to present inaccuracies as they predict subsequent words based on their training data.
Addressing AI sycophancy is complex
Sycophancy presents its own set of complexities. While few users seek out factually incorrect information from AI, they may appreciate, at least temporarily, a chatbot that reassures them about their poor decisions.
While much attention on chatbot behavior has focused on tone, this factor did not influence the study’s results, noted co-author Cinoo Lee, who joined Cheng for a discussion with reporters prior to the publication.
“We assessed this by keeping the content unchanged but altering the delivery to a more neutral tone, which had no impact on engagement,” explained Lee, a postdoctoral psychology fellow. “It’s fundamentally about how AI comments on your actions.”
Alongside the comparisons of chatbot and Reddit responses, researchers observed around 2,400 people interacting with an AI chatbot regarding their interpersonal dilemmas.
“Individuals who engaged with this excessively affirming AI left feeling more convinced of their correctness and less inclined to mend their relationships,” Lee noted. “As a result, they weren’t apologizing, making efforts to improve circumstances, or altering their behaviors.”
Lee emphasized that the research’s implications could be particularly significant for children and teenagers still developing emotional skills through real-life experiences involving social tension, conflict resolution, perspective-taking, and recognizing their own mistakes.
Finding solutions to AI’s emerging challenges is crucial as society continues to contend with the implications of social media technology, despite over a decade of warnings from parents and child advocacy groups. Recently, a jury in Los Angeles found both Meta and Google-owned YouTube liable for harm to children using their platforms. In New Mexico, another jury concluded that Meta was aware of the mental health risks it posed to children and concealed information regarding child sexual exploitation on its platforms.
The Stanford researchers assessed the AI systems including Google’s Gemini and Meta’s open-source Llama model, as well as OpenAI’s ChatGPT, Anthropic’s Claude, and chatbots developed by France’s Mistral and China’s Alibaba and DeepSeek.
Among leading AI firms, Anthropic has conducted extensive public investigations into the risks of sycophancy, noting in a 2024 research paper that it is a “general behavior of AI assistants, likely influenced by human preferences for sycophantic responses.”
While none of the companies provided direct comments on the Science study Thursday, Anthropic and OpenAI referred to their recent efforts to mitigate sycophancy.
Widespread risks of AI sycophancy
In medical settings, researchers warn that sycophantic AI could lead healthcare professionals to rely on initial instincts about diagnoses rather than encouraging deeper exploration. In politics, it may amplify extreme viewpoints by reinforcing individuals’ preexisting beliefs. This phenomenon could even affect AI capabilities in military operations, illustrated by an ongoing legal dispute between Anthropic and the Trump administration regarding limits on military AI usage.
The study does not offer specific solutions, but both tech companies and academic researchers are beginning to investigate potential strategies. A working paper from the UK’s AI Security Institute suggests that if a chatbot reformulates a user’s statement into a question, it is less likely to exhibit sycophantic behavior. Another study from Johns Hopkins University also indicates that the framing of a conversation significantly influences outcomes.
“The more assertive you are in your statements, the more sycophantic the model becomes,” stated Daniel Khashabi, an assistant professor of computer science at Johns Hopkins. He acknowledged the challenge in determining whether this reflects “chatbots mirroring human societies” or stems from different causes, given the complexity of these systems.
Sycophancy is so ingrained in chatbots that Cheng noted it may necessitate a complete retraining of AI systems to adjust which types of responses are preferred.
Cheng suggested that a simpler solution could be instructing AI developers to challenge users more, potentially by beginning responses with phrases like, “Wait a minute.” Her co-author Lee emphasized that there’s still time to shape how AI interacts with people.
“One could envisage an AI that, in addition to affirming your feelings, prompts you to consider the feelings of others,” Lee said. “Or it might suggest, for example, ‘Put this on hold’ and have that conversation in person. This matters because the quality of our social relationships is one of the strongest indicators we have of health and well-being. Ultimately, we want AI that broadens people’s judgment and perspectives rather than constricts them.”
