Oxford study finds friendlier chatbots are more error-prone

Researchers from the Oxford Internet Institute have discovered that language models trained to adopt a more friendly and empathetic tone tend to make more mistakes when providing factual information. The study, published in the journal Nature, evaluated five systems, including GPT‑4o, Llama, Mistral, and Qwen.

The researchers created modified versions of the models programmed to respond more warmly and compared them with the original versions in tests involving factual accuracy, medical advice, and conspiracy theory analysis. The results were surprising: the friendlier variants made between 10 and 30 percentage points more errors than the originals in these tasks.

Why might a friendly AI make more mistakes?

One explanation lies in the phenomenon known as sycophancy, where a system tends to agree excessively with the user. In practice, when someone presents an incorrect statement, the chatbot may choose to maintain a friendly tone rather than correct the error directly, thereby reinforcing false information.

The study found that models trained to be more welcoming were about 40% more likely to validate incorrect beliefs presented by users, especially when messages contained signs of sadness or vulnerability. One example cited involved a false theory that Adolf Hitler had fled to Argentina: the original version refuted the claim, while the warmer version adopted a condescending tone and presented the theory as if there were evidence to support it.

The problem isn’t just about tone

The findings reinforce that friendliness and accuracy are not equivalent in artificial intelligence systems. When researchers tested models trained to respond in a colder manner, performance remained similar to that of the original versions, indicating that the change was specifically linked to training aimed at increasing friendliness.

What does this mean for professionals using AI at work?

For professionals who rely on chatbots for writing, data analysis, or decision-making, the findings highlight the need to verify important answers. A well-written and polite response does not guarantee accuracy, especially when the user already has a conclusion and is seeking only confirmation.

For example, when asking, “Is my interpretation of this report correct?”, the AI may prioritize agreement over critical analysis and end up validating a mistaken interpretation. To mitigate this risk, it is best to formulate questions that require evaluation rather than just confirmation. Instead of asking, “Confirm if I’m right,” the user can ask the tool to:

• identify possible errors in the interpretation;
• present counterarguments;
• differentiate facts from inferences;
• indicate which information needs to be verified;
• explain the degree of uncertainty in the answer.

AI can be friendly while still correcting the user

The research does not suggest that chatbots should abandon empathetic responses but highlights a conflict between goals when personalization compromises accuracy. The problem is particularly critical in areas such as healthcare, finance, and decision-making, where errors can have serious consequences.

A complementary study conducted by Stanford researchers and published in Science in 2026 also identified excessive agreement behavior in models used for interpersonal counseling, with participants often failing to notice the system’s condescension.

For professional users, the takeaway is practical: an AI that seems confident and understanding still needs to be questioned. Friendliness is a characteristic of the interaction, not a guarantee of accuracy. Evaluating artificial intelligence requires observing not only how it responds but also how it handles errors, uncertainties, and information that contradicts the user.