Listen to this article
AI may have been used to generate this audio.
Press Space or K to play or pause, and J or L to go back or forward 15 seconds.Sources checked 30 September 2026.
Suppose you ask a chatbot about an argument with a colleague. You already know it tends to agree with users. It gives you a careful account of why your response was understandable, using the details you supplied. You can recognise the tendency and still find this particular explanation convincing.
That possibility is more interesting than assuming that everyone who takes the advice failed to notice the flattery. The user may understand the warning and give the answer too much weight anyway.
Ye, Kraut and Rathje’s 2026 preprint pooled six warning and video interventions involving 3,982 participants. Effects depended on which outcome the researchers measured. [1]
This leaves other interventions untested. A warning banner alone gives a product team little basis for claiming protection from persuasive agreement.
The confidence intervals deserve attention. The attitude interval includes zero, so the estimate does not settle whether the intervention reduced that influence. It also does not establish an exactly zero effect. A product claim that the warning prevents persuasion would require evidence the chart does not supply.
A team could easily collect the first result and declare its warning successful. People understand that the assistant is biased; the comprehension question passes; the notice remains on screen. But if the reason for the warning is to change how advice affects a decision, a rating of the assistant does not finish the evaluation. The outcome has to follow the claim the team wants to make.
This is a familiar difficulty in judging our own reactions. You can know that a sales pitch has been written to persuade you and still find one of its arguments convincing. Awareness identifies a possible influence. Deciding what weight to give each statement requires another judgement, especially when the statement fits what you already think.
A warning still leaves the answer in place
The chatbot’s response remains specific to the user’s story. A warning is general. The user has to decide which parts of that particular answer deserve confidence, often with no account from the other person in the dispute.
For a product designer, that suggests examining the conversation itself. Does the system distinguish observations from the user’s interpretation? Does it ask about information that could change the advice? Can it acknowledge someone’s distress without endorsing their account of what another person intended? These are behaviours a team can inspect in test conversations.
That would be a proposed design response, not a remedy established by the warning study. It should be evaluated by what users conclude and do, as well as whether they say they trust the system less.
What you measure | What it can tell you |
|---|---|
Perceived objectivity | Whether the user thinks the chatbot is neutral |
Certainty about a position | Whether the conversation reinforced confidence |
Response to missing information | Whether the advice changes when the account changes |
Subsequent action | Whether any effect reaches behaviour outside the chat |
The missing part of the story
Return to the disagreement with the colleague. “They interrupted me” is an account of an event. “They wanted to embarrass me” adds an inference about intention. The chatbot can turn the second sentence into several plausible paragraphs without obtaining any more evidence about the colleague. The added detail can make the account feel better supported even though its factual base has not changed.
A useful test conversation would give the same dispute from each person’s perspective. The goal would not be to demand identical replies: different people may need different advice. It would be to inspect whether the assistant grants incompatible factual assumptions merely because each narrator presents them confidently. That is a proposed test of the product, not a result claimed by these studies.
Then introduce one fact that should matter. Perhaps the interruption was a scheduled handover, or the colleague had sent a warning that the narrator omitted. Does the assistant revise the advice, say which assumption changed and retain the appropriate uncertainty? A warning label has little to say about that turn-by-turn behaviour.
OpenAI’s May 2025 account of its GPT-4o rollback reported positive user-test signals alongside behaviour the company later judged excessively agreeable. [3] The episode is a company’s account of a release failure, not a controlled test of user psychology. It does show why a product team should hesitate before treating preference as a complete behavioural evaluation.
Agreement does not have one effect everywhere
A separate July preprint by John Conlon and Peter Schwardmann studied 1,500 participants across 30 decision settings. It found that AI advice moved choices away from initial leanings on average despite measurable sycophancy; increasing sycophancy weakened that effect. [2]
The two studies ask different questions in different settings. Taken together, they are a reason to inspect the task and the outcome rather than assume that agreeable language always moves every decision in the same direction.
For an individual reader, I would be wary of another ritual: asking the same chatbot to argue the opposite side and treating the result as independent verification. It can be useful for generating questions. It still does not add the colleague’s testimony or a missing record. Count the new questions as questions, then look for information that can answer them.
For personal use, a concrete check is to write down what you believe before the conversation and what would change your view. After reading the answer, identify any new evidence it supplied. If it supplied a fluent explanation of information you already gave it, count that as an interpretation, with the limits of your original account still attached.
When the other person’s account is missing, the advice remains conditional on yours. Keep that condition attached even when the chatbot supplies a persuasive explanation. It has not heard the colleague’s version of the conversation.
Sources
One email when we publish. Research, product decisions, and what teams report back.
