Sources checked 11 October 2026.
Take a pen and draw a bicycle. Not a good one. Two wheels are already on the page; add the frame, the pedals and the chain.
In 2006 Rebecca Lawson gave that sheet to 81 people at a university open day in England. Most of them rarely or never cycled, though across her studies 97% of the non-cyclists had learned to ride. Over 40% of the non-experts made at least one error, both in the drawing and in a multiple-choice version of the same question. [1]
The commonest mistake looped the chain around both wheels, which would stretch it every time the rider steered. Another joined the frame to the front wheel, so the bicycle could not turn at all.
Before the test, the non-experts put their knowledge of how bicycles work at 4.6 out of 7. Telling a second group that the task was about function did not help; they made more errors, not fewer. Only 9% of the cycling experts made an error in the drawing, so the knowledge is there to be had. It just does not arrive by riding. [1]
That gap between feeling you know and being able to show it has a name. It matters more now that a clear explanation of almost anything is one prompt away.
The feeling drops when you try
Leonid Rozenblit and Frank Keil called it the illusion of explanatory depth. In their studies at Yale, people rated how well they understood everyday devices, a zipper, a flush toilet, a cylinder lock, on a 7-point scale. Then they wrote a step-by-step explanation of how each one works, answered a question that needed the mechanism (how would you pick this lock?), and read an expert's explanation. They rated themselves again after each step. [2]
The ratings fell after the writing and again after the question. Twelve other students then scored the written explanations. Their scores were significantly lower than the writers' first two self-ratings and no different from the later ones. [2]
After trying, people placed themselves roughly where outside judges placed them. Nobody taught them anything between the first rating and the third. The effort to explain did the correcting.
Two details from the paper are worth keeping. The illusion was much smaller for facts such as capital cities and for film plots, and absent for procedures, where you find out at once whether you know them. And the best predictor of overconfidence for an item was the ratio of visible to hidden parts. [2] We see the outside of a zipper every day and take that for knowing the inside.
A fluent answer from a model is all visible parts.
What a clear answer does to the feeling
The worry predates chat assistants. In 2015 Matthew Fisher, Mariel Goddu and Keil reported nine experiments in which searching the internet raised people's estimates of their own knowledge. Their summary: people "mistake access to information for their own personal understanding". [3]
An assistant makes that access smoother, and recent studies point the same way.
Mark Steyvers and colleagues showed 301 people answers from GPT-3.5, PaLM2 and GPT-4o and asked how likely each one was to be correct. People were far worse at telling right answers from wrong ones than the models' own confidence scores were. Longer explanations raised their confidence without making them any better at it. [4]
Daniela Fernandes and colleagues asked 246 people to solve 20 logical reasoning problems from the LSAT with ChatGPT. They scored 12.98 on average and estimated 16.50. Those who knew more about AI were more confident and less accurate about their own performance. [5] (The figures are from the preprint; the study was later published in Computers in Human Behavior.)
A 2026 conference study asked 102 students to explain how things such as zippers work after reading the same short texts. One group got the texts as ChatGPT answers to questions they had put; another got them with no source. The ChatGPT group overestimated their ability to explain the most, and wrote shorter, weaker explanations. [6] It is a small study with pre-written answers, so I read it as a direction rather than a size.
And in a field experiment with nearly a thousand high-school maths students, standard GPT-4 access lifted practice scores by 48% and cut scores on a later exam, taken without AI, by 17%. The students did not perceive that they had learned less. [7] We looked at that study in an earlier article. [8]
None of this says AI makes people stupid. It says something narrower. The feeling of understanding is cheap to produce, and a clear answer produces it whether or not anything was learned.
Draw the bicycle before you close the tab
The same procedure that exposes the illusion is how understanding gets built.
In Henry Roediger and Jeffrey Karpicke's 2006 experiment, 180 students learned a prose passage by reading it four times, or by reading it once and then recalling it three times. Five minutes later the readers remembered more. A week later the order flipped: the recall group kept 61% of the ideas, the readers 40%. The readers were also the most confident they would remember it. [9]
Explaining works the same way. In Michelene Chi's 1994 study, eighth graders prompted to explain each line of a text on the circulatory system to themselves gained 32% from pre-test to post-test, against 22% for classmates who read it twice. [10] It was a small study, 24 students, but it is the habit the bicycle test asks for.
A line usually credited to Einstein says that if you cannot explain something simply, you do not understand it. I could not find a documented source for it. The documented version is about Richard Feynman. David Goodstein asked him why spin one-half particles obey Fermi-Dirac statistics. Feynman promised a freshman lecture on it, came back a few days later and said: "I couldn't reduce it to the freshman level. That means we don't really understand it." [11]
So this is the routine I would run before closing a chat that taught me something. It is Rozenblit and Keil's procedure, pointed at yourself.
If the second number is lower than the first, nothing has gone wrong. It is the first accurate reading you have had, and the gap is the part of the answer you still have to learn.
The next time an answer feels clear, close the tab and draw the bicycle.
Sources
[1] Lawson: The science of cycology. Memory and Cognition 34(8), 2006; author's copy, archived.
[2] Rozenblit and Keil: The misunderstood limits of folk science. Cognitive Science 26(5), 2002.
[3] Fisher, Goddu and Keil: Searching for explanations. Journal of Experimental Psychology: General 144(3), 2015.
[4] Steyvers et al.: What large language models know and what people think they know. Nature Machine Intelligence, January 2025.
[5] Fernandes et al.: Performance and metacognition disconnect when reasoning in human-AI interaction. Preprint v2; published as "AI makes you smarter but none the wiser" in Computers in Human Behavior 175, 2026.
[6] Aslanov, Felmer and Guerra: Overconfidence without understanding?. Proceedings of the Cognitive Science Society 48, 2026.
[7] Bastani et al.: Generative AI without guardrails can harm learning. PNAS 122(26), June 2025.
[8] Fkra: The chatbot that lifted practice scores 48% and cut exam scores 17%. Our earlier article on the same study.
[9] Roediger and Karpicke: Test-enhanced learning. Psychological Science 17(3), 2006.
[10] Chi, de Leeuw, Chiu and LaVancher: Eliciting self-explanations improves understanding. Cognitive Science 18(3), 1994.
[11] Goodstein and Goodstein: Feynman's lost lecture. Engineering and Science 59(3), Caltech, 1996.
One email when we publish. Research, product decisions, and what teams report back.
