Listen to this article
AI may have been used to generate this audio.
Press Space or K to play or pause, and J or L to go back or forward 15 seconds.Sources checked 11 October 2026.
On one task in a 2023 experiment with 758 Boston Consulting Group consultants, the people working without AI reached the right answer about 84.5% of the time. Those given GPT-4 got there about 70% of the time, and those given GPT-4 plus a short guide to prompting, about 60%. Their written recommendations were rated higher anyway, whether the answer was right or wrong. [1]
On 18 other tasks, ones the model handled well, the same consultants finished 12.2% more of them, 25.1% faster, at more than 40% higher quality. [1] On the hard task the model did not refuse or hedge. It answered in the same confident prose, and nothing in that prose told the reader which kind of task they were on.
That is where every team using these tools now sits. Producing an answer costs almost nothing. Deciding whether to trust it, what it left out and what happens after you act on it is still expensive, and that part lands on you. A survey of 319 knowledge workers by Microsoft and Carnegie Mellon researchers put it plainly: generative AI "shifts the nature of critical thinking toward information verification, response integration, and task stewardship". [2]
The tools for that work are old. In 1943 the psychologist Kenneth Craik argued that a mind carrying a "small-scale model" of external reality can "try out various alternatives" before it acts. [3] Fifty years later Charlie Munger told a business school audience, "You've got to have models in your head", and more than one or two, because with too few "you'll torture reality so that it fits your models". [4]
Here are five worth keeping close when the machine has an answer ready. None of them is new. Each one is worth more now than when answers were scarce.
1. The map is not the territory
Alfred Korzybski's phrase is a reminder that every description leaves something out. A benchmark score is a map of a model. So is a leaderboard rank, a demo, and the model's own explanation of its answer.
The sharpest version came from the anthropologist Marilyn Strathern in 1997: "When a measure becomes a target, it ceases to be a good measure." She noted that the idea already had a name, Goodhart's law, after the economist Charles Goodhart. [5]
AI benchmarks are measures under heavy pressure. A 2025 analysis of about two million battles on Chatbot Arena, a widely watched public leaderboard, found that Meta tested 27 private variants there before releasing Llama 4. Google and OpenAI had each received an estimated fifth of all the arena's data, while 83 open-weight models together received under a third. [6] The authors showed that training on arena data lifts scores on arena-style questions. Whether it helps on your questions is a separate matter.
Contamination is the quieter version. When researchers at Scale AI wrote 1,205 fresh grade-school maths problems matched to a popular test set, some model families scored up to 8 points lower on the new ones, while the frontier models scored about the same on both. [7] Part of the old score was a map of what those models had already seen.
Before you trust a number about a model, ask what it was measured on and whether anyone had a reason to aim at it. Then try the model on a sample of your own work. Nobody has optimised for that.
2. Know where your circle of competence ends
Warren Buffett, in his 1996 letter to shareholders: "The size of that circle is not very important; knowing its boundaries, however, is vital." [8]
The BCG experiment is a picture of two circles. The model's is jagged: strong on one task, wrong on a neighbouring one that looks just as hard. The researchers called it the jagged technological frontier. [1] Stanford's 2026 AI Index gives a blunter example: a model earned a gold medal at the International Mathematical Olympiad, yet the best model reads analogue clocks correctly 50.1% of the time. [9]
So there are two boundaries to know now. Your own tells you whether you could catch a wrong answer. The model's you can only learn by checking its work on tasks where you already know the answer.
Where the two circles overlap, the tool makes you faster and you can check it. Where you are outside your own circle and believe you are inside the model's, you cannot tell a good answer from a fluent one. The consultants' polished wrong recommendations live there.
3. Invert
Munger again, in a 2007 commencement speech: problems "are easier to solve if you turn around in reverse". If you want to help India, he said, do not ask how to help India; ask "What's doing the worst damage in India?" and how to avoid it. [10]
Inversion is the cheapest use of an assistant I know. Ask it how your plan fails, not whether the plan is good. Assistants lean towards agreeing with you, as two earlier pieces here covered [11], and asking for the failure turns that tendency around.
The premortem is the formal version: before a project starts, the team imagines it has already failed and writes down why. It usually travels with a statistic, that imagining the failure has already happened improves people's ability to find its causes by 30%, credited to a 1989 study by Deborah Mitchell, Jay Russo and Nancy Pennington. I could not find that number in the paper's abstract.
What the abstract does report is more useful. Setting the event in the past rather than the future "showed little influence". Treating the outcome as certain did change the explanations: they were longer and contained more episodic reasons, specific chains of events. [12] The force is in saying "it failed", not "it might fail".
For an AI project, someone has already collected the failure list. RAND interviewed 65 experienced data scientists and engineers in 2023 and drew out five root causes; the first, a misunderstood problem, was the most common. [13] Start your premortem from theirs.
RAND's root cause | The premortem question |
|---|---|
The problem was misunderstood, or the model optimised for the wrong metric | Who agreed on what success means, and is it the number we are measuring? |
The data was not there | Do we have the data today, labelled, for this exact task? |
The technology came before the problem | Would we still run this project without the AI? |
No infrastructure to run it | Who maintains it in month seven? |
The problem was too hard for AI | Has anything solved a smaller version of it? |
4. Ask what happens next
Howard Marks calls this second-level thinking. "First-level thinking is simplistic and superficial, and just about everyone can do it." [14] The second level asks what follows, and what follows that.
In 1865 the economist William Stanley Jevons saw that more efficient steam engines did not save coal: "It is wholly a confusion of ideas to suppose that the economical use of fuel is equivalent to a diminished consumption. The very contrary is the truth." [15] On 27 January 2025, the day a cheaper DeepSeek model knocked AI stocks, Microsoft's chief executive Satya Nadella posted: "Jevons paradox strikes again!" [16]
The numbers bear him out. Epoch AI found that the price of reaching a fixed level of performance fell by between 9 and 900 times a year, depending on the benchmark, with a median of 50. [17]
Use went the other way. Google says it processed 9.7 trillion tokens a month across its products in May 2024, and over 3.2 quadrillion in May 2026. [18]
The first-level conclusion is that AI is getting cheap. The second-level one is that your bill depends on how much you use, and cheap tokens invite more use: agents that run for an hour, reasoning models that write thousands of tokens before they answer. Andreessen Horowitz, which tracked the price fall, noted that OpenAI's o1 cost the same per output token as GPT-3 had at launch, $60 per million. [19] A given capability gets cheaper; the frontier does not. Budget per finished task, not per token.
The same question applies to people. If the work that used to train juniors is handed to a model, the saving shows up this year and the cost shows up when you need seniors. [20]
5. Start from the base rate
Daniel Kahneman and Dan Lovallo wrote in 1993 that "decision makers have a strong tendency to consider problems as unique", and that optimistic forecasts come from an inside view "which anchors predictions on plans and scenarios". [21] The remedy is the outside view: before you look at your own plan, look at what usually happens to plans like it.
Bent Flyvbjerg has built the largest collection of that evidence, more than 16,000 projects. [22] His own summary of it is below.
Reference classes for AI projects are younger and thinner, and the popular ones need checking before you borrow them. The widely repeated claim that 95% of organisations get zero return from generative AI comes from a preliminary MIT-affiliated report built on 52 interviews, 153 survey responses and a review of public cases, with a narrow definition of success. [24] RAND's "more than 80 percent of AI projects fail" is a figure the report quotes from earlier coverage, not one it measured. [13] Gartner's warnings, that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025 and over 40% of agentic AI projects cancelled by the end of 2027, are forecasts. [25]
None of those is a precise base rate. They all point the same way, and that is enough to change the default question from "will this work?" to "why would ours be the exception?"
What I think
A model can run any of these five if you ask it. It will write you a premortem, list base rates, trace second-order effects. What it will not do is notice that this is the moment to ask. That noticing, the small alarm when an answer arrives too easily, is the part that stays with you. It is what mental models were for all along.
They are not a checklist for every prompt. They are questions that should fire when the stakes are real:
What was this measured on, and who was aiming at it?
Am I inside my own circle here, and have I checked the model on a task where I know the answer?
If this has failed in six months, what does the post-mortem say?
It worked. What happens next, and after that?
What usually happens to projects like this one?
Pick one and run it on the next answer you were about to accept.
Sources
[1] Dell'Acqua et al.: Navigating the jagged technological frontier. Harvard Business School working paper 24-013, 2023; since published in Organization Science, March 2026.
[2] Lee et al.: The impact of generative AI on critical thinking. CHI 2025; a self-reported survey, so it shows association, not cause.
[3] Craik: The Nature of Explanation, 1943. Passage reproduced by Farnam Street.
[4] Munger: A lesson on elementary, worldly wisdom. USC Business School, 1994; transcript.
[5] Strathern: "Improving ratings": audit in the British University system. European Review 5(3), 1997.
[6] Singh et al.: The Leaderboard Illusion. Preprint, 2025.
[7] Zhang et al.: A careful examination of large language model performance on grade school arithmetic. Preprint, version 4; the first version reported drops of up to 13%.
[9] Stanford HAI: The 2026 AI Index Report. Top takeaways.
[10] Munger: 2007 USC Law School commencement address. Transcript.
[11] Fkra: A friendlier chatbot can give worse answers. See also our piece on why a warning about AI flattery fell short.
[12] Mitchell, Russo and Pennington: Back to the future. Journal of Behavioral Decision Making 2(1), 1989; abstract.
[13] Ryseff, De Bruhl and Newberry: The root causes of failure for artificial intelligence projects. RAND, 2024.
[14] Marks: It's not easy. Oaktree memo, September 2015.
[15] Jevons: The Coal Question, 1865. Chapter VII, excerpt hosted by Yale's Energy History project.
[17] Epoch AI: LLM inference prices have fallen rapidly but unequally across tasks. March 2025.
[18] Google: Sundar Pichai at I/O 2026. The 2025 points are from the I/O 2025 keynote and the Q2 and Q3 2025 earnings posts.
[19] Appenzeller: Welcome to LLMflation. Andreessen Horowitz, November 2024.
[20] Fkra: If juniors stop getting hired, where do seniors come from?
[21] Kahneman and Lovallo: Timid choices and bold forecasts. Management Science 39(1), 1993; abstract.
[22] Flyvbjerg: Towards a theory of modular natives. Preprint, 2026; describes the database.
[23] Flyvbjerg: How Big Things Get Done, in his own summary. Next Big Idea Club, 2023.
[24] MIT NANDA: The GenAI Divide, State of AI in Business 2025. Preliminary report, July 2025.
[25] Gartner: 30% of generative AI projects abandoned after proof of concept (July 2024) and over 40% of agentic AI projects cancelled (June 2025). Press releases; forecasts.
One email when we publish. Research, product decisions, and what teams report back.
