Listen to this article
AI may have been used to generate this audio.
Press Space or K to play or pause, and J or L to go back or forward 15 seconds.Sources checked 1 October 2026.
There is a small category of work where opening the assistant can take as much attention as doing the task. A calculation, a short message, a spelling check. You decide what to ask, inspect the answer and then put it back where the work belongs.
Sometimes that is still worthwhile. You might want a second opinion or help getting started. It becomes a productivity claim when you say it saved time, and that claim can be checked.
In a May preprint covering three preregistered studies (2,691 participants), Sunny Yu and colleagues found that people overestimated AI’s efficiency gains on simple tasks. Predicted use was 33% of tasks and observed use 47%, measured in different samples. [1]
The developers who felt faster
In METR’s randomised trial, 16 experienced developers completed 246 tasks with early-2025 tools. Their expectations and subsequent impressions diverged from measured time. [2] A team deciding whether to buy assistance would reach different conclusions depending on which of those answers it collected.
The setting matters. One possible explanation is that a developer familiar with a large repository already knows where to make a small edit, while explaining that context to an assistant takes work. That explanation would need testing. A person learning a new library or building a prototype starts with different information, so the same assistance could have a different value.
The follow-up makes the story less convenient. In February 2026, METR said its newer experiment had become difficult to interpret because developers selected which tasks to submit, sometimes avoided the no-AI condition, and struggled to report time during parallel work. METR called the estimates an unreliable signal of current productivity. [3] There is no clean before-and-after trend to draw from those two studies.
That matters for anyone using the first finding as a slogan. A careful measurement can establish a result in one setting and later become harder to repeat as behaviour changes. The right response is to improve the experiment and keep its scope attached. Keeping an old percentage while silently updating the model names would manufacture evidence.
Count the whole task
For a practical check, start the clock when the task begins and stop it when the result is usable. Include gathering context, prompting, reviewing and moving the answer into the document or system where it will be used. Measuring only the interval while the model generates text leaves out work the person still performs.
Quality belongs in the comparison. A quicker answer that needs correction tomorrow has moved work into a different session. A slower answer that prevents a costly mistake can be a good result. Record the acceptance condition before timing the task so that speed does not silently replace the original goal.
This is a proposed personal audit, not a replication of the study. For recurring work, alternate between methods on comparable tasks. Avoid doing the exact same task twice and calling the second attempt faster because of the tool. You already know more the second time.
Where the benefit was observed
The published Generative AI at Work study followed a staggered rollout to 5,172 customer-support agents. It estimated a 15% average increase in issues resolved per hour, with larger gains among less experienced and lower-skilled workers. [4] That outcome concerns support work and a particular deployment. It should not be plotted beside METR’s task-time result as though both measure one general AI effect.
The comparison is still instructive. Useful assistance may arrive where a worker needs information they do not yet have at hand. Unhelpful assistance may interrupt a process the worker already knows well. These are possible explanations to test in a particular organisation, not a rule that novices always benefit and experts never do.
A productivity claim therefore needs a unit of work. Resolved cases per hour, accepted code changes and a person’s impression of an easier day answer different questions. Even elapsed time and active human time can separate: an agent might take longer to finish while freeing the person to do something else. Record both when that distinction affects the decision.
There is also a selection problem in the personal audit. If you use the assistant only for jobs you expect it to handle well, a comparison with all your unaided work will include differences in the jobs. If you use it only when stuck, it may look worse for the same reason. A useful record includes why you chose assistance, not only how the assisted task ended.
An easier task can still be worth paying for
Time is not the only useful outcome. Someone might use assistance because writing in a second language is tiring, because the first blank page is difficult, or because they need an accessible way to complete the work. Those reasons deserve their own assessment. Forcing every benefit into “hours saved” makes the account less accurate.
The paper leaves longer-term habits unresolved. [1] A finding from simple tasks should stay attached to those tasks when a manager uses it to make a decision.
A manager asking staff for estimated hours saved will receive estimates. Those can help identify work worth examining, but they should not automatically become a financial return in a presentation. A small observation of the complete task can be more useful than an exact-looking total compiled from guesses.
For a task you repeat this week, write down your expected time before you start. Keep the finished result and the elapsed time beside that prediction. After several attempts, decide whether the assistant helped you finish sooner, produced a better result or made the task easier to begin. Use the benefit you observed when you describe it.
Sources
One email when we publish. Research, product decisions, and what teams report back.
