Listen to this article
AI may have been used to generate this audio.
Press Space or K to play or pause, and J or L to go back or forward 15 seconds.Sources checked 5 October 2026.
A person can pay for an app because it solved one urgent problem. That transaction is useful evidence. It does not tell you whether the problem will return next week, whether they will remember the app, or whether another tool already included in their work can handle it.
Subscription products have to live through all of those questions. A convincing demonstration only needs a good first session.
RevenueCat’s 2026 report found weaker subscription retention in its AI app group. The platform sample mainly covers 2025. These are subscription counts, and the observational comparison cannot establish that AI caused cancellations. [1]
Build around the next occasion
Consider a hypothetical tool that turns meeting notes into a project plan. The first demonstration can look excellent. A messy page becomes a neat set of tasks, with owners and dates. A team could spend its next month improving that transformation and still learn very little about why a customer would renew.
The next meeting creates different work. Someone changed a date in the task tracker. Two people disagree about the owner. A task is finished, but the source notes still say it is open. The user needs the app to reconcile that situation without losing the decisions already made.
Those details are awkward to demonstrate in a launch video because they require a before and an after. They are also where the app can become part of a working routine.
Product discovery should include watching the second occasion on which someone needs the result.
For this imaginary product, I would ask a small group to use it across consecutive meetings. I would watch what they export, where they edit the output and which corrections they repeat. A repeated correction is a more specific development task than a request to make the assistant smarter.
Observation | Possible explanation | Next product check |
|---|---|---|
Pays once and leaves | The need was a one-off | Ask when the problem last occurred |
Returns but copies everything elsewhere | The useful work happens in another tool | Watch the hand-off into that tool |
Repeats the same correction | A recurring requirement is missing | Test that correction across several sessions |
Cancels after completing the task | The product did its job | Consider pricing that fits occasional use |
Follow the meeting-plan example one step further. The customer has corrected an owner in the task tracker. At the next meeting, the assistant sees the old owner in the notes and confidently changes it back. A product that looked useful in an isolated session has now created cleanup work. Better prose generation would not repair that behaviour. The product needs a rule for which system owns the decision.
That is a product-development question with a fairly concrete answer to investigate. Show the customer the conflicting records. Ask which one they treat as authoritative and when they expect the application to ask permission. Then test the next meeting, including a deliberate conflict. Success would mean the plan remains usable as other people change it.
There is a reason to resist turning this immediately into a broad “memory” feature. The user may need one remembered decision with a source and a way to correct it. A general promise that the assistant remembers everything would add behaviour the team has not yet learned how to govern.
What to measure before adding another feature
A renewal metric needs a defined denominator. People who installed the app, started a trial and became paying subscribers are different groups. Monthly and annual plans also expose customers to different payment decisions. Keep those cohorts separate when you compare them.
Then attach a product event to the payment data. Did the customer complete the recurring task? Did they use the result in their work? If an app generates a plan that nobody follows, counting generated plans will make the product look healthier than the customer’s behaviour suggests.
Google’s HEART work separates experience measures such as retention and task success and asks teams to connect goals to observable signals before choosing metrics. [2] For our meeting assistant, the goal might be keeping a usable plan across meetings. The signal would be a customer accepting the reconciled changes. The metric could count eligible repeat meetings in which that happened, with correction work recorded separately.
Choosing that denominator takes work. A customer who had no second meeting has not failed a repeat-meeting test. A person who returned only to copy a missing date has not necessarily completed the recurring job. Instrumentation should preserve those differences rather than give every visit the same meaning.
A small experiment can then answer a narrow question. Compare the existing reconciliation behaviour with a version that shows conflicting changes before applying them. Keep the generation model fixed if that is not what you are testing. Look at accepted plans, corrections and use at the next eligible meeting. The experiment might show that confirmation adds friction without helping. That would be useful to know before building a larger system around it.
Cancellation can also be a sensible outcome. Someone buying a tool for a single application, a house move or a short project may have no reason to stay. For that customer, a subscription might be the questionable part of the product. A lower renewal rate alone cannot tell the team which explanation applies.
A cancelling customer can show the team where the product stopped being useful. Ask them to walk through their last successful session and the first time they chose something else. The next development task may be in the hand-off between the app and the rest of their work.
Sources
One email when we publish. Research, product decisions, and what teams report back.
