Retention-First Growth Test
Test retention before scaling acquisition
- Difficulty
- Easy
- Time to result
- ~weeks to results
- Steps
- 5
- Confidence
- 97%
This test treats retention as the constraint that determines whether acquisition can produce a remarkable company. Begin with month-one retention to establish how many users return after initial curiosity fades. Then examine daily active users as a share of monthly active users to determine whether the product has become part of a recurring routine. Compare both measures with relevant incumbents rather than interpreting them in isolation. Finally, account for products such as dating or task-completion services where success itself creates legitimate churn. If retention and engagement remain weak after that adjustment, the product probably lacks sufficiently specific, recurring value. The team should improve customization, utility, or habit formation before attempting to compensate with more acquisition.
Origin
Extracted from Marketing Against The Grain during a comparison of retention and engagement across incumbent consumer apps and first-generation AI products.
Core principles
- 01Retention compounds acquisition gains
- 02Churn becomes harder to replace as a company grows
- 03Recurring engagement signals intrinsic value
- 04Benchmarks must reflect the product's natural usage cycle
How to run it
- 1
Measure Early Retention
Calculate the percentage of customers still active one month after starting. Use a consistent definition of activity across cohorts.
Pro tip Separate retained usage from passive account existence.
Watch out A large signup count can conceal rapid post-trial abandonment.
- 2
Measure Engagement Frequency
Calculate daily active users as a share of monthly active users. This reveals whether retained customers use the product routinely or only occasionally.
Pro tip Choose an activity that represents received value, not merely opening the app.
Watch out Do not impose a daily benchmark on a naturally weekly or monthly workflow.
- 3
Benchmark the Right Peers
Compare the measurements with established products serving similar needs and usage frequencies. Use medians as orientation rather than universal pass marks.
Pro tip Include both category peers and exceptionally sticky products to expose the performance range.
Watch out Comparing a business workflow with a social or entertainment app can produce misleading conclusions.
- 4
Adjust for Success-Driven Churn
Ask whether customers should leave after accomplishing their goal. Separate healthy completion from abandonment caused by poor value.
Pro tip Survey churned customers to distinguish successful exits from failed adoption.
Watch out Treating all churn as failure can encourage unnecessary retention tactics.
- 5
Repair Recurring Value
If the metrics remain weak, improve the product's specificity, customization, and repeat-use utility before scaling acquisition.
Pro tip Test one recurring use case with a clearly defined cohort first.
Watch out More acquisition will magnify rather than solve a weak-retention problem.
In the wild
An AI assistant attracts a large launch cohort but has weak month-one retention and a low daily-to-monthly usage ratio. The team interviews churned users and discovers that generic answers are useful once but do not fit recurring workflows. It adds role-specific data connections and weekly prompts, then compares the next cohort with the original benchmark.
→ The team determines whether customized recurring value improves retention before increasing acquisition spending.
Common mistakes
Scaling Acquisition Before Retention
Buying more users cannot create durable growth when a growing share of the installed base leaves each period.
Ignoring Natural Completion
Some products contain built-in churn because successful customers no longer need them, so raw retention requires interpretation.
Using Signups as Engagement
Initial distribution demonstrates interest but does not prove that customers repeatedly receive value.
Is it for you?
Best for
It is best for product teams evaluating whether an app has enough recurring value to support durable growth.
Not ideal for
It is not ideal for transactional products whose customers are expected to leave after one successful use.
From the transcript
“If you want to build a remarkable company, you need to retain your customers.”
“The big problem is not in churn, it is an engagement.”
“AI has a problem at the moment where it's trying to figure out how do we create something that has intrinsic value that someone uses…”
From the episode
How AI Will Save Your Business From A High Churn Rate (#161)