Unvalidated Tactic Due-Diligence Rule
Limit investment in promising tactics until evidence validates the underlying theory
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 5
- Confidence
- 92%
The rule begins by separating facts from community interpretations: what has the platform explicitly confirmed, and what is merely the market’s best guess? The team then measures downside by estimating implementation cost, reversibility, and the consequences of a false assumption. Rather than launching a sweeping project, it runs a bounded test with predetermined evidence for success or failure. Resource commitment increases only when platform guidance or observed results validate the mechanism. Kieran Flanagan grounds the rule in an early-career SEO mistake: he persuaded a global company to undertake months of PageRank sculpting work based on an industry theory that was later withdrawn. Applied consistently, the method turns uncertainty into staged learning and protects teams from expensive confidence unsupported by evidence.
Origin
Kieran Flanagan derived the lesson from advocating a three-to-six-month PageRank sculpting project before learning that the underlying SEO theory was wrong.
Core principles
- 01Distinguish industry consensus from verified guidance
- 02Scale investment in proportion to evidence
- 03Prefer reversible experiments under uncertainty
- 04Wait for real-world data before committing major resources
- 05Record the assumptions that would invalidate a tactic
How to run it
- 1
Separate facts from assumptions
List what the platform has officially confirmed and distinguish it from tactics inferred by vendors or practitioners.
Pro tip Attach a source and date to each claimed fact.
Watch out Repeated industry commentary does not turn an unverified theory into platform guidance.
- 2
Estimate exposure
Calculate the time, cost, disruption, and reputational impact if the tactic proves ineffective.
Pro tip Pay particular attention to work that touches an entire site, customer database, or communications system.
Watch out Technical elegance can make a speculative tactic feel safer than it is.
- 3
Design a bounded test
Apply the tactic to a limited, reversible scope that can generate useful evidence without committing the full organization.
Pro tip Preserve a comparison group or baseline whenever possible.
Watch out A test that changes everything at once cannot isolate the tactic’s actual effect.
- 4
Define validation criteria
Specify the observable result, measurement period, and external confirmation required before expanding the tactic.
Pro tip Write down what evidence would cause the team to stop, not only what would encourage expansion.
Watch out Do not reinterpret ambiguous results merely to justify work already performed.
- 5
Scale only after evidence
Increase investment when the test, subsequent data, or authoritative guidance supports the proposed mechanism.
Pro tip Reassess after major platform updates because previously valid tactics can become obsolete.
Watch out Temporary success does not guarantee that a platform will tolerate the tactic permanently.
In the wild
A marketing team hears that separate subdomains may contain the impact of Gmail spam enforcement. Instead of rebuilding all email infrastructure, it moves one noncritical newsletter stream, authenticates it, monitors complaints, and documents whether providers treat its reputation independently. The team postpones a company-wide migration until more provider guidance and field data emerge.
→ The company gains operational evidence without betting its entire communications system on an unvalidated interpretation.
Early in his marketing career, Kieran advocated restructuring a global company’s site around PageRank sculpting after reading an industry theory about directing authority with nofollow links. The company undertook months of work, after which the source published that the theory was wrong.
→ The experience became a warning to limit major commitments until uncertain technical claims have been validated.
Common mistakes
Confusing a best guess with a fact
A widely discussed response to a platform change may still lack endorsement or reliable field evidence.
Launching at full organizational scale
Large implementation projects amplify the cost of being wrong before the team has learned whether the mechanism works.
Ignoring reversibility
Teams often evaluate potential upside without asking how quickly and safely they can undo the change.
Is it for you?
Best for
It is best for teams responding to new algorithms, platform rules, technical theories, or emerging industry playbooks.
Not ideal for
It is not ideal when immediate legal, security, or platform compliance requires decisive action regardless of incomplete evidence.
From the transcript
“you should definitely like be very careful about the amount of work you take gone with things that are still yet to be validated”
“these are these are massive changes”
“it's not a validated solution it's not one that Google has said you should do”
From the episode
Did Google Just Kill Email Marketing? (#184)