Exact-or-Close Answer Rule
Match deterministic systems or AI to the precision the decision requires
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 6
- Confidence
- 98%
The rule begins by asking whether a use case requires an exact answer or can benefit from a close, correctable answer. Exact recurring decisions—such as deterministic routing—belong with data science, business rules, and workflow automation because errors must not accumulate. LLMs fit interactions where an approximate first response creates value and the user or system can provide feedback, clarification, or escalation. Cost changes the decision: exact personalization may be too expensive for every low-value interaction, while an LLM can deliver a useful approximation at scale. Teams therefore evaluate required precision, cost, consequence, and recoverability before choosing the technology. The result is not “AI everywhere,” but deliberate allocation of deterministic and probabilistic methods to the problems each can solve responsibly.
Origin
Kipp Bodnar introduced the distinction while explaining the AI principles adopted by HubSpot’s marketing organization, with Kieran Flanagan adding cost as a necessary axis. Extracted from Marketing Against The Grain.
Core principles
- 01Required precision determines the appropriate technical method.
- 02Exact recurring decisions need deterministic data and workflow systems.
- 03LLMs suit interactions where approximation can be corrected through feedback.
- 04Close answers are valuable when they make personalized service economically scalable.
- 05Cost and consequence must be considered alongside answer quality.
How to run it
- 1
Specify the answer
Define exactly what the system must decide, recommend, generate, or communicate.
Pro tip Write the expected output and its consumer in one sentence.
Watch out A vague use case makes precision requirements impossible to assess.
- 2
Assess error consequences
Determine what happens if the answer is incomplete, approximate, inconsistent, or wrong.
Pro tip Separate inconvenient errors from irreversible or systemic ones.
Watch out Do not use average accuracy to dismiss rare but severe failures.
- 3
Test recoverability
Ask whether a user or system can correct the answer through another interaction or escalation.
Pro tip Explicit feedback loops make close-answer use cases safer.
Watch out An answer is not recoverable merely because someone could discover the error later.
- 4
Compare service cost
Estimate the cost and scalability of producing exact versus close answers for the relevant audience.
Pro tip Include the value of expanding personalized service to customers previously given generic treatment.
Watch out Cheap approximation is not valuable when the error cost exceeds the savings.
- 5
Choose the system
Use deterministic data science or workflow automation for mandatory exactness and LLMs for useful, recoverable approximation.
Pro tip Combine methods when structured rules can bound an LLM’s role.
Watch out Do not force every use case into a single technology stack.
- 6
Monitor and escalate
Measure error patterns and route uncertain or consequential cases to an exact process or human reviewer.
Pro tip Set explicit confidence and escalation thresholds.
Watch out A feedback loop is ineffective if nobody reviews its failures.
In the wild
A company keeps lead routing in deterministic workflow automation because every lead must reach the correct sales representative consistently. It uses an LLM for initial support responses because customers can clarify their need, reject an insufficient response, or escalate to a person.
→ Each use case receives the level of precision and feedback its consequences require.
Common mistakes
Using an LLM for mandatory exactness
Probabilistic output is unsuitable when every recurring decision must be correct and consistent.
Demanding exactness everywhere
The cost of exact personalization can leave smaller customers with generic or no service when a close answer would be useful.
Ignoring recoverability
Two answers with similar accuracy can carry very different risk depending on whether errors can be corrected in the interaction.
Is it for you?
Best for
It is best for teams choosing between LLMs, data science, workflow automation, and conventional software for a customer or operational use case.
Not ideal for
It is not ideal for high-stakes domains where even an approximate answer requires formal risk analysis beyond a simple decision rule.
From the transcript
“if you need an exact answer, you need data science.”
“If you need a possibly exact answer or a close or approximate answer, then you can use AI in a large language model”
“it cost me a lot less money to give you close answers”
From the episode
Use This A.I. Marketing Strategy To Grow Your Business In 2023 (#114)