Capability Boundary Testing
Test familiar failures first, then push outward until the model breaks.
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 5
- Confidence
- 94%
Capability Boundary Testing evaluates a new AI model by beginning with a battery of known failure cases and progressively increasing difficulty. Each task should have an observable pass condition, such as following an exact output structure or producing sentences with precisely seven words. Passing familiar tests establishes that capabilities have improved, but the evaluation continues until the model begins making errors or inventing information. That failure boundary matters more than a few impressive demonstrations because it defines where human verification, fallback logic, or task restrictions become necessary. The resulting output is not a binary judgment that the model is good or bad, but a practical map of which task conditions are dependable and which remain unsafe.
Origin
Nathan Labenz described using this approach while privately testing GPT-4 months before its release on Marketing Against The Grain.
Core principles
- 01Start with tasks previous models failed.
- 02Use precise, objectively checkable constraints.
- 03Treat early success as evidence, not proof.
- 04Search deliberately for the boundary where reliability collapses.
How to run it
- 1
Collect Historical Failures
List tasks that earlier models could not complete reliably, including formatting, counting, reasoning, or citation tasks.
Pro tip Prioritize failures that matter to a real production workflow.
- 2
Define Exact Success
Specify what a passing result looks like before running each test. Prefer constraints that can be checked mechanically.
Pro tip Use exact schemas, word counts, or known references when possible.
Watch out A vague impression of quality makes comparison unreliable.
- 3
Run the Initial Battery
Test each use case with minimal intervention to determine whether the new model can solve it directly.
Pro tip Preserve prompts and outputs for comparison with older models.
- 4
Push Beyond Early Wins
Increase ambiguity, novelty, length, or factual difficulty after the model succeeds.
Pro tip Probe domains where neither you nor the model can rely on routine facts.
Watch out Early valid answers can lull evaluators into trusting later fabricated ones.
- 5
Map the Failure Boundary
Record where reliability begins to fall and establish verification or fallback requirements for that region.
Pro tip Describe the boundary in terms of task conditions, not a single average score.
In the wild
A video platform asks a model to produce creative copy inside a rigid, machine-readable structure that previous models could not follow without examples or fine-tuning. The evaluator first checks exact structural compliance, then increases the script complexity and input length until formatting failures appear.
→ The team identifies whether zero-shot generation can replace or supplement its fine-tuned production model.
An analyst asks for references on a familiar technical topic, verifies every returned link, and then repeats the exercise with increasingly obscure topics. Valid early links do not end the test; fabricated references reveal the model's factual boundary.
→ Citation-bearing outputs receive verification rules matched to their risk.
Common mistakes
Stopping After Impressive Results
A few flawless outputs can conceal hallucinations that emerge only when the task moves beyond common material.
Using Subjective Pass Criteria
Judging whether an answer merely feels good makes it difficult to locate a reproducible capability boundary.
Is it for you?
Best for
Teams evaluating whether a newly released model is reliable enough for specific workflows.
Not ideal for
Situations where outputs have no objective success criteria or cannot be independently verified.
From the transcript
“And I just started running them down one by one.”
“This one, boom, seven, seven, seven, seven, seven, seven, you know, just like flawless uh performance.”
“It's just that that boundary of where it starts to hallucinate is much further out.”
From the episode
GPT-4 Beta User Reveals What Jobs It Will Destroy In 2023 with Nathan Labenz (#103)
Nathan Labenz