Incremental Prompt Refinement Loop
Improve AI outputs through small, observable prompt changes
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 6
- Confidence
- 99%
The loop treats prompting as an incremental experiment rather than a one-shot command. Begin with a quick, simple request to observe the model's baseline response. Use redo several times to explore different outputs produced from the same instructions. Then edit the original prompt and add only the detail needed to correct an observed weakness. Rules specify what the model should or should not do, while approved input-output examples demonstrate what success looks like. Testing after each small change reveals whether that intervention actually improved the result and prevents an enormous prompt from becoming impossible to debug. Over successive iterations, the user learns the model's tendencies and develops a compact, reusable prompt that reliably produces a strong starting point for human review.
Origin
Extracted from Marketing Against The Grain during Dan Shipper's practical master class on using ChatGPT effectively.
Core principles
- 01The first response is only one possibility
- 02Simple initial prompts reveal the model's baseline behavior
- 03Small changes make cause and effect observable
- 04Rules should address demonstrated mistakes
- 05Examples communicate quality more clearly than vague preferences
How to run it
- 1
Establish a simple baseline
Write the shortest prompt that expresses the task and generate an initial result. Use it to discover what the model already understands and where it fails.
Pro tip Ask for a small sample before attempting an entire large deliverable.
Watch out A long speculative prompt can hide which instructions are useful.
- 2
Explore with redo
Generate several responses from the unchanged prompt to inspect the natural range of possibilities.
Pro tip Compare recurring strengths and failures rather than choosing solely by first impression.
Watch out Do not treat the first response as the model's fixed capability.
- 3
Edit the original prompt
Return to the initial instruction and change it directly instead of building a long chain of corrective follow-ups.
Pro tip Save promising prompt versions so you can compare them later.
Watch out Long conversational correction chains can make the effective instructions difficult to understand.
- 4
Add one targeted rule
When a specific mistake appears, add a concise instruction describing the preferred behavior or prohibited behavior.
Pro tip Phrase the rule so it can be evaluated in the next output.
Watch out Adding many rules simultaneously prevents you from knowing which change caused the result.
- 5
Demonstrate good output
Add one approved input-output pair that resembles the transformation you want the model to perform.
Pro tip Introduce examples individually and observe how each changes the output.
Watch out Fifty examples added at once can obscure bad examples and waste context.
- 6
Repeat and stabilize
Continue generating, inspecting, and making small changes until the output is consistently useful enough to enter a human review process.
Pro tip Turn a stable prompt into a shared template for recurring tasks.
Watch out Reliable-looking output still requires factual and contextual checking.
In the wild
A marketer first asks ChatGPT to turn a podcast transcript into a social post. After several redos reveal generic openings, the marketer edits the original prompt to prohibit generic hooks. One approved transcript-post pair is then added. Each revision is tested independently until the prompt consistently identifies a specific claim and produces usable drafts.
→ The marketer learns which instructions affect quality and creates a reusable prompt without unnecessary complexity.
Common mistakes
Accepting the first response
One generation samples only one part of the available possibility space and may substantially underrepresent what the model can produce.
Writing the giant prompt first
Starting with extensive detail produces a difficult-to-debug result because the user cannot identify which instructions helped or harmed it.
Adding examples in bulk
Introducing many examples simultaneously makes their individual effects invisible and may inject inconsistent patterns.
Is it for you?
Best for
People building repeatable prompts who need to learn what materially improves the model's output.
Not ideal for
High-risk tasks where repeated model outputs cannot be safely evaluated by a knowledgeable human.
From the transcript
“you should be pressing redo whenever you do a prompt and like just go through the space of possibilities”
“you want to start really simple you want to add complexity over time”
“doing it one at a time you can see what changes and you can start to learn like what's working what's useful what's not”
From the episode
How I Use ChatGPT To Grow My 54,000+ Subscriber Newsletter