AI Website Copilot Experiment
Infer visitor jobs and surface the most relevant next action immediately
- Difficulty
- Advanced
- Time to result
- ~months to results
- Steps
- 7
- Confidence
- 98%
Treat website navigation as a recommendation problem. Begin with reasonable content suggestions for an unknown visitor, then collect a few high-signal inputs such as company size, current goal, and professional role. Combine those answers with observed site interactions and other permitted data sources to rank the pages most likely to help. Present the recommendations with relevance indicators and allow visitors to provide feedback or refine their profile. The primary test is whether personalized routing increases conversion by getting people to the right information or action faster. Time on site, page consumption, and other engagement measures provide secondary evidence. This approach also creates the learning foundation for increasingly dynamic experiences, but the initial experiment should remain a visible recommendation layer rather than immediately replacing familiar navigation behavior.
Origin
Extracted from Marketing Against The Grain as HubSpot previewed its website copilot and AI-generated recommendation experiment.
Core principles
- 01Scarcer website traffic increases the value of every visit
- 02Visitors should reach relevant information with fewer clicks
- 03Recommendations can guide visitors who cannot articulate a precise query
- 04Personalization should improve measurable behavior rather than merely appear intelligent
How to run it
- 1
Define Visitor Jobs
Identify the main outcomes visitors seek, such as researching a solution, evaluating products, learning a skill, or requesting a demo.
Pro tip Describe jobs in visitor language rather than internal product categories.
Watch out Product-centric labels may not match how first-time visitors understand their needs.
- 2
Create the Unknown-Visitor Baseline
Offer a small set of broadly useful recommendations before collecting personal information.
Pro tip Use aggregate behavior and strategic priority to create a sensible default set.
Watch out Do not pretend that generic recommendations are deeply personalized.
- 3
Collect Minimal Intent Signals
Ask for a few inputs that materially improve prediction, such as company size, goal, and role.
Pro tip Request only signals that change the resulting recommendations.
Watch out Long profiling forms recreate the navigation friction the copilot is meant to remove.
- 4
Rank and Explain Recommendations
Score candidate content for relevance and present the highest-ranked pages or actions clearly.
Pro tip A relevance indicator can communicate the system's uncertainty and help users compare options.
Watch out False precision can undermine trust if scores are not calibrated.
- 5
Capture Feedback and Behavior
Collect explicit relevance feedback and observe clicks, conversions, page consumption, and time on site.
Pro tip Combine thumbs-up or thumbs-down signals with actual downstream behavior.
Watch out A click does not prove that the recommended page satisfied the visitor's job.
- 6
Run a Controlled Conversion Test
Release the copilot to selected pages or visitors and compare conversion with the existing navigation experience.
Pro tip Measure several conversion types while naming one primary decision metric in advance.
Watch out Changing traffic allocation or page design mid-test can make results difficult to interpret.
- 7
Progress Toward Dynamic Experiences
Use validated recommendation data to test personalized modules and eventually more adaptive page structures.
Pro tip Preserve familiar interaction patterns until evidence shows visitors are ready for a larger change.
Watch out Do not leap from a recommendation pilot to fully generated pages without proving accuracy, trust, and operational control.
In the wild
HubSpot's prototype first showed three general pages to an unknown visitor. The visitor could then specify a company of 11 to 25 employees, a goal of researching business solutions, and a marketing role. The copilot ranked a marketing-software page as 94% relevant and offered a direct path to it.
→ The visitor received a more targeted next step without manually exploring the full static navigation.
Common mistakes
Replacing Navigation Too Early
Users have deeply learned website conventions, so removing familiar navigation before proving the alternative can create confusion.
Measuring Engagement as the Goal
More pages or longer sessions may indicate continued confusion; conversion should remain the primary success measure.
Collecting Excessive Profile Data
A long personalization questionnaire increases friction and may outweigh the time saved by better recommendations.
Is it for you?
Best for
It is best for content-rich or product-rich websites where visitors struggle to navigate many pages and organic traffic is increasingly scarce.
Not ideal for
It is not ideal for very small websites with one obvious visitor path or insufficient traffic for a meaningful controlled experiment.
From the transcript
“And one of the ways that we're gonna do that is by getting exceptional at understanding who is coming to the website, what they are…”
“It is essentially a smarter version of our navigation that is trained on various data sets that is looking to based on an unknown visitor's…”
“The first KPI is really conversion rate.”
From the episode
How We Hacked Hubspot With Ai To Make Free Money