How to tell if a page change helped your AI visibility
By the Why Won't AI team · Published · Last updated
As of November 2026, the fair test is a before and after on the same questions. Save a baseline, record exactly what you changed and when, confirm that crawlers can read the new page, then ask the same questions on the same assistants on the same schedule. A new citation after a change is an observation. It is not proof that the change caused it.
Pick one outcome before you edit
| The change | The outcome to watch |
|---|---|
| A new comparison page | Cited in answers to that comparison question |
| A corrected price or feature list | Correct facts in answers to questions that name you |
| A page that crawlers could not read before | That page cited at all |
| A new page for one platform | Named in answers to questions about that platform |
Freeze the plan
Write down the questions, the assistants, the country, the schedule and how you count failures before you publish. Do not change any of them during the test.
Record the change exactly
- The page address and what changed on it.
- The date and time it went live.
- Anything else that changed at the same time, such as other pages, links or listings.
Make sure the change can be seen
- Check that crawlers get the new version, not an old or empty copy.
- Put the page in your sitemap and link to it from another page.
- Tell search engines about it (see the indexing guide).
- Allow time. A new page can take days or weeks to be crawled and indexed.
Compare like with like
Compare only the questions and assistants that you collected both before and after. If an answer failed in one period, leave that slot out of both. Do not count a failed request as a loss.
Report one of four results
- It improved.
- Nothing changed.
- It got worse.
- We cannot tell yet.
All four are useful. A test that shows no change saves you from repeating work that did not help.
Our own test
We published guides for Lovable and Base44 sites on November 3 and 4, 2026. Our baseline for those buyer questions is 0 mentions in 10 answers. We ask the same questions every week from November 11 and will compare the results at week 2 and week 4, including if nothing changes.
Questions people ask
How long should I wait after a change?
Wait until crawlers have read the new page and it is indexed, then measure over several weeks.
What if a request fails during the test?
Leave that question and assistant out of both the before and the after.
Can I prove a change caused a result?
Not with a before and after alone. You can show what changed after, and rule out other changes you know about.
What this does not tell you
How long to wait in your category, how many runs you need before you trust a change, and what to try next if nothing moves.