Research
Studies from our own scans: which AI crawlers sites let in, the most common AI search issues, and which sites AI answers name. Each study states its method and sample size.
What AI crawlers get from 129 websites (November 2026)
In our scans of 129 sites, 75% of browser-rendered sites sent AI crawlers an empty homepage. Server-rendered sites averaged 82 of 100.
Read the studyWhich site builders AI crawlers can read: the AI App Builder Index (November 2026)
52% of browser-rendered Lovable sites we scanned send AI crawlers an empty homepage, against 6% of server-rendered Lovable sites and 3% of Next.js sites.
Read the studyWhat robots.txt tells AI crawlers on 130 websites (November 2026)
127 of 130 sites we scanned had a robots.txt file. Only 3 blocked an AI crawler from the whole site, and none blocked Googlebot or Bingbot.
Read the studyThe most common AI search issues on 129 websites (November 2026)
83% of the 129 sites we scanned do not open the homepage with a direct answer, and 64% have weak meta descriptions. The full list, from our scan data.
Read the studyHow often AI names a small website: 90 buyer questions (November 2026)
We asked Perplexity one buyer question for each of 90 sites. It named the site in 9 answers (10%) and named other businesses in 52 (58%).
Read the study
Want to see your own numbers? Run a free scan.