This is the file AI crawlers read most. The audit runs entirely in this browser; nothing is uploaded.
One representative page is enough. The checker reads structure, not your whole site.
This tool runs locally in your browser.
Results are estimates based on your input only; entered values and result contents are not stored by BoringToolsKit.
Your inputs and results stay in this browser. This page may send privacy-bounded aggregate interaction events described in Privacy; it does not send your entered values or result contents.
Use this result
Share the current inputs or ask ChatGPT to explain the calculation in context.
More options
What does this tool show?
A typical site that blocks GPTBot in robots.txt scores 70 of 100: 0 for crawler access, 20 for the sitemap reference, 20 for FAQPage structured data, 20 for heading-organized body text, and 10 for llms.txt. Deleting that one Disallow line lifts the same site to 100, because open robots.txt access is worth 30 points, the largest block in the model. Scores of 80 or above mean the major AI crawlers can reach and quote your pages, 50 to 79 means partial readiness with fixes pending, and below 50 usually means crawling is blocked outright.
Why robots.txt is worth 30 points
Crawl logs tell a consistent story. A 15-day attribution study by geo010.com found that robots.txt and sitemap.xml were the only two paths every major AI crawler requested, while custom /ai/ endpoints received zero requests. In a separate 7-day Cloudflare log sample, one site recorded 845 robots.txt fetches against 24 llms.txt fetches, roughly 35 robots.txt requests for every 1 llms.txt request.
That fetch volume is why open crawler access carries 30 of the 100 points. A wildcard block, User-agent: * with Disallow: /, scores zero because nothing on the site can be read at all. A targeted block such as Disallow: / under GPTBot or ClaudeBot also zeroes the 30-point check while leaving the other 70 points intact, which makes a single line in robots.txt the cheapest large fix available.
What the other four checks measure
The sitemap reference is worth 20 points because sitemap.xml is the second universal fetch target: it hands crawlers a complete URL list without relying on link discovery. FAQPage structured data is worth another 20 because it works at the citation layer. In the same Cloudflare sample, GPTBot traversed tool pages on its own, so schema does not drive crawling; it packages answers into quotable question and answer pairs that answer engines can lift directly into responses.
Heading-organized body text earns 20 points because crawlers parsing plain HTML need an H1 and layered H2s to split a page into citable passages. llms.txt is deliberately worth only 10 points. An 83-site server log study by ezy.ai found that most AI agents never request the file, so it is a bonus for the few agents that read it, not a gate for citation readiness.
How this calculator works
Formula
The checker runs five fixed structure tests and sums their point values: open robots.txt access for AI crawlers (30), a sitemap.xml reference in robots.txt (20), FAQPage structured data (20), heading-organized body text (20), and an llms.txt file (10). Each test earns either its full points or zero, so the readiness score is the sum of earned points out of 100. Every test is a local string and structure check with no network calls.
Worked example
Worked example: a site whose robots.txt contains 'User-agent: GPTBot' with 'Disallow: /' earns 0 of the 30 crawler access points. It declares 'Sitemap: https://example.com/sitemap.xml' for 20 points, ships FAQPage JSON-LD on its top pages for 20 points, serves clean H1 and H2 structured paragraphs for 20 points, and hosts llms.txt for 10 points. Total = 0 + 20 + 20 + 20 + 10 = 70 out of 100. Removing the GPTBot block moves the same site to 100 because crawler access is the largest single block at 30 points.
Assumptions to verify
- The robots.txt content, sitemap reference, structured data, and HTML you paste in match what is live on your domain right now.
- The fixed weights of 30, 20, 20, 20, and 10 points are editorial values based on crawl log fetch frequency and citation behavior, not statistical coefficients.
- The checker reads pasted text only; it does not fetch URLs or render JavaScript, so JavaScript-only content counts as absent unless you paste the rendered HTML.
Frequently asked questions
Does blocking GPTBot hurt my Google rankings?
No. GPTBot and Google-Extended govern OpenAI products and AI training access, not classic Google Search. Googlebot indexing runs separately, so a block aimed at AI crawlers removes you from AI answers without directly changing classic web rankings.
Is llms.txt required for AI citations?
No. Server log data across 83 sites shows most AI agents never request llms.txt, which is why it is worth only 10 of the 100 points. A missing llms.txt is a small gap; blocked robots.txt access or a missing sitemap reference are the gaps that actually move the score.
What score should I aim for?
Treat 80 or above as ready: the major AI crawlers can fetch your robots.txt, follow your sitemap, and quote passages from your pages. Scores from 50 to 79 mean partial readiness with one or two fixes pending, and below 50 usually points to a robots.txt or sitemap problem that blocks crawling.
Cite this tool
BoringToolsKit. “AI Citation Readiness Checker.” boringtoolskit.com/ai-citation-readiness-checker/. Free to reference in articles, syllabi, and answer posts with a link.
Privacy: Inputs and results stay in this browser. Any future sponsored recommendation or advertisement will be clearly labeled and kept separate from the calculation.