AI Discovery Readiness Audit Checklist

An AI discovery readiness audit should test the public conditions a website can influence: crawler policy, response delivery, initial HTML, answer clarity, structured data, identity, evidence quality, remediation, and fresh re-checking. It should not claim access to private provider indexes or promise that passing the checklist will produce citations.

How to use this checklist

Use one record per origin and scope:

scheme + hostname + port + selected paths + audit time + method version

Do not assume that the apex domain represents www, documentation, support, shop, app, regional, or staging hostnames. Select the pages that matter to the user journey and the prompts you expect them to support.

For every item, record one of these states:

• Observed: directly collected from the public resource.
• Documented: supported by a current primary source.
• Simulated: produced by a controlled diagnostic request.
• Heuristic: a deterministic or bounded quality signal requiring interpretation.
• Inferred: a conclusion derived from other evidence.
• Manual check: important but not safely automatable.
• Unavailable: the check could not be completed under the declared limits.

Do not silently convert unavailable evidence into failure.

1. Scope and critical pages

• Record the canonical origin being audited.
• List important alternate hosts and subdomains separately.
• Select the homepage, product or service page, documentation entry point, About page, and one high-intent guide where applicable.
• Record the user or business question each selected page should answer.
• Exclude private or unauthorized resources.
• Record the audit time, methodology version, and crawler-registry version.

Why it matters: A broad domain verdict can hide that the important answer page lives on a different hostname or behaves differently from the homepage.

2. Crawler purpose and policy

• Identify the exact provider token from current official documentation.
• Classify it as search discovery, training, user-triggered retrieval, ordinary search, data-use control, or uncertain.
• Retrieve the deployed robots.txt from the correct origin.
• Resolve the rule for each selected path.
• Review wildcard and token-specific groups.
• Check for duplicate groups, misspelled tokens, and contradictory rules.
• Record the owner’s intended policy where available.
• Distinguish an intentional training restriction from an unintended search restriction.

Boundary: robots.txt expresses policy. It is not authentication and does not prove that a provider retrieved or used a page.

See AI Search Crawlers vs Training Crawlers.

3. Public response and delivery

For every selected page:

• Record initial and final URL.
• Record HTTP status.
• Preserve the redirect chain.
• Record response content type.
• Check for a bot-management or JavaScript challenge.
• Check for 403, 429, timeout, redirect loop, or oversized-response conditions.
• Record relevant response headers.
• Compare ordinary and crawler-like requests only through bounded, authorized diagnostics.
• Label crawler-like requests as simulated rather than provider-observed.
• Check delivery on critical paths beyond the homepage, not only the root URL.

Why it matters: A policy can allow a token while a CDN or WAF still blocks the request. A homepage-only check can also miss a delivery problem confined to one important page — the OpenForBots audit can test up to four extra paths under a default and a simulated automated-client profile; see delivery diagnostics for selected paths. A flagged path also comes with a short guidance note on what to check; see suggested fixes and remediation guidance.

4. Initial HTML and page structure

• Confirm the initial response contains meaningful text.
• Confirm one descriptive title and one primary H1.
• Check that essential content is not available only after an interaction.
• Confirm important links are present in the initial document.
• Inspect canonical URL and language declarations.
• Check that navigation reaches the selected page through crawlable links.
• Record whether client rendering materially changes the content.

Boundary: Weak initial HTML is a portability risk, not proof that every answer engine fails to render the page.

5. Content and answer clarity

• State the primary entity, offer, and audience near the top.
• Provide a direct answer before extended background.
• Use descriptive headings that match real user decisions.
• Support important claims with primary sources or original evidence.
• Separate fact, interpretation, recommendation, and unknown.
• Show publication, update, or verification dates where facts can change.
• Explain limitations and counterexamples.
• Remove contradictory product descriptions across important pages.
• Provide an obvious next action.

Quality test: A careful reader should be able to quote the direct answer, locate its evidence, and identify what the page does not claim.

6. Structured data

• Confirm JSON-LD parses as valid JSON.
• Check that types match visible page content.
• Use Organization and WebSite consistently where appropriate.
• Use page-level types such as Article, TechArticle, SoftwareApplication, or BreadcrumbList only when supported by the page.
• Keep names, URLs, logos, dates, and identifiers consistent.
• Avoid unsupported ratings, reviews, prices, or claims.
• Check duplicate or conflicting entities.
• Link the page to the canonical organization where useful.

Boundary: Valid structured data can reduce ambiguity but does not force a provider to use or cite it.

7. Entity identity and trust

• Use one canonical product name and organization name.
• Publish a clear About page.
• Show an official contact and correction route.
• Link only genuine official profiles.
• Keep ownership and authorship visible.
• Provide methodology for research or diagnostic claims.
• Publish privacy, security, and terms boundaries appropriate to the product.
• Resolve same-name ambiguity with consistent URLs and descriptions.
• Avoid fictional awards, testimonials, clients, or certifications.

Why it matters: Machines and people both need enough evidence to distinguish the right entity and evaluate the source.

Run the AI Structure and Identity Checker to inspect the public homepage signals described in this section.

8. llms.txt and other optional resources

• Check whether /llms.txt exists and returns readable text.
• Confirm its summary matches the public site.
• Check links for canonical, successful destinations.
• Keep it concise and curated rather than copying the sitemap.
• Remove stale, private, or unsupported entries.
• Label the convention as optional and evolving.
• Do not treat absence as a core access failure.

See Does llms.txt Improve AI Visibility?.

9. Evidence and prioritization

For every finding:

• Preserve a stable finding ID.
• Link the conclusion to raw evidence IDs.
• Show the source URL and observation time.
• Record evidence class and confidence boundary.
• Explain user or business impact without exaggeration.
• Define the smallest useful remediation.
• Record dependencies and ordering.
• Define the evidence required to verify resolution.
• Keep raw evidence available without forcing it into the main decision view.

Prioritize using dependency and consequence, not an arbitrary aggregate score.

10. Remediation and fresh re-check

• Apply only authorized changes.
• Preserve the original evidence record.
• Record the claimed change separately from verification state.
• Repeat the same compatible checks after deployment.
• Compare the same origin, token, path, and evidence class.
• Mark a condition resolved only when fresh evidence supports it.
• Keep external-outcome limitations after technical resolution.
• Record regressions and rule-version changes.

A ticket marked “done” is not a verified fix.

11. Outcome observation, kept separate

• Record the exact prompt.
• Record engine, model or product label, mode, language, location, and date.
• Record mention, citation, cited URL, and factual accuracy separately.
• Preserve negative and partial results.
• Repeat material tests under comparable conditions.
• Do not attribute a citation change to one remediation without controlled evidence.

Read AI Visibility Tracking vs Technical Readiness.

Copyable checklist

# AI discovery readiness audit

## Scope
- [ ] Origin and critical paths recorded
- [ ] Method and registry versions recorded

## Access
- [ ] Crawler token and purpose verified
- [ ] robots.txt retrieved and path rule resolved
- [ ] Public delivery, redirects, headers, and status checked

## Content
- [ ] Meaningful initial HTML present
- [ ] Direct answer, entity, offer, and audience clear
- [ ] Important claims supported and dated

## Structure and identity
- [ ] Structured data matches visible content
- [ ] Canonical identity and official profiles consistent
- [ ] Trust, contact, and correction routes available

## Remediation and verification
- [ ] Findings link to evidence
- [ ] Minimal remediation defined
- [ ] Fresh compatible re-check completed
- [ ] External-outcome limitations preserved

Definition of a useful result

A useful audit does not need to end with “pass” or “fail.” It can conclude:

• a material public barrier was observed;
• a policy and delivery result disagree;
• the page is accessible but unclear;
• identity or source evidence is incomplete;
• no material public barrier was found under the tested conditions;
• a check was unavailable and requires manual review.

These conclusions lead to better work than a single unexplained readiness number.

Run the OpenForBots public audit to apply this checklist to a public domain, or review the published methodology before using it in a client workflow.