Get your site cited in AI answers

What changed, and what did not

Answer engines such as ChatGPT, Claude, Perplexity and Google's AI features read the web and compose a reply. Some answer from what they learned in training. Many fetch live pages at the moment of the question and cite them.

What did not change: they still need to reach your page, understand it and find it credible. Everything that makes a page good for search (clear headings, honest titles, real expertise) still counts.

What did change: a crawler for training is not the same as a crawler that fetches a page to answer a user. Your robots.txt can treat them differently, and most sites have never decided which they want.

Answerability in Growth Score is the score for this: whether AI answers can read, quote and cite your site. It reads which AI crawlers your robots.txt allows, whether an llms.txt exists, and whether your pages are marked up to be quoted.

Step 1: decide who gets in

Open your robots.txt and look for rules naming AI crawlers. The names to know:

• GPTBot: collects pages that may be used to train OpenAI models.
• OAI-SearchBot: ChatGPT search.
• ChatGPT-User: fetches a page when a person asks ChatGPT to open it.
• ClaudeBot: Anthropic's crawler. Claude-User and Claude-SearchBot cover user requests and search.
• PerplexityBot: Perplexity's index.
• Google-Extended: a control token that governs whether Google uses your content for its Gemini models. It does not affect ordinary search ranking.

These names change, so check each company's own documentation before you rely on this list.

The decision is yours, and it is a business one. If you want to be recommended, allow the search and user-fetch crawlers. If you do not want your writing used for training, block the training crawlers and say so explicitly. The mistake is having a blanket rule from years ago that blocks everything by accident.

Step 2: read your own file the way a crawler does

A rule only works if you wrote what you meant. Two traps catch people:

• A broad Disallow under User-agent: * that quietly applies to every assistant that has no rule of its own.
• A rule for one named crawler that is more specific than the general one, so the general rule you were relying on never fires.

The free AI crawler check reads your robots.txt for each of these crawlers and tells you which can train on your writing, which can open a page to answer or cite it, and which rule blocks each one. Run it, then read the answer once and decide whether it matches your intent.

Then make the policy visible. A short comment above the AI rules, naming the date and the reason, saves the next person from undoing it.

Step 3: write the answer first

Assistants quote what is easy to lift. A page that buries the point in the fourth paragraph gets paraphrased badly or skipped for one that states it plainly.

For every page that should be cited, add a quotable answer near the top: one or two sentences that stand on their own, answer the question in the heading, and name the subject in full instead of saying "it". Then support it below.

A pattern that works for guides and help pages:

1. 1A question as the heading, in the words a person would ask.
2. 2The answer in the first one or two sentences, with the key fact in them.
3. 3The detail, the steps, the exceptions and the evidence after.

Use real headings and short sections so a machine can tell where an answer starts and ends. Define your terms on first use. Be specific: the date a policy changed, the plan a feature belongs to, the exact limit. Vague pages are not quoted because there is nothing safe to quote.

Step 4: mark up what the page is

Structured data is a block of labelled facts in your page (usually JSON-LD) that says what the page is: an article, a product, an organisation, a set of questions with answers. It does not guarantee a citation. It removes guesswork about who wrote the page, when, and about what.

Start with the basics:

• Organization on your home page: name, logo, links to your real profiles.
• Article: headline, author, dates.
• Product or Offer where you sell something, with the true price.
• FAQPage only on a page whose main content is a list of questions and answers.

Mark up only what a visitor can see on the page. Invented or hidden facts get ignored at best and penalised at worst. The schema and share-tag check reads your JSON-LD and names what is missing or broken, and the schema builder makes valid blocks from a form.

Step 5: consider an llms.txt, and be honest about it

llms.txt is a plain text file at yoursite.com/llms.txt, written in Markdown, that lists your most useful pages with a one-line description each. The idea is to give a language model a clean map instead of making it dig through menus and scripts.

Be realistic. It is an emerging convention, not a standard every assistant honours, and nobody can promise it changes whether you are cited. It costs an hour. It forces you to decide which ten pages matter most, and it may also help people who read your docs through an assistant.

If you write one: a title line, a short summary, then grouped links with a sentence each. Link only to pages that load, and keep it short. The llms.txt check finds malformed entries and links a tool could not follow.

Step 6: prove it is working

You cannot manage what you cannot see. Two things to watch:

• Referrals from assistants. Visits from chat.openai.com, perplexity.ai, claude.ai and similar show up as referrers in your analytics, and some assistants add a tag to the links they send. Group them into one segment so the trend is visible.
• The answers themselves. Once a month, ask three assistants the five questions a buyer would ask. Note whether you were named, whether the facts were right and who was named instead.

When a fact is wrong, fix the source page first: make the correct fact easy to find and easy to quote, in plain words near the top. When a competitor is cited and you are not, read their page and ask what it states that yours implies.

The Surfil pixel records AI referrals separately, with no cookies, so the trend is there without building a report by hand.

Mistakes that cost citations

Most losses are self-inflicted:

• Blocking every crawler in an old robots.txt and forgetting.
• Putting the real content behind a script that never runs for a crawler.
• Writing pages that never state a plain fact, only hints and slogans.
• Using one page for ten topics, so no heading answers a single question.
• Letting facts go stale: old pricing, retired features, a policy that changed.

Fix them in that order. Growth Score reads the access and markup signals daily, so you hear about a flipped rule the day it happens. The free Growth Diagnosis shows where your site stands now.