How to optimise your site for Claude search in 2026

Decorative title card illustration for Claude search optimisation article

Allow Claude-SearchBot and Claude-User in robots.txt (keep ClaudeBot blocking optional if you want to opt out of training), rewrite your priority pages so the answer sits in the first paragraph, split those pages into small self-contained HTML blocks, add Organisation, Person, Article and FAQ schema with named bylines, publish an llms.txt file, then run ten priority prompts through Claude with web search switched on to see what gets cited.

That's the full list. Everything below explains why each step matters and how fast it pays off.

Access (1 to 3 days to take effect): Anthropic's crawlers respect robots.txt directives, so a wrong rule here blocks everything downstream.
Structure (1 to 2 weeks): Answer-first paragraphs and question-format headings give Claude something clean to lift.
Entity clarity (2 to 4 weeks): Schema and consistent naming stop Claude misattributing your content to someone else.
Discovery (days, but compounding): An llms.txt file signals your priority pages to ingestion systems.
Monitoring (ongoing): Ten test prompts run weekly tell you whether any of this actually worked.

Pro Tip: Run Cited's free audit before you touch a single line of code. It flags exactly which of these five items is broken on your site, so you're not guessing where to start.

Key takeaways

Claude citations depend on crawler access, answer-first structure and clean entity schema working together, not any single fix in isolation.

Point	Details
Fix access first	Allow Claude-SearchBot and Claude-User in robots.txt before making any structural changes.
Rewrite for extraction	Move the direct answer to the first paragraph and split long sections into single-idea blocks.
Pin your entity	Add Organisation, Person, Article and FAQ schema with consistent names across your site.
Publish llms.txt	A low-cost discovery signal that still differentiates sites from the majority without one.
Test with real prompts	Run 10 to 20 priority prompts weekly and track which pages and blocks get cited.
Start with a free audit	Cited's free audit at cited.best/audit checks access, structure, schema and citation fitness in one pass.

Where does Claude actually get the content it cites?
How does Claude decide which page to cite?
What's the step-by-step framework for optimising for Claude?
What should your robots.txt and llms.txt actually say?
How do you format content so Claude can lift it precisely?
How do you test whether your Claude citations are actually improving?
What does a Claude visibility audit typically find, and how fast can you fix it?
How should you handle Claude's citations inside user queries?
What pitfalls cause pages to lose Claude visibility?
What does successful Claude optimisation look like in practice?
How does Claude optimisation fit with ChatGPT, Perplexity and Gemini work?
Why fast fixes and slow brand-building both matter
How Cited helps you act on all of this
Sources
FAQ

Where does Claude actually get the content it cites?

Claude pulls from two very different places, and confusing them wastes weeks of effort. The first is training data recall: knowledge baked into the model during a training run, which only updates when Anthropic ships a new model version. You cannot influence this on a useful timescale. If your brand isn't in the training set, no amount of on-page tweaking this quarter changes that. The second path is live: Claude's web search and tool-use features fetch and cite pages in real time.

This second path is where your optimisation budget belongs, because it moves on a crawl cycle, not a model release cycle.

Training data recall: slow, backwards-looking, and outside your control until Anthropic trains again.
Live web search (web_search tool): Claude issues a search, retrieves results, and can cite them within the same session, sometimes within hours of a page going live if the crawler has already indexed it.
search_result blocks in RAG workflows: developers building on the Anthropic platform can return structured search_result JSON with source, title and cited_text fields, giving Claude a programmatic citation path rather than a synthesised guess.

The practical implication: stop treating Claude optimisation as a wait-for-the-next-model exercise. Most of the citation opportunity sits in the live search and tool-use layer, which behaves far more like traditional technical SEO, responsive to crawl access and page structure, than like a slow-moving training pipeline. Anthropic's web_search tool also supports developer-side controls such as allowed_domains and blocked_domains, meaning some of your visibility depends on decisions made in someone else's integration, not just your own site.

How does Claude decide which page to cite?

Claude favours small, self-contained blocks of text that answer one question cleanly, because it cites at the block level rather than pulling fragments from the middle of a sprawling paragraph. A 400-word section with three ideas tangled together gives Claude nothing precise to lift. A 60-word answer to "What is a search_result block?" gives it exactly one citable unit.

Four signals drive the decision:

Quotability: short, complete text blocks that stand alone without needing the paragraph before or after them.
Recency and corroboration: content that's current and echoed by other credible sources tends to outrank stale, unsupported claims.
Entity clarity: schema markup that names the organisation and author precisely, so Claude attributes the claim to you rather than merging it with a similarly-named competitor.
Recommendation strength: how often your brand or page is mentioned elsewhere on the web, which shapes both training-time recall and live citation preference.

Pro Tip: Write your answer paragraph as if it were the only sentence Claude will ever read from your page. If it doesn't stand up alone, split it or rewrite it.

Practitioner analysis backs this up directly: sites that split content into focused, server-rendered blocks see more precise citation behaviour explained in this practical guide than sites relying on long, unstructured sections. The same platform documentation notes that when developers return search_result blocks, mapping the cited_text field to a specific claim improves traceability and reduces the risk of Claude synthesising an answer without a clean source to point to.

What's the step-by-step framework for optimising for Claude?

Run this as a one-week sprint, then fold steps five and six into a weekly cadence. SearchScore's guide to Claude optimisation recommends this exact order, access first, then structure, then entity, then footprint, because fixing structure before access wastes effort on pages Claude can't reach anyway.

Audit robots.txt and your CDN or WAF. Check for wildcard "Claude*" or "block AI bots" rules that catch Claude-SearchBot and Claude-User alongside ClaudeBot.
Pick 5 to 10 priority pages. Choose the pages that answer your highest-value queries, then rewrite the opening paragraph so it states the direct answer first.
Split long sections into focused blocks. Each block should answer one question, ideally under a question-format H2 or H3.
Add structured schema and named bylines. Organisation, Person, Article and FAQ schema, with content rendered server-side rather than injected by client-side JavaScript.
Publish llms.txt. Point it at your priority pages, and if you control a search or RAG integration, configure allowed_domains to include your own site.
Run 10 priority prompts in Claude with web search enabled. Record which pages get cited and which specific blocks get quoted, then iterate.

The team most likely to see fast results is the one that treats this as a checklist, not a project plan. Step 1 alone, removing an accidental wildcard block, can restore access within a single crawl cycle. Steps 2 to 4 take longer because they touch content and templates, typically two to four weeks depending on your CMS. Step 6 is where you find out whether any of it worked, so don't skip it or leave it for "later".

Assign one owner per step, not one owner for the whole framework.
Re-run step 6 weekly for the first month, then monthly once citations stabilise.
Keep a simple log of prompt, cited page, and cited block so you can spot patterns.

What should your robots.txt and llms.txt actually say?

Anthropic runs three named agents, ClaudeBot for training, Claude-SearchBot for search indexing, and Claude-User for live user-triggered fetches, and all three honour robots.txt directives. If you want to opt out of training while staying visible in search and live citations, disallow ClaudeBot specifically and leave Claude-SearchBot and Claude-User open. Document that choice somewhere your engineering team will actually find it, because the next CDN update or bot-management vendor renewal is exactly when a wildcard rule quietly reappears.

Watch CDN and WAF "block AI bots" toggles. These are frequently enabled by default after a security review and catch retrieval agents alongside scrapers you actually wanted to stop.
Watch JavaScript challenges. A challenge page that's trivial for a human can look identical to a hard block from a crawler's perspective.
Verify by IP range and behaviour, not user-agent string alone, since user-agent headers can be spoofed and aren't proof of a genuine Anthropic visit.
Check server logs for 429 or 403 responses to Anthropic agents in the days after any infrastructure change, not just the day of.

Pro Tip: Set a calendar reminder to re-check robots.txt every time your CDN vendor ships a "security update". That's the single most common moment a wildcard block gets reintroduced without anyone noticing.

Industry analysis of this exact failure pattern found that many sites accidentally blocked retrieval agents through blanket rules meant to stop training crawlers, not search or live fetch. The fix is granular allow rules, not a blanket toggle in either direction.

How do you format content so Claude can lift it precisely?

Server-render the answer sentence in your opening paragraph. If your CMS injects the lead paragraph via client-side JavaScript, Claude's fetch may see an empty shell rather than your answer, which is one of the most common and least visible failures on modern content platforms.

Use question-format H2s that match how people actually phrase queries.
Keep each paragraph to one idea, since a block that mixes two claims gives Claude nothing clean to quote.
Add Article, Organisation, Person and FAQ schema, and keep names identical across your site, your directory listings, and your author bios.
Publish llms.txt: audits show a large share of sites still don't have one, and SearchScore's guide treats it as a genuine discovery advantage rather than a nice-to-have.

How do you test whether your Claude citations are actually improving?

Build a fixed set of 10 to 20 priority prompts covering your core topics, phrased the way a real user would ask them, not the way you'd phrase a keyword.

Run each prompt with web search enabled and record which pages get cited and which specific text block gets quoted.
Cross-reference citations against your server logs to confirm the crawl actually happened after your changes, not before.
Check daily during your rollout week, then drop to weekly once results settle.
Keep the same prompt set over time so you're comparing like with like, not chasing a moving target.

This is the only step that tells you whether steps 1 to 6 of the framework worked, so don't treat it as optional polish.

What does a Claude visibility audit typically find, and how fast can you fix it?

Cited's free audit checks four things per Anthropic agent: whether ClaudeBot, Claude-SearchBot and Claude-User can actually reach your pages, whether your answers are extractable, whether your schema is complete, and how citation-ready your content actually is.

The recurring pattern across audits: a wildcard agent block, an answer buried three paragraphs in, no named author, and one long unbroken content block where five should exist.

Access fixes (removing wildcard blocks): days.
Structural fixes (answer-first rewrites, block splitting): weeks.
Training-path improvements (broader brand recall): quarters, tied to Anthropic's own release cycle.

Pro Tip: Fix access before structure, every time. A perfectly written answer-first page that Claude can't fetch produces zero citations.

Run the free audit now if you haven't already, it takes a few minutes and tells you which of these three categories your site actually falls into.

How should you handle Claude's citations inside user queries?

When someone asks Claude a question and it cites your page, the citation carries your source name and often a direct quote pulled from your cited_text block. That's a level of exposure traditional search results never gave you: the user sees your specific words, attributed to you, inside the answer itself. This changes what "ranking" means. You're not competing for a blue link position; you're competing to be the exact sentence Claude decides to quote.

Hands arranging glass data panels representing content structuring

Write for that reality. Your answer paragraph should read correctly in isolation, because it may appear stripped of its surrounding context inside someone else's conversation. Avoid phrasing that depends on "as mentioned above" or "see the table below", since those references become meaningless once Claude lifts the sentence out.

Keep facts and figures consistent across every page that touches the same topic. If your pricing page says one figure and your FAQ says another, Claude has no reliable way to know which is current, and may cite the wrong one or decline to cite either. Consistency across your own site is a citation-fitness signal in its own right, not just good housekeeping.

Finally, treat schema as your defence against misattribution. If two organisations share a similar name, precise Organisation and Person schema is what stops Claude crediting your competitor for your own reporting or data. Where you've published original statistics or research, make the sourcing explicit in the text itself, not just in a footnote Claude's fetch might not render.

What pitfalls cause pages to lose Claude visibility?

The most common self-inflicted pitfall is the wildcard robots.txt block, usually added during a security tidy-up with no intention of blocking search or live fetch, but catching Claude-SearchBot and Claude-User anyway. The second most common: JavaScript-rendered answer paragraphs that read fine in a browser but arrive empty when Claude's agent fetches the raw HTML.

There's no evidence of a "Claude penalty" in the way Google has algorithmic penalties for manipulative SEO. What looks like de-ranking is almost always de-access: a page that used to get cited stops appearing because something upstream, a CDN change, a bot-management vendor update, a new WAF rule, quietly started returning a block or a challenge page to Anthropic's agents.

Watch for these specific failure modes:

Silent CDN updates that add AI-bot blocking as a default-on security feature.
Content that contradicts itself across pages, which makes Claude less confident in citing either version.
Answers that need surrounding context to make sense, since an isolated citation with no context reads as unreliable or gets skipped.
Missing or inconsistent author and organisation names, which risks misattribution rather than a clean citation to you.
Stale content with no recency signal, when a corroborating, more recent source exists elsewhere.

None of these require a manual reconsideration or an appeal. They require a log check, a robots.txt review, and a rewrite. That's a far more solvable problem than a Google penalty ever was, provided you catch it.

What does successful Claude optimisation look like in practice?

The clearest pattern across sites that gain Claude citations isn't a single dramatic change. It's the combination of the same three fixes, applied consistently: open access for the right agents, an answer-first paragraph, and clean schema.

A site that previously buried its core answer under three paragraphs of scene-setting, then moved that answer to the first sentence and split the surrounding content into question-format blocks, tends to see citations return within the following crawl cycle rather than months later, precisely because the access and structure fixes both operate on a fast timescale. Sites that only fix structure while leaving a wildcard robots.txt block in place see no improvement at all, because Claude never reaches the improved page to notice the change.

The pattern holds in reverse too. A technically well-configured site with open access but content written as long, undifferentiated narrative paragraphs still underperforms, because Claude has nothing precise enough to cite. Access without structure is as ineffective as structure without access. Both have to move together.

What ties genuine case-level improvement together is measurement discipline: running the same set of test prompts before and after a change, rather than assuming a fix worked because it felt thorough. Sites that skip this step often can't tell whether their citation rate improved or whether they simply got lucky on a handful of prompts. The ones that track it consistently can point to specific pages and specific blocks that started getting quoted, and specific ones that stopped, which is the only reliable evidence that an optimisation effort actually worked.

How does Claude optimisation fit with ChatGPT, Perplexity and Gemini work?

The good news for teams already doing generative engine optimisation elsewhere: the core fixes overlap heavily. Answer-first structure, clean schema, named authorship and server-rendered content all help visibility across ChatGPT, Perplexity, Gemini and Copilot, not just Claude. If you've already restructured content for one of these platforms, you've done most of the work for Claude too.

Where the platforms diverge is the technical layer. Each has its own named crawlers and its own robots.txt conventions, which means a blanket "allow all AI bots" or "block all AI bots" rule is the wrong tool regardless of which platform you're optimising for. You need agent-specific rules for each: Claude's three agents behave differently from OpenAI's or Google's crawlers, and lumping them together in one directive is how sites accidentally lose visibility on a platform they never meant to block.

llms.txt is the other area worth treating as cross-platform. It's not a Claude-specific file, and publishing one signals priority content to any AI system that checks for it, making it one of the lower-effort, higher-leverage items on this entire list.

Practically, this means your monitoring should run in parallel across platforms rather than in isolation. If you're tracking Claude citations weekly, add the same cadence for the other engines you care about, using the same priority prompt set adapted slightly for each platform's phrasing conventions. A page that gets cited by Claude but not by Perplexity is telling you something specific about that platform's corroboration or recency preferences, and that comparison is only visible if you're measuring both at once. For deeper background on the wider discipline this sits inside, see what generative engine optimisation actually covers.

How does Claude optimisation fit with ChatGPT, Perplexity and Gemini work? — overview diagram

Why fast fixes and slow brand-building both matter

Robots.txt and answer-first rewrites are plumbing. Fix them and you can see citation changes within weeks, sometimes days. But Claude's training-time recall of your brand depends on something slower: how often and how consistently you're mentioned across the web, under one consistent name, over months and years.

Teams that only chase the fast fixes plateau, because live search citations can't compensate for a model that's never encountered your brand in training. Teams that only invest in long-term footprint building miss citations they could have had this quarter for the cost of an hour's robots.txt review.

Run both. Every model update resets some of the training-path work, so the slow lever needs permanent attention, not a one-off campaign.

How Cited helps you act on all of this

Most of what's above takes an engineering ticket, a content rewrite, and someone with the time to test it properly, which is exactly where teams stall. Cited runs the diagnostic work for you: a free audit that checks crawler access per Anthropic agent, answer extractability, schema completeness and citation fitness, then hands you a prioritised fix list instead of a generic score.

If the audit flags issues worth fixing, Cited's Technical Fixes package (a one-off £495) covers the access and structure work described in the framework above: robots.txt correction, answer-first rewrites, schema implementation. For ongoing work across a growing content library, the AI Optimised subscription (£995 a month) covers continuous monitoring and iteration, the weekly prompt testing this article recommends, done as a managed service rather than a task on your own list. Larger organisations with multiple domains or complex CDN setups get a custom Enterprise engagement.

Read the Cited methodology to see exactly how the six dimensions of AI citability are scored, then book a call to talk through what your audit found and what fixing it would involve. Start with the free audit if you haven't run one yet.

Sources

Read Anthropic's own crawler and robots.txt guidance and the search_result documentation for the technical detail behind this article. SearchScore's Claude SEO guide and Menra's crawler explainer cover the same ground from a practitioner angle, and Cited's content optimisation guide expands on answer-first writing.

FAQ

Can Claude optimise SEO?

Claude can help draft and analyse SEO content, but it doesn't optimise your site for search itself. Getting cited by Claude requires the technical and structural fixes covered above, which Cited's audit is built to identify.

How do I make my Claude usage more efficient?

For citation purposes, run the same fixed set of priority prompts each time you test, and enable web search so you're seeing live citation behaviour rather than training-data recall.

Which Claude setup is best for search-style queries?

Claude's web search and web_search tool paths, which return structured search_result blocks with citation metadata, are the relevant path for live citations, rather than the model's underlying training data.

Is SEO dead or evolving for 2026?

Traditional SEO isn't dead, but it's evolving to include generative engine optimisation: the same access, structure and entity fundamentals now decide whether Claude, ChatGPT and Perplexity cite you, not just whether Google ranks you.

What's the fastest fix for low Claude visibility?

Checking robots.txt for wildcard blocks against Claude-SearchBot and Claude-User usually delivers the fastest result, often within days, which is exactly what Cited's free audit checks first.