Extension Domains:
Earning Topical Authority Beyond Your Core Domain.

A Note on Terms: SEO, AEO, and GEO

This paper treats three disciplines as one connected system rather than three separate workstreams:

• SEO (Search Engine Optimization) earns visibility in traditional ranked results — Google, Bing — through technical health, on-page relevance, and off-page authority.
• AEO (Answer Engine Optimization) structures content so AI-powered answer surfaces — Google AI Overviews, AI Mode, Bing Copilot — select it as the direct, quoted, or summarized answer.
• GEO (Generative Engine Optimization) is the broader discipline of earning citation and synthesis inclusion inside LLM-generated responses — ChatGPT, Perplexity, Gemini, Claude — where the underlying mechanism is retrieval and generation rather than ranked placement.

The practical distinction matters less than it sounds: as this paper covers throughout, the technical and editorial foundations that earn traditional rankings are largely the same foundations that earn AI citations. Every section below is annotated for which discipline it primarily serves, and most serve all three at once.

Executive Summary

Every established domain eventually hits a ceiling on certain query clusters. Sometimes it's a legacy content structure that doesn't fit a new topic. Sometimes it's brand perception — a B2B software company can't easily rank for consumer-facing educational content under the same domain. Sometimes it's technical debt: a CMS that can't support the content format a topic needs (interactive tools, deep reference documentation, community-generated content).

An Extension Domain is a deliberately built secondary property, distinct from throwaway or "PBN-style" domains, whose entire value proposition rests on being better than what already ranks — better researched, better designed, better structured for the way both humans and AI systems consume information. It is not a shortcut. It is a second front-line asset that has to independently earn trust across SEO, AEO, and GEO simultaneously.

This paper covers:

1. What legitimately distinguishes an Extension Domain from a manipulative "domain network" play — and from its mirror-image failure mode, site reputation abuse
2. When an Extension Domain is the right call (and when it isn't)
3. Information architecture and technical foundations, including the AI-crawler access stack and cryptographic bot verification
4. Content depth and topical authority strategy, grounded in controlled-experiment evidence
5. Design and UX standards that must be met
6. Optimizing for AI answer engines and LLM-mediated search specifically, using an evidence-weighted factor framework
7. Governance, disclosure, and risk management
8. The 2026 regulatory and enforcement landscape
9. Measurement framework

1. The Core Distinction

The dividing line isn't "how many domains you operate." It's what generates the ranking or citation signal.

There are, in practice, three patterns worth distinguishing — not two. Most guidance stops at contrasting a legitimate Extension Domain with a manufactured-consensus network. That comparison is necessary but incomplete, because it only covers one direction of the underlying failure: building new, low-value properties that all point back to (or duplicate) the same claim. The opposite direction — borrowing authority that a single, already-trusted domain earned, by loading it with third-party content that trades on that trust — is exactly what Google's site reputation abuse policy targets, and it is worth naming here because the same underlying test (independent merit vs. borrowed signal) applies in both directions.

Extension Domain (legitimate)	Manufactured-consensus network	Site reputation abuse ("parasite SEO")
Independent merit: original research, superior UX, genuine external citations	Manufactured agreement between properties you control	Third-party content riding on a host site's already-earned ranking signals
None — a new property earns its own trust	Outward: many controlled domains simulate consensus around one claim	Inward: third-party content borrows one trusted domain's authority
Disclosed or discoverable brand relationship; cross-linking is transparent	Relationship hidden; designed to look like independent corroboration	Often a licensing or revenue-share arrangement, disclosed or not
Deeper, better, or differently-targeted than the main domain could support	Often thin, repurposed, or designed purely to pad document counts for a query cluster	Frequently produced by a third party with editorial and topical distance from the host's core subject
None — it's just a well-run second property	Reputational and platform-level penalties; in AI-search contexts, this is information manipulation, not SEO	Manual action against the host domain itself — the specific pages, and sometimes the section, lose ranking ability

Google formalized site reputation abuse as a distinct spam category in March 2024 and, in November 2024, sharpened the policy in a way every Extension Domain operator should internalize: first-party editorial oversight does not cure it. If the primary purpose of third-party content hosted on a domain is to exploit that domain's already-established ranking signals, disclosed editorial involvement does not exempt it. This closes the obvious loophole — "we reviewed it before publishing" — and reframes the real test as one of intent and value, not process. It's the same test this paper applies to Extension Domains, run in the opposite direction: a domain (new or established) has to independently deserve the signal it receives.

If you can't describe, in one sentence, what independent audience need or query intent your Extension Domain serves — independent of wanting to "show up more" — you're likely building the wrong kind of asset, regardless of which direction the borrowed authority would flow.

2. When an Extension Domain Is (and Isn't) the Right Call

Good reasons to launch one:

• A distinct audience segment with different intent, vocabulary, or trust expectations than your core domain's audience (e.g., a developer-tools company launching a consumer-education property)
• A content format your core CMS/IA genuinely can't support well (interactive calculators, large reference libraries, UGC/community features)
• A regulatory, editorial, or brand-safety reason to separate certain content (e.g., a financial brand separating licensed advisory content from general commentary)
• Geographic or language-market separation that benefits from local hosting, ccTLD signals, and market-specific editorial voice
• A genuinely distinct content architecture required for AEO/GEO performance in a query space — for example, a reference property built around dense, self-contained, directly-answerable definitional content that would sit awkwardly inside a core site built for narrative or sales-oriented content

Bad reasons (signals you should fix the main domain instead):

• "We can't rank for X on the main domain" with no deeper diagnosis of why — often this is a content-quality or IA problem, not a domain problem
• Wanting more total properties appearing in a result set or AI-generated answer for the same claim
• Avoiding the work of building genuine authority by instead building volume
• Wanting more surface area specifically to influence what an AI answer engine synthesizes or cites — as of May 2026, this sits inside Google's own definition of search spam (see Section 8), not a gray area

3. Information Architecture and Technical Foundations (SEO + AEO + GEO)

An Extension Domain should be engineered with at least the same technical rigor as a flagship property — the technical bar has, if anything, gone up, because it now has to satisfy traditional crawlers, a dozen-plus distinct AI crawlers, and (increasingly) cryptographic identity checks simultaneously.

Domain and hosting

• Use a dedicated, purpose-fit domain name — avoid names engineered purely to contain exact-match keywords with no brand logic; modern ranking and LLM-training-adjacent systems both discount this.
• Independent, fast, properly-configured hosting. Core Web Vitals matter as much here as on the primary domain.
• Clear robots.txt, XML sitemaps, and canonical tag discipline from day one.

For detailed implementation, see HGM's Technical SEO Architecture and Website Speed Optimization Guide.

Structured data

• Full schema.org markup appropriate to content type: Article, FAQPage, HowTo, Product, Organization, BreadcrumbList as relevant.
• sameAs and Organization schema that transparently links back to the parent brand where a relationship exists — this is the opposite of hiding common ownership; disclosed affiliation is a trust signal, not a liability.
• Author markup (Person schema, bylines, credentials) — increasingly important as both search engines and LLM-based systems weight demonstrable expertise (E-E-A-T).
• JSON-LD must be present in the raw server response, not only in the rendered DOM.

See HGM's Schema Markup Guide for implementation-level detail across content types.

Crawlability for traditional and AI systems

The crawler landscape an Extension Domain has to satisfy has gotten materially more complicated since this paper's first edition. It's no longer "Googlebot and everything else." As of 2026, an active enterprise site can expect crawl traffic from eleven or more distinct AI agents in a given month, each with a different purpose, a different rendering capability, and its own robots.txt directive.

The distinction that matters most: training crawlers vs. inference crawlers.

Training crawlers	Inference crawlers
Scheduled batch run	Live user query
LLM training dataset → model weights	RAG retrieval → single query response
Days to weeks	Real-time, on-demand
Indirect — shapes model knowledge	Direct — URL shown to the user
Almost never (Google-Extended is the notable exception)	Rarely, with partial exceptions
GPTBot, ClaudeBot / anthropic-ai, Meta-ExternalAgent	OAI-SearchBot, PerplexityBot, YouBot

Google-Extended is worth calling out on its own: it is entirely separate from Googlebot, governs only Gemini and AI Overviews inclusion (blocking it has zero effect on ordinary Google Search rankings), and is the only major LLM crawler with confirmed full JavaScript rendering capability. Every other major training crawler — GPTBot, ClaudeBot, Meta-ExternalAgent — fetches raw HTML only. A page that Google Search Console shows as fully indexed is not evidence that these crawlers can read it; a client-side-rendered React or Vue page can be completely invisible to them even while it ranks normally in Google. Server-side rendering or static generation for GEO-priority pages isn't optional if inclusion in non-Google model training matters to the brand.

For up-to-date crawler profiles, see HGM's LLMs Crawler Handbook and AI Crawler Budgets guide.

The three-file AI access stack. Comprehensive crawler management now runs through three distinct files, each doing a different job:

• robots.txt — access control. Permits or denies specific crawlers at the path level. Voluntary; enforcement is honor-system plus IP blocking as a fallback.
• llms.txt — content-relevance signaling. A Markdown index at the domain root pointing AI agents to a site's highest-value, most LLM-readable pages, each with a one-line description of what it covers. As of 2026, adoption is confirmed at Perplexity AI and Anthropic; OpenAI and Google have acknowledged the format without a formal adoption commitment.
• AI-priority sitemap — crawl discovery with freshness metadata (<lastmod>), feeding the pre-computed vector indexes that hybrid systems like Google AI Overviews increasingly rely on rather than crawling live per query.

A caveat worth stating plainly, because llms.txt gets disproportionate attention relative to its measured impact: in Cyrus Shepard's May 2026 evidence-weighted analysis of AI citation factors (covered fully in Section 6), llms.txt scored 2.0 out of 10 — the lowest of 23 measured factors. It's worth implementing anyway, since it's close to a zero-risk, non-zero-return action and costs an afternoon of engineering time — but it shouldn't come at the expense of the much higher-evidence work covered in Section 6.

HGM publishes its own llms.txt as a live, working example of the format.

Cryptographic crawler verification: the layer beyond robots.txt. robots.txt is a request, not a proof — any scraper can claim to be GPTBot or ClaudeBot in its user-agent string, and plenty do. The emerging answer is Web Bot Auth, a Cloudflare-led protocol now under formal standardization: the IETF chartered a dedicated WebBotAuth working group in early 2026, building on RFC 9421 (HTTP Message Signatures). Under the standard, each bot operator generates an Ed25519 keypair, publishes the public half at a well-known JWKS directory it controls, and cryptographically signs every outbound request with a Signature-Agent header. A site can then verify the signature instead of trusting a spoofable string — cryptographic certainty in place of an honor system.

Adoption is still uneven — Cloudflare, Amazon, Akamai, OpenAI, Anthropic, and Perplexity all support it in some form, and it has been adopted as the authentication layer underneath emerging agentic-commerce protocols (Visa's and Mastercard's agent-payment standards both build on it), but Google's core indexing crawler was still unsigned as of mid-2026, even as its separate AI-browsing agent had begun publishing keys. For an Extension Domain built to last past 2026, it's worth tracking now: it will eventually be the difference between confidently granting an actual AI agent elevated access and unknowingly handing that same access to anything that copies its user-agent string.

• Keep canonical URLs stable — AI systems that cache or periodically re-crawl content penalize churn.

4. Content Depth and Topical Authority (SEO + AEO + GEO)

This is where "deserves stronger content" becomes concrete practice — and it's the section with the clearest controlled-experiment evidence behind it.

Topical mapping, not keyword lists. Build a full topic graph for the query space you're entering — the entities, subtopics, and relationships a genuine subject-matter expert would consider necessary to "cover" the space. Tools: entity extraction from top-performing pages, "People Also Ask" mining, community forums (Reddit, specialized forums) for real user vocabulary and unanswered questions, and query-level data pulled directly from Search Console.

For building this pipeline, see HGM's Search Console API & Apps Script guide.

Depth over breadth. A smaller number of comprehensive, well-maintained pages consistently outperforms a large number of thin pages — both in traditional rankings and in likelihood of being the source an LLM synthesizes from or cites.

Original contribution, backed by controlled evidence. The single highest-leverage thing an Extension Domain can do: contribute something that does not already exist elsewhere — original data, original research, a proprietary framework, first-hand case studies, or expert interviews. This isn't just intuition. The 2024 Princeton study that coined the term "Generative Engine Optimization" tested nine distinct content interventions across roughly 10,000 queries in a controlled benchmark (GEO-bench). The headline results are directly relevant to how an Extension Domain should spend its editorial effort:

• Keyword stuffing — the classic SEO-era tactic — produced negligible or even negative effects on AI citation likelihood.
• Adding statistics, citing sources, adding expert quotations, and fluency optimization were the strongest performers, each improving visibility by roughly 30–40% relative to an unoptimized baseline.
• The single most striking finding — call it the equalizer effect — was specific to citations: pages that started in a low-visibility position (the study used position 5) saw the largest relative gain from adding citations, on the order of a 115% visibility increase. Sources that already ranked at the top saw comparatively little additional lift.

That equalizer effect is the empirical case for this entire paper's thesis. An Extension Domain, by definition, starts without the years of accumulated authority its established competitors have. The Princeton findings suggest that's exactly the situation in which genuine citation density — original data, cited sources, expert quotation — produces the largest relative payoff. The lever with the most leverage is most available to the newest, least-established property in the query space, provided the content is genuinely substantive rather than performatively citation-heavy.

Freshness with integrity. Update content when the underlying facts change, not on an arbitrary schedule designed to appear "active." Superficial timestamp updates without substantive changes are increasingly detectable and discounted.

Editorial standards. Named authors with real credentials, visible editorial policy, correction/update logs, and citations to primary sources. These are exactly the signals both human readers and AI-answer systems use to weight a source's reliability — and they're also, not coincidentally, the exact signals Google's spam policies use in the opposite direction to distinguish genuine content from site reputation abuse.

HGM's AI SEO Agency work centers on building the entity and knowledge-graph completeness that makes an original contribution legible to both traditional and generative systems.

5. Design and UX Standards

"Better design" should mean measurable UX quality, not just aesthetics:

• Performance: sub-2.5s LCP, minimal CLS, fast TTFB — non-negotiable given both user experience and ranking impact. This threshold has gotten stricter in practice, not looser: inference crawlers like OpenAI's OAI-SearchBot treat server response time as a hard cutoff rather than a ranking input — a slow time-to-first-byte on a live, query-triggered fetch means the crawler abandons the request and the AI engine simply cites the next available source instead. A slow Extension Domain isn't just ranked lower; on inference-crawled surfaces, it can be functionally invisible.
• Readability: clear typographic hierarchy, scannable structure (headers, summaries, tables), appropriate for how people actually read technical or educational content (F-pattern scanning, summary-first structures).
• Accessibility: WCAG 2.2 AA as a baseline — this is both an ethical baseline and a ranking-adjacent signal.
• Navigation clarity: a genuinely useful IA — topic hubs, clear breadcrumbs, related-content modules driven by actual topical relationships, not just internal-linking-for-SEO patterns.
• Mobile-first execution, since most discovery — including AI-assistant-mediated discovery on mobile — happens off-desktop.

Good design, in this context, is itself a trust signal: it communicates that a real team maintains the property, which is one of the implicit heuristics both users and AI-summarization systems use when weighing source credibility.

For specific engineering guidance, see HGM's Website Speed Optimization Guide.

6. Optimizing Specifically for AI Answer Engines and LLM-Mediated Search (AEO + GEO)

This is the newer discipline layered on top of traditional SEO. Some of what applies transfers directly; some is distinct — and as of 2026, the "distinct" part is no longer a matter of opinion. It's been measured.

What transfers, with evidence behind it. Topical authority, E-E-A-T signals, structured data, and page performance all still matter — LLM-based answer systems (whether retrieval-augmented chat products or AI Overviews) still rely heavily on traditional ranking signals to select what to retrieve and cite in the first place. This isn't just an assertion: large-scale correlation research from Ahrefs found that 38% of AI Overview citations come directly from a domain's top-10 Google ranking, with the overlap continuing to increase for pages ranked beyond position 10. Winning traditional search is still most of the way to winning AI citation.

An evidence-weighted view of what's different. In May 2026, Cyrus Shepard (founder of Zyppy SEO) published the first attempt to move GEO advice from opinion to evidence: a meta-analysis synthesizing 54 published experiments, patents, and case studies into 23 scored ranking factors, each rated on repeatability across studies, strength of evidence, and official platform documentation or patent support. The scored list is a genuinely useful prioritization tool for an Extension Domain deciding where to spend engineering time:

The practical read: extractability and query-answer match are doing more work than any AI-specific technical file. The original discipline this paper describes — answer-first structure, self-contained paragraphs, genuine citation density — scores far better on the evidence than the AI-specific plumbing (llms.txt, in particular) that gets a disproportionate share of attention in GEO commentary.

Extractability over persuasiveness. LLMs summarizing a page favor content with clear, self-contained, directly-answerable statements — a well-formed paragraph that fully answers "what is X" or "how does X work" is more likely to be extracted in spirit than content that builds an argument across many paragraphs.

Answer-first structure. Lead sections with a direct answer, then expand with nuance, caveats, and supporting detail. This mirrors how LLMs are trained to synthesize: the direct claim first, context after.

Genuine citation density (not manufactured). Content that itself cites primary, authoritative, independent sources is more likely to be treated by an LLM as reliable and worth surfacing — the opposite of trying to be one of many manufactured "corroborating" sources. This is also, as of May 2026, an explicit line in Google's own spam policy — covered fully in Section 8.

Consistency across the open web, achieved honestly. LLM training and retrieval systems do pick up on whether a claim is corroborated across multiple independent domains. The legitimate way to earn this is to produce work good enough that independent third parties choose to cite it — press coverage, academic citation, backlinks from unaffiliated authoritative sites — not to simulate that corroboration internally.

Monitor AI-specific crawlers and citations directly. Track referrer traffic and mentions from AI Overviews, Perplexity, ChatGPT search, and similar surfaces; the business case for doing this is concrete — Seer Interactive's research found that being cited in Google's AI Overviews correlates with roughly 120% more organic clicks per impression and a 41% increase in paid clicks compared to queries where a brand isn't cited.

For tracking setup, see HGM's AI Visibility & Monitoring resource and the Search Intelligence Hub platform's Unified AI Search Visibility Score.

For implementation support, see HGM's Answer Engine Optimization and Generative Engine Optimization practices.

7. Governance, Disclosure, and Risk Management

• Disclose brand relationships. Footer/about-page disclosure of common ownership between an Extension Domain and its parent brand protects against both platform penalties and reputational risk if the relationship is later "discovered" and looks concealed.
• Confirm the value flows the right direction. Section 1 drew the distinction between an Extension Domain and site reputation abuse; in practice, that means periodically re-answering one question honestly: is this property earning its own signal, or is it (in either direction) trading on signal it hasn't earned? Google's clarification that disclosed first-party oversight does not cure site reputation abuse if the underlying intent is exploitative means "we disclosed it" is necessary but not sufficient.
• Avoid identical or near-identical content across domains. If content must appear on both properties, canonicalize clearly and don't rely on spun/rewritten duplicates — this is a hallmark of the pattern you want to avoid, and both search engines and LLM-training pipelines increasingly detect near-duplicate content clusters.
• Set a real editorial bar and budget. An Extension Domain that isn't resourced to actually be better than the main domain in its niche will underperform and can dilute brand trust. Treat it as a genuine product, not a satellite.
• Monitor how AI systems represent the brand, not just the content. This is a newer duty than the others on this list, and Section 8 covers why: a German court has now held an AI provider directly liable for a false AI-generated claim about a business, on the theory that the generated answer is the provider's own statement rather than a passive reflection of search results. That cuts two ways for an Extension Domain operator — it's a reason for real optimism that AI providers have sharper incentive to get citations right, and it's a reason to actively monitor for AI-generated misrepresentation rather than assuming no news is good news.
• Sunset discipline. If the domain stops serving a genuine, independent audience need, retire or consolidate it rather than let it become a low-value legacy property.

HGM's AI Governance resource covers building this into a repeatable internal review process rather than an ad hoc one.

8. The 2026 Regulatory and Enforcement Landscape

Everything above has gotten measurably higher-stakes in 2026. Three developments, taken together, mean the difference between a genuine Extension Domain and a manipulative one is no longer just a best-practice distinction — it's increasingly a legal and policy one.

Google's spam policy now names AI-answer manipulation directly

On May 15, 2026, Google revised the foundational definition in its Search spam policies. The previous language defined spam as techniques to deceive users or manipulate Google's systems into ranking content highly. The revised definition adds a second, parallel clause: attempting to manipulate the generative AI responses Google shows in Search is now spam under the same policy, full stop. That folds AI Overviews and AI Mode into every existing spam category — site reputation abuse, scaled content abuse, link spam, cloaking — rather than leaving AI-answer manipulation as an unaddressed gray zone. Industry reporting has flagged tactics like biased listicles and "recommendation poisoning" (content engineered to bias which option an AI engine recommends) as newly-explicit risk areas, with enforcement running through the same automated and manual spam-action pipeline Google already uses, up to and including deindexing.

A separate, unrelated systems update to Google's SpamBrain detection infrastructure rolled out June 24–26, 2026; Google confirmed that update specifically does not target site reputation abuse and introduces no new spam categories — it's worth distinguishing the two events, since only the May 15 change is a change in policy scope.

The practical takeaway for this paper's thesis: the "genuine citation density, not manufactured" principle in Section 6 was already the right approach on the merits. As of May 2026, it's also the only approach that isn't a direct, named policy violation.

The EU's ongoing site reputation abuse investigation

Separately, the European Commission opened a formal Digital Markets Act investigation on November 13, 2025, focused specifically on how Google's site reputation abuse policy is applied to news publishers. The regulatory concern is narrower than it might sound: the Commission is examining whether the policy, in practice, penalizes a "common and legitimate way" for publishers to monetize their sites — sponsored content and branded partner hubs — rather than only catching genuine manipulation, which could raise fair-treatment questions under the DMA's gatekeeper obligations.

Google has defended the policy as necessary to protect search quality and has argued more broadly that DMA compliance changes have degraded the product for European users and businesses. Google submitted a remedies proposal on May 6, 2026, adjusting how the policy applies to news domains; the Commission's spokesperson publicly characterized that offer as insufficient two days later. As of this writing, the matter remains open, with no publicly communicated resolution date. It's a useful reminder that even the "obviously bad" side of this paper's core distinction — trading on borrowed authority — is not a fully settled question at the policy level; where regulators ultimately draw the line for legitimate commercial content on publisher sites is still being actively contested.

A German court has held an AI provider liable for what its AI said

On May 28, 2026, the Regional Court of Munich I (case no. 26 O 869/26) issued a ruling with real relevance to any brand that depends on being accurately represented in AI-generated answers. Two Munich-based publishing companies had been falsely linked, in Google's AI Overviews, to scams and subscription traps — connections that did not appear in any of the sources the AI cited. The court's reasoning is the significant part: it held that an AI Overview constitutes Google's own "independent, new, and substantive statement," not a passive reflection of third-party search results, and that the liability shield German courts have historically extended to search engines (liable only once notified of an obvious rights violation) therefore does not apply. Google is appealing and has characterized the underlying errors as narrow rather than systemic; the ruling is not yet final. It is nonetheless considered a plausible first-of-its-kind precedent for generative AI answer liability more broadly, not just for Google specifically.

For an Extension Domain — or any brand — the relevance runs in two directions. First, as AI providers face growing legal exposure for what their systems say, they have a sharper institutional incentive to ground generated answers in verifiable, well-sourced, checkable content, which structurally rewards exactly the editorial rigor described in Sections 4 and 6. Second, brand monitoring now has a genuine (if early-stage) legal dimension alongside the reputational one: knowing how AI systems describe your Extension Domain and parent brand is no longer purely a marketing concern.

None of this changes the paper's underlying thesis. It raises the cost of getting it wrong, and the reward for getting it right.

9. Measurement Framework

Metric category	Examples
Authority	Referring domains (unique, unaffiliated), citations in press/academic sources, brand mentions off-domain
AI visibility	Presence/attribution in AI Overviews, Perplexity, ChatGPT/Claude search answers; "share of AI voice" tools
Engagement quality	Time on page, scroll depth, return visits — proxies for genuine usefulness
Technical health	Core Web Vitals, crawl error rate, indexation ratio, AI-crawler access logs
Business outcome	Assisted conversions attributable to the extension domain, audience segment growth independent of the core domain; externally, Seer Interactive's research associates AI Overview citation with roughly 120% more organic clicks per impression and 41% more paid clicks as a benchmark for what's at stake

When prioritizing engineering backlog against this framework, weight effort by evidence strength rather than by how much attention a tactic gets. Shepard's 23-factor scoring from Section 6 is a useful input here directly: a well-supported factor scored 9 or above (URL accessibility, search rank, query-answer match) earns backlog priority ahead of a contested or weakly-evidenced one (structured data at 5.6, llms.txt at 2.0) — implement the low-scoring items because they're cheap, not because they're the priority.

Avoid vanity metrics like raw page count or raw domain count in a "network" — these correlate with the manipulative pattern, not with genuine value. HGM's Search Intelligence Hub is one example of a system built to track this full framework — technical health, entity strength, and citation probability — in one place rather than across a half-dozen disconnected tools.

Supplementary: The Entity-Knowledge Graph Intersection

Beyond the technical and content foundations, Extension Domains must operate within the broader entity-knowledge graph ecosystem that both search engines and LLMs use to understand topical relationships.

Google's Knowledge Graph and similar systems used by AI providers (including proprietary entity graphs at OpenAI, Anthropic, and Perplexity) rely on entity recognition and relationship mapping. An Extension Domain that aligns its content with recognized entities and their relationships earns stronger signals across all three disciplines.

Entity Density and Semantic Coherence

Recent research from the HGM Search Intelligence Hub indicates that pages with high entity density (a high ratio of known entities to total tokens) and strong semantic coherence (entities that naturally cluster around a central topic) are more likely to be cited in AI-generated answers. This is because LLM retrieval systems weight content that aligns with their internal knowledge graph representations.

For Extension Domains, this means:

• Explicit entity references: Use precise, canonical entity names. For example, "Search Engine Optimization" rather than "SEO" in first references, and link to authoritative definitions where possible.
• Relationship articulation: Explicitly state relationships between entities. "Google's AI Overviews rely on retrieval-augmented generation (RAG) systems" establishes the relationship between Google, AI Overviews, and RAG.
• Internal linking as relationship signaling: Link between semantically related pages on your Extension Domain to establish entity relationships for crawlers and AI systems.

Supplementary: AI Crawler Economics and Cost Management

As AI crawler traffic increases, the economics of hosting an Extension Domain shift. What was once a negligible cost can become a significant line item.

In 2026, enterprise sites report that AI crawler traffic can account for 15-30% of total server requests, with training crawlers (GPTBot, ClaudeBot) performing deep, expensive crawls that consume significant bandwidth and compute resources. Inference crawlers (OAI-SearchBot, PerplexityBot) are less frequent but require low-latency responses to qualify for citation inclusion.

Cost-Aware Crawler Management

• Rate limiting: Implement per-crawler rate limits using the Crawl-Delay directive in robots.txt or server-level rate limiting. Balance accessibility with cost control.
• Cache strategy: Serve cached responses to training crawlers where freshness is less critical. Inference crawlers require real-time responses but can be served from edge caches.
• Selective access: Use llms.txt to guide AI crawlers to high-value pages, reducing the surface area they need to crawl for adequate coverage.
• Bot verification: Implement Web Bot Auth to differentiate legitimate AI crawlers from scrapers that mimic them, preventing unnecessary traffic from unauthorized sources.

For detailed guidance, see HGM's AI Crawler Budgets guide.

Supplementary: Future-Proofing Your Extension Domain

The 2026 landscape is not static. Several emerging trends will shape how Extension Domains are evaluated in the coming years.

Agentic AI and Autonomous Search

The rise of agentic AI systems — autonomous agents that perform multi-step tasks — will change how content is discovered and cited. These agents may conduct preliminary research, synthesize findings, and execute actions based on content they retrieve. Extension Domains that provide structured, verifiable, and actionable content will be favored.

Semantic Fingerprinting

Emerging semantic fingerprinting technologies allow AI systems to identify content sources based on unique linguistic patterns rather than just URL or domain. This means an Extension Domain that develops a distinctive editorial voice may be more consistently identified and cited across different AI platforms.

Cryptographic Attestation

Beyond Web Bot Auth, broader cryptographic attestation frameworks are emerging. These allow content publishers to certify the provenance and integrity of their content, making it easier for AI systems to trust and cite Extension Domain content with confidence.

Conclusion

The sustainable version of a multi-domain content strategy is the one that treats each Extension Domain as an independently deserving product: its own audience thesis, its own editorial standard, its own reason to exist that isn't just "more surface area." Done this way, an Extension Domain earns exactly the kind of real, independent external corroboration that a manufactured domain network — or a site trading on borrowed authority in the other direction — can only simulate. It holds up whether the entity evaluating it is a human reader, a search engine's ranking system, or an LLM synthesizing an answer from across the web — and, increasingly, whether the entity evaluating it is a regulator or a court.

That's a higher bar than it was when this paper was first written. It's also, on the evidence in Section 4, a bar that rewards a genuinely new, well-built property more than it rewards an entrenched one coasting on accumulated authority. Build for that.

Frequently Asked Questions