Markdown for AI agents
Every page, as clean Markdown an AI can read.
Markdown has become the lingua franca for AI agents. The Seogence Knowledge Layer gives each of your pages a clean, structured Markdown copy — served at a direct URL — so agents like ChatGPT, Claude, Gemini and Perplexity can read your content without parsing HTML or executing JavaScript.
Building an integration? See the Knowledge Layer reference. Wondering how this compares to Cloudflare? See the full comparison.
What it is
A second, machine-readable copy of every page.
Your site is built for human eyes — layout, scripts, styling. An AI agent has to dig the meaning out of all of it. The Knowledge Layer does the translation once: every page gets a clean Markdown document (with its schema.org structured data embedded) at a direct URL. Same content, a readable version underneath — what a screen reader did for people, for the machines that now read the web.
How agents access it
Four URL patterns, one content-negotiation header.
/KnowledgeLayer/{domainKey}/{contentHash}.mdThe canonical, content-addressed Markdown for a specific page snapshot./KnowledgeLayer/{domainKey}/by-url?u=…The page-stable resolver — maps any page URL to its latest Markdown, no content hash needed. Add&redirect=1for a 302 to the cacheable.mdURL./KnowledgeLayer/{domainKey}/llms.txtThe per-domain index — one entry point listing every page as a Markdown link. New to the format? See What is llms.txt./KnowledgeLayer/{domainKey}/organization.mdThe bundle’s typed organization node — who is behind the site (name, website, contact, official profiles), per Google’s Open Knowledge Format (v0.1 draft).
The resolver honours Accept: text/markdown (and text/*, */*, or no preference). A request that explicitly excludes Markdown — e.g. Accept: application/json — gets 406 Not Acceptable.
# Resolve any page URL to its clean Markdown (content negotiation)
curl "https://seogence.com/KnowledgeLayer/yoursite.com/by-url?u=https://yoursite.com/product" \
-H "Accept: text/markdown"# Discover everything the site exposes, as one index
curl "https://seogence.com/KnowledgeLayer/yoursite.com/llms.txt"# Prefer a cacheable, content-addressed URL? Ask for a redirect
curl -I "https://seogence.com/KnowledgeLayer/yoursite.com/by-url?u=https://yoursite.com/product&redirect=1"
# -> 302 Location: /KnowledgeLayer/yoursite.com/9f8e2c....mdOutput format
Frontmatter, clean body, and your structured data — inline.
Each document opens with YAML frontmatter (title, url, description, image…) plus typed OKF keys (type, okf_version, resource, timestamp, schema_type), then the page body converted to Markdown with navigation, scripts and styling stripped, then your schema.org JSON-LD preserved verbatim in a fenced block.
---
title: "Luxury Decor Bowl"
url: "https://yoursite.com/product"
description: "Handmade in Sweden."
image: "https://yoursite.com/bowl.jpg"
type: product
okf_version: "0.1"
resource: "https://yoursite.com/product"
timestamp: 2026-06-28T09:14:03.000Z
schema_type: "Product"
---
# Luxury Decor Bowl
Handmade luxury-decor bowls, finished by hand. Food-safe, one of a kind.
## Structured data
```json
{
"@context": "https://schema.org",
"@type": "Product",
"name": "Luxury Decor Bowl",
"offers": { "@type": "Offer", "price": "24.00", "priceCurrency": "USD" }
}
```
---
*Structured content by [Seogence](https://seogence.com).*Now an OKF bundle
Typed for what reads the web next.
Your Knowledge Layer is now an OKF bundle — conformant with Google’s Open Knowledge Format (v0.1 draft). Every page’s frontmatter carries a required type, and a typed organization.md node describes who is behind the site. To be plain about it: no public crawler consumes OKF bundles today, and Google says OKF has no effect on Search ranking — it is a format, not a platform, and it works alongside your schema.org JSON-LD and llms.txt with nothing replaced.
Response headers
Built for agents — with the token savings, quantified.
| Header | Value | Purpose |
|---|---|---|
Content-Type | text/markdown; charset=utf-8 | The markdown MIME type agents request. |
Access-Control-Allow-Origin | * | Permissive CORS so cross-origin agents can fetch it. |
Content-Signal | ai-train=yes, search=yes, ai-input=yes | Explicitly permits AI training, search, and agent input. |
x-markdown-tokens | 342 | Estimated tokens of the Markdown being served. |
x-original-tokens | 5188 | Estimated tokens of the original HTML — the savings, quantified. |
Cache-Control | public, max-age=3600 | Shared caching for one hour. |
X-Robots-Tag | noindex | On per-page markdown (it mirrors the live page); the llms.txt index stays indexable. |
x-markdown-tokens vs x-original-tokens lets a client measure the savings before it spends a single token. (Example values shown.)
Discovery
Found automatically — even without JavaScript.
The embed injects a <link rel="alternate"> on each page. For non-JS crawlers, paste one static tag into your <head> — it points at your llms.txt index and works on every page with no per-page templating.
<link rel="alternate" type="text/markdown"
href="https://seogence.com/KnowledgeLayer/yoursite.com/llms.txt"
title="Markdown for AI agents">Enable it
Off by default. One snippet, any host.
The Knowledge Layer is opt-in per domain — turn it on in your Seogence admin (Domains → markdown copy). Add the Seogence snippet once and it works on WordPress, Shopify, Wix, Next.js, React, or plain HTML. It re-reads itself when your content changes, by content hash, so the AI copy never goes stale.
Questions
Common questions.
How do AI agents read my site as Markdown?
They fetch the page-stable resolver or the per-domain llms.txt index. The resolver maps any page URL to its clean Markdown, and honours the Accept: text/markdown content-negotiation header.
Do I have to change my pages?
No. You add the Seogence snippet once. The Knowledge Layer generates a clean Markdown copy of each page as visitors open it — or of every page at once if you import your sitemap — and keeps it current as your content changes.
Does it work without the agent executing JavaScript?
Yes. The Markdown and llms.txt are served directly at a URL — no HTML to parse, no JavaScript to run. Paste the discovery <link> into your <head> so non-JS crawlers find the index too.
What is the Open Knowledge Format (OKF)?
A draft format (v0.1) published by Google Cloud that represents knowledge as a bundle of typed Markdown documents. The Knowledge Layer conforms: every page carries typed OKF frontmatter and the bundle includes an organization.md node. To be clear: Google says OKF has no effect on Search ranking, and no public crawler consumes OKF bundles today — it is forward-compatible AI-readiness that works alongside your JSON-LD and llms.txt, with nothing replaced.
Try it
See your page as Markdown — the way an AI reads it.
Enter any live page. Seogence analyzes it and shows the clean Markdown copy — with the structured data it would add — so you can see exactly what AI agents would consume.