Documentation

Knowledge Layer

A Markdown copy of each page a visitor opens, served over plain HTTP, no JavaScript required to read it — plus the mechanism that lets a crawler find it in the first place.

What is served#

Three kinds of document, all plain Markdown:

Per-page documents — one Markdown file per captured page, at a content-addressed URL: the same content always resolves to the same URL, and a URL resolver exists for callers that only know the page’s live address. These carry the full OKF frontmatter block described below and ship X-Robots-Tag: noindex.
organization.md — a typed node stating who is behind the site. Its frontmatter carries type, title, resource and okf_version.
llms.txt — the per-domain index, listing every known page as a Markdown link and pointing at the organization node. Pages only: a URL that still carries a query string once tracking parameters are stripped is application state, not a page, and is listed only if your own sitemap declared it. It is a plain Markdown document with no frontmatter block, and it is the one resource served without X-Robots-Tag: noindex, because it is the one we want found.

Per-page documents carry additive OKF (Open Knowledge Format) v0.1 frontmatter — a type mapped from the page’s schema.org type, a stable resource URL, and a capture timestamp. OKF is a draft format: no public web crawler consumes OKF bundles today, and it has no effect on search rankings. It works alongside schema.org JSON-LD and llms.txt — nothing is replaced.

Why discovery needs its own step#

Common AI crawlers — GPTBot, ClaudeBot and PerplexityBot among them — fetch raw HTML and do not execute JavaScript. A page that only advertises its Markdown copy through a script-injected tag is invisible to them. Two mechanisms close that gap, both delivered as plain HTML markup rather than anything JavaScript-dependent:

A static link tag, included alongside the <script> in install lines copied for domains with Markdown delivery enabled — present in the raw HTML of every page carrying that block, with no script execution required to see it. It is a sibling of the script tag, not part of it, and existing installs are not updated in place: if you pasted your line before enabling Markdown delivery, re-copy it, or paste the single link line next to the script you already have.
An optional /llms.txt redirect on your own domain, pointing at your Knowledge Layer index. This matters most for single-page-app hosts: a catch-all rewrite otherwise returns your app shell with a 200 for /llms.txt, which looks like a valid file to a crawler and is not one.

Serving /llms.txt from your own domain#

The convention expects the file at your site root. Point https://your-site.com/llms.txt at your Knowledge Layer index with a redirect — a 302 where your host offers one, since the target contains your identifier and a temporary redirect stays easy to change.

Host	Where
WordPress	The Redirection plugin (Tools → Redirection), or a `template_redirect` hook in your theme.
Squarespace	Developer tools → URL mappings, one line.
Cloudflare	Rules → Redirect Rules, matching `http.request.uri.path eq "/llms.txt"`.
Netlify	A line in `_redirects`.
Vercel	A `redirects` entry in `vercel.json` — set `statusCode: 302`, since the default is a permanent 308.
nginx / Apache	`location = /llms.txt { return 302 …; }`, or `Redirect temp` with mod_alias.
Shopify	Override the template, not a URL redirect — Shopify serves its own `/llms.txt`, and admin redirects only fire on URLs that would otherwise 404. Add `templates/llms.txt.liquid`.
Wix	Turn off the built-in llms.txt first (SEO → Privacy & Control), then use the URL Redirect Manager. Wix issues `301` only.
GitHub Pages	No server-side redirects exist. Commit a static `llms.txt` instead, and rely on the link tag for freshness.

Verifying it yourself#

# The static tag is in the raw HTML — no JS execution required:
curl -s https://your-site.com/ | grep 'rel="alternate" type="text/markdown"'

# The index it points at serves Markdown:
curl -s https://seogence.com/KnowledgeLayer/yoursite.com/llms.txt | head

What this does and does not claim#

Discoverability is not consumption. These mechanisms make the Knowledge Layer available to a crawler that chooses to fetch it; they cannot make any AI system fetch, ingest, or cite what it finds, and none of them affect search rankings.