Project LLMs.txt

Project LLMs.txt helps AI interpret your site more accurately. It implements the emerging llms.txt standard, the AI-era equivalent of robots.txt, and publishes the structured Markdown files that ChatGPT, Claude, Perplexity, Gemini, and other AI assistants look for when they access your site.

Visitors read HTML. AI assistants read Markdown. Project LLMs.txt bridges the gap.

What It Does

Project LLMs.txt scans the content you’ve already written and produces three things:

1. /llms.txt – a clean, structured index at your site root listing every page, post, and product with a short description, formatted per the llms.txt spec.
2. /llms-full.txt – the same index with the full Markdown body of every page inlined after it, so assistants have the complete context in one fetch.
3. /llms/slug.md (Pro) – per-page Markdown files so AI assistants can retrieve a single page without pulling the whole corpus.

Everything is generated from your existing content. No re-writing, no rebuilding.

Why It Matters

Markdown is the format AI reads most cleanly. Most websites are a tangle of HTML with navigation, ads, cookie banners, page-builder wrappers, etc., that AI tools have to fight through to find the actual content. When an AI agent or assistant visits your site and finds a clean /llms.txt, it gets a structured index of everything on your site in one fetch, already in the format it reads best. Whether it came looking for the file or simply landed there, the result is the same: a complete, accurate picture of your site instead of a pile of noisy markup.

The llms.txt standard is still early, as no major search engine has announced official support yet, but a growing ecosystem of AI agents, coding assistants, and developer tooling already understands it. Companies like Stripe, Vercel, Cloudflare, and Anthropic publish their own llms.txt files themselves.

Publishing now is cheap insurance. The file costs nothing to generate and keeps your site ready as the ecosystem matures.

How It Works

1. Scans your posts, pages, products, and custom post types
2. Reads your published content (blocks, shortcodes, and page-builder output), with full-page scraping available for anything your theme renders outside the editor
3. When scraping, extracts the main content zone using a Readability-style algorithm
4. Converts that HTML to clean, structured Markdown
5. Assembles the results into /llms.txt, /llms-full.txt, and (Pro) per-page /llms/*.md files at your site root

Key Features

• Flexible content extraction – Reads your published content (blocks, shortcodes, and page-builder output like Divi and Visual Composer) by default, with an optional full-page scrape for sections your theme, or a builder like Elementor, renders outside the editor.
• WooCommerce first-class support – Products include structured price, SKU, stock status, variations, customer reviews, and related/upsell products.
• Clean, structured output – Built-in Markdown templates for every content type produce well-formatted output out of the box, with no setup. (Editing the templates is a Pro feature.)
• Physical file or live endpoint – Write actual files to your site root (fastest for crawlers) or serve dynamically via WordPress rewrite rules.
• Automatic backups – Before each regeneration, the previous llms.txt and llms-full.txt are saved (the five most recent are kept), so a bad generation never costs you a good file.
• Smart content selector – Auto-detects your main content zone. Override with a CSS selector for custom themes.
• WP-CLI integration – wp llmstxt generate, status, list, and clear for automation and CI pipelines.
• Respects your SEO settings – Pages marked noindex or nofollow in Yoast, Rank Math, SEOPress, AIOSEO, or Slim SEO are excluded from the public files automatically.
• Bulk-import friendly – Public regeneration hooks let WP All Import or any custom script rebuild your files after a bulk import or migration, so your llms.txt never drifts from your content.
• No tracking, no cookies – No third-party scripts, no analytics, no cookies. Nothing about your site or visitors is sent anywhere. Runs entirely on your server, GPL-licensed.

Pro Features

Pro is a separate plugin, distributed from projectllmstxt.com – none of its functionality is included (or locked) in this free plugin.

• Auto-regenerate on publish – Files rebuild in the background, usually within about a minute of saving content (with a 5-minute debounce for bulk edits).
• Custom templates per content type – Edit and save templates; bulk-regenerate all matching pages with one click.
• Individual .md files – Physical files at /llms/slug.md, served directly by Apache/Nginx.
• Post editor AI Preview – See and edit exactly what AI will read, right from the post editor.
• Per-page Markdown overrides – Hand-tune one page without touching the rest. Free users can preview each page’s Markdown; editing requires Pro.
• Custom field mapping – Pull ACF fields and any post meta into your output as {{acf_*}} and {{meta_*}} template variables. Sensitive keys (passwords, tokens, API keys) are blocked from output by default.
• Agency license – Unlimited sites.

A note on special characters

The generated files are UTF-8, which every AI assistant and crawler reads correctly. If accented letters or symbols look garbled when you open llms.txt directly in a browser, your web server is serving .txt files without a charset declaration – the file itself is fine. Ask your host to add AddDefaultCharset UTF-8, or switch to Live Endpoint mode, which always declares the charset.

Bundled assets

The admin interface bundles the Figtree and JetBrains Mono typefaces, both licensed under the SIL Open Font License 1.1. Nothing is loaded from third-party servers.

Links

• llms.txt standard
• Plugin homepage
• Documentation