chancerylaneproject

Markdown for Agents and Statistics converts your WordPress content to Markdown and serves it 
to AI agents and language model tools that request it via HTTP content negotiation 
(<code>Accept: text/markdown</code>).
The Chancery Lane Project is a charity that helps organisations reduce emissions using the power of legal documents and processes. We’ve published this plugin as we believe that making content more legible for AI Agents makes a meaningful difference to their energy usage – not only by reducing the amount of tokens required (by up to 90% over HTML) to consume the content, but also minimising the server resources required to render, process and display pages at source.
How it works:
<ol>
<li>Posts and taxonomy archive pages are converted to Markdown and saved as static 
files on disk inside <code>wp-content/uploads/</code>.</li>
<li>When a visitor (or AI agent) requests a page with <code>Accept: text/markdown</code> in 
the HTTP headers, WordPress serves the pre-generated <code>.md</code> file directly — 
no page render required.</li>
<li>A <code><link rel="alternate" type="text/markdown"></code> tag is added to each page’s 
 so agents can discover Markdown versions automatically.</li>
</ol>
Features:
<ul>
<li>Content negotiation (<code>Accept: text/markdown</code>, <code>?output_format=md</code>, or known AI User-Agents)</li>
<li>Taxonomy archive support — category, tag, and custom taxonomy term pages served as Markdown post listings</li>
<li>Automatic Markdown generation on post save; taxonomy archives auto-update when any post in the term changes</li>
<li>AJAX bulk generation with live progress counter — no page timeouts on large sites</li>
<li>Per-post-type field configuration — choose which meta/ACF fields go in frontmatter or body</li>
<li>ACF support with dot notation for nested group fields (e.g. <code>group.subfield</code>)</li>
<li>Content fields option — use ACF fields as the body content instead of post_content</li>
<li>Manifest generation with content hashes and change tracking per post type</li>
<li>Incremental export — only re-export changed documents (<code>--incremental</code>)</li>
<li>Delta file (<code>changes.json</code>) for RAG system sync</li>
<li>Access statistics — logs AI agent requests with a dedicated stats admin page</li>
<li>Optional frontmatter fields — hierarchy (parent/ancestors/children IDs), author display name, root-relative featured image paths</li>
<li>Topics section — appends a <code>## Topics</code> section with linked taxonomy terms to the Markdown body</li>
<li>Export preview — preview generated Markdown inline in the post editor without writing to disk</li>
<li>WP-CLI commands: <code>generate</code>, <code>generate-taxonomies</code>, <code>prune-stats</code>, <code>status</code>, <code>delete</code></li>
<li>Fully unit-tested</li>
</ul>

Markdown for Agents and Statistics

Serve pre-generated Markdown files to AI agents via HTTP content negotiation.

Selvakumar Duraipandian

Markdown for AI Agents is a lightweight WordPress plugin that enables HTTP content negotiation for your site’s content. When a client (like an AI agent or a custom script) requests a page with the <code>Accept: text/markdown</code> header, the plugin intercepts the request and returns a clean, structured Markdown representation of the post or page content.
This is ideal for AI crawlers, RAG (Retrieval-Augmented Generation) systems, and non-browser clients that prefer machine-friendly text over complex HTML.
Important note: This plugin is primarily a developer/integration tool. Human visitors browsing your site will never see any difference — the Markdown output is only served when explicitly requested via the <code>Accept: text/markdown</code> HTTP header. Normal browser requests always receive the standard HTML page.
Key Features:
<ul>
<li>Automatically detects <code>Accept: text/markdown</code> headers.</li>
<li>Converts HTML content to clean Markdown using the League HTMLToMarkdown library.</li>
<li>Strips away theme layout, navigation, headers, footers, and sidebars — serving only the main content.</li>
<li>Adds useful HTTP response headers: <code>Content-Type: text/markdown</code>, <code>Vary: Accept</code>, and <code>X-Markdown-Word-Count</code>.</li>
<li>Respects WordPress visibility rules and filters.</li>
<li>No configuration required — works out of the box for posts, pages, and custom post types.</li>
</ul>
<h3>How It Works</h3>
This plugin uses a standard web technique called HTTP content negotiation. The same URL on your site can serve different representations of the same content depending on what the client asks for:
<ul>
<li>A regular browser sends <code>Accept: text/html</code> → receives your normal HTML page.</li>
<li>An AI agent sends <code>Accept: text/markdown</code> → receives a clean Markdown version of the same page.</li>
</ul>
No extra URLs, no duplicate content, no configuration needed. The plugin hooks into WordPress’s <code>template_redirect</code> action, detects the <code>Accept</code> header, captures the rendered HTML, converts it to Markdown, and returns it with appropriate headers.
<h3>Why Markdown for AI Agents?</h3>
When building RAG (Retrieval-Augmented Generation) applications or AI pipelines that ingest web content, HTML is extremely noisy. A typical WordPress page contains thousands of tokens worth of HTML tags, inline styles, navigation menus, scripts, and layout markup — none of which carries meaning for an AI model.
Serving clean Markdown instead can reduce token consumption by up to 60%, which means:
<ul>
<li>Lower API costs — fewer tokens ingested when loading pages into vector stores or LLM pipelines.</li>
<li>Faster processing — less text for the model to parse, filter, and discard.</li>
<li>Better retrieval accuracy — higher signal-to-noise ratio improves the quality of RAG results.</li>
<li>Simpler pipelines — no need for custom HTML stripping logic on the client side; the plugin handles it server-side.</li>
</ul>
Any AI agent, crawler, or ingestion script that sends <code>Accept: text/markdown</code> in its request header will automatically receive the clean Markdown version — no extra URLs, no separate endpoints, no changes to your content workflow.

Markdown for AI Agents

Serve clean Markdown versions of WordPress content to AI agents using HTTP content negotiation.

Markdown for Agents and Statistics vs Markdown for AI Agents