Hamza Mairaj

BerqWP

Berq Markdown makes your WordPress content instantly consumable by AI assistants, LLM crawlers (GPTBot, ClaudeBot, Googlebot-AI), and Markdown-aware tools — without duplicating your content or creating separate pages.
Appending <code>.md</code> to any public page or post URL (e.g. <code>https://example.com/about.md</code>) returns a clean, structured Markdown file with YAML frontmatter. The plugin handles content from all common sources: Gutenberg blocks, classic editor, shortcodes, and custom PHP templates.
How it works
When a visitor or crawler requests <code>/about.md</code>, the plugin intercepts the request, finds the matching WordPress post, converts the HTML content to Markdown, and serves it with the correct <code>Content-Type: text/markdown</code> header — all without creating any new database entries or files.
Key Features
<ul>
<li>Automatic .md URLs — any public post or page URL + <code>.md</code> returns clean Markdown</li>
<li>YAML frontmatter — every .md file includes <code>title</code>, <code>description</code>, <code>url</code>, <code>date_modified</code>, <code>author</code>, and <code>language</code></li>
<li>Full content compatibility — works with Gutenberg blocks, classic editor, shortcodes, and custom PHP page templates</li>
<li>HTML to Markdown conversion — pure PHP, no external dependencies; handles headings, paragraphs, lists, links, images, tables, code blocks, and more</li>
<li>Internal link rewriting — links to internal pages are rewritten to their <code>.md</code> equivalent automatically</li>
<li>URL exclusions with wildcards — exclude specific URL patterns using <code>*</code> as a wildcard (e.g. <code>/cart/*</code>, <code>*/checkout*</code>)</li>
<li>Post type selection — choose which post types (posts, pages, custom post types) are served as Markdown</li>
<li>Transient caching — optional cache with configurable TTL to reduce server load at scale</li>
<li><code><link></code> tag injection — injects a <code><link rel="alternate" type="text/markdown"></code> tag in every page’s <code><head></code> so crawlers can discover the .md version</li>
<li><code>[btrmd313_md_link]</code> shortcode — renders a link to the current page’s <code>.md</code> file, opens in a new tab</li>
<li>Multilingual support — detects language via Polylang or WPML, adds <code>language</code> and <code>alternates</code> to frontmatter and emits <code>hreflang</code> link tags</li>
<li>Public content rendering — always renders content as a non-logged-in visitor, so member-only sections are not exposed</li>
<li>Admin settings page — all options configurable under Settings > Berq Markdown, with a Settings link on the Plugins page</li>
</ul>

Berq Markdown – Markdown Pages for AI & LLM Crawlers

Serve any WordPress page or post as a clean Markdown file optimized for AI and LLM crawlers by appending .md to the URL.

Selvakumar Duraipandian

Markdown for AI Agents is a lightweight WordPress plugin that enables HTTP content negotiation for your site’s content. When a client (like an AI agent or a custom script) requests a page with the <code>Accept: text/markdown</code> header, the plugin intercepts the request and returns a clean, structured Markdown representation of the post or page content.
This is ideal for AI crawlers, RAG (Retrieval-Augmented Generation) systems, and non-browser clients that prefer machine-friendly text over complex HTML.
Important note: This plugin is primarily a developer/integration tool. Human visitors browsing your site will never see any difference — the Markdown output is only served when explicitly requested via the <code>Accept: text/markdown</code> HTTP header. Normal browser requests always receive the standard HTML page.
Key Features:
<ul>
<li>Automatically detects <code>Accept: text/markdown</code> headers.</li>
<li>Converts HTML content to clean Markdown using the League HTMLToMarkdown library.</li>
<li>Strips away theme layout, navigation, headers, footers, and sidebars — serving only the main content.</li>
<li>Adds useful HTTP response headers: <code>Content-Type: text/markdown</code>, <code>Vary: Accept</code>, and <code>X-Markdown-Word-Count</code>.</li>
<li>Respects WordPress visibility rules and filters.</li>
<li>No configuration required — works out of the box for posts, pages, and custom post types.</li>
</ul>
<h3>How It Works</h3>
This plugin uses a standard web technique called HTTP content negotiation. The same URL on your site can serve different representations of the same content depending on what the client asks for:
<ul>
<li>A regular browser sends <code>Accept: text/html</code> → receives your normal HTML page.</li>
<li>An AI agent sends <code>Accept: text/markdown</code> → receives a clean Markdown version of the same page.</li>
</ul>
No extra URLs, no duplicate content, no configuration needed. The plugin hooks into WordPress’s <code>template_redirect</code> action, detects the <code>Accept</code> header, captures the rendered HTML, converts it to Markdown, and returns it with appropriate headers.
<h3>Why Markdown for AI Agents?</h3>
When building RAG (Retrieval-Augmented Generation) applications or AI pipelines that ingest web content, HTML is extremely noisy. A typical WordPress page contains thousands of tokens worth of HTML tags, inline styles, navigation menus, scripts, and layout markup — none of which carries meaning for an AI model.
Serving clean Markdown instead can reduce token consumption by up to 60%, which means:
<ul>
<li>Lower API costs — fewer tokens ingested when loading pages into vector stores or LLM pipelines.</li>
<li>Faster processing — less text for the model to parse, filter, and discard.</li>
<li>Better retrieval accuracy — higher signal-to-noise ratio improves the quality of RAG results.</li>
<li>Simpler pipelines — no need for custom HTML stripping logic on the client side; the plugin handles it server-side.</li>
</ul>
Any AI agent, crawler, or ingestion script that sends <code>Accept: text/markdown</code> in its request header will automatically receive the clean Markdown version — no extra URLs, no separate endpoints, no changes to your content workflow.

Markdown for AI Agents

Serve clean Markdown versions of WordPress content to AI agents using HTTP content negotiation.

Berq Markdown – Markdown Pages for AI & LLM Crawlers vs Markdown for AI Agents