Whizz People

<p><strong>Whizz MD</strong> automatically creates a <code>.md</code> version of every published post and page on your WordPress site, served dynamically without storing any files on disk.</p>
<p>AI-powered search engines and LLM crawlers (GPTBot, ClaudeBot, PerplexityBot, Googlebot) increasingly prefer clean, structured content. This plugin gives them exactly that — a Markdown file with rich YAML frontmatter containing all your SEO signals, served at a predictable URL like <code>/your-post-slug.md</code>.</p>
<h4>How It Works</h4>
<ol>
<li>Any published post or page automatically gets a <code>.md</code> endpoint</li>
<li>AI crawlers discover it via a <code><link rel="alternate"></code> tag in your page <code><head></code>, a dedicated <code>/md-sitemap.xml</code>, and crawlable anchor links in your page content</li>
<li>The Markdown is generated on the fly — no files written to disk, no database bloat</li>
<li>Full YAML frontmatter is included with all SEO metadata</li>
</ol>
<h4>What Gets Generated</h4>
<p><strong>YAML Frontmatter includes:</strong></p>
<ul>
<li><code>title</code> — SEO title (from Yoast, Rank Math, AIOSEO, or post title)</li>
<li><code>slug</code> — post slug</li>
<li><code>date</code> and <code>modified</code> — publication and last modified dates</li>
<li><code>author</code> — author display name</li>
<li><code>meta_description</code> — SEO meta description (from SEO plugin or auto-generated)</li>
<li><code>focus_keyword</code> — primary keyword (from Yoast/Rank Math)</li>
<li><code>canonical_url</code> — points back to your HTML page</li>
<li><code>og_title</code> and <code>og_description</code> — Open Graph fields</li>
<li><code>og_image</code> — featured image URL</li>
<li><code>schema_type</code> — auto-detected (Article, HowTo, FAQPage, WebPage)</li>
<li><code>categories</code> and <code>tags</code></li>
<li><code>reading_time</code> and <code>word_count</code></li>
<li><code>robots</code> — respects your SEO plugin’s noindex settings</li>
<li><code>lang</code> — site language (WPML/Polylang compatible)</li>
</ul>
<p><strong>Markdown Body includes:</strong></p>
<ul>
<li>H1–H6 headings with anchor ID preservation</li>
<li>Bold, italic, strikethrough, inline code</li>
<li>Fenced code blocks with language detection</li>
<li>Ordered and unordered lists (nested)</li>
<li>GitHub Flavored Markdown (GFM) tables</li>
<li>Blockquotes, images with alt text, links (relative <span aria-hidden="true" class="wp-exclude-emoji">→</span> absolute)</li>
<li>Gutenberg block HTML stripped cleanly</li>
</ul>
<h4>URL Patterns</h4>
<ul>
<li><code>/your-post-slug.md</code> — clean pretty permalink</li>
<li><code>/parent/child-page.md</code> — works with nested pages</li>
<li><code>/?p=123&format=md</code> — query string fallback</li>
</ul>
<h4>AI Crawler Discovery</h4>
<p>The plugin automatically:</p>
<ul>
<li>Adds <code><link rel="alternate" type="text/markdown"></code> to every page <code><head></code></li>
<li>Registers <code>/md-sitemap.xml</code> listing all <code>.md</code> URLs</li>
<li>Appends explicit <code>Allow: /*.md</code> rules to <code>robots.txt</code> for all major AI crawlers</li>
<li>Injects a crawlable <code><a></code> link on each page so robots can follow it naturally</li>
</ul>
<h4>SEO Plugin Compatibility</h4>
<ul>
<li>✅ Yoast SEO</li>
<li>✅ Rank Math</li>
<li>✅ All in One SEO (AIOSEO)</li>
<li>✅ WPML</li>
<li>✅ Polylang</li>
<li>✅ Gutenberg (Block Editor)</li>
<li>✅ Classic Editor</li>
</ul>
<h4>No Duplicate Content Risk</h4>
<p>The <code>.md</code> endpoint serves <code>Content-Type: text/markdown</code>, not <code>text/html</code>. An HTTP-level <code>Link: rel="canonical"</code> header points back to the original HTML page, and <code>X-Robots-Tag: noindex, follow</code> prevents Google from indexing the raw Markdown as a competing webpage — while still allowing AI crawlers to read it.</p>
<h4>Why This Matters</h4>
<p>LLMs and AI search engines crawl the web to build their knowledge. Clean, structured Markdown with rich YAML frontmatter is the ideal format for these systems to parse, index, and cite your content — giving you organic visibility in AI-powered search results, summaries, and citations.</p>


Whizz MD — Markdown Endpoints for AI Crawlers

Serve every WordPress post and page as a clean .md file with full SEO frontmatter — discoverable by AI crawlers and search engines on the fly.

apasionados

<p>This plugin adds lines to the virtual robots.txt file (that WordPress creates automagically if the file is not present physically on the server) to block the archive.org bots.</p>
<blockquote>
<p>Please be aware of what this plugin does exactly: If you activate the plugin it will add some lines to the robots.txt file to tell the <a href="https://archive.org/" rel="nofollow ugc">Archive.org bots</a> <strong>not to crawl and index</strong> your page. This means that they won’t store a backup of your site and you won’t be able to browse the different versions of the website through time.</p>
<p>Please keep in mind that if you activate this plugin and your site is deleted from the archive.org index, you can’t undo this. If you remove the plugin and the block of the archive.org bot, they’ll start again crawling and indexing the website from that moment, but all the older information is lost. <strong>Be carefull and think if this is what you want to do!</strong>. If you mess it nobody will not be able to help you (neither we or archive.org).</p>
</blockquote>
<p><strong>Please only activate this plugin if you know what you’re doing</strong>.</p>
<h4>What can I do with this plugin?</h4>
<p>This plugin adds the necessary lines to the virtual robots.txt file that WordPress creates automagically to block the archive.org bots.</p>
<h4>What ideas is this plugin based on?</h4>
<p>None. We needed a fast solution to add these lines to the robots.txt file and this was the solution we coded.</p>
<h4>System requirements</h4>
<p>PHP version 5.5 or greater.</p>
<h4>Block archive.org bots robots.txt Plugin in your Language!</h4>
<p>The first release is avaliable in English and Spanish. In the “languages” folder we have included the necessary files to translate this plugin.</p>
<p>If you would like the plugin in your language and you’re good at translating, please use the <a href="https://translate.wordpress.org/projects/wp-plugins/block-archive-org-robots-txt/" rel="nofollow ugc">native WordPress Translation</a> functionality.</p>
<p>New to Translating a plugin? First read through the <a href="https://make.wordpress.org/polyglots/handbook/tools/glotpress-translate-wordpress-org/" rel="nofollow ugc">Translator Handbook</a>, then select your locale at <a href="https://translate.wordpress.org/" rel="nofollow ugc">Translating WordPress</a> and finally go to the <a href="https://translate.wordpress.org/projects/wp-plugins/block-archive-org-robots-txt/" rel="nofollow ugc">translation page for this plugin</a> to translate it.</p>
<h4>Further Reading</h4>
<p>You can access the description of the plugin in Spanish at: <a href="https://apasionados.es/blog/bloquear-el-bot-de-archive-org-a-traves-de-robots-txt-wordpress-plugin-7752/" rel="nofollow ugc">Block archive.org bots robots.txt en español</a>.</p>
<h3>Contact</h3>
<p>For further information please send us an <a href="https://apasionados.es/contacto/index.php?desde=wordpress-org-block-archive-org-robots-txt" rel="nofollow ugc">email</a>.</p>


Block Archive.org via WordPress robots.txt

Blocks the archive.org bots through the WordPress virtual robots.txt file.

Whizz MD — Markdown Endpoints for AI Crawlers vs Block Archive.org via WordPress robots.txt