Erik

<p>The <strong>md4AI</strong> plugin provides a powerful, multi-layered approach to Generative Engine Optimization (GEO).</p>
<h3>1. Automatic AI Bot Detection (Zero-Config)</h3>
<p>md4AI automatically detects requests from a comprehensive list of known AI agents (like GPTBot, ClaudeBot, Google-Extended, etc.) and serves them a lightweight, on-the-fly Markdown version of your content instead of the standard HTML theme. This ensures a clean, parsable data feed for LLMs right out of the box.</p>
<h3>2. Per-Post Custom Markdown (Metabox)</h3>
<p>The plugin adds a new metabox to your post and page editor, giving you full control over the AI-optimized version of your content.<br />
Serving an md version of the website is particular useful if your website has editors like Elementor / Beaver Builder / Divi Builder / other div bloated editors because bots usually skips complex and too much nested content.</p>
<ul>
<li><strong>Generate Custom Markdown:</strong> Convert your post’s content into Markdown with one click.</li>
<li><strong>Manually Edit:</strong> Modify and enhance the Markdown to be exactly as you want AIs to see it.</li>
<li><strong>(Optional) Enhance with AI:</strong> If you also use the <strong><a href="https://wordpress.org/plugins/ai-services/" rel="ugc">AI Services</a></strong> plugin, you can use the “Generate with AI” button to automatically add FAQs, discussion questions, or key takeaways extracted from your post.</li>
<li><strong>Serve Enhanced Content:</strong> When an AI bot visits this <em>specific post</em>, it will be served your new, custom-tailored Markdown version, giving it far richer data.</li>
<li><strong>Automatic Fallback:</strong> If you don’t create a custom version, the plugin falls back to the automatic on-the-fly conversion.</li>
</ul>
<h3>3. Site-Wide AI Instructions (llms.txt)</h3>
<p>The plugin adds a new main page under ‘Utilities’ with dedicated tabs for configuration and insights. From here, you can create and manage a <code>llms.txt</code> file for your entire site.</p>
<ul>
<li><strong>What is <code>llms.txt</code>?</strong> This is a file, similar to <code>robots.txt</code>, that provides general instructions, context, and useful links to AI crawlers. It’s a new standard to help guide LLMs in understanding your site’s content and purpose.</li>
<li><strong>Manage Content:</strong> Use the settings page to write your <code>llms.txt</code> content, which will then be served automatically to bots that look for it.</li>
<li><strong>Learn More:</strong> You can find more details on this new standard at <a href="https://llmstxt.org/" rel="nofollow ugc">https://llmstxt.org/</a>.</li>
</ul>
<p>This combination gives you full control, from automatic optimization to granular, AI-enhanced, per-post content and site-wide AI directives.</p>
<h3>4. GEO/AIO Insights & Validation</h3>
<p>The dedicated GEO/AIO Insights tab provides a unique validation mechanism for your site’s AI presence, offering measurable scores (0-100) on how well your brand is understood by Generative Engines.</p>
<ul>
<li><strong>The Validation Process</strong>: We ask a Large Language Model (LLM) di completare una “scheda di conoscenza” sul tuo sito (ad esempio, il nome del brand, gli argomenti principali e le categorie di prodotto).</li>
<li><strong>Ground Truth Check</strong>: The AI response is immediately compared with real data extracted from your WordPress database (e.g., site title, author names, WooCommerce categories)</li>
<li><strong>Final Score</strong>: Based on this cross-check, the insight provides a score for brand identity and technical relevance, highlighting exactly where the AI provided incorrect or insufficient data (hallucinations or missing data).</li>
</ul>


md4AI

Optimise content for generative engines (GEO) by serving custom Markdown and a site-wide llms.txt.

Anshul

<p>The AI Crawler and SEO Manager by akbansa plugin generates an <code>llms.txt</code> file for crawler discovery and logs visits by AI-based crawlers such as GPTBot, ClaudeBot, and others. It also provides an analytics page in the WordPress admin dashboard to view crawler activity by path, URL, and timestamp.</p>
<h3>Key Features:</h3>
<ul>
<li>Generates <code>llms.txt</code> file for crawler discovery.</li>
<li>Removes <code>llms.txt</code> on plugin uninstall</li>
<li>Schedules automatic generation of <code>llms.txt</code> daily at midnight (frequency configurable: daily, twice daily, hourly)</li>
<li>Uses atomic file writes for <code>llms.txt</code> and <code>sitemap.xml</code> for safety</li>
<li>Integrates with Yoast SEO, Rank Math, or All in One SEO for metadata (falls back to DB if not present)</li>
<li>Admin panel to select/exclude categories and content types for <code>llms.txt</code></li>
<li>Logs every AI crawler request from the frontend (user agent, crawl info, page URL, IP, referrer)</li>
<li>Filters crawlers using <code>config.json</code> and stores all details in a dedicated DB table</li>
<li>Analytics dashboard in admin to view recent crawler activity</li>
<li>Robust AJAX security and error handling</li>
<li>Detects and logs visits from AI crawlers based on their User-Agent.</li>
<li>Stores crawler logs in a database for analysis.</li>
<li>Supports popular AI-based crawlers (e.g., OpenAI, Anthropic, GPTBot, etc.).</li>
<li>Option to customize the list of crawlers via a <code>config.json</code> file.</li>
</ul>
<h3>Short Description</h3>
<p>Advanced AI-powered SEO and analytics for WordPress. Generates llms.txt for AI crawler discovery, logs AI bot visits, and provides comprehensive analytics dashboard.</p>
<h3>Usage</h3>
<ul>
<li>After activation, the plugin will automatically start generating the <code>llms.txt</code> file and log visits from AI-based crawlers.</li>
<li>The crawler logs are stored in the database for analysis.</li>
<li>You can view analytics by navigating to <strong>LLMS Crawlers</strong> in the WordPress admin menu.</li>
</ul>
<h3>Admin Analytics</h3>
<p>The admin panel will display a list of the most recent crawler visits, showing the following information:<br />
– Path<br />
– URL<br />
– Crawler Name<br />
– Timestamp</p>
<h3>Support</h3>
<p>If you encounter any issues, please feel free to reach out via the plugin support page or create an issue on the GitHub repository.</p>
<h3>License</h3>
<p>This plugin is licensed under the GPLv2 or later.</p>
<h3>New Features</h3>
<ol>
<li><strong>Admin Settings Filter</strong>: You can now filter active pages, products, categories, and posts for generating the <code>llms.txt</code>.</li>
<li><strong>Sitemap Handling</strong>: If a <code>sitemap.xml</code> exists, it will be used to get the list of pages. If not, the database will be used as a fallback.</li>
<li><strong>SEO Plugin Integration</strong>: The plugin integrates with Yoast SEO, All in One SEO, and Rank Math to fetch metadata for the <code>llms.txt</code> file.</li>
<li><strong>Frequency Setting</strong>: You can choose the frequency for regenerating the <code>llms.txt</code> file (daily, weekly, monthly, hourly).</li>
<li><strong>Traffic Capture and AI Agent Logging</strong>: A front-end script captures traffic, sends it to the backend, and logs AI agents based on a <code>config.json</code> file.</li>
<li><strong>Sitemap Update</strong>: The <code>sitemap.xml</code> is automatically updated with the absolute path to the <code>llms.txt</code> and the updated timestamp.</li>
</ol>


AI Crawler and SEO Manager by akbansa

The AI Crawler and SEO Manager by akbansa plugin generates an llms.txt file for crawler discovery and logs visits by AI-based crawlers such as GPTBot, …

md4AI vs AI Crawler and SEO Manager by akbansa