Md Atiar Rahman Ovi

Prepare your WordPress site for the AI Search Era.
As search engines evolve into Answer Engines (like ChatGPT Search, Perplexity, and Google Gemini), they rely less on HTML and more on structured, raw text data. The <code>llms.txt</code> standard is the new “robots.txt” for AI—a file specifically designed to tell Large Language Models (LLMs) what your site is about and where to find your best content.
WEO LLMS & AI Crawler Guide automatically generates a perfect, dynamic <code>llms.txt</code> file for your site. It creates a categorized “Resume” of your website that helps AI bots understand your structure instantly.
<h3>🚀 Key Features</h3>
<ul>
<li>Categorized Content Output: Automatically groups your content by type (e.g., <code>## Post</code>, <code>## Page</code>, <code>## Products</code>) so AI understands your site structure.</li>
<li>Automated Sitemap Detection: Automatically detects and lists your <code>sitemap.xml</code> at the top of the file for deeper crawling.</li>
<li>Markdown Optimization: Converts your content titles, links, and excerpts into clean Markdown (<code>- [Title](Link) - Excerpt</code>) optimized for LLM token limits.</li>
<li>Robots.txt Integration: Automatically adds a link to your <code>llms.txt</code> inside your site’s <code>robots.txt</code> file so crawlers can find it.</li>
<li>Custom Control: Choose exactly which Post Types (Pages, Posts, Products, Custom Types) to include.</li>
<li>Custom Intro/Outro: Add specific instructions for AI bots (e.g., “This site is the authority on Technical SEO…”).</li>
<li>Performance Caching: Built-in caching ensures the file loads instantly without slowing down your server.</li>
</ul>
<h3>💡 Why do you need this?</h3>
If you want your content to appear in AI answers (ChatGPT, Claude, Perplexity), you need to make it easy for them to read. A standard HTML page is full of noise (menus, sidebars, popups). An <code>llms.txt</code> file provides a clean, noise-free map of your content.
This plugin handles the technical heavy lifting, ensuring your site is “AI Ready” in seconds.

WEO LLMS & AI Crawler Guide


Awesome. Just install the plugin and it start crawling.


Plug and play

The professional architect for your llms.txt file. Control how AI agents (ChatGPT, Claude, Perplexity) consume your website content with a structured, …

Alex Oreshkin

Keep your site lightning fast while making your content perfectly readable to Large Language Models (LLMs) and AI agents.
This plugin allows you to easily manage your AI crawler protocol directly from the WordPress dashboard. It serves the file dynamically and securely, adhering to advanced security protocols necessary for enterprise hosting environments. Version 3.0.1 aligns with 2026 AEO standards including mission statements and extended context support.
<h4>Features</h4>
<ul>
<li>Green Web Post-Meta Caching (NEW): Utilizes a “Fragment Caching” architecture to pre-render AI data. Reduces server CPU overhead by over 90% during map generation, strictly aligning with 2026 AEO efficiency standards. Includes a silent background “Janitor” task to automatically migrate existing content.</li>
<li>Crawler Analytics Log: Track how many times your <code>llms.txt</code> file is accessed and exactly which AI bots are visiting. Features a detailed daily log (in your local timezone) that automatically expunges data older than 30 days to keep your database light.</li>
<li>Granular Taxonomy Controls: Filter your auto-generated AI map by specific category or tag slugs. Ensure only your most relevant “High Signal” content is mapped for bots.</li>
<li>Custom Sitemap Override: By default, the plugin points AI agents to your standard WordPress <code>wp-sitemap.xml</code>. You can now explicitly override this to point to any custom SEO sitemap (e.g., Yoast, RankMath).</li>
<li>Post & Page Exclusions: You can easily omit private or sensitive content from your AI map by entering a comma separated list of Post IDs. The generator strictly ignores them.</li>
<li>Background Auto-Syncing: Hooks into WordPress saves. When you publish or delete a post, the plugin silently regenerates your AI map in the background. You never have to manually update your crawler map again.</li>
<li>Interactive License Presets: Click to instantly apply standard data scraping licenses (CC-BY-4.0, Proprietary/No-AI, MIT, etc.) with helpful, plain English explanations showing exactly what rules apply to bots.</li>
<li>Expanded Bot Targeting Library: Easily construct your crawler protocols using quick-add toggle buttons for major and lesser-known AI agents.</li>
<li>Modular 3-Tier Architecture: Manage your Targeted Bots, Auto-Generated AI Map, and Custom Instructions in completely separate fields. Your manual prompts are safe from automated overwrites.</li>
<li>Virtual File Technology: Acts exactly like a physical text file to AI crawlers, but requires zero file write permissions on your server.</li>
<li>Live Dashboard Preview: See exactly what your <code>llms.txt</code> file looks like in real-time as you change settings, featuring a high-contrast terminal theme.</li>
<li>Custom Post Type Selector: Dynamically choose which content types (Portfolios, Products, Events, etc.) appear in your AI map.</li>
<li>Site Mission Statement: Provide context for AI models via a dedicated AEO mission statement block.</li>
<li>llms-full.txt Support: Automatically generates and serves <code>llms-full.txt</code> (Extended Context) for deep ingestion of concatenated Markdown content.</li>
</ul>

Virtual LLMS.TXT Generator

Safely creates a virtual llms.txt dynamically. Features interactive bot targeting, map auto-syncing, and decoupled custom instructions.

WEO LLMS & AI Crawler Guide vs Virtual LLMS.TXT Generator