Processby

The plugin allows you to create and edit the robots.txt file on your site.
<h4>Features</h4>
<ul>
<li>Works with multisite network on Subdomains;</li>
<li>An example of the correct file for WordPress;</li>
<li>Works out of the box;</li>
<li>Totally Free.</li>
</ul>

Robots.txt Editor

Lightweight and perfect for multisite installations.

But in documentation need to say where the option/editor is installed (Settings > Read options).

Perfect!

Alex Oreshkin

Keep your site lightning fast while making your content perfectly readable to Large Language Models (LLMs) and AI agents.
This plugin allows you to easily manage your AI crawler protocol directly from the WordPress dashboard. It serves the file dynamically and securely, adhering to advanced security protocols necessary for enterprise hosting environments. Version 3.0.1 aligns with 2026 AEO standards including mission statements and extended context support.
<h4>Features</h4>
<ul>
<li>Green Web Post-Meta Caching (NEW): Utilizes a “Fragment Caching” architecture to pre-render AI data. Reduces server CPU overhead by over 90% during map generation, strictly aligning with 2026 AEO efficiency standards. Includes a silent background “Janitor” task to automatically migrate existing content.</li>
<li>Crawler Analytics Log: Track how many times your <code>llms.txt</code> file is accessed and exactly which AI bots are visiting. Features a detailed daily log (in your local timezone) that automatically expunges data older than 30 days to keep your database light.</li>
<li>Granular Taxonomy Controls: Filter your auto-generated AI map by specific category or tag slugs. Ensure only your most relevant “High Signal” content is mapped for bots.</li>
<li>Custom Sitemap Override: By default, the plugin points AI agents to your standard WordPress <code>wp-sitemap.xml</code>. You can now explicitly override this to point to any custom SEO sitemap (e.g., Yoast, RankMath).</li>
<li>Post & Page Exclusions: You can easily omit private or sensitive content from your AI map by entering a comma separated list of Post IDs. The generator strictly ignores them.</li>
<li>Background Auto-Syncing: Hooks into WordPress saves. When you publish or delete a post, the plugin silently regenerates your AI map in the background. You never have to manually update your crawler map again.</li>
<li>Interactive License Presets: Click to instantly apply standard data scraping licenses (CC-BY-4.0, Proprietary/No-AI, MIT, etc.) with helpful, plain English explanations showing exactly what rules apply to bots.</li>
<li>Expanded Bot Targeting Library: Easily construct your crawler protocols using quick-add toggle buttons for major and lesser-known AI agents.</li>
<li>Modular 3-Tier Architecture: Manage your Targeted Bots, Auto-Generated AI Map, and Custom Instructions in completely separate fields. Your manual prompts are safe from automated overwrites.</li>
<li>Virtual File Technology: Acts exactly like a physical text file to AI crawlers, but requires zero file write permissions on your server.</li>
<li>Live Dashboard Preview: See exactly what your <code>llms.txt</code> file looks like in real-time as you change settings, featuring a high-contrast terminal theme.</li>
<li>Custom Post Type Selector: Dynamically choose which content types (Portfolios, Products, Events, etc.) appear in your AI map.</li>
<li>Site Mission Statement: Provide context for AI models via a dedicated AEO mission statement block.</li>
<li>llms-full.txt Support: Automatically generates and serves <code>llms-full.txt</code> (Extended Context) for deep ingestion of concatenated Markdown content.</li>
</ul>

Virtual LLMS.TXT Generator

Safely creates a virtual llms.txt dynamically. Features interactive bot targeting, map auto-syncing, and decoupled custom instructions.

Robots.txt Editor vs Virtual LLMS.TXT Generator