aicrawlerguard.com

<p>AI Crawler Guard detects popular AI-focused crawlers and lets you block or allow them from scraping your WordPress site. Everything runs locally—rules, decisions, logs, and history stay on your server.</p>
<ul>
<li>Built-in signatures for common AI bots, plus optional custom patterns</li>
<li>Global and per-crawler allow/block controls</li>
<li>Local crawler registry with first/last seen timestamps and hit counters</li>
<li>Lightweight logs with optional retention cap</li>
<li>No external calls or remote storage</li>
</ul>
<h3>Requirements</h3>
<ul>
<li>WordPress 5.8 or newer</li>
<li>PHP 7.4+</li>
<li>Works with Classic Editor, Gutenberg, WooCommerce, and major managed hosts</li>
<li>Respects wp-admin, wp-login, REST API, cron, and CLI contexts automatically</li>
</ul>
<h3>Performance</h3>
<ul>
<li>Detection runs once per front-end request (small O(n) UA scan)</li>
<li>No blocking of social preview bots (Facebook, Meta, Twitter, LinkedIn) by default</li>
<li>Assets load only on plugin admin pages and stay under 10 KB</li>
<li>Logs stay capped by your configured limit with auto-pruning</li>
</ul>
<h3>Privacy</h3>
<p>All detection logic and data storage live entirely inside WordPress. No data is sent to third-party services in v1.</p>


AI Crawler Guard

Detect and control AI crawler access with local logging and history.

Processby

<p>The plugin allows you to create and edit the robots.txt file on your site.</p>
<h4>Features</h4>
<ul>
<li>Works with multisite network on Subdomains;</li>
<li>An example of the correct file for WordPress;</li>
<li>Works out of the box;</li>
<li>Totally Free.</li>
</ul>


Robots.txt Editor

Lightweight and perfect for multisite installations.

But in documentation need to say where the option/editor is installed (Settings > Read options).

AI Crawler Guard vs Robots.txt Editor