aicrawlerguard.com

<p>AI Crawler Guard detects popular AI-focused crawlers and lets you block or allow them from scraping your WordPress site. Everything runs locally—rules, decisions, logs, and history stay on your server.</p>
<ul>
<li>Built-in signatures for common AI bots, plus optional custom patterns</li>
<li>Global and per-crawler allow/block controls</li>
<li>Local crawler registry with first/last seen timestamps and hit counters</li>
<li>Lightweight logs with optional retention cap</li>
<li>No external calls or remote storage</li>
</ul>
<h3>Requirements</h3>
<ul>
<li>WordPress 5.8 or newer</li>
<li>PHP 7.4+</li>
<li>Works with Classic Editor, Gutenberg, WooCommerce, and major managed hosts</li>
<li>Respects wp-admin, wp-login, REST API, cron, and CLI contexts automatically</li>
</ul>
<h3>Performance</h3>
<ul>
<li>Detection runs once per front-end request (small O(n) UA scan)</li>
<li>No blocking of social preview bots (Facebook, Meta, Twitter, LinkedIn) by default</li>
<li>Assets load only on plugin admin pages and stay under 10 KB</li>
<li>Logs stay capped by your configured limit with auto-pruning</li>
</ul>
<h3>Privacy</h3>
<p>All detection logic and data storage live entirely inside WordPress. No data is sent to third-party services in v1.</p>


AI Crawler Guard

Detect and control AI crawler access with local logging and history.

Marios Alexandrou

<p>Virtual Robots.txt is an easy (i.e. automated) solution to creating and managing a robots.txt file for your site. Instead of mucking about with FTP, files, permissions ..etc, just upload and activate the plugin and you’re done.</p>
<p>By default, the Virtual Robots.txt plugin allows access to the parts of WordPress that good bots like Google need to access. Other parts are blocked.</p>
<p>If the plugin detects an existing XML sitemap file, a reference to it will be automatically added to your robots.txt file.</p>


Virtual Robots.txt

Awesome, a simple solution to a common issue (site content managers who want to hide certain pages from Google search results).

Just a little note: in my case, the existing Sitemap created by the plugin "Google (XML) Sitemaps Generator" by Arne Brachhold wasn't detected.

Efficient and unobstrusive

I like the fact that it's so clean. Thanks for building it!

Does the Job

Virtual Robots.txt automatically creates a robots.txt file for your site. Your robots.txt file can be easily edited from the plugin settings page.

AI Crawler Guard vs Virtual Robots.txt