aicrawlerguard.com

<p>AI Crawler Guard detects popular AI-focused crawlers and lets you block or allow them from scraping your WordPress site. Everything runs locally—rules, decisions, logs, and history stay on your server.</p>
<ul>
<li>Built-in signatures for common AI bots, plus optional custom patterns</li>
<li>Global and per-crawler allow/block controls</li>
<li>Local crawler registry with first/last seen timestamps and hit counters</li>
<li>Lightweight logs with optional retention cap</li>
<li>No external calls or remote storage</li>
</ul>
<h3>Requirements</h3>
<ul>
<li>WordPress 5.8 or newer</li>
<li>PHP 7.4+</li>
<li>Works with Classic Editor, Gutenberg, WooCommerce, and major managed hosts</li>
<li>Respects wp-admin, wp-login, REST API, cron, and CLI contexts automatically</li>
</ul>
<h3>Performance</h3>
<ul>
<li>Detection runs once per front-end request (small O(n) UA scan)</li>
<li>No blocking of social preview bots (Facebook, Meta, Twitter, LinkedIn) by default</li>
<li>Assets load only on plugin admin pages and stay under 10 KB</li>
<li>Logs stay capped by your configured limit with auto-pruning</li>
</ul>
<h3>Privacy</h3>
<p>All detection logic and data storage live entirely inside WordPress. No data is sent to third-party services in v1.</p>


AI Crawler Guard

Detect and control AI crawler access with local logging and history.

apasionados

<p>This plugin adds lines to the virtual robots.txt file that WordPress creates automagically if the file is not present physically on the server to block the OpenAI ChatGPT-User bot that is used by plugins in ChatGPT to crawl websites. Here is the <a href="https://platform.openai.com/docs/plugins/bot" rel="nofollow ugc">information about the ChatGPT-User bot</a>. It also blocks the the <a href="https://platform.openai.com/docs/gptbot" rel="nofollow ugc">GPTBot user agent</a> that may potentially be used to improve future models.</p>
<blockquote>
<p>Please be aware of what this plugin does exactly: If you activate the plugin it will add some lines to the robots.txt file to tell Open AI Chat GPT bots (ChatGPT-User + GPTBot) to <strong>not to crawl and index</strong> parts of this website.</p>
</blockquote>
<p><strong>Please only activate this plugin if you know what you’re doing</strong>.</p>
<h4>What can I do with this plugin?</h4>
<p>This plugin adds the necessary lines to the virtual robots.txt file that WordPress creates automagically to block the OpenAI ChatGPT-User bot that is used by plugins in ChatGPT to crawl websites + the GPTBot user agent that may potentially be used to improve future models.</p>
<h4>What ideas is this plugin based on?</h4>
<p>None. After the launch of the OpenAI ChatGTP plugins that crawl websites, we needed a fast solution to add these lines to the robots.txt file and this was the solution we coded.</p>
<p>In August 2023 OpenAI added information about the GPTBot so we added this user-agent to the plugin.</p>
<h4>What is the robots.txt file?</h4>
<p>The robots.txt file is a plain text file located at the root folder of a domain (or subdomain) which tells web crawlers (like Googlebot) what parts of the website they should access and index.</p>
<p>The first thing a search engine crawler looks at when it is visiting a page is the robots.txt file and it controls how search engine spiders see and interact with the web pages.</p>
<h4>System requirements</h4>
<p>PHP version 7.4 or greater.<br />
We require 7.4 or higher because we believe that everybody should be running a modern PHP version. When releasing this plugin <a href="https://wordpress.org/about/requirements/" rel="ugc">WordPress recommends PHP 7.4 or higher</a>. We would recommend 8.0 or higher as 7.4 has no security updates since 28 Nov 2022.</p>
<h4>How to get this plugin in your Language!</h4>
<p>The first release is avaliable in English and Spanish. In the “languages” folder we have included the necessary files to translate this plugin.</p>
<p>If you would like the plugin in your language and you’re good at translating, please use the <a href="https://translate.wordpress.org/" rel="nofollow ugc">native WordPress Translation</a> functionality.</p>
<p>New to Translating a plugin? First read through the <a href="https://make.wordpress.org/polyglots/handbook/tools/glotpress-translate-wordpress-org/" rel="nofollow ugc">Translator Handbook</a>, then select your locale at <a href="https://translate.wordpress.org/" rel="nofollow ugc">Translating WordPress</a> and finally go to the <a href="https://translate.wordpress.org/projects/wp-plugins/block-ecommerce-assets-via-robots-txt/" rel="nofollow ugc">translation page for this plugin</a> to translate it.</p>
<h4>Further Reading</h4>
<p>You can access the description of the plugin in Spanish at: <a href="https://apasionados.es/blog/" rel="nofollow ugc">Block Chat GPT via robots.txt en español</a>.</p>
<h3>Contact</h3>
<p>For further information please send us an <a href="https://apasionados.es/contacto/index.php?desde=wordpress-org-block-chatgpt-assets-via-robots-txt" rel="nofollow ugc">email</a>.</p>


Block Chat GPT via robots.txt

Blocks the ChatGPT-User bot that is used by plugins in ChatGPT to crawl websites through the WordPress virtual robots.txt file

AI Crawler Guard vs Block Chat GPT via robots.txt