A smart WordPress plugin that automatically blocks malicious bots and crawlers that ignore your site's robots.txt file.

EncodeDotHost

Neil Batchelor

<p>EDH Bad Bots is an intelligent bot detection and blocking system that protects your WordPress site from unwanted crawlers and malicious bots. Unlike traditional blocking methods that rely on user agent strings (which can be easily spoofed), this plugin uses a honeypot technique to identify and block bots that don’t respect your site’s <code>robots.txt</code> directives.</p>
<h3>Key Features</h3>
<ul>
<li><strong>Automatic Bot Detection</strong>: Identifies bad bots using a hidden trap URL technique</li>
<li><strong>Smart Blocking System</strong>: Blocks misbehaving bots with configurable duration (default 30 days)</li>
<li><strong>Advanced DNS Resolution</strong>: PTR record lookups with DNS over HTTPS (DoH) support for hostname identification</li>
<li><strong>Dual-Level Blocking</strong>: Server-level <code>.htaccess</code> blocking AND PHP-level blocking for maximum effectiveness</li>
<li><strong>Configurable Blocking Methods</strong>: Choose between <code>.htaccess</code> blocking (Apache) or PHP-only blocking (Nginx compatible)</li>
<li><strong>IP Whitelist Management</strong>: Protect trusted IPs from ever being blocked</li>
<li><strong>Enhanced Admin Interface</strong>: Clean dashboard with hostname display, manual hostname updates, and debug tools</li>
<li><strong>Background Processing</strong>: Automated hostname resolution via WordPress cron jobs</li>
<li><strong>Legitimate Crawler Protection</strong>: Synchronous FCrDNS verification prevents Googlebot and similar crawlers from being blocked, even if robots.txt is misconfigured</li>
<li><strong>robots.txt Health Check</strong>: Daily cron monitors for a physical robots.txt missing the trap Disallow rule and alerts the admin</li>
<li><strong>Database Optimization</strong>: Automatic cleanup of expired blocks to maintain performance</li>
<li><strong>Security-First Design</strong>: All forms include proper nonce verification and user capability checks</li>
</ul>
<h3>How It Works</h3>
<p>The plugin implements a sophisticated honeypot system:</p>
<ol>
<li><strong>Trap URL Generation</strong>: Creates a unique, hidden URL specific to your domain</li>
<li><strong>Robots.txt Integration</strong>: Automatically adds a <code>Disallow</code> rule for the trap URL</li>
<li><strong>Hidden Link Placement</strong>: Places an invisible link to the trap URL in your site’s footer</li>
<li><strong>Bot Detection</strong>: When bad bots ignore robots.txt and follow the hidden link, they’re identified</li>
<li><strong>Automatic Blocking</strong>: Detected bot IPs are blocked with configurable duration and immediate effect</li>
<li><strong>Hostname Resolution</strong>: PTR record lookups identify the hostname/organization behind blocked IPs</li>
<li><strong>Legitimate Crawler Guard</strong>: If a request to the trap URL carries a Googlebot, Bingbot, or similar User-Agent, FCrDNS is run synchronously before any block decision. Verified crawlers are whitelisted on the spot; spoofed UAs are blocked normally.</li>
<li><strong>robots.txt Health Check</strong>: A daily cron fetches and validates the site’s robots.txt. If a physical file is missing the Disallow rule, an admin notice is shown until the issue is resolved.</li>
</ol>
<h3>Configuration</h3>
<h3>Admin Dashboard</h3>
<p>Access the plugin dashboard at <strong>Tools > Bad Bots</strong> in your WordPress admin:</p>
<h4>Whitelisted IPs Tab</h4>
<ul>
<li>Add IP addresses that should never be blocked</li>
<li>Remove IPs from the whitelist</li>
<li>View all currently whitelisted addresses with timestamps</li>
</ul>
<h4>Blocked Bots Tab</h4>
<ul>
<li>View all currently blocked IP addresses with hostnames</li>
<li>See when each IP was blocked and when the block expires</li>
<li>Manually update missing hostnames for better identification</li>
<li>Force refresh all hostnames to clear cache and re-resolve</li>
<li>Debug hostname resolution issues (when WP_DEBUG is enabled)</li>
<li>Manually unblock IPs if needed</li>
</ul>
<h4>Options Tab</h4>
<ul>
<li><strong><code>.htaccess Blocking</code></strong>: Enable/disable server-level IP blocking via <code>.htaccess</code> file</li>
<li><strong>Block Duration</strong>: Configure how many days to block detected bots</li>
<li>Configure blocking method based on your server setup (Apache vs Nginx)</li>
<li>Server-level blocking bypasses caching for immediate effect</li>
</ul>
<h4>Help Tab</h4>
<ul>
<li>Detailed explanation of how the plugin works</li>
<li>Best practices for managing IPs</li>
<li>Information about <code>.htaccess</code> blocking options</li>
<li>Unique trap URL for caching plugin exclusion</li>
</ul>
<h3>Requirements</h3>
<ul>
<li>WordPress 6.2 or higher</li>
<li>PHP 7.4 or higher</li>
<li>MySQL 5.6 or higher</li>
<li>Apache server (for <code>.htaccess</code> blocking) or Nginx (PHP-only blocking)</li>
<li>Writable <code>.htaccess</code> file (if using Apache server-level blocking)</li>
</ul>
<h3>Technical Details</h3>
<h3>Database Tables</h3>
<p>The plugin creates two custom database tables:</p>
<ul>
<li><code>wp_edhbb_blocked_bots</code>: Stores blocked IP addresses with expiration dates and hostnames</li>
<li><code>wp_edhbb_whitelisted_ips</code>: Stores permanently whitelisted IP addresses</li>
</ul>
<h3>DNS Resolution System</h3>
<p>The plugin includes an advanced DNS lookup system:</p>
<h4>DNS over HTTPS (DoH) Support</h4>
<ul>
<li><strong>Primary providers</strong>: Cloudflare DNS, Google DNS</li>
<li><strong>Secure queries</strong>: HTTPS-encrypted DNS requests for enhanced privacy</li>
<li><strong>Fallback system</strong>: Automatic fallback to traditional DNS methods</li>
</ul>
<h4>PTR Record Lookups</h4>
<ul>
<li><strong>Reverse DNS</strong>: Converts IP addresses to hostnames for better identification</li>
<li><strong>IPv4 and IPv6 support</strong>: Full support for both IP versions</li>
<li><strong>Caching</strong>: Results cached for 1 hour to improve performance</li>
<li><strong>Background processing</strong>: Automated hostname resolution via WordPress cron</li>
</ul>
<h3>Blocking Methods</h3>
<p>The plugin offers two blocking approaches:</p>
<h4>1. Server-Level Blocking (<code>.htaccess</code>)</h4>
<ul>
<li><strong>Default method</strong> for Apache servers</li>
<li>Blocks IPs at the server level before WordPress loads</li>
<li>Bypasses caching plugins for immediate effect</li>
<li>More efficient and faster blocking</li>
<li>Automatically manages <code>.htaccess</code> file with unique markers</li>
<li>Safe cleanup on plugin deactivation</li>
</ul>
<h4>2. PHP-Level Blocking</h4>
<ul>
<li><strong>Alternative method</strong> for Nginx or when <code>.htaccess</code> is unavailable</li>
<li>Blocks IPs during WordPress initialization</li>
<li>Compatible with all web servers</li>
<li>May be affected by caching plugins</li>
<li>No server configuration files modified</li>
</ul>
<h3>Security Features</h3>
<ul>
<li><strong>Nonce Verification</strong>: All forms use WordPress nonces for CSRF protection</li>
<li><strong>Capability Checks</strong>: Only users with <code>manage_options</code> capability can access admin features</li>
<li><strong>Input Sanitization</strong>: All user inputs are properly sanitized and validated</li>
<li><strong>SQL Injection Protection</strong>: All database queries use prepared statements</li>
<li><strong>Safe <code>.htaccess</code> Management</strong>: Uses unique markers and automatic cleanup</li>
</ul>
<h3>Performance Optimization</h3>
<ul>
<li><strong>Automatic Cleanup</strong>: Expired blocks are automatically removed from the database</li>
<li><strong>Efficient Queries</strong>: Database operations are optimized for minimal performance impact</li>
<li><strong>Smart Loading</strong>: Admin assets only load on the plugin’s admin page</li>
<li><strong>Server-Level Blocking</strong>: <code>.htaccess</code> blocking prevents blocked requests from reaching PHP</li>
<li><strong>Whitelist Filtering</strong>: Whitelisted IPs are excluded from <code>.htaccess</code> rules automatically</li>
<li><strong>DNS Caching</strong>: Hostname lookups cached to reduce DNS query overhead</li>
<li><strong>Background Processing</strong>: Hostname resolution runs in background to avoid delays</li>
</ul>
<h3>API Hooks</h3>
<h3>Actions</h3>
<ul>
<li><code>plugins_loaded</code>: Plugin initialization</li>
<li><code>init</code>: Early request blocking check</li>
<li><code>template_redirect</code>: Bot trap detection</li>
<li><code>wp_footer</code>: Hidden link injection</li>
<li><code>admin_menu</code>: Admin page registration</li>
<li><code>admin_notices</code>: robots.txt misconfiguration warning</li>
<li><code>edhbb_update_hostnames_cron</code>: Background hostname resolution (hourly)</li>
<li><code>edhbb_check_robots_txt_cron</code>: robots.txt Disallow validation (daily)</li>
</ul>
<h3>Filters</h3>
<ul>
<li><code>robots_txt</code>: Adds disallow rule to robots.txt</li>
<li><code>edhbb_trusted_crawler_domains</code>: Customise the FCrDNS hostname suffixes used to identify legitimate crawlers (e.g. <code>.googlebot.com</code>)</li>
<li><code>edhbb_trusted_crawler_ua_patterns</code>: Customise the User-Agent tokens that trigger a synchronous FCrDNS check on trap hits (e.g. <code>googlebot</code>, <code>bingbot</code>)</li>
</ul>
<h3>File Structure</h3>
<pre><code>`
</code></pre>
<p>edh-bad-bots/<br />
├── admin/<br />
│   └── views/<br />
│       └── admin-display.php    # Admin interface HTML<br />
├── assets/<br />
│   ├── css/<br />
│   │   └── admin-style.css      # Admin page styling<br />
│   └── js/<br />
│       └── admin-script.js      # Admin page JavaScript<br />
├── includes/<br />
│   ├── class-edhbb-admin.php    # Admin functionality<br />
│   ├── class-edhbb-blocker.php  # Bot detection and blocking<br />
│   ├── class-edhbb-database.php # Database operations<br />
│   └── class-edhbb-dnslookup.php # DNS/PTR lookup system<br />
├── edh-bad-bots.php            # Main plugin file<br />
├── LICENSE<br />
└── readme.txt<br />
    `</p>
<h3>Contributing</h3>
<p>Contributions are welcome! Please feel free to submit a Pull Request.</p>
<h3>Development Setup</h3>
<ol>
<li>Clone the repository to your WordPress plugins directory</li>
<li>Ensure you have a WordPress development environment running</li>
<li>Activate the plugin and test your changes</li>
</ol>
<h3>License</h3>
<p>This project is licensed under the GPL v3 or later.</p>
<h3>Author</h3>
<p><strong>EncodeDotHost</strong><br />
– Website: <a href="https://encode.host" rel="nofollow ugc">https://encode.host</a><br />
– GitHub: <a href="https://github.com/EncodeDotHost/edh-bad-bots" rel="nofollow ugc">@EncodeDotHost</a></p>
<h3>Support</h3>
<p>For support, please visit the support forum: <a href="https://wordpress.org/support/plugin/edh-bad-bots/" rel="ugc">https://wordpress.org/support/plugin/edh-bad-bots/</a></p>


Alternatives to EDH Bad Bots

Top alternatives