crawlantix

<p>Crawlantix AI Bot Tracker monitors visits from 30+ AI crawlers and gives you visibility into how bots interact with your site. See which bots visit, what pages they crawl, and catch misbehaving bots with a built-in logging-only honeypot. Lightweight, privacy-first, and fully functional out of the box.</p>
<p><strong>Free forever — no artificial limits on bot detection.</strong></p>
<p><strong>Tracked Bots (30+):</strong></p>
<ul>
<li>GPTBot / ChatGPT-User / OAI-SearchBot (OpenAI)</li>
<li>ClaudeBot / Claude-Web / anthropic-ai (Anthropic)</li>
<li>Googlebot / Google-Extended (Google / Gemini)</li>
<li>bingbot (Microsoft / Copilot)</li>
<li>PerplexityBot (Perplexity)</li>
<li>DeepSeek, Qwen (Alibaba), Mistral AI</li>
<li>Applebot-Extended (Apple Intelligence)</li>
<li>Meta-ExternalAgent (Meta AI)</li>
<li>Bytespider (ByteDance)</li>
<li>CCBot (Common Crawl), Amazonbot, YouBot, DuckDuckBot</li>
<li>AI2Bot, Diffbot, Timpibot, PetalBot</li>
<li>SemrushBot, AhrefsBot, DataForSeoBot, MJ12bot, DotBot</li>
</ul>
<p><strong>Features:</strong></p>
<h4>Lightweight Bot Detection (30+ Bots)</h4>
<p>Hooks into WordPress <code>init</code> with a priority-1 action. Bails immediately on non-AI User-Agents — zero performance cost for human visitors. All 30+ bots tracked.</p>
<h4>Dashboard with Charts</h4>
<p>Clean dashboard with trend chart and provider breakdown pie chart (Chart.js, bundled locally). Summary cards show total visits, unique bots, pages crawled, and daily averages. No external dependencies.</p>
<h4>Bot Activity Table</h4>
<p>Dedicated tab showing all detected bots with visits, bytes transferred, 24h sparklines, verification status, and honeypot hit counts.</p>
<h4>Crawled Pages</h4>
<p>See which pages bots visit most, which bots crawl them, and when they were last seen.</p>
<h4>Honeypot Endpoint (Logging Only)</h4>
<p>A CSS-hidden, aria-hidden, rel=nofollow link is injected in the footer. Only raw link-extracting bots will follow it. Visits are logged for transparency. Active defense (blocking, tarpit, rate-limit, decoy, shadowban) is reserved for the paid build.</p>
<h4>Bot Verification</h4>
<p>Reverse DNS (FCrDNS) verification for major bots — confirms Googlebot, GPTBot, ClaudeBot, etc. are actually who they claim to be.</p>
<h4>Privacy First</h4>
<p>IP addresses are SHA-256 hashed with a per-install salt before storage. Raw IPs are never saved. Includes WordPress Privacy API exporter and eraser hooks so data-subject access and erasure requests can flow through the standard WordPress Tools <span aria-hidden="true" class="wp-exclude-emoji">→</span> Personal Data workflow.</p>
<h4>AI Discovery Layer</h4>
<p>Serves <code>ai-plugin.json</code> (a discovery manifest that tells visiting AI agents the site is monitored) and <code>llms.txt</code> / <code>llms-full.txt</code> (text content authored by the admin via WordPress pages with slugs <code>llms-txt</code> and <code>llms-full-txt</code>) at the site root.</p>
<h4>Data Retention</h4>
<p>Bot visit data is retained for 30 days. Older records are automatically pruned via WP-Cron.</p>
<h3>External Services</h3>
<p>This plugin connects to the following external services:</p>
<h4>Reverse DNS Lookups</h4>
<p>The bot verification feature performs reverse DNS (FCrDNS) lookups using PHP’s <code>gethostbyaddr()</code> and <code>gethostbyname()</code> functions to verify that bots are who they claim to be (e.g., confirming a request claiming to be Googlebot actually originates from Google’s network). These lookups send the bot’s IP address to your server’s configured DNS resolver and the authoritative DNS servers for the IP address’s reverse DNS zone. Under some privacy regimes, IP addresses may be considered personal data. This feature runs automatically when a known AI bot visits your site and cannot currently be disabled via the admin UI (a filter hook <code>crawlantix_enable_verification</code> is available for developers).</p>
<p>No other external services, third-party APIs, or remote requests are used by this plugin. All analytics data is stored locally in your WordPress database. Chart.js is bundled locally — no CDN requests are made.</p>
<h3>Privacy Policy</h3>
<p>Crawlantix AI Bot Tracker is designed with privacy as a core principle:</p>
<ul>
<li><strong>Bot traffic only.</strong> The plugin only tracks automated bot traffic identified by User-Agent strings. Human visitors are not tracked and no cookies are set.</li>
<li><strong>No raw IP addresses stored.</strong> All IP addresses are SHA-256 hashed with a per-install random salt before storage (or AUTH_SALT when defined in <code>wp-config.php</code>). The original IP address cannot be recovered from the hash. Note: pseudonymous IP hashes may still be considered personal data under GDPR.</li>
<li><strong>Data stored per bot visit:</strong> IP hash, User-Agent string, requested URL, HTTP referrer URL, request method, timestamp, and derived fields (bytes transferred, bot verification status). Referrer URLs may contain personal data depending on the referring site.</li>
<li><strong>WordPress Privacy API integration.</strong> The plugin registers exporter and eraser callbacks with WordPress core, so data-subject access and erasure requests filed through Tools <span aria-hidden="true" class="wp-exclude-emoji">→</span> Personal Data flow correctly.</li>
<li><strong>No external data transmission.</strong> All analytics data remains in your local WordPress database. No data is sent to Crawlantix or third-party services. The only external communication is DNS lookups for bot verification (see External Services above).</li>
<li><strong>Data retention controls.</strong> Bot visit data is automatically pruned after 30 days. Administrators can delete all collected data on uninstall via the “Delete Data on Uninstall” Settings toggle.</li>
</ul>
<p>For sites that require a formal privacy policy disclosure, you may note: “We use the Crawlantix AI Bot Tracker plugin to monitor automated AI bot traffic to our site. This plugin records bot User-Agent strings, pseudonymous IP hashes, pages visited, referrer URLs, and timestamps for detected bot traffic only. Raw IP addresses are cryptographically hashed before storage. No human visitor data is collected.”</p>
<h3>Premium Version</h3>
<p>Crawlantix also offers paid tiers at <a href="https://crawlantix.com" rel="nofollow ugc">crawlantix.com</a> for site owners who need active bot defense in addition to monitoring. The paid build adds the following features on top of everything in this free version:</p>
<h4>Protect tier</h4>
<ul>
<li>Active honeypot responses: HTTP 403 block, tarpit (random 5–25s delay with worker-exhaustion safeguards), rate limit 429, decoy content, shadowban.</li>
<li>Auto-block of repeat honeypot offenders, with configurable thresholds.</li>
<li>Per-IP response rules — apply a chosen response strategy to specific IP hashes (up to 200 rules).</li>
<li>Custom honeypot paths (up to 5) with a reserved-route safety list.</li>
<li>Email alerts for honeypot hits and parameter explosion patterns.</li>
<li>Robots.txt trap entries that catch non-compliant scrapers.</li>
<li>Optional override that suppresses the WordPress core <code>/wp/v2/users</code> REST endpoints (username-enumeration hardening, off by default and easy to opt back in).</li>
</ul>
<h4>Optimize tier</h4>
<ul>
<li>Full REST API at <code>/wp-json/ai-tracker/v1/</code> with API key authentication and 13 endpoints (status, stats, page, trends, bots, top-pages, report, export, alerts, honeypot, crawled-pages, etc.).</li>
<li>GeoIP location tracking with MaxMind GeoLite2.</li>
<li>Crawl Analytics tab with deeper traffic-quality metrics.</li>
<li>Extended data retention up to 365 days.</li>
</ul>
<h4>Scale tier</h4>
<ul>
<li>Backup & restore — export all data as JSON; import with merge or replace modes.</li>
<li>Unlimited retention.</li>
<li>Priority support.</li>
</ul>
<p>The paid build is a drop-in upgrade: same plugin slug, same database tables, same option keys, so all your historical bot data carries over with no migration step on your part.</p>


Crawlantix AI Bot Tracker

Track 30+ AI crawlers visiting your site with real-time dashboards, logging-only honeypot, and bot verification.

Erik

gardenboi

<p>Are you unsatisfied with your current antispam solution for Contact Form 7? It might be using an ineffective method to combat the specific type of bot attacks you’re facing. Fortunately, I have a solution for you!<br />
Antispam for Contact Form 7 is a simple yet highly effective plugin that protects your mailbox from bot flooding. Say goodbye to tedious configurations and captchas, which often lead to reduced conversions and inconvenience for genuine users. Our plugin utilizes a combination of on-page and off-page bot traps, along with an auto-learning mechanism powered by a statistical “Bayesian” spam filter called B8.<br />
CF7-AntiSpam seamlessly integrates with <a href="https://wordpress.org/plugins/flamingo/" rel="ugc">Flamingo</a> and enhances its functionality. When both plugins are installed, Flamingo gains additional controls, and an extra dashboard widget is enabled.</p>
<h3>SETUP</h3>
<p>Basic – Install and go! No configuration, keys, or registrations are required to activate the antispam protection. In this case, some protections, such as fingerprinting, language checks, and honeypots, will be enabled.<br />
Advanced – For CF7A to properly analyze the email content using its dictionary, it needs to parse the input message field of your form. To notify the antispam to check this field, you’ll need to add a “marker” to each contact form on your website. Simply add ‘flamingo_message: “[your-message]”‘ in the additional settings panel of each contact form you want to secure. This process follows the same method used with Flamingo. While this step may seem tedious, it is required for advanced text statistical analysis. Without it, the B8 filter cannot be enabled.<br />
GeoIP – (Optional) If you need to restrict which countries or languages can email you, you can enable this functionality. To enable GeoIP, you’ll need to agree to the GeoLite2 End User License Agreement and sign up for GeoLite2 Downloadable Databases. This will provide you with the required key to download the database. For detailed instructions, please refer to the dedicated section in the cf7-antispam plugin settings.</p>
<h3>Antispam Available Tests</h3>
<p>✅ Browser Fingerprinting<br />
✅ Language checks (Geo-ip, http headers and browser)<br />
✅ Honeypot<br />
️✅ Honeyform*<br />
✅ Domain Name System Blackhole List (DNSBL)<br />
✅ Blocklists (with automatic ban after N failed attempts, user defined ip exclusion list)<br />
✅ Hidden fields with encrypted unique hash<br />
✅ Time elapsed (with min/max values)<br />
✅ Prohibited words in message/email and user agent<br />
✅ B8 statistical “Bayesian” spam filter<br />
✅ High Entropy / Gibberish checks<br />
✅ Identity protection<br />
✅ Webmail protection</p>
<h3>Extends Flamingo and turns it into a spam manager!</h3>
<p>With this plugin, you can now review emails and train B8 to identify spam and legitimate messages. This feature proves useful, especially during the initial stages when some spam emails may slip through.<br />
Already using Flamingo? Even better! Just remember to add ‘flamingo_message: “[your-message]”‘ to the advanced settings (similar to other Flamingo labels) before activating the plugin. Alternatively, you can explore the advanced options and select “rebuild dictionary.”<br />
Upon activating CF7A, all previously collected emails will be parsed, and B8 will learn and develop its vocabulary. This pre-trained algorithm gives you a head start. How cool is that?<br />
Additional Notes:<br />
– A new column has been added to the right side of the Flamingo inbound page, displaying the level of spaminess for each email.<br />
– If you unban an email on the Flamingo “inbound” page, the corresponding IP will be removed from the blocklist. However, marking an email as spam will not blocklist the IP again.<br />
– Before activating this plugin, please make sure to mark all spam emails as spam in the Flamingo inbound section. This auto-training process will help the B8 algorithm.<br />
– If you receive a spam message, please avoid deleting it from the “ham” section. Instead, place it in the spam section to teach B8 how to differentiate between spam and legitimate messages.</p>
<h3>B8 statistical “Bayesian” Filter</h3>
<p>Originally created by <a href="https://en.wikipedia.org/wiki/Gary_Robinson" rel="nofollow ugc">Gary Robinson</a> <a href="https://www.linuxjournal.com/article/6467" rel="nofollow ugc">b8 is a statistical “Bayesian”</a> spam filter implemented in PHP.<br />
The B8 filter is a foundational example of <strong>Machine Learning (ML)</strong> for text classification, representing an early, yet powerful, statistical approach in Natural Language Processing (NLP). This approach precedes feature-weighting methods like <strong>TF-IDF</strong>, which in turn paved the way for modern deep learning architectures, such as <strong>Transformers</strong> and <strong>GPT</strong>.<br />
The filter tells you whether a text is spam or not, using statistical text analysis. What it does is: you give b8 a text and it returns a value between 0 and 1, saying it’s ham when it’s near 0 and saying it’s spam when it’s near 1. See <a href="https://nasauber.de/opensource/b8/readme.html#how-does-it-work" rel="nofollow ugc">How does it work?</a> for details about this.<br />
To be able to distinguish spam and ham (non-spam), b8 first has to learn some spam and some ham texts. If it makes mistakes when classifying unknown texts or the result is not distinct enough, b8 can be told what the text actually is, getting better with each learned text.<br />
This takes place on your own server without relying on third-party services.<br />
More info: <a href="https://nasauber.de/opensource/b8/" rel="nofollow ugc">nasauber.de</a></p>
<h3>Identity protection</h3>
<p>To fully protect the forms, it may be necessary to enable a couple of additional controls, because bots use the public data of the website to spam on it.<br />
– The first is user related and denies those who are not logged in the possibility of asking (sensitive) information about the user via wp-api and the protection for the xmlrpc exploit wordpress.<br />
– The second one is the WordPress protection that will obfuscate sensitive WordPress and server data, adding some headers in order to enhance security against xss and so on.<br />
Will be hidden the WordPress and WooCommerce version (wp_generator, woo_version), pingback (X-Pingback), server (nginx|apache|…) and php version (X-Powered-By), enabled xss protection headers (X-XSS-Protection), removes rest api link from header (but it will only continue to work if the link is not made public).</p>
<h3>Mailbox Protection (Multiple Send)</h3>
<p>Enhance email security by enabling the “Multiple Send” feature, which prevents consecutive email submissions to the user’s mailbox. This measure is effective in thwarting automated spam attempts and ensures a secure communication environment.</p>
<h3>Security & Privacy: A Local-First Approach</h3>
<p>AntiSpam for Contact Form 7 is built with your security and privacy as the <strong>top priority</strong>. Unlike many modern anti-spam solutions that rely on external cloud services or third-party subscriptions, our plugin is designed to run <strong>entirely on your own WordPress installation</strong>.</p>
<ul>
<li><strong>100% Local Processing:</strong> All anti-spam logic, checks, and data processing are performed directly on your server. <strong>No data is ever sent to, or stored by, any external third-party service</strong> (including ours).</li>
<li><strong>Not a Software as a Service (SaaS):</strong> This plugin is a standalone, self-contained software solution, not an interface to a paid or subscription-based external service. Once installed, it works autonomously.</li>
<li><strong>Enhanced Security:</strong> Since there is <strong>no central server or external API endpoint</strong> to communicate with, your website is immune to potential risks associated with centralized services, such as Single Point of Failure or data breach risks.</li>
</ul>
<p>You retain complete control and ownership over the security of your Contact Form 7 submissions.</p>
<h3>Privacy Notices</h3>
<p>AntiSpam for Contact Form 7 only processes the IP but doesn’t store any personal data directly from the user input. However, it creates a dictionary of spam and ham (non-spam) words in the WordPress database.<br />
This dictionary is built from words found in the submitted messages, meaning it <strong>may contain words that were part of the user’s e-mail message or personal data</strong>. This data is “degenerated,” which means the words might be normalized or altered before being stored.<br />
The sole purpose of this word collecting is to build a dictionary used for local, decentralized spam detection.</p>
<h3>Support</h3>
<p>Community support: via the <a href="https://wordpress.org/support/plugin/contact-form-7-antispam/" rel="ugc">support forums</a> on wordpress.org<br />
Bug reporting (preferred): file an issue on <a href="https://github.com/erikyo/contact-form-7-antispam" rel="nofollow ugc">GitHub</a></p>
<h4>Contribute</h4>
<p>We love your input! We want to make contributing to this project as easy and transparent as possible, whether it’s:</p>
<ul>
<li>Reporting a bug</li>
<li>Testing the plugin with different user agent and report fingerprinting failures</li>
<li>Discussing the current state, features, improvements</li>
<li>Submitting a fix or a new feature</li>
</ul>
<p>We use GitHub to host code, to track issues and feature requests, as well as accept pull requests.<br />
By contributing, you agree that your contributions will be licensed under its GPLv2 License.</p>
<p>My goal is to create an antispam that protects cf7 definitively without relying on external services. And free for everyone.<br />
if you want to help me, <a href="https://github.com/erikyo/contact-form-7-antispam" rel="nofollow ugc">GitHub</a> is the right place 😉</p>
<h3>copyright</h3>
<p>AntiSpam for Contact Form 7, Copyright 2021 Codekraft Studio<br />
AntiSpam for Contact Form 7 is distributed under the terms of the GNU GPL</p>
<p>This program is free software: you can redistribute it and/or modify<br />
it under the terms of the GNU General Public License as published by<br />
the Free Software Foundation, either version 3 of the License, or<br />
(at your option) any later version.</p>
<p>This program is distributed in the hope that it will be useful,<br />
but WITHOUT ANY WARRANTY; without even the implied warranty of<br />
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.<br />
See the LICENSE file for more details.</p>
<h4>Resources</h4>
<ul>
<li>Contact Form 7 and Flamingo © 2021 Takayuki Miyoshi,<a href="https://it.wordpress.org/plugins/contact-form-7/" rel="nofollow ugc">LGPLv3 or later</a></li>
<li>B8 https://nasauber.de/opensource/b8/, © 2021 Tobias Leupold, <a href="https://gitlab.com/l3u/b8/-/tree/ab26daa6b293e6aa059d24ce7cf77af6c8b9b052/LICENSES" rel="nofollow ugc">LGPLv3 or later</a></li>
<li>GeoLite2 <a href="https://www.maxmind.com/en/geolite2/eula" rel="nofollow ugc">license</a></li>
<li>GeoIP2 PHP API <a href="https://github.com/maxmind/GeoIP2-php" rel="nofollow ugc">GeoIP2-php</a></li>
<li>chart.js https://www.chartjs.org/, © 2021 Chart.js <a href="https://github.com/chartjs/Chart.js/graphs/contributors" rel="nofollow ugc">contributors</a>, <a href="https://github.com/chartjs/Chart.js/blob/master/LICENSE.md" rel="nofollow ugc">MIT</a></li>
<li>Sudden Shower in the Summer, Public domain, Wikimedia Commons https://commons.wikimedia.org/wiki/File:Sudden_Shower_in_the_Summer_(5759500422).jpg</li>
</ul>
<h3>Contibutions</h3>
<p>Mirek Długosz – <a href="https://github.com/erikyo/cf7-antispam/pull/30" rel="nofollow ugc">#30</a> fixes a crash that occurred when analysing flamingo metadata<br />
MeliEve – <a href="https://github.com/wp-blocks/cf7-antispam/pull/42" rel="nofollow ugc">#42</a> Fix “internal_server_error” when message is empty<br />
MeliEve – <a href="https://github.com/wp-blocks/cf7-antispam/pull/61" rel="nofollow ugc">#61</a>  Handle deferrer script loading<br />
Zodiac1978 – <a href="https://github.com/wp-blocks/cf7-antispam/pull/67" rel="nofollow ugc">#67</a> Remove warning for unsafe email configuration w/o protection<br />
JohnHooks – <a href="https://github.com/wp-blocks/cf7-antispam/pull/61" rel="nofollow ugc">#66</a> Readme + plugin env<br />
sdellenb – <a href="https://github.com/wp-blocks/cf7-antispam/pull/163" rel="nofollow ugc">#66</a> Fix $reason parameter for calling cf7a_ban_by_ip</p>
<h3>Special thanks</h3>
<p>This project is tested with BrowserStack. <a href="https://www.browserstack.com/" rel="nofollow ugc">Browserstack</a></p>
<h3>MaxMind GeoIP2</h3>
<p>This plugin on demand can enable GeoLite2 created by MaxMind, available from <a href="https://www.maxmind.com" rel="nofollow ugc">https://www.maxmind.com</a><br />
While enabled you may <strong>have to mention it in the privacy policy</strong> of your site, depending on the law regulating privacy in your state!<br />
* GeoIP2 databases <a href="https://www.maxmind.com/en/accounts/current/geoip/downloads" rel="nofollow ugc">GeoLite2 Country</a></p>
<h3>DNSBL servers privacy policies</h3>
<ul>
<li>dnsbl-1.uceprotect.net <a href="http://www.uceprotect.net/en/index.php?m=13&s=0" rel="nofollow ugc">www.uceprotect.net license</a></li>
<li>dnsbl-2.uceprotect.net <a href="http://www.uceprotect.net/en/index.php?m=13&s=0" rel="nofollow ugc">www.uceprotect.net license</a></li>
<li>dnsbl-3.uceprotect.net <a href="http://www.uceprotect.net/en/index.php?m=13&s=0" rel="nofollow ugc">www.uceprotect.net license</a></li>
<li>dnsbl.sorbs.net <a href="http://www.sorbs.net/information/faq/" rel="nofollow ugc">sorbs.net license</a></li>
<li>zen.spamhaus.org <a href="https://www.spamhaus.org/organization/dnsblusage/" rel="nofollow ugc">spamhaus.org license</a></li>
<li>bl.spamcop.net <a href="https://www.spamcop.net/fom-serve/cache/297.html" rel="nofollow ugc">spamcop.net license</a></li>
<li>b.barracudacentral.org <a href="https://www.barracuda.com/company/legal/trust-center/data-privacy/privacy-policy" rel="nofollow ugc">barracudacentral.org privacy-policy</a></li>
<li>dnsbl.dronebl.org <a href="https://dronebl.org/docs/faq" rel="nofollow ugc">dronebl.org</a></li>
<li>all.spamrats.com <a href="https://spamrats.com/tos.php" rel="nofollow ugc">spamrats.com tos</a></li>
<li>bl.ipv6.spameatingmonkey.net <a href="https://spameatingmonkey.com/faq" rel="nofollow ugc">spameatingmonkey.net</a></li>
</ul>
<h3>Inspirations, links</h3>
<ul>
<li>Nikolai Tschacher <a href="https://incolumitas.com/pages/BotOrNot/" rel="nofollow ugc">incolumitas.com</a></li>
<li>Antoine Vastel <a href="https://github.com/antoinevastel/fpscanner" rel="nofollow ugc">fp-scanner</a>/<a href="https://github.com/antoinevastel/fp-collect" rel="nofollow ugc">fp-collect</a></li>
<li>Niespodd <a href="https://github.com/niespodd/browser-fingerprinting" rel="nofollow ugc">niespodd</a></li>
<li>Thomas Breuss <a href="https://gist.github.com/tbreuss/74da96ff5f976ce770e6628badbd7dfc" rel="nofollow ugc">tbreuss</a></li>
<li>Domain Name System-based blackhole list <a href="https://en.wikipedia.org/wiki/Domain_Name_System-based_blackhole_list" rel="nofollow ugc">wiki</a></li>
<li>dnsbl list <a href="https://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists" rel="nofollow ugc">wiki</a></li>
</ul>


AntiSpam for Contact Form 7

A trustworthy antispam plugin for Contact Form 7. Wave goodbye to spam and keep your inbox clean!

Crawlantix AI Bot Tracker vs AntiSpam for Contact Form 7