aicrawlerguard.com

AI Crawler Guard detects popular AI-focused crawlers and lets you block or allow them from scraping your WordPress site. Everything runs locally—rules, decisions, logs, and history stay on your server.
<ul>
<li>Built-in signatures for common AI bots, plus optional custom patterns</li>
<li>Global and per-crawler allow/block controls</li>
<li>Local crawler registry with first/last seen timestamps and hit counters</li>
<li>Lightweight logs with optional retention cap</li>
<li>No external calls or remote storage</li>
</ul>
<h3>Requirements</h3>
<ul>
<li>WordPress 5.8 or newer</li>
<li>PHP 7.4+</li>
<li>Works with Classic Editor, Gutenberg, WooCommerce, and major managed hosts</li>
<li>Respects wp-admin, wp-login, REST API, cron, and CLI contexts automatically</li>
</ul>
<h3>Performance</h3>
<ul>
<li>Detection runs once per front-end request (small O(n) UA scan)</li>
<li>No blocking of social preview bots (Facebook, Meta, Twitter, LinkedIn) by default</li>
<li>Assets load only on plugin admin pages and stay under 10 KB</li>
<li>Logs stay capped by your configured limit with auto-pruning</li>
</ul>
<h3>Privacy</h3>
All detection logic and data storage live entirely inside WordPress. No data is sent to third-party services in v1.

AI Crawler Guard

Detect and control AI crawler access with local logging and history.

zicygp

Zicy AI Access Monitor helps site owners understand when known AI crawlers and AI assistant retrieval agents access their WordPress content.
The plugin logs self-identifying AI-related User-Agents, summarizes raw visits into daily totals, and provides an admin dashboard for reviewing AI access patterns across your site. It is designed for AI visibility, AI search optimization, and content monitoring workflows.
What it tracks:
<ul>
<li>Search crawlers — AI systems that may crawl content for search or answer retrieval workflows, such as OAI-SearchBot and PerplexityBot</li>
<li>Training crawlers — AI-related crawlers that may access content for model training or dataset-building workflows, such as GPTBot, ClaudeBot, CCBot, Amazonbot, Bytespider, CohereBot, Mistral-ai, and Meta-WebIndexer</li>
<li>User proxies — AI assistant fetchers that retrieve pages when responding to user requests, such as ChatGPT-User, Claude-User, Perplexity-User, Gemini-Deep-Research, Google-Agent, and BingPreview</li>
<li>AI control agents — AI-related User-Agents such as Google-Extended that may represent AI data-use control behavior rather than ordinary search crawling</li>
</ul>
How it works:
<ol>
<li>Matching visits are recorded as temporary raw logs.</li>
<li>URLs are normalized to reduce duplicates caused by common tracking parameters.</li>
<li>IP addresses are anonymized by default before being stored.</li>
<li>Each night at 02:15 site time, raw logs older than today are summarized into daily totals.</li>
<li>Raw logs that have been rolled up are deleted.</li>
<li>Daily summary records are retained so you can review historical AI access activity.</li>
</ol>
In your WordPress admin dashboard:
<ul>
<li>View daily summaries of known AI bot and AI assistant access</li>
<li>See today’s raw logs before the next rollup</li>
<li>Filter by date, bot, type, or signature</li>
<li>Review top pages accessed by AI assistant user proxies</li>
<li>Compare week-over-week activity for top pages</li>
<li>See vendor diversity indicators for accessed pages</li>
<li>Export daily summary data as CSV</li>
<li>Receive weekly email summaries with top accessed pages</li>
<li>Generate copy-ready AI Tracking Briefs for selected pages</li>
</ul>
Why it matters:
<ul>
<li>See which known AI systems are accessing your content</li>
<li>Identify which pages AI assistants retrieve most often</li>
<li>Support AI search optimization and AI visibility reporting</li>
<li>Understand differences between crawler, training, control, and user-triggered AI access</li>
<li>Build structured page briefs that can be used in AI visibility workflows</li>
</ul>
Bot allowlist includes:
<ul>
<li>OpenAI: GPTBot, OAI-SearchBot, ChatGPT-User</li>
<li>Anthropic: ClaudeBot, Claude-User, Claude-SearchBot</li>
<li>Perplexity: PerplexityBot, Perplexity-User</li>
<li>Google: Google-Extended, Gemini-Deep-Research, Google-Agent</li>
<li>Microsoft: BingPreview</li>
<li>Amazon: Amazonbot</li>
<li>Common Crawl: CCBot</li>
<li>ByteDance: Bytespider</li>
<li>Cohere: CohereBot</li>
<li>Mistral: Mistral-ai</li>
<li>Meta: Meta-WebIndexer</li>
</ul>
Developers can extend or modify the allowlist with the <code>gp_aibal_ai_bots</code> filter.
Important detection note:
This plugin detects only self-identifying User-Agents. It does not guarantee that every visit is from the claimed company or system, because User-Agent headers can be spoofed. The plugin should be used as a visibility and reporting aid, not as a security or access-control tool.
<h3>Privacy</h3>
Zicy AI Access Monitor is designed to keep raw request-level data temporary.
The plugin stores matching AI-related access events in your WordPress database, anonymizes IP addresses by default, rolls raw logs into daily summaries, and deletes rolled-up raw logs. Daily summaries are retained for reporting. The plugin does not send logged access data to Zicy or any third-party service.
Site owners are responsible for ensuring that their use of access logging complies with applicable privacy laws and their own privacy policy.

Zicy AI Access Monitor

Monitor known AI crawlers and AI assistant fetches, with daily rollups, CSV exports, weekly summaries, and page briefs.

AI Crawler Guard vs Zicy AI Access Monitor