bentannenbaum

<p>Search is moving inside AI assistants. When someone asks ChatGPT, Claude, Perplexity or Google’s AI a question, those assistants quietly fetch real web pages — and <strong>those fetches are the new impressions</strong>. Most analytics tools never show them, because AI crawlers don’t run JavaScript and don’t appear in client-side analytics.</p>
<p><strong>Aiso AI Crawler Analytics reads the requests that actually hit your server</strong> and turns them into a clean, agency-ready dashboard right inside wp-admin. Think of it as the “AI visibility” companion to your security log — focused entirely on AI search performance instead of threats.</p>
<h4>What you get</h4>
<ul>
<li><strong>Headline KPIs</strong> — total AI-crawler hits, “chat impressions” (live fetches during AI answers), distinct crawlers, and pages crawled.</li>
<li><strong>By AI platform</strong> — ChatGPT (OpenAI), Claude (Anthropic), Perplexity, Google/Gemini, Microsoft Copilot, Meta AI and more.</li>
<li><strong>By crawler / user-agent</strong> — the exact bots (GPTBot, ChatGPT-User, OAI-SearchBot, ClaudeBot, Claude-User, PerplexityBot, Googlebot, Bingbot, Bytespider, Applebot…).</li>
<li><strong>By purpose</strong> — training vs. citation-indexing vs. <strong>live chat impressions</strong> vs. search vs. agentic browsing, so you know whether a bot is just training a model or actively quoting you in an answer.</li>
<li><strong>By page</strong> — which URLs AI assistants read most, with per-page impression counts.</li>
<li><strong>Over time</strong> — a daily activity chart so you can see trends and spikes.</li>
<li><strong>Live hit log</strong> — recent crawler visits with country and a source-IP verification badge that flags possible spoofed user-agents.</li>
<li><strong>CSV export</strong> — hand clients a report in one click.</li>
</ul>
<h4>Built for agencies</h4>
<p>Connect the plugin to a free Aiso account and every site you manage reports into one central place, so your AI-visibility data is kept long-term and viewable across your whole portfolio. Not an agency? The plugin works completely on its own — install it and the dashboard fills up locally with <strong>nothing sent anywhere</strong>.</p>
<h4>Privacy-first</h4>
<p>Visitor IPs are anonymised by default (stored only as a salted hash). The plugin records AI-crawler requests, not your human visitors, and never transmits any data externally until you explicitly connect an Aiso account.</p>
<h3>External services</h3>
<p>This plugin connects to the Aiso API (https://www.getaiso.com) <strong>only when you enable syncing by connecting an account</strong>. It is used to (a) optionally generate an API key for your site and (b) store your AI-crawler hits centrally so they are retained long-term and can be viewed across multiple sites.</p>
<ul>
<li><strong>When data is sent:</strong> only while “Send data to Aiso” is enabled in Settings, and only for AI-crawler hits (not human visitors).</li>
<li><strong>What is sent:</strong> the requested path and URL, the crawler’s user-agent, the detected bot and company, an optionally-hashed source IP, a two-letter country code, the HTTP status, a timestamp, your site URL, and your Aiso API key.</li>
<li><strong>What is not sent:</strong> human-visitor analytics, post content, or personal data of your readers.</li>
</ul>
<p>By connecting an account you agree to Aiso’s terms and privacy policy:</p>
<ul>
<li>Terms: https://www.getaiso.com/quick_links/terms_of_service</li>
<li>Privacy policy: https://www.getaiso.com/quick_links/privacy_policy</li>
</ul>
<p>If you never connect an account, the plugin makes no external requests and stores all data locally on your own server.</p>


Aiso AI Crawler Analytics

See which AI assistants — ChatGPT, Claude, Perplexity, Google and more — crawl your site, broken down by assistant, crawler, purpose and page.

apasionados

<p>This plugin adds lines to the virtual robots.txt file that WordPress creates automagically if the file is not present physically on the server to block the Common Crawl bot (CCBot) that is used to train AI algorithms like ChatGPT (in version 3.5 it makes up 60% of the training data mix) to crawl websites. Here is the <a href="https://commoncrawl.org/big-picture/frequently-asked-questions/" rel="nofollow ugc">information about the Common Crawl bot (CCBot)</a>.</p>
<blockquote>
<p>Please be aware of what this plugin does exactly: If you activate the plugin it will add some lines to the robots.txt file to tell Common Crawl bots to <strong>not to crawl and index</strong> parts of this website.</p>
</blockquote>
<p><strong>Please only activate this plugin if you know what you’re doing</strong>.</p>
<h4>What can I do with this plugin?</h4>
<p>This plugin adds the necessary lines to the virtual robots.txt file that WordPress creates automagically to block the Common Crawl bot (CCBot) is used to train AI algorithms like ChatGPT to crawl websites.</p>
<h4>What ideas is this plugin based on?</h4>
<p>Our plugin to <a href="https://wordpress.org/plugins/block-chat-gpt-via-robots-txt/" rel="ugc">block the OpenAI ChatGTP bot</a> after the launch of the OpenAI ChatGPT plugins that crawl websites.</p>
<h4>What is the robots.txt file?</h4>
<p>The robots.txt file is a plain text file located at the root folder of a domain (or subdomain) which tells web crawlers (like Googlebot) what parts of the website they should access and index.</p>
<p>The first thing a search engine crawler looks at when it is visiting a page is the robots.txt file and it controls how search engine spiders see and interact with the web pages.</p>
<h4>System requirements</h4>
<p>PHP version 7.4 or greater.<br />
We require 7.4 or higher because we believe that everybody should be running a modern PHP version. When releasing this plugin <a href="https://wordpress.org/about/requirements/" rel="ugc">WordPress recommends PHP 7.4 or higher</a>. We would recommend 8.0 or higher as 7.4 has no security updates since 28 Nov 2022.</p>
<h4>How to get this plugin in your Language!</h4>
<p>The first release is avaliable in English and Spanish. In the “languages” folder we have included the necessary files to translate this plugin.</p>
<p>If you would like the plugin in your language and you’re good at translating, please use the <a href="https://translate.wordpress.org/" rel="nofollow ugc">native WordPress Translation</a> functionality.</p>
<p>New to Translating a plugin? First read through the <a href="https://make.wordpress.org/polyglots/handbook/tools/glotpress-translate-wordpress-org/" rel="nofollow ugc">Translator Handbook</a>, then select your locale at <a href="https://translate.wordpress.org/" rel="nofollow ugc">Translating WordPress</a> and finally go to the <a href="https://translate.wordpress.org/projects/wp-plugins/block-ecommerce-assets-via-robots-txt/" rel="nofollow ugc">translation page for this plugin</a> to translate it.</p>
<h4>Further Reading</h4>
<p>You can access the description of the plugin in Spanish at: <a href="https://apasionados.es/blog/" rel="nofollow ugc">Block Common Crawl bot (CCBot) via robots.txt en español</a>.</p>
<h3>Contact</h3>
<p>For further information please send us an <a href="https://apasionados.es/contacto/index.php?desde=wordpress-org-block-chatgpt-assets-via-robots-txt" rel="nofollow ugc">email</a>.</p>


Block Common Crawl via robots.txt

Blocks the Common Crawl bot (CCBot) that gathers data that is used to train AI algorithms like ChatGPT through the WordPress virtual robots.txt file

Aiso AI Crawler Analytics vs Block Common Crawl via robots.txt