apasionados

This plugin adds lines to the virtual robots.txt file that WordPress creates automagically if the file is not present physically on the server to block the Common Crawl bot (CCBot) that is used to train AI algorithms like ChatGPT (in version 3.5 it makes up 60% of the training data mix) to crawl websites. Here is the <a href="https://commoncrawl.org/big-picture/frequently-asked-questions/" rel="nofollow ugc">information about the Common Crawl bot (CCBot)</a>.
<blockquote>
Please be aware of what this plugin does exactly: If you activate the plugin it will add some lines to the robots.txt file to tell Common Crawl bots to not to crawl and index parts of this website.
</blockquote>
Please only activate this plugin if you know what you’re doing.
<h4>What can I do with this plugin?</h4>
This plugin adds the necessary lines to the virtual robots.txt file that WordPress creates automagically to block the Common Crawl bot (CCBot) is used to train AI algorithms like ChatGPT to crawl websites.
<h4>What ideas is this plugin based on?</h4>
Our plugin to <a href="https://wordpress.org/plugins/block-chat-gpt-via-robots-txt/" rel="ugc">block the OpenAI ChatGTP bot</a> after the launch of the OpenAI ChatGPT plugins that crawl websites.
<h4>What is the robots.txt file?</h4>
The robots.txt file is a plain text file located at the root folder of a domain (or subdomain) which tells web crawlers (like Googlebot) what parts of the website they should access and index.
The first thing a search engine crawler looks at when it is visiting a page is the robots.txt file and it controls how search engine spiders see and interact with the web pages.
<h4>System requirements</h4>
PHP version 7.4 or greater. 
We require 7.4 or higher because we believe that everybody should be running a modern PHP version. When releasing this plugin <a href="https://wordpress.org/about/requirements/" rel="ugc">WordPress recommends PHP 7.4 or higher</a>. We would recommend 8.0 or higher as 7.4 has no security updates since 28 Nov 2022.
<h4>How to get this plugin in your Language!</h4>
The first release is avaliable in English and Spanish. In the “languages” folder we have included the necessary files to translate this plugin.
If you would like the plugin in your language and you’re good at translating, please use the <a href="https://translate.wordpress.org/" rel="nofollow ugc">native WordPress Translation</a> functionality.
New to Translating a plugin? First read through the <a href="https://make.wordpress.org/polyglots/handbook/tools/glotpress-translate-wordpress-org/" rel="nofollow ugc">Translator Handbook</a>, then select your locale at <a href="https://translate.wordpress.org/" rel="nofollow ugc">Translating WordPress</a> and finally go to the <a href="https://translate.wordpress.org/projects/wp-plugins/block-ecommerce-assets-via-robots-txt/" rel="nofollow ugc">translation page for this plugin</a> to translate it.
<h4>Further Reading</h4>
You can access the description of the plugin in Spanish at: <a href="https://apasionados.es/blog/" rel="nofollow ugc">Block Common Crawl bot (CCBot) via robots.txt en español</a>.
<h3>Contact</h3>
For further information please send us an <a href="https://apasionados.es/contacto/index.php?desde=wordpress-org-block-chatgpt-assets-via-robots-txt" rel="nofollow ugc">email</a>.

Block Common Crawl via robots.txt

Blocks the Common Crawl bot (CCBot) that gathers data that is used to train AI algorithms like ChatGPT through the WordPress virtual robots.txt file

Esteban

Do you want to keep your WordPress blog updated automatically with fresh, SEO-optimized content? 📈 With BotWriter, artificial intelligence takes care of writing and publishing articles for you! 
You can go on vacation or stop worrying about your blog:
🔹 Automatically publish content every day / every week 
🔹 Generate 100% original articles with AI images 
🔹 NEW: Choose from 7+ AI text providers (OpenAI, Claude, Gemini, Mistral, Groq…) 
🔹 NEW: 8 image providers including Flux 2, Nano Banana Pro & DALL-E 3 
🔹 NEW: Article templates for consistent, professional content 
🔹 Optimize your blog to rank higher on Google 
🔹 Set it up in minutes and let AI do the rest
In this video, we’ll show you how to set up BotWriter in just a few steps so your blog stays active effortlessly. Take advantage of AI-powered automation and streamline your content strategy! 🚀 
<iframe loading="lazy" class="youtube-player" width="750" height="422" src="https://www.youtube.com/embed/PatljFLDNwI?version=3&rel=1&showsearch=0&showinfo=1&iv_load_policy=1&fs=1&hl=en-US&autohide=2&wmode=transparent" allowfullscreen="true" style="border:0;" sandbox="allow-scripts allow-same-origin allow-popups allow-presentation allow-popups-to-escape-sandbox"></iframe>
BotWriter has been designed to simplify content creation for blogs, websites, and online stores. Version 2.0 introduces multi-provider architecture—choose the best AI for each job. Use GPT-5 for creative writing, Claude for technical content, or Gemini for multilingual articles. Pair them with cutting-edge image models like Flux 2 Pro, Google’s Nano Banana, or Stable Diffusion.
BotWriter is capable of creating entire blog content in minutes! Check out the example: 
<iframe loading="lazy" class="youtube-player" width="750" height="422" src="https://www.youtube.com/embed/MpJ0KHKRYi8?version=3&rel=1&showsearch=0&showinfo=1&iv_load_policy=1&fs=1&hl=en-US&autohide=2&wmode=transparent" allowfullscreen="true" style="border:0;" sandbox="allow-scripts allow-same-origin allow-popups allow-presentation allow-popups-to-escape-sandbox"></iframe>
How to Auto-Generate Articles from an RSS Feed | BotWriter + NASA RSS 
<iframe loading="lazy" class="youtube-player" width="750" height="422" src="https://www.youtube.com/embed/7aq0496XwY0?version=3&rel=1&showsearch=0&showinfo=1&iv_load_policy=1&fs=1&hl=en-US&autohide=2&wmode=transparent" allowfullscreen="true" style="border:0;" sandbox="allow-scripts allow-same-origin allow-popups allow-presentation allow-popups-to-escape-sandbox"></iframe>
<h3>Key Features</h3>
🆕 Multi-Provider AI Text Generation 
Choose from 7 different AI providers for text generation: 
* OpenAI: GPT-5, GPT-4.1, GPT-4o series 
* Anthropic: Claude Sonnet 4.5, Claude Haiku 
* Google: Gemini 2.5 Pro, Gemini 2.5 Flash 
* Mistral: Mistral Large, Mistral Small 
* Groq: Llama 3.3 70B (ultra-fast inference) 
* OpenRouter: Access to 200+ models through one API 
* DeepSeek: Cost-effective alternatives
⛽ Multi-Provider AI Image Generation 
5 image providers with 50+ models: 
* OpenAI DALL-E: GPT-Image-1, DALL-E 3 
* Fal.ai: Flux 2 Pro/Dev, Nano Banana Pro (Google Gemini), Seedream 4.5 
* Replicate: Flux 1.1 Pro, Ideogram V3, Imagen 4 
* Stability AI: Stable Diffusion 3.5, SDXL 
* Cloudflare Workers AI: Flux Schnell (fast & affordable)
⛽ Article Templates 
Create professional, consistent content with reusable templates: 
* Pre-configured prompts for different content types 
* SEO-optimized structures 
* Custom style guidelines 
* One-click template application to any task
Scheduled Content Automation 
BotWriter automates your content creation workflow. Schedule tasks to publish AI-generated articles on specific days and times—daily, weekly, or custom schedules. Perfect for maintaining a consistent publishing calendar without manual intervention.
Multiple Content Sources 
* AI-Generated Content: Create 100% original articles from scratch using custom prompts and AI keywords 
* RSS Feed Integration: Transform RSS feeds into unique, rewritten articles automatically 
* News Aggregation: Pull trending news stories and rewrite them for your niche 
* Super Tasks: Generate entire content series—create multiple related articles in bulk with a single configuration
Smart Content Generation 
Each scheduled task runs automatically via WordPress cron, generating: 
* SEO-optimized titles and content 
* Relevant tags and categories 
* AI-generated featured images (configurable per task) 
* Custom post lengths (from short to comprehensive articles)
Instant Publishing with “Write Now” 
Need content immediately? The “Write Now” feature creates and publishes a single post instantly with real-time progress tracking—no waiting for scheduled cron jobs.
Full Customization 
* Choose from multiple AI text and image models per task 
* Configure image size and quality 
* Customize writing style, tone, and narrative voice 
* Set post status (publish, draft, pending, private) 
* Control publishing frequency and timing
Your API Keys, Your Control 
BotWriter is 100% free. You only pay the AI providers directly for API usage—BotWriter doesn’t charge subscription fees or take a cut of your API costs.
<h3>External Services</h3>
This plugin relies on third-party external services to generate text and images automatically, as well as to retrieve related content for your blog.
<pre><code>Service's Terms
 https://wpbotwriter.com/services-terms/

Privacy Policy
https://wpbotwriter.com/privacy-policy/
</code></pre>
Below is a summary of the external services used and the data they handle:
1. BotWriter Platform (wpbotwriter.com) 
These endpoints are part of the service provided by our platform: 
– Activation: <code>https://wpbotwriter.com/public/activation.php</code> – Validates and activates the plugin 
– Generate Posts: <code>https://wpbotwriter.com/public/redis_api_cola.php</code> and <code>redis_api_finish.php</code> – Process content generation 
– User Data: <code>https://wpbotwriter.com/public/getUserData.php</code> – Subscription data for Pro users 
– Email Confirmation: <code>https://wpbotwriter.com/public/envio_email_confirmacion.php</code> – Optional email registration
2. Text Generation Providers 
When configured, the plugin connects to these services for text generation: 
– OpenAI API – <a href="https://openai.com/terms/" rel="nofollow ugc">Terms</a> | <a href="https://openai.com/privacy/" rel="nofollow ugc">Privacy</a> 
– Anthropic API (Claude) – <a href="https://www.anthropic.com/legal/consumer-terms" rel="nofollow ugc">Terms</a> | <a href="https://www.anthropic.com/legal/privacy" rel="nofollow ugc">Privacy</a> 
– Google AI (Gemini) – <a href="https://ai.google.dev/terms" rel="nofollow ugc">Terms</a> | <a href="https://policies.google.com/privacy" rel="nofollow ugc">Privacy</a> 
– Mistral AI – <a href="https://mistral.ai/terms/" rel="nofollow ugc">Terms</a> | <a href="https://mistral.ai/privacy/" rel="nofollow ugc">Privacy</a> 
– Groq – <a href="https://groq.com/terms-of-use/" rel="nofollow ugc">Terms</a> | <a href="https://groq.com/privacy-policy/" rel="nofollow ugc">Privacy</a> 
– OpenRouter – <a href="https://openrouter.ai/terms" rel="nofollow ugc">Terms</a> | <a href="https://openrouter.ai/privacy" rel="nofollow ugc">Privacy</a>
3. Image Generation Providers 
When configured, the plugin connects to these services for image generation: 
– OpenAI DALL-E – <a href="https://openai.com/terms/" rel="nofollow ugc">Terms</a> | <a href="https://openai.com/privacy/" rel="nofollow ugc">Privacy</a> 
– Fal.ai – <a href="https://fal.ai/terms" rel="nofollow ugc">Terms</a> | <a href="https://fal.ai/privacy" rel="nofollow ugc">Privacy</a> 
– Replicate – <a href="https://replicate.com/terms" rel="nofollow ugc">Terms</a> | <a href="https://replicate.com/privacy" rel="nofollow ugc">Privacy</a> 
– Stability AI – <a href="https://stability.ai/terms-of-service" rel="nofollow ugc">Terms</a> | <a href="https://stability.ai/privacy-policy" rel="nofollow ugc">Privacy</a> 
– Cloudflare Workers AI – <a href="https://www.cloudflare.com/terms/" rel="nofollow ugc">Terms</a> | <a href="https://www.cloudflare.com/privacypolicy/" rel="nofollow ugc">Privacy</a>
Data Transmitted: The plugin sends user-supplied prompts, configuration parameters, and API keys when requesting content generation. Your API keys are stored encrypted in your WordPress database and transmitted securely to the respective providers.
By using this plugin, you acknowledge that it communicates with these third-party and external services, and that data is transmitted as described above. Please review the respective Terms of Service and Privacy Policies for each service.

BotWriter – Free AI Content Generator

Multi-provider AI content automation for WordPress. SEO-ready posts with GPT-5, Claude, Gemini + images from DALL-E, Flux, Nano Banana Pro & more.

Block Common Crawl via robots.txt vs BotWriter – Free AI Content Generator