chrisguitarguy

George Pattichis

<p>WordPress, by default, includes a simple robots.txt file that’s dynamically generated from within the WP application. This is great, but how do you easily change the content?</p>
<p>Enter <strong>WP Robots Txt</strong>, a plugin that adds an additional field to the “Reading” admin page where you can do just that. No manual coding or file editing required!</p>
<p>Simply visit https://your-site.com/wp-admin/options-reading.php and you can control the contents of your https://your-site.com/robots.txt</p>
<p><a href="https://wordpress.org/plugins/wp-robots-txt/#developers" rel="ugc">Changelog</a></p>


WP Robots Txt

WP Robots Txt Allows you to edit the content of your robots.txt file.

apasionados

<p>This plugin adds lines to the virtual robots.txt file that WordPress creates automagically if the file is not present physically on the server to block the OpenAI ChatGPT-User bot that is used by plugins in ChatGPT to crawl websites. Here is the <a href="https://platform.openai.com/docs/plugins/bot" rel="nofollow ugc">information about the ChatGPT-User bot</a>. It also blocks the the <a href="https://platform.openai.com/docs/gptbot" rel="nofollow ugc">GPTBot user agent</a> that may potentially be used to improve future models.</p>
<blockquote>
<p>Please be aware of what this plugin does exactly: If you activate the plugin it will add some lines to the robots.txt file to tell Open AI Chat GPT bots (ChatGPT-User + GPTBot) to <strong>not to crawl and index</strong> parts of this website.</p>
</blockquote>
<p><strong>Please only activate this plugin if you know what you’re doing</strong>.</p>
<h4>What can I do with this plugin?</h4>
<p>This plugin adds the necessary lines to the virtual robots.txt file that WordPress creates automagically to block the OpenAI ChatGPT-User bot that is used by plugins in ChatGPT to crawl websites + the GPTBot user agent that may potentially be used to improve future models.</p>
<h4>What ideas is this plugin based on?</h4>
<p>None. After the launch of the OpenAI ChatGTP plugins that crawl websites, we needed a fast solution to add these lines to the robots.txt file and this was the solution we coded.</p>
<p>In August 2023 OpenAI added information about the GPTBot so we added this user-agent to the plugin.</p>
<h4>What is the robots.txt file?</h4>
<p>The robots.txt file is a plain text file located at the root folder of a domain (or subdomain) which tells web crawlers (like Googlebot) what parts of the website they should access and index.</p>
<p>The first thing a search engine crawler looks at when it is visiting a page is the robots.txt file and it controls how search engine spiders see and interact with the web pages.</p>
<h4>System requirements</h4>
<p>PHP version 7.4 or greater.<br />
We require 7.4 or higher because we believe that everybody should be running a modern PHP version. When releasing this plugin <a href="https://wordpress.org/about/requirements/" rel="ugc">WordPress recommends PHP 7.4 or higher</a>. We would recommend 8.0 or higher as 7.4 has no security updates since 28 Nov 2022.</p>
<h4>How to get this plugin in your Language!</h4>
<p>The first release is avaliable in English and Spanish. In the “languages” folder we have included the necessary files to translate this plugin.</p>
<p>If you would like the plugin in your language and you’re good at translating, please use the <a href="https://translate.wordpress.org/" rel="nofollow ugc">native WordPress Translation</a> functionality.</p>
<p>New to Translating a plugin? First read through the <a href="https://make.wordpress.org/polyglots/handbook/tools/glotpress-translate-wordpress-org/" rel="nofollow ugc">Translator Handbook</a>, then select your locale at <a href="https://translate.wordpress.org/" rel="nofollow ugc">Translating WordPress</a> and finally go to the <a href="https://translate.wordpress.org/projects/wp-plugins/block-ecommerce-assets-via-robots-txt/" rel="nofollow ugc">translation page for this plugin</a> to translate it.</p>
<h4>Further Reading</h4>
<p>You can access the description of the plugin in Spanish at: <a href="https://apasionados.es/blog/" rel="nofollow ugc">Block Chat GPT via robots.txt en español</a>.</p>
<h3>Contact</h3>
<p>For further information please send us an <a href="https://apasionados.es/contacto/index.php?desde=wordpress-org-block-chatgpt-assets-via-robots-txt" rel="nofollow ugc">email</a>.</p>


Block Chat GPT via robots.txt

Blocks the ChatGPT-User bot that is used by plugins in ChatGPT to crawl websites through the WordPress virtual robots.txt file

WP Robots Txt vs Block Chat GPT via robots.txt