apasionados

<p>This plugin adds lines to the virtual robots.txt file that WordPress creates automagically if the file is not present physically on the server to block some WooCommerce assets and Search pages that should not be indexed.</p>
<blockquote>
<p>Please be aware of what this plugin does exactly: If you activate the plugin it will add some lines to the robots.txt file to tell search engine bots to <strong>not to crawl and index</strong> parts of this website.</p>
</blockquote>
<p>This plugin blocks certain pages from WooCommerce (“add-to-cart”, “orderby”, “filter”, cart, account, checkout, …) from being crawled by search engines. Crawling these links normally requires a lot of CPU, memory and bandwidth because they are not cacheable and/or create crawling loops (while they are useless). Optimizing your robots.txt when running a WooCommerce online store allocates more processing power for pages that really matter.</p>
<p>This plugin also blocks the search pages from WordPress from getting indexed.</p>
<p><strong>Please only activate this plugin if you know what you’re doing</strong>.</p>
<h4>What can I do with this plugin?</h4>
<p>This plugin adds the necessary lines to the virtual robots.txt file that WordPress creates automagically to block to block WooCommerce assets that should not be indexed and the Search pages.</p>
<h4>What ideas is this plugin based on?</h4>
<p>None. We needed a fast solution to add these lines to the robots.txt file and this was the solution we coded.</p>
<h4>What is the robots.txt file?</h4>
<p>The robots.txt file is a plain text file located at the root folder of a domain (or subdomain) which tells web crawlers (like Googlebot) what parts of the website they should access and index.</p>
<p>The first thing a search engine crawler looks at when it is visiting a page is the robots.txt file and it controls how search engine spiders see and interact with the web pages.</p>
<h4>System requirements</h4>
<p>PHP version 7.1 or greater.<br />
We require 7.1 or higher because we believe that everybody should be running a modern PHP version. When releasing this plugin <a href="https://wordpress.org/about/requirements/" rel="ugc">WordPress recommends PHP 7.4 or higher</a>.</p>
<h4>How to get this plugin in your Language!</h4>
<p>The first release is avaliable in English and Spanish. In the “languages” folder we have included the necessary files to translate this plugin.</p>
<p>If you would like the plugin in your language and you’re good at translating, please use the <a href="https://translate.wordpress.org/projects/wp-plugins/block-ecommerce-assets-via-robots-txt/" rel="nofollow ugc">native WordPress Translation</a> functionality.</p>
<p>New to Translating a plugin? First read through the <a href="https://make.wordpress.org/polyglots/handbook/tools/glotpress-translate-wordpress-org/" rel="nofollow ugc">Translator Handbook</a>, then select your locale at <a href="https://translate.wordpress.org/" rel="nofollow ugc">Translating WordPress</a> and finally go to the <a href="https://translate.wordpress.org/projects/wp-plugins/block-ecommerce-assets-via-robots-txt/" rel="nofollow ugc">translation page for this plugin</a> to translate it.</p>
<h4>Further Reading</h4>
<p>You can access the description of the plugin in Spanish at: <a href="https://apasionados.es/blog/" rel="nofollow ugc">Block WooCommerce Assets via robots.txt en español</a>.</p>
<h3>Contact</h3>
<p>For further information please send us an <a href="https://apasionados.es/contacto/index.php?desde=wordpress-org-block-woocommerce-assets-robots-txt" rel="nofollow ugc">email</a>.</p>


Block eCommerce Assets via robots.txt

Blocks some WooCommerce assets and Search pages that should not be indexed through robots.txt

Viktor Szépe

<h4>This plugin needs more documentation!</h4>
<p>You can edit your robots.txt and add remote content to it.<br />
E.g. you have several sites and want to use a centralized robots.txt.</p>
<h4>Features</h4>
<ul>
<li>Include or exclude WordPress’ own robots.txt (core function)</li>
<li>Include or exclude plugins – e.g. sitemap plugins – output to robots.txt (filter output)</li>
<li>Include or exclude a remote text file (the common part)</li>
<li>Include or exclude custom records from the settings page (the site specific part)</li>
</ul>
<h4>Where is robot.txt?</h4>
<p>WordPress handles robots.txt as a virtual URL – just the same way as posts and pages.</p>
<p>So when you browse to <code>https://example.com/robots.txt</code> WordPress generates robots.txt on the fly.</p>
<h4>TODO</h4>
<ul>
<li>add more description here</li>
<li>add a video too</li>
<li>add an admin notice for subdir installs (robots.txt is useless in a subdir)</li>
<li>‘At least one “Disallow” field must be present in the robots.txt file.’ – check for that</li>
</ul>
<h4>Links</h4>
<p>Development of this plugin goes on on <a href="https://github.com/szepeviktor/multipart-robotstxt-editor" rel="nofollow ugc">GitHub</a>.</p>


Multipart robots.txt editor

Customize your site's robots.txt and include remote content to it

Block eCommerce Assets via robots.txt vs Multipart robots.txt editor