Booter - Bots & Crawlers Manager is a preventative measure (treatment in advance) and treatment of damages caused by crawlers and bots.

Ilan Firsov

uPress

ezCache

Lightning-fast WordPress optimization — page caching, Redis Object Cache, Critical CSS, WebP images, CSS/JS optimization, and 15+ performance tools.

uPress Link

uPress Link is a companion plugin for the WordPress hosting manager at https://www.upress.io

Mondo Design

<p>Booter – Bots & Crawlers Manager is a preventative measure (treatment in advance) and treatment of damages caused by crawlers and bots.<br />
The plugin uses a number of existing technologies which are known by crawlers and bots and takes them one step forward – smartly and almost completely automatically.<br />
To allow the plugin to function correctly, you must follow the instructions and manually enter some data (which must be done by a human being to avoid errors).</p>
<h4>At the prevention level</h4>
<ul>
<li>Booter allows you to manage and create an advanced dynamic robots.txt file.</li>
<li>View a 404 error log to see the most common bad links.</li>
<li>Blocking bad bots that cause high server loads due to very frequent page crawls, or are used to search for security vulnerabilities.</li>
</ul>
<h4>At the treatment level</h4>
<ul>
<li>Booter allows you to limit the amount of requests from crawlers and bots, if or when they exceed the specified amount of requests per minute, it will be rejected for a specified period of time.</li>
<li>Rejecting links that we do not want in the fastest way, not by just blocking but by sending the appropriate HTTP status code to make search engines forget them.</li>
</ul>
<h4>Instructions for use in case of damage treatment</h4>
<ol>
<li>Activate the plugin.</li>
<li>Enable the 404 error log option.</li>
<li>Set the access rate limit.</li>
<li>Watch the 404 log, try to find common parts in the URLs that repeats most often.</li>
<li>Enter the common parts to the “reject links” page, and ensure the rejection code is 410.</li>
<li>Clear the 404 error log.</li>
<li>Repeat the process once every few hours until the 404 error log remains blank.</li>
<li>Check the status of your website’s index coverage every few days.</li>
</ol>


Alternatives to Booter – Bots & Crawlers Manager

Top alternatives