ABCdatos

In a nutshell: You don’t want robots wasting your resources and slowing down your site if they don’t give you some kind of direct or indirect benefit. An optimal configuration will maximize your SEO results and reduce your operating costs.
You’ll need less than a minute to set your site characteristics and get a customized robots.txt configuration installed, matching your needs and easily updated without headaches. Let us to apply our experience in maintaining and optimizing sites. A clear knowledge of what we want to do and why, moved us to change the decission method from human created robots.txt to an automated system, things are simpler as they seems. Here comes to help this plugin, doing all the magic for you.
Some of your site specs will be automatically detected to reduce the manual config needs to the minimum. A few simple questions about your site will be enough to vast majority of sites.
<h3>Update procedure</h3>
Any change required to the robots.txt is applied in real time when operating in virtual file mode. If you are working in real file mode, after updating the plugin version you should click the Save Changes button in the configuration screen to regenerate the robots.txt file.

Magic robots.txt

This plugin automatically creates a robots.txt analyzing your site to improve your Google ranking and site performance.

Jeff Starr

<blockquote>
✨ Trap bad bots in a virtual black hole
</blockquote>
Important: Do NOT use this plugin on sites with caching. <a href="https://wordpress.org/support/topic/important-do-not-use-on-sites-with-caching/" rel="ugc">Learn more »</a>
👾 Bye bye bad bots..
Bad bots are the worst. They do all sorts of nasty stuff and waste server resources. The Blackhole plugin helps to stop bad bots and save precious resources for legit visitors.
👾 How does it work?
First the plugin adds a hidden trigger link to the footer of your pages. You then add a line to your robots.txt file that forbids all bots from following the hidden link. Bots that then ignore or disobey your robots rules will crawl the link and fall into the trap. Once trapped, bad bots are denied further access to your WordPress site.
I call it the “one-strike” rule: bots have one chance to obey your site’s robots.txt rule. Failure to comply results in immediate banishment. The best part is that the Blackhole only affects bad bots: human users never see the hidden link, and good bots obey the robots rules in the first place. Win-win! 🙂
<blockquote>
✨ Add a blackhole trap to help stop bad bots
</blockquote>
Important: Do NOT use this plugin on sites with caching. <a href="https://wordpress.org/support/topic/important-do-not-use-on-sites-with-caching/" rel="ugc">Learn more »</a>
👾 Features
<ul>
<li>Easy to set up</li>
<li>Squeaky clean code</li>
<li>Focused and modular</li>
<li>Lightweight, fast and flexible</li>
<li>Built with the WordPress API</li>
<li>Works with other security plugins</li>
<li>Easy to reset the list of bad bots</li>
<li>Easy to delete any bot from the list</li>
<li>Regularly updated and “future proof”</li>
<li>Blackhole link includes “nofollow” attribute</li>
<li>Plugin options configurable via settings screen</li>
<li>Works silently behind the scenes to protect your site</li>
<li>Whitelists all major search engines to never block</li>
<li>Focused on flexibility, performance, and security</li>
<li>Email alerts with WHOIS lookup for blocked bots</li>
<li>Complete inline documentation via the Help tab</li>
<li>Provides setting to whitelist any IP addresses</li>
<li>Customize the message displayed to bad bots 😉</li>
<li>One-click restore the plugin default options</li>
<li>Does NOT use or require any .htaccess rules</li>
</ul>
Blackhole for Bad Bots protects your site against bad bots, spammers, scrapers, scanners, and other automated threats.
<blockquote>
✨ Not using WordPress? Check out the <a href="https://perishablepress.com/blackhole-bad-bots/" rel="nofollow ugc">standalone PHP version of Blackhole</a>!
</blockquote>
👾 Whitelist
By default, this plugin does NOT block any of the major search engines (user agents):
<ul>
<li>AOL.com</li>
<li>Baidu</li>
<li>Bingbot/MSN</li>
<li>DuckDuckGo</li>
<li>Googlebot</li>
<li>Teoma</li>
<li>Yahoo!</li>
<li>Yandex</li>
</ul>
These search engines (and all of their myriad variations) are whitelisted via user agent. So are a bunch of other “useful” bots. They always are allowed full access to your site, even if they disobey your robots.txt rules. This list can be customized in the plugin settings. For a complete list of whitelisted bots, visit the Help tab in the plugin settings (under “Whitelist Settings”).
<blockquote>
✨ Check out <a href="https://plugin-planet.com/blackhole-pro/" rel="nofollow ugc">Blackhole Pro</a> and level up with advanced features!
</blockquote>
👾 Exclusive Pro Features
<ul>
<li>Option to disable for logged-in users</li>
<li>Threshold control (number of allowed hits)</li>
<li>Custom email alerts</li>
<li>Custom messages for blocked bots</li>
<li>Custom redirect for blocked bots</li>
<li>Custom blackhole trigger links</li>
<li>Complete inline documentation</li>
<li>Block bots based on user agent</li>
<li>Block bots based on IP address</li>
<li>Whitelist/allow bots by user agent</li>
<li>Whitelist/allow bots by IP address</li>
<li>Redirect whitelisted bots</li>
<li>Set custom HTTP Status Code</li>
<li>Full-featured Bad Bot Log with paging, sorting, and field search</li>
<li>Manually add bad bots to the Bad Bot Log</li>
<li>Geo/IP location lookups for each bad bot</li>
<li>Logs number of blocked hits for each bot</li>
</ul>
..plus everything the free version can do and more.
<blockquote>
✨ Learn more and <a href="https://plugin-planet.com/blackhole-pro/" rel="nofollow ugc">get Blackhole Pro »</a>
</blockquote>
👾 Privacy
User Data: This plugin automatically blocks bad bots. When bad bots fall into the trap, their IP address, user agent, and other request data are stored in the WP database. No other user data is collected by this plugin. At any time, the administrator may delete all saved data via the plugin settings.
Services: This plugin does not connect to any third-party locations or services.
Cookies: This plugin does not set any cookies.
Credit: Header Image Courtesy NASA/JPL-Caltech.
Blackhole for Bad Bots is developed and maintained by <a href="https://x.com/perishable" rel="nofollow ugc">Jeff Starr</a>, 15-year <a href="https://plugin-planet.com/" rel="nofollow ugc">WordPress developer</a> and <a href="https://books.perishablepress.com/" rel="nofollow ugc">book author</a>.
👾 Support development
I develop and maintain this free plugin with love for the WordPress community. To show support, you can <a href="https://monzillamedia.com/donate.html" rel="nofollow ugc">make a donation</a> or purchase one of my books:
<ul>
<li><a href="https://wp-tao.com/" rel="nofollow ugc">The Tao of WordPress</a></li>
<li><a href="https://digwp.com/" rel="nofollow ugc">Digging into WordPress</a></li>
<li><a href="https://htaccessbook.com/" rel="nofollow ugc">.htaccess made easy</a></li>
<li><a href="https://wp-tao.com/wordpress-themes-book/" rel="nofollow ugc">WordPress Themes In Depth</a></li>
<li><a href="https://books.perishablepress.com/downloads/wizards-collection-sql-recipes-wordpress/" rel="nofollow ugc">Wizard’s SQL Recipes for WordPress</a></li>
</ul>
And/or purchase one of my premium WordPress plugins:
<ul>
<li><a href="https://plugin-planet.com/bbq-pro/" rel="nofollow ugc">BBQ Pro</a> – Blazing fast WordPress firewall</li>
<li><a href="https://plugin-planet.com/blackhole-pro/" rel="nofollow ugc">Blackhole Pro</a> – Automatically block bad bots</li>
<li><a href="https://plugin-planet.com/banhammer-pro/" rel="nofollow ugc">Banhammer Pro</a> – Monitor traffic and ban the bad guys</li>
<li><a href="https://plugin-planet.com/ga-google-analytics-pro/" rel="nofollow ugc">GA Google Analytics Pro</a> – Connect WordPress to Google Analytics</li>
<li><a href="https://plugin-planet.com/head-meta-pro/" rel="nofollow ugc">Head Meta Pro</a> – Ultimate Meta Tags for WordPress</li>
<li><a href="https://plugin-planet.com/rest-pro-tools/" rel="nofollow ugc">REST Pro Tools</a> – Awesome tools for managing the WP REST API</li>
<li><a href="https://plugin-planet.com/simple-ajax-chat-pro/" rel="nofollow ugc">Simple Ajax Chat Pro</a> – Unlimited chat rooms</li>
<li><a href="https://plugin-planet.com/usp-pro/" rel="nofollow ugc">USP Pro</a> – Unlimited front-end forms</li>
</ul>
Links, tweets and likes also appreciated. Thank you! 🙂

Blackhole for Bad Bots

Отличный инструмент

Simple. I install this every time I spin up a new WordPress site.

Simple

Blackhole is a WordPress security plugin that detects and traps bad bots in a virtual black hole, where they are denied access to your entire site.

Magic robots.txt vs Blackhole for Bad Bots