Viktor Szépe

<h4>This plugin needs more documentation!</h4>
You can edit your robots.txt and add remote content to it. 
E.g. you have several sites and want to use a centralized robots.txt.
<h4>Features</h4>
<ul>
<li>Include or exclude WordPress’ own robots.txt (core function)</li>
<li>Include or exclude plugins – e.g. sitemap plugins – output to robots.txt (filter output)</li>
<li>Include or exclude a remote text file (the common part)</li>
<li>Include or exclude custom records from the settings page (the site specific part)</li>
</ul>
<h4>Where is robot.txt?</h4>
WordPress handles robots.txt as a virtual URL – just the same way as posts and pages.
So when you browse to <code>https://example.com/robots.txt</code> WordPress generates robots.txt on the fly.
<h4>TODO</h4>
<ul>
<li>add more description here</li>
<li>add a video too</li>
<li>add an admin notice for subdir installs (robots.txt is useless in a subdir)</li>
<li>‘At least one “Disallow” field must be present in the robots.txt file.’ – check for that</li>
</ul>
<h4>Links</h4>
Development of this plugin goes on on <a href="https://github.com/szepeviktor/multipart-robotstxt-editor" rel="nofollow ugc">GitHub</a>.

Multipart robots.txt editor

Customize your site's robots.txt and include remote content to it

error

Welcome to a whole new way of keeping your blog, forum, guestbook, wiki or 
content management system free of link spam. Bad Behavior is a PHP-based 
solution for blocking link spam and the robots which deliver it.
Thousands of sites large and small, like SourceForge, GNOME, the U.S. 
Department of Education, and many more, trust Bad Behavior to help reduce 
incoming link spam and malicious activity.
Bad Behavior complements other link spam solutions by acting as a gatekeeper, 
preventing spammers from ever delivering their junk, and in many cases, from 
ever reading your site in the first place. This keeps your site’s load down, 
makes your site logs cleaner, and can help prevent denial of service 
conditions caused by spammers.
Bad Behavior also transcends other link spam solutions by working in a 
completely different, unique way. Instead of merely looking at the content of 
potential spam, Bad Behavior analyzes the delivery method as well as the 
software the spammer is using. In this way, Bad Behavior can stop spam attacks 
even when nobody has ever seen the particular spam before.
Bad Behavior is designed to work alongside existing spam prevention services 
to increase their effectiveness and efficiency. Whenever possible, you should 
run it in combination with a more traditional spam prevention service.
Bad Behavior works on, or can be adapted to, virtually any PHP-based Web 
software package. Bad Behavior is available natively for WordPress, MediaWiki, 
Drupal, ExpressionEngine, and LifeType, and people have successfully made it 
work with Movable Type, phpBB, and many other packages.
Installing and configuring Bad Behavior on most platforms is simple and takes 
only a few minutes. In most cases, no configuration at all is needed. Simply 
turn it on and stop worrying about spam!
The core of Bad Behavior is free software released under the GNU Lesser General 
Public License, version 3, or at your option, any later version.
<h3>Release Notes</h3>
<h4>Bad Behavior 2.2 Known Issues</h4>
<ul>
<li>
Bad Behavior 2.2 requires MySQL 5.0 or later and PHP 5.2 or later.
</li>
<li>
CloudFlare users must enable the Reverse Proxy option in Bad Behavior’s 
settings. See the documentation for further details.
</li>
<li>
Bad Behavior is unable to protect internally cached pages on MediaWiki. 
Only form submissions will be protected.
</li>
<li>
When upgrading from version 2.0.19 or prior on MediaWiki and WordPress, 
you must remove the old version of Bad Behavior from your system manually 
before manually installing the new version. Other platforms are not 
affected by this issue.
</li>
<li>
Bad Behavior on WordPress requires version 3.1 or later. Users of older 
versions should upgrade WordPress prior to installing Bad Behavior.
</li>
<li>
On WordPress when using WP-Super Cache, Bad Behavior must be enabled in 
WP-Super Cache’s configuration in order to protect PHP Cached or Legacy 
Cached pages. Bad Behavior cannot protect mod_rewrite cached (Super Cached) 
pages.
</li>
<li>
When using Bad Behavior in conjunction with Spam Karma 2, you may see PHP 
warnings when Spam Karma 2 displays its internally generated CAPTCHA. This 
is a design problem in Spam Karma 2. Contact the author of Spam Karma 2 for 
a fix.
</li>
</ul>

Bad Behavior

Although sometimes it can be tricky if you get a false positive, those are pretty rare (haven't seen one in a couple years). Works well to keep bad actors off your site.

works nicely

Bad Behavior prevents spammers from ever delivering their junk, and in many
cases, from ever reading your site in the first place.

Multipart robots.txt editor vs Bad Behavior