Viktor Szépe

<h4>This plugin needs more documentation!</h4>
<p>You can edit your robots.txt and add remote content to it.<br />
E.g. you have several sites and want to use a centralized robots.txt.</p>
<h4>Features</h4>
<ul>
<li>Include or exclude WordPress’ own robots.txt (core function)</li>
<li>Include or exclude plugins – e.g. sitemap plugins – output to robots.txt (filter output)</li>
<li>Include or exclude a remote text file (the common part)</li>
<li>Include or exclude custom records from the settings page (the site specific part)</li>
</ul>
<h4>Where is robot.txt?</h4>
<p>WordPress handles robots.txt as a virtual URL – just the same way as posts and pages.</p>
<p>So when you browse to <code>https://example.com/robots.txt</code> WordPress generates robots.txt on the fly.</p>
<h4>TODO</h4>
<ul>
<li>add more description here</li>
<li>add a video too</li>
<li>add an admin notice for subdir installs (robots.txt is useless in a subdir)</li>
<li>‘At least one “Disallow” field must be present in the robots.txt file.’ – check for that</li>
</ul>
<h4>Links</h4>
<p>Development of this plugin goes on on <a href="https://github.com/szepeviktor/multipart-robotstxt-editor" rel="nofollow ugc">GitHub</a>.</p>


Multipart robots.txt editor

Customize your site's robots.txt and include remote content to it

msfreed

<p>Thin Content Manager helps you easily identify thin content pages (i.e. posts and pages with minimal textual content) on your site and block them from search engines. The plugin calculates the body word count for each post/page on your site (disregarding words contained within HTML tags and shortcodes) and displays the information on a new ‘Thin Content Manager’ page within the Settings section of your WordPress admin panel. The admin page then provides a simple point-and-click interface to automatically insert robots noindex and nofollow tags – <meta name=”robots” content=”noindex, nofollow”> – into the head section of pages you select. These tags tell Google and other search engine spiders to ignore those pages, which is important for improving your site’s quality score and trust ranking for SEO purposes.</p>
<p>This plugin works with with single site installs only.</p>


Thin Content Manager

See the body word count to identify pages with thin content, then select pages to insert robots noindex,nofollow tags into.

Multipart robots.txt editor vs Thin Content Manager