Arefly

<p>Log the Search Engine Robots Visits of your Webite.</p>
<p>取得搜索引擎的蜘蛛訪問網站的記錄</p>
<p>And you can find your log at http://YOUR_BLOG_URL/robots_log.txt</p>
<h4>Translators</h4>
<ul>
<li>Chinese, Simplified (zh_CN) – <a href="http://www.arefly.com/" rel="nofollow ugc">Arefly</a></li>
<li>Chinese, Traditional (zh_TW) – <a href="http://www.arefly.com" rel="nofollow ugc">Arefly</a></li>
<li>English (en_US) – <a href="http://www.arefly.com" rel="nofollow ugc">Arefly</a></li>
</ul>
<p>If you have created your own language pack, or have an update of an existing one, you can send <a href="https://codex.wordpress.org/Translating_WordPress" rel="nofollow ugc">gettext PO and MO files</a> to <a href="http://www.arefly.com/about/" rel="nofollow ugc">Arefly</a> so that I can bundle it into WP Robots Log. You can download the latest <a href="https://plugins.svn.wordpress.org/wp-robots-log/trunk/lang/wp-robots-log.pot" rel="nofollow ugc">POT file</a>.</p>


WP Robots Log

Log the Search Engine Robots Visits of your Webite. 取得搜索引擎的蜘蛛訪問網站的記錄

Viktor Szépe

<h4>This plugin needs more documentation!</h4>
<p>You can edit your robots.txt and add remote content to it.<br />
E.g. you have several sites and want to use a centralized robots.txt.</p>
<h4>Features</h4>
<ul>
<li>Include or exclude WordPress’ own robots.txt (core function)</li>
<li>Include or exclude plugins – e.g. sitemap plugins – output to robots.txt (filter output)</li>
<li>Include or exclude a remote text file (the common part)</li>
<li>Include or exclude custom records from the settings page (the site specific part)</li>
</ul>
<h4>Where is robot.txt?</h4>
<p>WordPress handles robots.txt as a virtual URL – just the same way as posts and pages.</p>
<p>So when you browse to <code>https://example.com/robots.txt</code> WordPress generates robots.txt on the fly.</p>
<h4>TODO</h4>
<ul>
<li>add more description here</li>
<li>add a video too</li>
<li>add an admin notice for subdir installs (robots.txt is useless in a subdir)</li>
<li>‘At least one “Disallow” field must be present in the robots.txt file.’ – check for that</li>
</ul>
<h4>Links</h4>
<p>Development of this plugin goes on on <a href="https://github.com/szepeviktor/multipart-robotstxt-editor" rel="nofollow ugc">GitHub</a>.</p>


Multipart robots.txt editor

Customize your site's robots.txt and include remote content to it

WP Robots Log vs Multipart robots.txt editor