netroby

WP-log-robots is a wordpress plugin, we could used it to loged the search engine robots activity.
after you had installed it . the WP-log-robots will log each robots activity and wrote it to log file .
if your site was : http://www.netroby.com. then the log file would be location at : http://www.netroby.com/robots_log.txt
<h3>Manual Installation</h3>
In case you want to make all steps manually:
<ol>
<li>
Upload to your plugins folder, usually <code>wp-content/plugins/</code> and unzip the file, it will create a <code>wp-content/plugins/wp-log-robots/</code> directory.
</li>
<li>
Make <code>wp-content</code> directory writeable by the web server.
</li>
<li>
Activate the plugin on the plugin screen.
</li>
</ol>
<h3>More Info</h3>
For more info, please visit http://www.netroby.com/ .
(For feedback/comments, please send an e-mail to: netroby[at]netroby.com ).

WP-log-robots

WP-log-robots is a wordpress plugin, we could used it to loged the search engine  robots activity.

Javier Casares

Fine-grained control over how search engines index and display your WordPress content. Apply 5 independent robots directives to 25 different page contexts with flexible implementation methods.
5 Robots Directives:
<ul>
<li>noindex: Prevent search engines from indexing the page</li>
<li>nofollow: Prevent search engines from following links on the page</li>
<li>noarchive: Prevent search engines from showing cached versions</li>
<li>nosnippet: Prevent search engines from showing text snippets in results</li>
<li>noimageindex: Prevent search engines from indexing images on the page</li>
</ul>
Implementation Methods:
<ul>
<li>HTML Meta Tags: Traditional method, easy to verify in page source (default)</li>
<li>HTTP Headers: More robust, works with all content types including PDFs and images</li>
<li>Both: Maximum compatibility for all scenarios</li>
</ul>
Control Levels:
<ul>
<li>Global Settings: Apply directives to 25 different page contexts (posts, pages, archives, etc.)</li>
<li>Granular Control (Optional): Override global settings for individual posts, pages, and custom post types via meta boxes in the editor</li>
</ul>
Perfect for:
<ul>
<li>Blocking indexing of attachment pages while allowing link following</li>
<li>Preventing duplicate content issues with flexible directive combinations</li>
<li>Controlling archive page indexing with granular control</li>
<li>Managing pagination SEO with independent settings</li>
<li>Protecting private content from search engine caching</li>
<li>Preventing snippet display while still indexing content</li>
</ul>
Main pages
<ul>
<li>Front Page: Block the indexing of the site’s front page.</li>
<li>Home: Block the indexing of the site’s home page.</li>
</ul>
Pages and Posts
<ul>
<li>Page: Block the indexing of the site’s pages.</li>
<li>Privacy Policy: Block the indexing of the site’s privacy policy page.</li>
<li>Single: Block the indexing of a post on the site.</li>
<li>Singular: Block the indexing of a post or a page of the site.</li>
</ul>
Taxonomies
<ul>
<li>Category: Block the indexing of the site categories. The lists where the posts appear.</li>
<li>Tag: Block the indexing of the site’s tags. The lists where the posts appear.</li>
</ul>
Dates
<ul>
<li>Date: Block the indexing when any date-based archive page (i.e. a monthly, yearly, daily or time-based archive) of the site. The lists where the posts appear.</li>
<li>Day: Block the indexing when a daily archive of the site. The lists where the posts appear.</li>
<li>Month: Block the indexing when a monthly archive of the site. The lists where the posts appear.</li>
<li>Time: Block the indexing when an hourly, “minutely”, or “secondly” archive of the site. The lists where the posts appear.</li>
<li>Year: Block the indexing when a yearly archive of the site. The lists where the posts appear.</li>
</ul>
Archives
<ul>
<li>Archive: Block the indexing of any type of Archive page. Category, Tag, Author and Date based pages are all types of Archives. The lists where the posts appear.</li>
<li>Author: Block the indexing of the author’s page, where the author’s publications appear.</li>
<li>Post Type Archive: Block the indexing of any post type page.</li>
</ul>
Pagination
<ul>
<li>Pagination: Block the indexing of the pagination, i.e. all pages other than the main page of an archive.</li>
</ul>
Search
<ul>
<li>Search: Block the indexing of the internal search result pages.</li>
</ul>
Attachments
<ul>
<li>Attachment: Block the indexing of an attachment document to a post or page. An attachment is an image or other file uploaded through the post editor’s upload utility. Attachments can be displayed on their own “page” or template. This will not cause the indexing of the image or file to be blocked.</li>
</ul>
Previews
<ul>
<li>Customize Preview: Block the indexing when a content is being displayed in customize mode.</li>
<li>Preview: Block the indexing when a single post is being displayed in draft mode.</li>
</ul>
Error Page
<ul>
<li>Error 404: This will cause an error page to be blocked from being indexed. As it is an error page, it should not be indexed per se, but just in case.</li>
</ul>
Important note: if you have any doubt about any of the following items it is best not to activate the option as you could lose results in the search engines.
<h3>Compatibility</h3>
<ul>
<li>WordPress: 5.7 – 7.1</li>
<li>PHP: 7.2 – 8.5</li>
</ul>
<h3>Security</h3>
This plugin adheres to the following security measures and review protocols for each version:
<ul>
<li><a href="https://developer.wordpress.org/plugins/" rel="nofollow ugc">WordPress Plugin Handbook</a></li>
<li><a href="https://developer.wordpress.org/plugins/wordpress-org/plugin-security/" rel="nofollow ugc">WordPress Plugin Security</a></li>
<li><a href="https://developer.wordpress.org/apis/security/" rel="nofollow ugc">WordPress APIs Security</a></li>
<li><a href="https://github.com/WordPress/WordPress-Coding-Standards" rel="nofollow ugc">WordPress Coding Standards</a></li>
<li><a href="https://wordpress.org/plugins/plugin-check/" rel="ugc">Plugin Check (PCP)</a></li>
</ul>
<h3>Privacy</h3>
<ul>
<li>This plugin does not collect any information about your site, your identity, the plugins, themes or content the site has.</li>
</ul>
<h3>Vulnerabilities</h3>
<ul>
<li>No vulnerabilities have been published up to version 2.0.1.</li>
<li>Version 2.0.0 includes proactive security hardening based on comprehensive security audit (see docs/SECURITY-2026-01-20.md).</li>
</ul>
Found a security vulnerability? Please report it to us privately at the <a href="https://github.com/javiercasares/noindex-seo/security/advisories/new" rel="nofollow ugc">noindex SEO GitHub repository</a>.

noindex SEO

Control search engine indexing with robots directives using HTML meta tags or HTTP headers.

WP-log-robots vs noindex SEO