chrisguitarguy

George Pattichis

WordPress, by default, includes a simple robots.txt file that’s dynamically generated from within the WP application. This is great, but how do you easily change the content?
Enter WP Robots Txt, a plugin that adds an additional field to the “Reading” admin page where you can do just that. No manual coding or file editing required!
Simply visit https://your-site.com/wp-admin/options-reading.php and you can control the contents of your https://your-site.com/robots.txt
<a href="https://wordpress.org/plugins/wp-robots-txt/#developers" rel="ugc">Changelog</a>

WP Robots Txt

WP Robots Txt Allows you to edit the content of your robots.txt file.

Rolf Allard van Hagen

Take control of your WordPress core XML Sitemap and add a Google News Sitemap. Compatible with WordPress Multisite, Polylang and WPML and most caching plugins. There are no static files created.
NEW: Connect your site to your Google Search Console and Bing Webmaster Tools to keep an eye on sitemap performance and possible errors. 
NEW: Submit your sitemaps right from the WordPress admin. Yes, you read that correctly: Sitemap Pings are back!
<h4>Features</h4>
<ul>
<li>Control the WordPress core XML Sitemaps.</li>
<li>Add a Google News Sitemap.</li>
<li>Connect your site to Google Search Console and Bing Webmaster Tools to keep an eye on sitemap performance and possible errors.</li>
<li>Submit/ping sitemaps to Google Search Console and Bing Webmaster Tools for fast reindexation.</li>
<li>Add custom robots.txt rules.</li>
</ul>
XML Sitemap
<ul>
<li>Option to disable the main XML Sitemap Index.</li>
<li>Select sitemaps to include in the sitemap index: post types, taxonomies, authors and google news.</li>
<li>Select which (custom) post types, taxonomy archives and users to include in respective sitemaps.</li>
<li>Exclude specific posts, pages and custom post type posts from sitemaps.</li>
<li>Add a custom sitemap with external URLs.</li>
<li>Add external sitemaps.</li>
<li>Choose the sitemap server/generator: WordPress core or Plugin.</li>
<li>Plugin server: Include featured images or attached images with title.</li>
<li>Choose to update Last modification date only on post modification (default) or also on comments.</li>
<li>Sitemap stylesheet templates can be overridden by theme template files.</li>
</ul>
Google News Sitemap
<ul>
<li>Optionally activate the Google News Sitemap.</li>
<li>Includes the required news sitemap tags: Publication name, language, title and publication date.</li>
<li>Set a News Publication Name (site name by default).</li>
<li>Supports custom post types.</li>
<li>Limit inclusion of certain post categories.</li>
<li>Google News Sitemap stylesheet template can be overridden by theme template files.</li>
</ul>
<h4>Pro Extensions</h4>
<a href="https://premium.status301.com/downloads/xml-sitemap-advanced/" rel="nofollow ugc">XML Sitemap Advanced</a>
<ul>
<li>Sitemap notifier: Automatic XML Sitemap Index update notification to Google and Bing.</li>
<li>Sitemap notifier log</li>
<li>Set a custom sitemap URL (slug)</li>
<li>Exclude posts and pages in the Bulk Editor.</li>
<li>Optionally include empty taxonomy term and empty author archives</li>
<li>Yoast SEO and Rank Math compatibility: exclude posts set to noindex</li>
</ul>
<a href="https://premium.status301.com/downloads/google-news-advanced/" rel="nofollow ugc">Google News Advanced</a>
<ul>
<li>Sitemap norifier: Automatic Google News Sitemap update notification to Google.</li>
<li>Sitemap notifier log</li>
<li>Multiple post types – Include more than one post type in the same News Sitemap.</li>
<li>Keywords – Add the keywords tag to your News Sitemap. Keywords can be created from Tags, Categories or a dedicated Keywords taxonomy.</li>
<li>Stock tickers – Add stock tickers tag to your News Sitemap. A dedicated Stock Tickers taxonomy will be available to manage them.</li>
</ul>
<h4>Privacy / GDPR</h4>
This plugin does not collect any user or visitor data nor set browser cookies. Using this plugin should not impact your site privacy policy in any way.
Data that is published
An XML Sitemap index, referencing other sitemaps containing your web site’s public post URLs of selected post types that are already public, along with their last modification date and associated image URLs, and any selected public archive URLs. 
An author sitemap can be included, which will contain links to author archive pages. These urls contain author/user slugs, and the author archives can contain author bio information. If you wish to keep this out of public domain, then deactivate the author sitemap and use an SEO plugin to add noindex headers to the respective archive pages.
A Google News Sitemap containing your web site’s public and recent (last 48 hours) URLs of selected news post type, along with their publication time stamp and associated image URL.
Data that is transmitted
when a connection between your site and Google Search Console is configured in this plugin, the sitemap index and google news sitemap can be submitted to Google. No data is transmitted to any third party.
<h4>Contribute</h4>
If you’re happy with this plugin as it is, please consider writing a quick <a href="https://wordpress.org/support/plugin/xml-sitemap-feed/reviews/#new-post" rel="ugc">rating</a> or helping other users out on the <a href="https://wordpress.org/support/plugin/xml-sitemap-feed" rel="ugc">support forum</a>.
If you wish to help build this plugin, you’re very welcome to <a href="https://translate.wordpress.org/projects/wp-plugins/xml-sitemap-feed/" rel="nofollow ugc">translate it into your language</a> or contribute code on <a href="https://github.com/RavanH/xml-sitemap-feed/" rel="nofollow ugc">Github</a>.
<h4>Credits</h4>
XML Sitemap Feed was originally based on the discontinued plugin Standard XML Sitemap Generator by Patrick Chia. Since then, it has been completely rewritten and extended in many ways.
NOTES:
<ol>
<li>
If you do not use fancy URL’s or you have WordPress site running in a subdirectory, a dynamic robots.txt will NOT be generated.
</li>
<li>
On large sites, it is advised to use a good caching plugin to improve your site and sitemap performance.
</li>
</ol>

XML Sitemap & Google News


I've been trying several plugins to generate a proper sitemap on our site. This one has a simple yet effective setup and really works like a charm with my CPT's.


Really works with custom post types


This is probably the best sitemap plugin available.


Best Sitemap Plugin

Take control of your WordPress core XML Sitemap and add a Google News Sitemap.

WP Robots Txt vs XML Sitemap & Google News