chrisguitarguy

George Pattichis

WordPress, by default, includes a simple robots.txt file that’s dynamically generated from within the WP application. This is great, but how do you easily change the content?
Enter WP Robots Txt, a plugin that adds an additional field to the “Reading” admin page where you can do just that. No manual coding or file editing required!
Simply visit https://your-site.com/wp-admin/options-reading.php and you can control the contents of your https://your-site.com/robots.txt
<a href="https://wordpress.org/plugins/wp-robots-txt/#developers" rel="ugc">Changelog</a>

WP Robots Txt

WP Robots Txt Allows you to edit the content of your robots.txt file.

citywanderer

Miina Sikk

Unblock CSS & JS for Googlebot plugin allows Googlebot to access the JavaScript and CSS files.
Google periodically sends to webmasters warnings that their JavaScript .js files and their CSS stylesheets are blocked – even when the webmasters have never explicitly done so. In fact, it is estimated that 85% of all users of Google webmaster tools have received such a warning.
Unblock CSS & JS for Googlebot solves this problem for you — and no configuration is needed. Just install and activate the plugin.
How does it work? It just adds in three lines to your robots.txt file to ensure the Google spider can get through.
You don’t want it anymore? Just uninstall and the added lines will be removed.
For more information and support, check out: http://www.wpsos.io/wordpress-plugin-unblock-css-js-for-googlebot/

Unblock CSS & JS for Googlebot

Modifies robots.txt to allow Googlebot access JS and CSS files.

WP Robots Txt vs Unblock CSS & JS for Googlebot