dyaylaci

DY LLMs.txt Tool allows you to take control of how AI crawlers (ChatGPT, Claude, Perplexity, Google-Extended) verify and index your content. Improve your AEO (Answer Engine Optimization) by generating a structured, machine-readable <code>llms.txt</code> file automatically.
Unlike a standard <code>robots.txt</code> which blocks bots, <code>llms.txt</code> creates a semantic map of your site, helping Large Language Models (LLMs) understand your content context better and reducing hallucinations.
<h3>🚀 Key Features</h3>
<ul>
<li>📈 AEO Ready: Prepare your website for the future of AI Search.</li>
<li>🛍️ WooCommerce Support: Automatically include Product categories and essential shop pages.</li>
<li>🧩 Perfect Integration: Fully compatible with Yoast SEO, Rank Math, and All in One SEO. It respects your <code>noindex</code> settings automatically.</li>
<li>🌍 Multilingual Ready: Seamless integration with WPML and Polylang. Generates separate endpoints for each language.</li>
<li>🛡️ Privacy First: Built-in PII Scrubber to mask emails and phone numbers from AI bots.</li>
<li>🤖 Bot Tracker: Monitor which AI agents (GPTBot, ClaudeBot, etc.) are accessing your file.</li>
<li>⚡ Performance: Offers both Dynamic (on-the-fly) and Static (Cached) generation modes for high-traffic sites.</li>
</ul>
<h3>💡 Why do you need this?</h3>
AI Search Engines don’t browse websites like humans. They need structured text data. This plugin bridges the gap between your WordPress content and AI models, ensuring your brand is represented correctly in AI-generated answers. 
Key features:
<ul>
<li>Entity Clustering: Group content by taxonomy (categories/tags) to create a structured index.</li>
<li>Bot Tracking: Track and report visits from popular AI user agents.</li>
<li>Smart Exclusion: Exclude content manually, by ID, or automatically when marked as <code>noindex</code> by SEO plugins.</li>
<li>Dynamic or Static generation: Generate on-the-fly or write a cached static file for performance.</li>
<li>Privacy controls: Optional masking of emails and phone numbers.</li>
<li>Multilingual support: Works with popular multilingual plugins and can generate separate indexes per language.</li>
</ul>

DY LLMs.txt Tool

Create and manage an llms.txt file for AI crawlers to improve AEO (Answer Engine Optimization) and provide an authoritative AI index for your site.

Javier Casares

Fine-grained control over how search engines index and display your WordPress content. Apply 5 independent robots directives to 25 different page contexts with flexible implementation methods.
5 Robots Directives:
<ul>
<li>noindex: Prevent search engines from indexing the page</li>
<li>nofollow: Prevent search engines from following links on the page</li>
<li>noarchive: Prevent search engines from showing cached versions</li>
<li>nosnippet: Prevent search engines from showing text snippets in results</li>
<li>noimageindex: Prevent search engines from indexing images on the page</li>
</ul>
Implementation Methods:
<ul>
<li>HTML Meta Tags: Traditional method, easy to verify in page source (default)</li>
<li>HTTP Headers: More robust, works with all content types including PDFs and images</li>
<li>Both: Maximum compatibility for all scenarios</li>
</ul>
Control Levels:
<ul>
<li>Global Settings: Apply directives to 25 different page contexts (posts, pages, archives, etc.)</li>
<li>Granular Control (Optional): Override global settings for individual posts, pages, and custom post types via meta boxes in the editor</li>
</ul>
Perfect for:
<ul>
<li>Blocking indexing of attachment pages while allowing link following</li>
<li>Preventing duplicate content issues with flexible directive combinations</li>
<li>Controlling archive page indexing with granular control</li>
<li>Managing pagination SEO with independent settings</li>
<li>Protecting private content from search engine caching</li>
<li>Preventing snippet display while still indexing content</li>
</ul>
Main pages
<ul>
<li>Front Page: Block the indexing of the site’s front page.</li>
<li>Home: Block the indexing of the site’s home page.</li>
</ul>
Pages and Posts
<ul>
<li>Page: Block the indexing of the site’s pages.</li>
<li>Privacy Policy: Block the indexing of the site’s privacy policy page.</li>
<li>Single: Block the indexing of a post on the site.</li>
<li>Singular: Block the indexing of a post or a page of the site.</li>
</ul>
Taxonomies
<ul>
<li>Category: Block the indexing of the site categories. The lists where the posts appear.</li>
<li>Tag: Block the indexing of the site’s tags. The lists where the posts appear.</li>
</ul>
Dates
<ul>
<li>Date: Block the indexing when any date-based archive page (i.e. a monthly, yearly, daily or time-based archive) of the site. The lists where the posts appear.</li>
<li>Day: Block the indexing when a daily archive of the site. The lists where the posts appear.</li>
<li>Month: Block the indexing when a monthly archive of the site. The lists where the posts appear.</li>
<li>Time: Block the indexing when an hourly, “minutely”, or “secondly” archive of the site. The lists where the posts appear.</li>
<li>Year: Block the indexing when a yearly archive of the site. The lists where the posts appear.</li>
</ul>
Archives
<ul>
<li>Archive: Block the indexing of any type of Archive page. Category, Tag, Author and Date based pages are all types of Archives. The lists where the posts appear.</li>
<li>Author: Block the indexing of the author’s page, where the author’s publications appear.</li>
<li>Post Type Archive: Block the indexing of any post type page.</li>
</ul>
Pagination
<ul>
<li>Pagination: Block the indexing of the pagination, i.e. all pages other than the main page of an archive.</li>
</ul>
Search
<ul>
<li>Search: Block the indexing of the internal search result pages.</li>
</ul>
Attachments
<ul>
<li>Attachment: Block the indexing of an attachment document to a post or page. An attachment is an image or other file uploaded through the post editor’s upload utility. Attachments can be displayed on their own “page” or template. This will not cause the indexing of the image or file to be blocked.</li>
</ul>
Previews
<ul>
<li>Customize Preview: Block the indexing when a content is being displayed in customize mode.</li>
<li>Preview: Block the indexing when a single post is being displayed in draft mode.</li>
</ul>
Error Page
<ul>
<li>Error 404: This will cause an error page to be blocked from being indexed. As it is an error page, it should not be indexed per se, but just in case.</li>
</ul>
Important note: if you have any doubt about any of the following items it is best not to activate the option as you could lose results in the search engines.
<h3>Compatibility</h3>
<ul>
<li>WordPress: 5.7 – 7.1</li>
<li>PHP: 7.2 – 8.5</li>
</ul>
<h3>Security</h3>
This plugin adheres to the following security measures and review protocols for each version:
<ul>
<li><a href="https://developer.wordpress.org/plugins/" rel="nofollow ugc">WordPress Plugin Handbook</a></li>
<li><a href="https://developer.wordpress.org/plugins/wordpress-org/plugin-security/" rel="nofollow ugc">WordPress Plugin Security</a></li>
<li><a href="https://developer.wordpress.org/apis/security/" rel="nofollow ugc">WordPress APIs Security</a></li>
<li><a href="https://github.com/WordPress/WordPress-Coding-Standards" rel="nofollow ugc">WordPress Coding Standards</a></li>
<li><a href="https://wordpress.org/plugins/plugin-check/" rel="ugc">Plugin Check (PCP)</a></li>
</ul>
<h3>Privacy</h3>
<ul>
<li>This plugin does not collect any information about your site, your identity, the plugins, themes or content the site has.</li>
</ul>
<h3>Vulnerabilities</h3>
<ul>
<li>No vulnerabilities have been published up to version 2.0.1.</li>
<li>Version 2.0.0 includes proactive security hardening based on comprehensive security audit (see docs/SECURITY-2026-01-20.md).</li>
</ul>
Found a security vulnerability? Please report it to us privately at the <a href="https://github.com/javiercasares/noindex-seo/security/advisories/new" rel="nofollow ugc">noindex SEO GitHub repository</a>.

noindex SEO

Control search engine indexing with robots directives using HTML meta tags or HTTP headers.

DY LLMs.txt Tool vs noindex SEO