ProAI

<p>ProAI TXT Manager for LLMS and Robots is a powerful WordPress plugin that automatically generates and maintains essential txt files for your website:</p>
<ul>
<li><strong>llms.txt</strong> – Provides structured information for AI language models about your site’s content, navigation, and structure</li>
<li><strong>robots.txt</strong> – Controls search engine crawler access and includes sitemap references</li>
</ul>
<h4>Key Features</h4>
<ul>
<li><strong>Automatic Generation</strong>: Creates both llms.txt and robots.txt files automatically</li>
<li><strong>AI-Optimized Content</strong>: Structures llms.txt content specifically for AI language models</li>
<li><strong>Auto-Regeneration</strong>: Updates files automatically when your content changes</li>
<li><strong>Custom Content</strong>: Add your own custom information to both files</li>
<li><strong>SEO Integration</strong>: Automatically detects and includes sitemaps from popular SEO plugins</li>
<li><strong>WordPress Standards</strong>: Follows WordPress security and coding standards</li>
<li><strong>Easy Management</strong>: Simple admin interface for all settings</li>
</ul>
<h4>What is llms.txt?</h4>
<p>The llms.txt file is a structured text file that provides AI language models with essential information about your website, including:</p>
<ul>
<li>Site information and description</li>
<li>Navigation structure</li>
<li>Recent content and categories</li>
<li>Contact information</li>
<li>Custom information you want AI models to know</li>
</ul>
<h4>What’s Included in Robots.txt</h4>
<ul>
<li>WordPress security best practices</li>
<li>Automatic sitemap detection (Yoast, RankMath, WordPress core)</li>
<li>Common WordPress disallow rules</li>
<li>Custom rules you specify</li>
<li>Crawl delay settings</li>
</ul>
<h4>Perfect For</h4>
<ul>
<li>Website owners who want to be AI-friendly</li>
<li>SEO professionals managing multiple sites</li>
<li>Developers building modern WordPress sites</li>
<li>Anyone wanting automated TXT file management</li>
</ul>
<h3>Technical Details</h3>
<h4>File Serving</h4>
<p>The plugin serves files virtually, meaning they don’t actually exist as physical files in your root directory unless you specifically choose to generate them. This prevents clutter and ensures the files are always up-to-date.</p>
<h4>Performance</h4>
<p>The plugin is designed to be lightweight and efficient. File generation only occurs when content changes, and the files are cached for optimal performance.</p>
<h4>Security</h4>
<p>The robots.txt generation follows WordPress security best practices, including:<br />
* Disallowing access to wp-admin (except admin-ajax.php)<br />
* Disallowing access to wp-includes<br />
* Disallowing access to plugin and theme directories<br />
* Preventing access to WordPress system files</p>
<h4>Compatibility</h4>
<ul>
<li>Works with all major SEO plugins</li>
<li>Compatible with multisite installations</li>
<li>Supports custom post types and taxonomies</li>
<li>Works with custom menu structures</li>
</ul>


ProAI TXT Manager for LLMS and Robots

Automatically generates and manages llms.txt and robots.txt files for your WordPress site with AI-optimized content.

Sean Mullins

<p><strong>LLMs.txt Curator</strong> helps site owners generate and curate a high-quality <code>llms.txt</code> file for AI assistants and retrieval systems, ensuring only relevant, well-described content is exposed to large language models.</p>
<p>It generates and maintains <code>llms.txt</code> and <code>llms-full.txt</code> — the emerging standard for telling AI systems (ChatGPT, Claude, Perplexity, Gemini, and others) which pages on your site matter most and what they contain.</p>
<p>Unlike auto-generators that dump every URL into a flat file, LLMs.txt Curator takes a <strong>curation-first approach</strong>. You choose the pages, organise them into sections, fill descriptions, override titles for AI, validate quality, and see exactly which AI bots are reading your file — all from a single interface.</p>
<h4>What makes this different</h4>
<p>Most llms.txt plugins treat the file as a static output. LLMs.txt Curator treats it as a living asset:</p>
<ul>
<li><strong>Quality Score</strong> — every generated file shows your coverage percentage, which pages have descriptions, and which still need attention.</li>
<li><strong>Description Suggestions</strong> — one click fills all missing descriptions from a five-step fallback chain (SEO meta <span aria-hidden="true" class="wp-exclude-emoji">→</span> excerpt <span aria-hidden="true" class="wp-exclude-emoji">→</span> Open Graph <span aria-hidden="true" class="wp-exclude-emoji">→</span> page content). Never overwrites what you’ve set manually.</li>
<li><strong>Change Detection</strong> — a banner appears in your admin when curated pages have been updated since the last generation, with a Regenerate Now button. Your llms.txt stays current without manual checking.</li>
<li><strong>Per-Page Title Override</strong> — set a different title for each page specifically for AI consumption, without touching your on-site SEO.</li>
<li><strong>Safety Mode</strong> — blocks generation if validation errors exist, preventing a broken file from going live. Includes an on-demand validator with plain-English results.</li>
<li><strong>AI Crawler Analytics</strong> — see which bots visited in the last 7 days, with a visual bar chart. OpenAI, Anthropic, Perplexity, Google — know who is actually reading your file.</li>
</ul>
<h4>Core features</h4>
<ul>
<li><strong>Drag-and-drop curation</strong> — reorder sections and pages visually; each section becomes an <code>##</code> heading per the spec</li>
<li><strong>llms-full.txt generation</strong> — the companion file with full Markdown content for each curated page</li>
<li><strong>Five SEO plugin integrations</strong> — Rank Math, Yoast SEO, All in One SEO, SEOPress, The SEO Framework</li>
<li><strong>Schema-aware descriptions</strong> — uses structured data before falling back through the description chain</li>
<li><strong>Scheduled regeneration</strong> — choose Instant (on publish), Daily, Weekly, or Manual</li>
<li><strong>WooCommerce support</strong> — SKU, price, stock, categories, dimensions; respects product visibility</li>
<li><strong>WordPress Multisite</strong> — network-activate across all sites; each site manages its own independent llms.txt; Network Admin overview with per-site and bulk regenerate</li>
<li><strong>Pre-built templates</strong> — Business, E-commerce, SaaS, Blog, Local Business</li>
<li><strong>Live preview</strong> — see your exact llms.txt output before saving</li>
<li><strong>Import / export</strong> — move your configuration between sites as JSON</li>
<li><strong>WP-CLI</strong> — <code>wp llms-txt regenerate</code>, <code>wp llms-txt status</code>, <code>wp llms-txt crawler-log</code></li>
<li><strong>REST API</strong> — <code>POST /wp-json/llms-txt/v1/regenerate</code>, <code>GET /wp-json/llms-txt/v1/status</code></li>
<li><strong>Atomic file writes</strong> — temp file <span aria-hidden="true" class="wp-exclude-emoji">→</span> rename; no half-written files served to bots</li>
<li><strong>ETag/304 caching</strong> — proper HTTP headers for CDN revalidation</li>
<li><strong>Subdirectory / Bedrock support</strong> — correctly finds the site root on non-standard installs</li>
<li><strong>Robots.txt reference</strong> — automatically adds a spec-compliant comment</li>
</ul>
<h4>Description Suggestions in detail</h4>
<p>When pages lack descriptions, AI systems get less context. The suggestion engine fills the gap automatically:</p>
<ol>
<li>Schema markup (Rank Math, custom <code>_schema_json</code>)</li>
<li>SEO plugin meta description</li>
<li>WordPress excerpt</li>
<li>Open Graph description (<code>_og_description</code> / <code>og_description</code>)</li>
<li>First 160 characters of post content</li>
</ol>
<p>Already-set descriptions are never touched. Pages that can’t be filled automatically are listed for manual review.</p>
<h4>Quality Score</h4>
<p>Every generated <code>llms.txt</code> ends with a coverage report:</p>
<pre><code>Quality Score: 94%
Pages included: 48
Pages with descriptions: 45
Pages missing descriptions: 3
</code></pre>
<p>This is visible to the AI systems reading your file, and to you in the Preview tab.</p>
<h4>AI Crawler Analytics</h4>
<p>Track 12 known bots: GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended, Applebot-Extended, Meta-ExternalAgent, Bytespider, CCBot, Cohere, DeepSeek, Amazonbot.</p>
<p>The 7-day analytics card shows a visual bar chart of recent activity. All-time totals are kept separately. IP addresses are anonymised before storage — last octet zeroed for IPv4, last 80 bits for IPv6. No data leaves your server.</p>
<h4>Safety Mode</h4>
<p>Before generation, the validator checks:</p>
<ul>
<li>Pages missing or unpublished</li>
<li>Duplicate URLs across sections</li>
<li>Noindex conflicts</li>
<li>Canonical mismatches</li>
<li>Password-protected pages</li>
<li>Thin content (< 100 words, no meta, no excerpt)</li>
<li>File size > 50 KB</li>
<li>Too many pages (> 80)</li>
</ul>
<p>When Safety Mode is on, errors block generation and the results are shown immediately. Warnings are surfaced but don’t block. Everything is explained in plain English.</p>
<h4>Scheduled regeneration</h4>
<ul>
<li><strong>Instant</strong> — regenerates ~30 seconds after any page is published or trashed (default)</li>
<li><strong>Daily</strong> — once per day, via WP-Cron recurring event</li>
<li><strong>Weekly</strong> — once per week</li>
<li><strong>Manual</strong> — only when you click Save & Generate</li>
</ul>
<h4>How to get started</h4>
<ol>
<li>Install and activate. The plugin auto-scans your content and creates initial sections.</li>
<li>Curate: drag, drop, add, remove. Aim for 20-60 pages that best represent your site.</li>
<li>Click <strong>Generate Missing Descriptions</strong> to fill gaps automatically.</li>
<li>Fix any warnings shown in the Safety Mode card.</li>
<li>Set the update mode that suits your workflow.</li>
<li>Save & Generate. Your files are live at <code>/llms.txt</code> and <code>/llms-full.txt</code>.</li>
<li>Enable the AI Crawler Log to see which bots start visiting.</li>
</ol>
<h4>WooCommerce integration</h4>
<p>When WooCommerce is active, the plugin automatically includes SKU, price, and stock status in <code>llms.txt</code> descriptions, and full product details in <code>llms-full.txt</code>. Products with “hidden” visibility are excluded, and you can optionally exclude out-of-stock products.</p>
<h4>Developer hooks</h4>
<ul>
<li><strong><code>llmscu_capability</code></strong> filter — override the required capability (default: <code>manage_options</code>)</li>
<li><strong><code>llmscu_post_limit</code></strong> filter — scanner post limit per type (default: 500)</li>
<li><strong><code>llmscu_full_word_limit</code></strong> filter — per-page word cap in <code>llms-full.txt</code> (default: unlimited)</li>
<li><strong><code>llmscu_regenerated</code></strong> action — fires after each successful regeneration with the content string</li>
</ul>


LLMs.txt Curator

Curate, score, and maintain your llms.txt with quality scoring, description suggestions, change detection, and AI crawler analytics.

ProAI TXT Manager for LLMS and Robots vs LLMs.txt Curator