Md Atiar Rahman Ovi

Prepare your WordPress site for the AI Search Era.
As search engines evolve into Answer Engines (like ChatGPT Search, Perplexity, and Google Gemini), they rely less on HTML and more on structured, raw text data. The <code>llms.txt</code> standard is the new “robots.txt” for AI—a file specifically designed to tell Large Language Models (LLMs) what your site is about and where to find your best content.
WEO LLMS & AI Crawler Guide automatically generates a perfect, dynamic <code>llms.txt</code> file for your site. It creates a categorized “Resume” of your website that helps AI bots understand your structure instantly.
<h3>🚀 Key Features</h3>
<ul>
<li>Categorized Content Output: Automatically groups your content by type (e.g., <code>## Post</code>, <code>## Page</code>, <code>## Products</code>) so AI understands your site structure.</li>
<li>Automated Sitemap Detection: Automatically detects and lists your <code>sitemap.xml</code> at the top of the file for deeper crawling.</li>
<li>Markdown Optimization: Converts your content titles, links, and excerpts into clean Markdown (<code>- [Title](Link) - Excerpt</code>) optimized for LLM token limits.</li>
<li>Robots.txt Integration: Automatically adds a link to your <code>llms.txt</code> inside your site’s <code>robots.txt</code> file so crawlers can find it.</li>
<li>Custom Control: Choose exactly which Post Types (Pages, Posts, Products, Custom Types) to include.</li>
<li>Custom Intro/Outro: Add specific instructions for AI bots (e.g., “This site is the authority on Technical SEO…”).</li>
<li>Performance Caching: Built-in caching ensures the file loads instantly without slowing down your server.</li>
</ul>
<h3>💡 Why do you need this?</h3>
If you want your content to appear in AI answers (ChatGPT, Claude, Perplexity), you need to make it easy for them to read. A standard HTML page is full of noise (menus, sidebars, popups). An <code>llms.txt</code> file provides a clean, noise-free map of your content.
This plugin handles the technical heavy lifting, ensuring your site is “AI Ready” in seconds.

WEO LLMS & AI Crawler Guide


Awesome. Just install the plugin and it start crawling.


Plug and play

The professional architect for your llms.txt file. Control how AI agents (ChatGPT, Claude, Perplexity) consume your website content with a structured, …

Ayuda WordPress

Fernando Tellado

VigIA (Spanish for “lookout” or “watchman”, incorporating “IA” – Spanish for “AI”) is a complete AI visibility toolkit for WordPress. Monitor 55+ AI crawlers, control access to your content, and optimize how AI systems discover and understand your site.
<h4>What does VigIA do?</h4>
<ul>
<li>Scores your AI visibility with a 100-point analyzer covering 20 checks across 5 categories</li>
<li>Tracks AI crawlers visiting your site (GPTBot, ClaudeBot, PerplexityBot, and 55+ others)</li>
<li>Provides detailed analytics with charts, statistics, and exportable reports</li>
<li>Blocks unwanted crawlers via PHP (403 response)</li>
<li>Manages robots.txt rules for AI crawlers with compliance monitoring</li>
<li>Sends email alerts about crawler activity (daily, weekly, or monthly)</li>
<li>Generates llms.txt files to help AI systems understand your site</li>
<li>Serves markdown endpoints for individual posts and pages (Markdown for Agents standard)</li>
<li>Generates JSON-LD structured data with Site Identity and AI Discovery signals</li>
<li>Exposes abilities for AI agents and automation tools (WordPress 6.9+)</li>
</ul>
<h4>Key Features</h4>
AI Visibility Analyzer 
* 100-point scoring system with letter grades (A+ to F) 
* 20 individual checks across 5 categories 
* Access & AI Discovery (37 pts): robots.txt, AI bot directives, Content Signals, llms.txt, sitemap, RSS feed 
* Structured Data & Semantic Context (25 pts): JSON-LD schemas, Open Graph, Twitter Cards, meta description, canonical URL 
* Content Structure & Readability (20 pts): heading hierarchy, semantic HTML5, image alt text, content/HTML ratio 
* AI Interaction & Distribution (8 pts): markdown delivery, AI share buttons 
* Access Performance (10 pts): TTFB measurement 
* Smart recommendations with direct links to VigIA features and plugin suggestions 
* Analyze any page on your site with URL autocomplete selector 
* Results cached for 24 hours with manual re-analyze option
Analytics Dashboard 
* Total visits, unique crawlers, and pages crawled statistics 
* Timeline chart with daily breakdown 
* Category distribution (AI Training, AI Search, AI Assistant, Data Scraper) 
* Top crawlers and most crawled pages tables with paginated navigation 
* <a href="https://wordpress.org/plugins/ai-share-summarize/" rel="ugc">AI Share & Summarize</a> integration: see share button clicks per page 
* Recent activity log with filters and paginated navigation 
* Period comparison functionality 
* CSV export
Crawler Blocking 
* Block crawlers via PHP with 403 Forbidden response 
* Quick block dropdown in analytics dashboard 
* Manage blocks from Extras page 
* Works on any server (Apache, Nginx, LiteSpeed, etc.)
Robots.txt Management 
* Add Disallow rules for AI crawlers 
* Visual preview of your robots.txt 
* Compliance monitoring: see which crawlers ignore your rules 
* One-click blocking for non-compliant crawlers 
* Works with both physical and virtual robots.txt
Email Alerts 
* Daily, weekly, or monthly reports 
* Three detail levels: Minimal, Normal, Complete 
* Non-compliant crawler warnings 
* Activity comparison with previous period
Markdown for Agents 
* Serve individual posts/pages as optimized markdown for AI agents 
* Dedicated .md URL endpoints (e.g., /your-post.md) 
* Accept: text/markdown content negotiation 
* Discoverability via link headers and HTML alternate tags 
* YAML frontmatter with title, date, author, categories, tags, and more 
* Respects blocking rules (blocked crawlers get 403) 
* Respects LLMs.txt exclusion filters 
* Analytics integration: tracks markdown requests per crawler 
* X-Markdown-Tokens response header 
* Follows the Cloudflare Markdown for Agents standard
LLMs.txt Generator 
* Select content by post type with one click 
* Filter by taxonomies (categories, tags, custom) 
* Manual include/exclude with AJAX search 
* Exclude by URL patterns (wildcards supported) 
* SEO plugin integration (auto-exclude noindex content) 
* Auto-regeneration (daily, weekly, monthly) 
* Robots.txt integration (add llms.txt and llms-full.txt references) 
* Generate llms.txt and llms-full.txt files 
* Full content or excerpt mode 
* Compatible with Yoast SEO, Rank Math, All in One SEO, SEOPress, and The SEO Framework
JSON-LD Structured Data 
* Generate WebSite and Organization/Person schema for site identity 
* AI Discovery: ReadAction pointers to llms.txt, llms-full.txt, and Markdown for Agents endpoints 
* Social profiles and sameAs links for brand identity across the web 
* SearchAction for Google sitelinks search box 
* Media library integration for logo selection 
* SEO plugin conflict detection (Yoast, Rank Math, AIOSEO, SEOPress, The SEO Framework) 
* Choose output page (front page or any published page) 
* Live JSON-LD preview with real-time updates 
* Smart integration with LLMs.txt and Markdown for Agents features
<h4>Supported AI Crawlers</h4>
VigIA monitors 55+ AI crawlers including:
<ul>
<li>OpenAI: GPTBot, OAI-SearchBot, ChatGPT-User</li>
<li>Anthropic: ClaudeBot, Claude-Web, Claude-SearchBot</li>
<li>Google: Google-Extended, GoogleOther, Gemini-Deep-Research</li>
<li>Perplexity: PerplexityBot, Perplexity-User</li>
<li>Meta: Meta-ExternalAgent, FacebookBot</li>
<li>Microsoft: BingBot</li>
<li>ByteDance: Bytespider</li>
<li>Amazon: Amazonbot</li>
<li>Apple: Applebot-Extended</li>
<li>And many more…</li>
</ul>
<h4>Privacy Focused</h4>
VigIA stores visitor data locally in your WordPress database. No data is sent to external servers.
<h3>Abilities API</h3>
VigIA is one of the first WordPress plugins to implement the <a href="https://developer.wordpress.org/apis/abilities-api/" rel="nofollow ugc">Abilities API</a> introduced in WordPress 6.9. This API allows AI agents, automation tools, and external systems to discover and interact with VigIA’s functionality in a standardized, secure way.
<h4>What are Abilities?</h4>
Abilities are self-contained units of functionality that VigIA exposes through WordPress’s central registry. Each ability has defined inputs, outputs, and permissions, making it easy for automation tools to understand and use them.
<h4>Available Abilities</h4>
VigIA registers the following abilities:
Analytics
<ul>
<li><code>vigia/get-crawler-stats</code> – Get statistics about AI crawler visits (total visits, unique crawlers, pages crawled)</li>
<li><code>vigia/get-top-crawlers</code> – Get a ranked list of most active AI crawlers</li>
<li><code>vigia/get-top-pages</code> – Get the most crawled pages on your site</li>
</ul>
Blocking
<ul>
<li><code>vigia/get-blocked-items</code> – List all blocked crawlers and IP addresses</li>
<li><code>vigia/block-crawler</code> – Block a crawler by User-Agent pattern</li>
<li><code>vigia/unblock-crawler</code> – Remove an existing block</li>
</ul>
Robots.txt
<ul>
<li><code>vigia/get-robots-rules</code> – Get current AI crawler rules in robots.txt</li>
<li><code>vigia/add-robots-disallow</code> – Add a Disallow directive for a crawler</li>
<li><code>vigia/remove-robots-rule</code> – Remove a robots.txt rule</li>
</ul>
<h4>Use Cases</h4>
<ul>
<li>Automated monitoring: AI agents can query crawler statistics and alert you to anomalies</li>
<li>Reactive blocking: Automation tools can block crawlers that repeatedly ignore robots.txt</li>
<li>External dashboards: Aggregate data from multiple WordPress sites with VigIA installed</li>
<li>WP-CLI integration: Future command-line access through the Abilities API</li>
<li>n8n / Make workflows: Build custom automation flows using VigIA’s abilities</li>
</ul>
<h4>Requirements</h4>
The Abilities API requires WordPress 6.9 or later. On older WordPress versions, VigIA works normally but abilities are not available.
<h3>Support</h3>
Need help or have suggestions?
<ul>
<li><a href="https://servicios.ayudawp.com" rel="nofollow ugc">Official website</a></li>
<li><a href="https://wordpress.org/support/plugin/vigia/" rel="ugc">WordPress support forum</a></li>
<li><a href="https://www.youtube.com/AyudaWordPressES" rel="nofollow ugc">YouTube channel</a></li>
<li><a href="https://ayudawp.com" rel="nofollow ugc">Documentation and tutorials</a></li>
</ul>
Love the plugin? Please leave us a 5-star review and help spread the word!
<h3>About AyudaWP</h3>
We are specialists in WordPress security, SEO, AI and performance optimization plugins. We create tools that solve real problems for WordPress site owners while maintaining the highest coding standards and accessibility requirements.

VigIA – AI Visibility, Analytics & Control

Monitor 55+ AI crawlers, control access via robots.txt, and boost your AI visibility with llms.txt, JSON-LD, Markdown for Agents & Visibility Score.

WEO LLMS & AI Crawler Guide vs VigIA – AI Visibility, Analytics & Control