Crusco Agentic Content
Introduction
Optimize Joomla for ChatGPT, Claude, AI Search & Agentic Web. Serve Markdown, YAML and llms.txt v2. Enforce copyright with TDMRep/Content Signals opt-outs.
Description
Traditional SEO is evolving rapidly with the emergence of the Agentic Web. AI crawlers and autonomous agents now retrieve website content that LLMs use to respond to user prompts.
If your content is buried under heavy layouts, scripts, tracking pixels, navigation menus, and other syntactic noise, AI models may misinterpret its meaning or context. This can lead to inaccurate AI-generated responses that may negatively affect how your brand and products are represented.
Crusco Agentic Content is a Joomla plugin that improves the Agent Readability of your website content, a key element of any AEO ( strategy and an entry point to the Agentic Web.
The plugin automatically removes unnecessary markup and converts your standard HTML into clean, AI-ready Markdown. It also helps protect your intellectual property using two opt-out copyright reservation protocols.
Make your Joomla content easier for AI systems to read, understand, and cite accurately.
Why Choose Crusco Agentic Content?
- Future-Proof Your Traffic: Traditional search is shrinking. Make sure your website content is read and interpreted correctly so that AI systems can cite it effectively.
- Cut AI Token Costs: Clean, optimized Markdown eliminates background noise, reducing the token payload and server resource consumption.
- Enforce Legal Control: Don’t let AI companies scrape your data for free. Prohibit unauthorized training using Content Signals and TDMRep opt-outs.
- llms.txt v2 Ready: Organize localized content indexes through a multilingual hub, delivering clean, structured maps for RAG systems.
How Crusco Agentic Content Outperforms Generic Solutions
Unlike standard HTML-to-Markdown conversion tools or simple llms.txt generators, which often output noisy code and generic links, Crusco Agentic Content is an enterprise-grade optimization suite that turns raw web content into clean, structured, AI-ready assets.
Key Differences
Extension compatibility
- Generic Solutions: Joomla Core components.
- Crusco Agentic Content: All extensions, templates, and page builders.
Page Builder & Custom Wrappers
- Generic Solutions: Fail or produce noisy layouts.
- Crusco Agentic Content: Custom CSS selector support.
Parsing Quality
- Generic Solutions: Basic, messy HTML conversion.
- Crusco Agentic Content: Advanced hybrid engine with structural fallbacks.
Token Cost Control
- Generic Solutions: High payload, with Base64 data, hidden CSS, banners, and unwanted modules left intact.
- Crusco Agentic Content: Active token stripping removes noise at the source, achieving up to 80% payload reduction, depending on page structure.
AI Copyright Protection
- Generic Solutions: None, or basic robots.txt.
- Crusco Agentic Content: CDSM (Copyright in the Digital Single Market) compliant. Content Signals and TDMRep opt-outs, supported by the EU AI Act’s GPAI provider obligations.
RAG System Readiness
- Generic Solutions: Plain text only.
- Crusco Agentic Content: Native YAML front matter with 28+ schema types, engineered for immediate integration into any retrieval-augmented generation (RAG) system.
AI content delivery
- Generic Solutions: Often set only a basic
Vary: Acceptheader, completely lacking .md sibling support or failing to implement it on the homepage. Relies on silent, messy HTML fallbacks when the requested format cannot be provided. - Crusco Agentic Content: Parses full Accept headers and quality values, returning an RFC 9110-compliant
406 Not Acceptableresponse when the requested representation cannot be served. Instead of a silent error, it serves a transparent response body listing the available formats (HTML and Markdown) alongside alternative Link headers that guide the crawler to the correct resource. It also includes full.mdsibling support with a dedicatedindex.mdendpoint for the homepage.
- Generic Solutions: Often set only a basic
llms.txt Specification Support
- Generic Solutions: Standard, single-language llms.txt output with zero support for sections or menu-level asset customization.
- Crusco Agentic Content: Support for the
llms.txtv2 specification, featuring structural section grouping, granular Joomla menu integration, and an automated multilingual hub that routes RAG systems to localized indexes (llms_it-IT.txt,llms_en-GB.txt, and so on).
Main Features
🛠️ Enterprise-Grade Markdown Engine
Built on top of the industry-standard League HTML-to-Markdown library, Crusco Agentic Content extends its capabilities with a specialized, custom-coded processing layer. Because every website is unique in its structure, CSS, frameworks, and installed extensions, Crusco Agentic Content gives you precise control over the exact data served to Large Language Models, helping reduce AI misinterpretation and hallucinations.
- Smart H1 Fallback: Automatically recovers missing H1 tags from template positions or falls back to the
titletag. - AI-Ready Page Anchors: Converts page layout anchors into a cross-platform Markdown format that preserves table-of-contents (TOC) navigation for LLMs.
- Process Media Tags: Because Markdown lacks native syntax for
videoandaudiotags, Crusco Agentic Content converts multimedia elements into Thumbnail Links, a cross-platform Markdown representation suitable for AI consumption. - Process Map Tags: Although
mapandareatags have no direct equivalent in Markdown, Crusco Agentic Content transforms image maps into a semantic, structured Markdown text block that language models can ingest to understand the image map and improve their overall comprehension. - Base64 Stripping: Optionally removes embedded images to save token space and prevent inaccurate AI answers.
- Hidden Element Purge: Removes content concealed with inline CSS, creating a leaner, more cost-effective AI input.
- CSS Selector Targeting: Clean up your content by removing cookie banners, CAPTCHAs, comment boxes, and any unwanted or low-value elements, including specific Joomla modules.
- JSON-LD Controls: Easily include or exclude Schema markup to remove syntactic noise according to your needs.
- Component Blacklisting: Exclude specific Joomla Core or third-party components from Markdown generation with one click.
🧩 Universal Page Builder & Extension Compatibility
By default, Crusco Agentic Content targets the core Joomla Component Area, precision-isolating your primary text from layout noise (menus, sidebars, and footers) to feed LLMs with pure Markdown.
If a layout, advanced Page Builder, or third-party extension bypasses native Joomla output structures, the plugin provides two alternative mechanisms to ensure perfect extraction:
- Custom CSS Selector: Define a custom wrapper block via CSS selector to isolate and extract text from specific layout zones only.
- Full HTML Page: Parse the entire HTML document while defining explicit structural exclusions (e.g.,
header,aside,footer,dialog) or filtering out specific classes and IDs.
🤖 Future-Proof Delivery
Ensure broad compatibility with AI agents and crawlers. It implements two well-established AI content delivery methods:
- HTTP Content Negotiation: Fully compliant with RFC 9110 directives. Achieves a perfect score on the acceptmarkdown.com “How AI-Ready is your URL?” test.
- Markdown mirror: Leverages the RFC 8288 standard to mirror your content. Inserts alternative Markdown links into both the HTML
headand HTTP headers for fast discovery.
⚡ SERP & CDN Safeguards
Engineered for Advanced SEO protection. Prevents keyword cannibalization between HTML and the Markdown mirror, and prevents caching mix-ups between human visitors and AI on CDNs or edge-caching layers.
🛡️ Security Header & Robots Preservation
Zero security compromises. Crusco Agentic Content serves Markdown while fully respecting your Joomla "HTTP Headers" system plugin configuration, including strict-transport-security and referrer-policy.
Robots restrictions are managed with smart, automated routing:
- Negotiated Markdown: Requests served from the original URL automatically mirror the exact
noindexrestrictions of the corresponding HTML page. This approach avoids sending conflicting signals (such as combiningnoindexwith a canonical tag), ensuring search engines process the exclusion directive cleanly without ambiguity. - Dedicated .md Endpoint: The standalone Markdown mirror strictly enforces a
noindexrule. This isolates alternative content views and prevents any risk of keyword cannibalization, ensuring your primary HTML pages retain full SEO priority.
📊 Structured YAML Metadata (Front Matter)
LLMs process information with massive accuracy when the context is declared in the very first lines. Crusco Agentic Content optionally injects a rich YAML Front Matter block at the top of every Markdown response to guide models with zero ambiguity.
- Token Optimization: YAML cuts complexity and token consumption compared to bulky JSON-LD. It can includes author identity (E-E-A-T), schema type, chronological freshness, and taxonomies into an immediately readable format for models.
- RAG System Routing: Empowers AI agents to filter, catalog, and validate your source's authority before text chunking even begins.
- 4 Preset Formatting Modes: Choose the style that fits your strategy: Accurate, Balanced, Token Saver, or Custom.
Supported Content Types (With Automated Fallback)
The plugin dynamically maps and falls back across 28+ schema types (from Article, Product, and Guide down to a universal WebPage wrapper). This guarantees the crucial type attribute is always present for LLM classification. Supported types:
- AdvertiserContentArticle
- Article
- Blog
- BlogPosting
- Book
- ComicStory
- Course
- CreativeWork
- Diet
- DiscussionForumPosting
- Event
- Guide
- HowTo
- JobPosting
- MobileApplication
- NewsArticle
- Product
- Recipe
- Report
- SatiricalArticle
- ScholarlyArticle
- SocialMediaPosting
- SoftwareApplication
- TechArticle
- Thesis
- VideoGame
- WebApplication
- WebPage.
🇪🇺 AI Policy & EU AI Act Compliance
Regain full legal and technical control over your data directly within your site's HTTP headers and inside the head tag, in compliance with the CDSM (Copyright in the Digital Single Market) Directive and the obligations set out by the EU AI Act for General-Purpose AI (GPAI) model providers, which legally binds AI providers to respect machine-readable copyright reservations.
- Global Guardrails (TDMRep & Content Signals): The plugin simultaneously activates two major global standards to set your opt-out reservation. It implements the TDMRep protocol for European copyright protection, alongside Content Signals syntax, the HTTP-based framework backed by Cloudflare and Thoughtworks, to dictate whether AI models can use your texts for real-time answers (RAG) or prohibit commercial training datasets.
- Machine-Readable TDM Policy URL: In compliance with the EU AI Act and the CDSM Directive, you can attach a formal policy URL to your network responses. This allows organizations to provide automated systems with precise terms, structured in a machine-readable ODRL profile following W3C specifications.
- Link Directives: Control whether AI agents can follow links within the Markdown output to explore topics further and cite sources, or restrict them to protect your link network.
.well-known/tdmrep.jsonendpoint: Choose whether to apply the AI policy rules on content usage not only to HTML and Markdown pages, but also to all static files, such as images, videos and PDFs.
📑 llms.txt v2 & Multilingual Hub
Support for the updated llms.txt v2 standard optimizes AI crawling and token efficiency with structural grouping and localized indexes.
- Organize Dedicated Sections for LLMs: Group your pages into logical, well ordered sections within your index.
- Joomla Menu Integration: Manage included pages, custom titles, and descriptions directly from native menu options.
- Multilingual Hub: Automatically links localized indexes like
llms_en-GB.txtandllms_it-IT.txtthrough a central root index.
Explore all features and in-depth guides in our official documentation.
Translations
The Extension is translated into the following languages:
- Italian
- English
- German
- Spanish
- French
- Russian
- Chinese
Crusco Agentic Content
- Version:
- 5.1.0
- Developer:
- HTML Crusco
- Last updated:
-
Nov 19 2026
1 day ago - Date added:
- Jul 15 2026
- License:
- GPLv2 or later
- Type:
- Free download
- Includes:
- p
- Compatibility:
- J5 J6
Share