What was announced
Jeremy Howard, the developer behind the llms.txt proposal, published a version 2 update to the specification on August 10, 2026. It is the first revision since the format launched in September 2024. The update adds standard link relations that give AI agents a reliable way to discover Markdown versions of web pages and the llms.txt files that describe them.
What changed
The original specification required sites to serve Markdown versions of pages by appending .md to the full URL — so /docs/tutorial.html would become /docs/tutorial.html.md. Agents had to guess this pattern or rely on the llms.txt file's link list to find Markdown content.
Version 2 introduces two changes to address that gap. First, it adds a second URL pattern: sites can now replace the file extension entirely, turning /docs/tutorial.html into /docs/tutorial.md. Both patterns are valid under the updated spec.
Second, and more significantly, v2 introduces two standard link relations for discoverability. A rel="alternate" attribute with type="text/markdown" provides a direct link from any HTML page to its Markdown counterpart. A rel="describedby" attribute points to the llms.txt file that covers the page. Sites can implement these as HTML <link> elements in the page head or as an HTTP Link: response header, which means server or CDN configuration can add them without modifying any templates.
Details and availability
The spec also formalizes how path-scoped llms.txt files work. A file at /docs/llms.txt covers all pages under /docs/, and the most specific file applies when multiple files exist in a path hierarchy. This allows sites that only control a subpath — such as a GitHub Pages project site — to participate fully.
The v2 update removes the mechanical semantics of the Optional section. In v1, the Optional section told context-expansion tooling what to omit when building an LLM prompt. That tooling is no longer part of the proposal. Optional sections remain allowed as a convention for secondary links, but they no longer carry processing instructions.
The background section now describes how agents actually use websites rather than predicting that they might. Howard notes that thousands of sites now publish llms.txt files, documentation platforms like Mintlify generate them automatically for hosted sites, and Google's Chrome team added an llms.txt check to Lighthouse's Agentic Browsing category when it was introduced in May 2026.
Context
The llms.txt proposal addresses a specific problem: web pages are built for people, wrapping information in navigation, ads, and JavaScript. Converting HTML back into clean text for language models is imprecise and token-costly. The specification provides a standardized way for sites to offer concise, LLM-friendly content that agents can process directly.
Adoption has grown since the original proposal. OpenAI, Anthropic, and Google's Gemini team all publish llms.txt files for their developer documentation. The format has become a de facto standard for AI-oriented content discoverability, even though Google's Search team has stated that support for llms.txt is not on their agenda.
What has not been confirmed
The specification does not describe how search engines or AI platforms will use llms.txt files for ranking, retrieval, or content selection. Google Search has not committed to reading or acting on the file. The update standardizes how sites expose Markdown content to agents but does not establish whether any particular AI system will preferentially use that content over HTML.
Sources
- The /llms.txt file, v2 — Full specification, modified August 10, 2026
- Changes — llms.txt — Changelog describing v2 revisions
