llms.txt explained: a machine-readable summary for LLMs
llms.txt is a proposed convention — a plain-text file at the root of a site that offers language models a curated, easy-to-parse summary of what the site contains and where the important content lives. It is an attempt to help models understand a site without wading through its full HTML.
Last updated
What llms.txt is meant to do
The idea behind llms.txt is that web pages are cluttered with navigation, scripts, and markup that make them noisy for a model to read. A single, human-curated file in Markdown can lay out the site’s purpose and link to its most important, clean content, giving a model a reliable map.
It is a proposal, not an established standard, and adoption by the major AI tools is far from universal. It should be understood as an emerging convention some publishers add hopefully, not a guaranteed channel into any particular assistant.
How it differs from robots.txt
robots.txt tells crawlers what they may and may not access; it is about permission and exclusion. llms.txt is the opposite in spirit — it is about curation and helpfulness, pointing a model toward the content you most want understood, in a form that is easy to read.
The two also sit at the same place, the site root, and are sometimes confused. One restricts, the other summarises. A site can sensibly have both, doing different jobs.
What it can and cannot do
At best, llms.txt makes a site easier for a cooperating model to parse and gives you some influence over how your content is summarised. It costs little to add and does no harm, which is part of its appeal.
What it cannot do is compel any model to read it, cite you, or represent you a particular way. It is a courtesy that helpful tools may honour, not a control. Treating it as a guaranteed lever would be a mistake; treating it as a low-cost, honest signal is fair.
Where Kirti fits
Kirti is being built to keep a brand’s machine-readable surfaces — including files like llms.txt where a brand chooses to use them — accurate and consistent with the rest of its presence, as part of the same audit-and-maintain loop.
The agent is in active development and not yet generally available, and llms.txt itself is an emerging proposal, so this guide describes an unsettled area rather than a fixed standard.