llms.txt is an evolving proposal for giving agents a concise Markdown guide to a website’s useful information. The author-maintained primary resource documents its format, changes, and integrations.
What the file does
It provides a project heading, optional context, and named links to useful resources. It can complement accessible HTML and text versions of pages. Its purpose is guidance to content, not Allow/Disallow rules.
What it does not establish
The proposal is not a universal indexing requirement or an access-control standard. Publishing the file does not prove that every assistant uses it or that your site will be cited. Separate documented tool support from results observed in your own tests.
Use robots.txt for cooperating crawler preferences, appropriate indexing controls for search, and authentication for private material. On answer.cloud, discovery files are generated by the publishing workflow.
See the implementation guide and crawl permissions guide.