llms.txt: your website, readable for AI.
Robots.txt tells crawlers where they may go. llms.txt tells AI models what your site is about. Small file, growing importance.
Definitie
llms.txt
llms.txt is a proposed web standard (2024): a markdown file at a fixed location (/llms.txt) that gives AI language models a concise, structured summary of a website — which pages exist, what they cover and where the key information lives. Where robots.txt governs what crawlers may do, llms.txt helps AI systems understand what is there.
Zie ook: Generative Engine OptimizationSelf-evolving website
Why would you want one?
AI search engines and assistants draw their answers from web content. An llms.txt makes it easy for them to summarise and cite your site correctly — resulting in better representation in AI answers. It costs nothing and can only help.
What does it look like?
An llms.txt is plain markdown: a title, a short description of the site and a grouped list of links with one line of context per page. Larger sites add an llms-full.txt with the full content as plain text. This site has one itself — see fwrd.digital/llms.txt for a live example.
Status of the standard
llms.txt is a proposal, not an official standard — no AI vendor has guaranteed support. But the cost is zero, adoption is growing and the major crawlers demonstrably fetch the file. Comparable to the early days of sitemap.xml: early adopters benefit longest.
Findable in Google and in AI.