`llms.txt` is a file format proposal introduced by the community in 2024, using a concise Markdown format to allow AI systems to quickly understand the structure and key content of a website. We have implemented it on this site, but we want to be clear upfront: Google has stated that it will not use `llms.txt`, and it is not a requirement for inclusion or citation. This article will teach you how to use it, and also tell you what to expect from it.
Jeremy Howard of Answer.AI proposed llms.txt in 2024-09 for a practical reason: navigation, ads and scripts clutter web pages and make them harder for language models with limited context to process. A clean Markdown summary can explain the site and point to its key pages. Its status matters: this is a community proposal, not an official standard of any search engine or AI platform. That distinction should guide how much effort you invest.
The documentation is intentionally simple, consisting of a single Markdown file located in the root directory of the website. The structure is as follows: H1 tag: Website name; Blockquote tag: A brief summary of the website; H2 tags: A list of links, with a description for each link. Additionally, there is an optional file called "llms-full.txt" that contains the complete page content. This file is intended for systems that can process long documents. Both files are plain text and do not require any special headers.
We use Route Handlers in Next.js App Router to provide both `/llms.txt` and `/llms-full.txt`. These files are not static, handwritten documents, but rather dynamically generated content pulled from the same data layer (TypeScript files containing service, case, and pricing information) that the main website uses. They are then output as static files during the build process using "force-static". This design solves the biggest problem with handwritten files: content drift. If prices or services change, the `llms.txt` file will automatically update during the next build, preventing the situation where the website and `llms.txt` have different information – which would provide outdated information to the AI system, making things worse than having no `llms.txt` file at all.
This is the part that most educational resources don't cover. Google's official documentation clearly states that AI functionality doesn't require any special technical expertise, and members of the search team have publicly stated that they don't use llms.txt. Whether other AI platforms read or have read this file, and whether this affects citations, remains unclear. We reviewed our server logs and found that some AI crawlers have accessed both files, but "being accessed" and "affecting citations" are two separate things, and the latter cannot be verified. Therefore, our conclusion is that llms.txt is a low-cost supplementary measure, not the core function of GEO – the true core remains the indexable content and original evidence.
Reasons to do it: Extremely low cost (one file), no known risks, and the potential for the platform to formally adopt it in the future. Reasons not to do it: If someone is asking you for a consulting fee for this, or trying to package it as "essential for any AI platform" to sell – this goes against the platform's official stance and can be used as a benchmark to assess the credibility of vendors. Our message to clients is consistent: do it, but it should be considered after technical SEO, content evidence, and measurement.
Three things to check: 1) Use `curl` or a browser to directly access `/llms.txt` and verify that the response is 200 and the content is up-to-date; 2) Ensure that the content filtering and sitemap are consistent – pages marked as "noindex" should not appear in `/llms.txt`, as this would effectively be handing over content you don't want to be exposed to AI; and 3) Regularly review the server logs to see which crawlers are actually accessing the content. This is the only firsthand data you have to determine if anyone is actually reading the content.
