What is llms.txt?
A plain-English definition, and the tools that work with it.
llms.txt is a proposed convention for a Markdown file at your domain root that gives AI systems a curated map of your site: a name, a one-line summary and grouped links. It is optional, and Google has stated its AI search does not use it.
- It is not robots.txt. robots.txt is a decades-old standard that genuinely controls crawler access, llms.txt only suggests content and carries no enforcement, so ignoring it has no consequence.
- The honest evidence is thin. No major AI operator documents llms.txt as a retrieval or ranking input, and sites that do well in AI answers overwhelmingly do so without one.
- It is still occasionally useful. Some coding assistants and documentation agents fetch it deliberately, which makes it most worthwhile for sites with developer docs.
- The format is a curated map, not an index. A 30-link file a model will actually read beats a 300-link dump. You already have a sitemap for completeness.
Tools for llms.txt
Free, private and in your browser. No sign-up.
llms.txt Generator
Generate a spec-shaped llms.txt from your sitemap or URL list, with an honest answer to the question every other generator dodges: Google has said its AI search doesn't use llms.txt. Here's who it actually helps, and a good file in ten seconds if you want one.
GigAI GEO Audit
One URL in, one AI-visibility report out: crawler access with Cloudflare detection, JS rendering, citability scored with published methodology, retrieval chunks, answer snippets and schema. Each finding linked to the free tool that fixes it.
XML Sitemap Generator
Turn a list of URLs into a valid sitemap.xml with per-URL changefreq, priority and lastmod, validated against the 50,000-URL limit and ready to submit to Google. 100% in your browser.
llms.txt problems we solve
Hit one of these? Here's the fix.
Frequently asked questions
Related terms
- robots.txtrobots.txt is a plain-text file at a site's root that tells search-engine crawlers which URLs they may or may not crawl. It's used to keep crawlers out of admin, duplicate or low-value areas and to point them at the sitemap, but it controls crawling, not privacy.
- XML sitemapAn XML sitemap is a file listing a website's URLs so search engines can discover and index them efficiently. It doesn't guarantee ranking, but it helps Google and Bing find every important page (especially on large or new sites) and speeds up indexing of fresh content.
- Generative Engine Optimization (GEO)Generative Engine Optimization (GEO) is the practice of making a website usable by AI answer engines like ChatGPT, Claude, Perplexity and Google's AI features, so they can crawl it, read it without JavaScript, and quote it as a source. It overlaps with SEO but optimizes for being cited rather than ranked.
- AI crawlerAn AI crawler is an automated bot that fetches web pages on behalf of an AI company, either to train a model, to build a search index that AI answers cite from, or to read a page live when a user asks about it. Each purpose is a separate bot with its own name in robots.txt.
Keep exploring
Related tools
Problems we solve
Definitions
From the blog
Common tasks
Try llms.txt Generator
Free, private and instant. Everything runs in your browser.