The Lab | Systems Ninjas

Should Your Website Have An llms.txt?

Uldis Zalcmanis

Published on: 03/09/2026

We measured 400 services from a national register and read what their sites tell AI systems. At least six of the fourteen verified llms.txt files were not written by the business, and every AI-crawler block we found came from a CDN default. Plus the statistic we are withdrawing.

Answers

109 Childcare Websites. 3 Block AI Crawlers. All 3 Blocks Came From Cloudflare.

Uldis Zalcmanis

Published on: 02/09/2026

I sampled 400 services from Australia's national childcare register, resolved 109 distinct websites, and read what each one tells AI systems. Three block an AI crawler and all three blocks came from Cloudflare. Exactly one llms.txt in the whole sample was written by a human. Full method, confidence intervals and limitations included.

Field Notes