109 childcare websites. 3 block AI crawlers. All 3 blocks came from Cloudflare.

I sampled 400 services from a national register, resolved 109 distinct websites, and read what each one tells AI systems. Three of them block an AI crawler. Not one of the three wrote the rule. This is the method, the numbers, and the correction that nearly cost me the headline.

01 / Why this sectorI needed a population I could actually define

Last week I pointed a scanner at my own website to find out what it tells AI systems. It told me a few things I did not know I had said, and I published the whole list.

That is a small embarrassing story on its own. The interesting question is how common it is.

To answer that I needed a sector where I could define the population properly, instead of picking sites that felt representative and calling it a sample. A real register. Complete, public, and maintained by somebody with a legal reason to keep it accurate.

Australian childcare has one. Every centre-based service in the country sits in a national register kept by the regulator... 18,164 rows on the day I took the snapshot, of which 17,687 were eligible for this study.

And childcare turns out to be close to a perfect test case for the question I actually care about. Which is not is this sector AI ready. It is:

Has anyone here made a decision at all?

These are small local businesses. Nobody running one has sat down of an evening to consider whether an AI company should be allowed to read their fees page. So if a decision exists on their website, somebody else made it. That is the thing worth measuring.

Everything below was pre-registered. I wrote the measurement spec and froze it before the first fetch, so I could not go hunting for whichever number turned out to be interesting.

02 / The denominatorThe first number is the one that limits every other number

The register carries 98 columns. Name, address, suburb, phone, quality ratings, opening hours for every day of the week.

It does not carry a website address. Not one. Neither do the regulator's own service search pages.

So every URL in this study was constructed by us. And that is exactly where a study like this leaks.

400 services sampled. 138 matched to a live site, which is 34.5%. Those 138 collapse to 109 distinct websites, because 16 domains serve several of the sampled centres and one domain serves seven of them. Every rate in this article is out of 109.

Then I hand-checked 40 of the matches to see how often the resolver was right.

26 of 40. 65% precision.

That is not a footnote, so it is not going at the bottom. About a third of the resolved set is not the population at all... 8 completely unrelated businesses, 5 parent organisations such as a parish rather than the centre's own site, and 1 I could not determine either way.

I am telling you this before the findings rather than after, because it decides how much weight the findings can carry. And because "unresolved" means MY method found nothing. It does not mean the centre has no website. Recall cannot be estimated this way, so I am not going to report it and neither should anybody quoting this.

03 / llms.txt21 files. Nothing like 21 decisions.

Quick definition, because this one is new enough that most people have not met it. llms.txt is a proposed convention: a plain text file at the root of your site telling AI systems what your business is and which pages matter. Think of robots.txt, but for meaning instead of permission.

21 of the 109 sites served one. That is 19.3%.

Which was about three times higher than I expected. And that is precisely the moment a number deserves to be distrusted rather than published.

So rather than sampling the numerator, I opened all 21 and read them.

Six were not the childcare business at all. Four were online stores, one was a speech-pathology practice, one was a farm. One more I could not identify.

Verified: 14 of 109. 12.8%.

That correction points the direction it does for a structural reason, and this is the most useful paragraph in the article.

Shopify and Wix generate an llms.txt automatically. So when my resolver picks the wrong domain and lands on an online store, that wrong domain is MORE likely to carry the file than a real childcare site is.

My errors do not cancel out. They push this specific number up.

Now the part that actually matters. Of the 14 real ones:

1. Three are Wix platform defaults. Each one carries the line "This site is powered by Wix and supports the Model Context Protocol".
2. Three are the same file with the brand name swapped. Three separately branded childcare groups serve an llms.txt containing an identical sentence about sparking a love of learning. I fetched all three and diffed them.
3. Two more sit in the same size and link-count class but I did not diff them, so they are counted as suspected and never as confirmed.
4. One was generated by an SEO plugin, which announces itself inside the file: "Generated by All in One SEO".
5. Four are unattributed. No generator string, no template match. Recorded as unknown rather than guessed.
6. One is plainly hand-authored. Bespoke prose, real detail about the centre, correct history. Somebody sat down and wrote it.

So at least 6 of the 14 were demonstrably not written by the business whose site they sit on. And in a 400-service sample of a regulated national register, exactly one person appears to have written this file themselves.

Quality, where a file exists: 16 of the 21 open with a valid Markdown heading. Of 63 links I sampled from inside them, 5 were already dead. Published once, never opened again.

04 / robots.txtEvery AI block I found was a vendor default

89 of the 109 sites served a readable robots.txt. 20 had none at all.

Three of the 89 block at least one AI crawler. That is 3.4%, and the interval around it is wide: 1.2% to 9.4%. Three is a small number and I am not going to dress it up.

Here is the part I did not expect.

All three of those files carry Cloudflare's own # BEGIN Cloudflare Managed content marker. All three block the identical eight agents, in the identical order.

Hand-written AI-crawler rules in the sample: zero.

Not one site in 89 made this decision itself. And 72 of the 89 contain no rule matching any AI crawler at all, which is not a policy either way. It is silence.

Two more zeros worth having.

Zero sites block OAI-SearchBot. That is the agent governing whether you show up in ChatGPT's search answers, and it is a completely different agent from GPTBot, which is training only. The managed list blocks the training one. So if a centre believed it had opted out of AI, it has opted out of the half that does not affect whether a parent can find it, and left the half that does. Nobody chose that trade. It arrived pre-made.

Zero blanket Disallow: /. Nobody has closed the door.

And the regulator does exactly the same thing. I read its robots.txt myself before writing this sentence: it disallows nine named agents, eight of them AI crawlers, inside the same Cloudflare managed block, and it adds Cloudflare's Content-Signal: ai-train=no policy line on top.

Which means the most sophisticated AI policy anywhere in this study, including the regulator's, was written by a CDN.

05 / The contradictionOne site does both at once

My spec said that under 20 cases I publish observations and no percentage. The set here is one, so here it is as an observation.

One site publishes an llms.txt inviting AI systems to come and read it, while its robots.txt blocks eight AI crawlers from doing that.

It is the cleanest possible illustration of the whole finding, because the owner did not author the contradiction. They published the llms.txt. Cloudflare's managed default published the block.

Two vendors. Opposite instructions. One website. Nobody in the middle.

06 / So whatThe recommendation is not the one you are expecting

Both halves of "AI readiness" in this sector are vendor artefacts rather than business decisions. The llms.txt files are mostly Wix defaults, one agency template replicated across three brands, and an SEO plugin. The crawler blocks are entirely Cloudflare's managed list.

One clearly hand-authored file. Zero hand-written crawler rules.

The sector has not opted out of AI crawling. It has also not opted in. A CDN made the call for the few who have one, and everybody else has no position, because nobody ever asked them for one.

Now, I want to head off the conclusion people will want to draw from this, because it is the wrong one. The useful version is not "go and add an llms.txt". Adding one puts you in exactly the same category as the three Wix defaults... a file on your server that you did not write and will never look at again.

The useful version is smaller than that, and it is free.

That is the entire recommendation. It costs nothing, I am not selling it, and it works whether or not you ever speak to anybody like me.