The worst defect a bot has arrives on a calendar, not in a diff.

Because nothing about it changed, and that is exactly the problem.

Think of a passport. Nothing about the passport changes. A date passes and it stops working.

Bots rot in three ways, and none of them show up in a list of what you changed. The AI engine underneath gets retired and quietly swapped for another. A fact it was told once stops being true. And a link somebody typed into its instructions stops working.

None of these show up as an error. The bot keeps answering. It just gets quietly worse, in a voice nobody chose.

01 / Cause 1Cause 1: the AI engine is retired, and the supplier picks what replaces it

This is the only fault we know of that arrives on a calendar instead of arriving in your work. You change nothing. A shutdown date passes. The AI engine behind the bot is swapped for one the platform picked, and everything about how it writes shifts a little. The tone. The length. How willing it is to guess when it does not know.

There is no error to read. There is no list of changes to look through, because nobody changed anything.

We watch for this because we had to learn to. The platform we build on told us about two rounds of retirements, the first on 2026-08-31 and the second on 2026-10-30, each naming which engines were going. So far, ordinary. Then we tried to check it.

We could not find either date published anywhere. The news reached us in an email. The supplier's public list of changes covers that period and says nothing about it. No help article mentions either date.

Two days after the first date, the supplier's own screens were still offering engines that were supposed to be gone. We checked by reading the code the site actually loads.

It offers engines that are retired. It also offers engines that the system refuses to run if you pick them.

So the supplier's own product is wrong in BOTH directions. And there is nothing a customer can use to confirm a retirement before it happens to them.

So the rule we work to is: plan for the date. Do not wait to be told, when there is nowhere for the telling to arrive.

02 / The inversionThe counter-intuitive part: not choosing is the exposed position

Everyone assumes the risky position is having your bot tied to an engine that is being switched off, and that leaving it on whatever the platform gives you is the safe one.

It is the other way round.

A bot that never chose an engine runs whatever the platform hands it. And what the platform hands out is exactly the thing they move when they retire something. It changes with no notice and leaves no record on your side at all.

A bot tied to a dying engine has something better: a date, and somebody who can do something about it before that date.

If you run a bot and nobody can tell you which engine it uses, you have already found the problem.

03 / No fix at allAnd one that has no fix at all

Depending on how a bot was built, the platform will not let you choose its engine at all. Ask it to, and it answers with this:

422  llm is only allowed when bot type is PROMPT_BASED_BOT

In plain words: this kind of bot is not allowed to pick an engine.

So those bots will be moved whenever the platform decides, and nobody has any way to stop it. We record them that way. We are careful never to write it up as something fixable.

A warning nobody can act on is how a safety check gets switched off by the people it keeps nagging.

04 / Cause 2Cause 2: a fact was typed in, instead of referenced

This one is ours to own, because we did it.

A bot's instructions can hold a fact directly ("book here: example.com/booking"). Or they can point at one place where that fact is kept, and read it from there.

Typing it in is quicker, and it works perfectly. Right up until the fact changes.

It is like writing your phone number on twenty noticeboards around town. Change your number and you have twenty things to find. You will not find them all.

When a web address was changed, four live bots turned out to be carrying their own private copies of the same link. One of them pointed at an address that had stopped existing altogether, and people were arriving at it from paid adverts.

Nobody edited those bots. Nobody broke them. The world moved and the copies stayed where they were.

So we now have a tool that checks this, instead of a note reminding us to. The rule is: never type a shared fact into a bot's instructions. Not a price, a date, an address, a link or an opening time.

Point at the one place it is kept. Then there is one place to change it, and one place to be wrong.

There are two different failures here, and they are worth keeping apart, because most tools only catch the first:

• A link that is already dead. Easy to spot.
• A copy that will die the next time somebody changes that link. A link checker will never see it, because today it works fine.

The second one is the one that hurts. The only way to find it is to go looking for typed-in copies where a pointer should be.

05 / Cause 3Cause 3: the bot learned something and wrote nothing down

The quietest of the three. A person tells the bot something. The bot uses it correctly, right there in the conversation, and saves none of it.

A week later an automatic message asks them for the very thing they already gave. Their file never learned what the conversation knew.

From where the customer sits, that is not a small annoyance. It feels like being ignored by a company that is not paying attention.

A bot that hears a fact and writes nothing down has not learned it. If your bot cannot write to the customer's file, its memory lasts exactly as long as one conversation and not a second longer.

06 / Honest maintenanceWhat honest maintenance looks like

Not "we will keep an eye on it". Four specific things, all checkable:

1. A chosen engine, or a written note saying why one cannot be chosen.
2. Zero typed facts. Every shared value referenced, not copied.
3. A version number the bot can tell you. Ours carry one, so you can ask a live bot which version it is running. Without that you are trying to fix something you cannot even name.
4. A dated check of every claim it makes about prices, opening hours and what is available. Those are the ones that leave a real person standing outside a locked door.

07 / Our withdrawalAnd a benchmark of ours we are currently withdrawing

We compared several AI engines using a live client bot's real instructions. It gave us clean, useful-looking numbers. Some of them were surprising in the way good findings usually are, including one where a newer and smaller engine came out worst.

Then we found the flaw.

A bot's instructions are a form with gaps in them. The gaps get filled in with real prices, real names and real links at the moment the bot answers. We fed the engines the blank form.

On this bot, what we tested was 38% of the real thing. Nothing failed. Nothing warned us. It handed back believable tables that measured how those engines behave with most of our actual rules missing.

So we are pulling those numbers until we run it again properly. The test tool now fills in the gaps first, refuses to run at all if it cannot read them, and tells us how many it filled in.

We are publishing that here for the same reason we publish everything else: a test record containing only wins is a brochure, and everyone reading it knows.