Six Ways AI Customer Service Fails, and Only Four Make the News

Four AI support failures are famous. A tribunal held an airline liable for what its chatbot said, a support bot invented a policy that did not exist and customers cancelled over it, a delivery company's bot swore at someone, and a dealership bot agreed to sell a car for a dollar.

All four are real, documented and worth reading. All four are also unusual. The two failure modes with no news story attached are the ones running in most deployments right now.

1. The invented policy

In April 2025, users of the code editor Cursor were being logged out unexpectedly — a session bug, a race condition on slow connections. Its AI support agent told them this was expected behaviour under a new policy limiting subscriptions to one device per account, described as a core security feature.

There was no such policy. The bot had constructed a plausible explanation for a bug. It spread through Hacker News and Reddit, users cancelled subscriptions over a rule that did not exist, and a cofounder eventually posted to acknowledge an incorrect response from a front-line AI support bot.

What makes this the most instructive of the four: the answer was fluent, confident, internally consistent and specific. Nothing about it read as a machine failing. It read as a support agent explaining a decision, which is exactly what customers cancel over.

The configuration that prevents it is refusal. A system that will say I do not know why that is happening, let me find out is worth more than one that resolves 5% more.

2. The binding statement

In February 2024 a Canadian tribunal ordered Air Canada to compensate a passenger who had followed bereavement fare guidance given by its chatbot — guidance that contradicted the airline's actual policy. The finding was that Air Canada is responsible for all information on its website, including chatbot responses.

This is the one with the widest reach, because it settles a question every deployment quietly depends on: whether the output is the company speaking. It is.

Which makes anything the AI can state about price, entitlement, refund windows or policy a commitment. The practical consequence is not more disclaimers — the ruling did not turn on a disclaimer — it is that the topics where being wrong creates an obligation should be answered from a fixed source or not at all.

3. Scope escape

Two documented cases. A Chevrolet dealership's chatbot was instructed to agree with everything the customer said and duly offered a Tahoe for one dollar. Klarna's support bot, in its first month, wrote Python when asked to.

Neither cost anyone much, and the pattern is the point: a general model wired to a support widget will do general-model things when someone tries. The defence is not better instructions. It is a system that can only take defined actions and can only speak about defined topics, which is an architecture choice made before purchase.

4. The tone break

In early 2024 DPD's chatbot, prompted by a customer, swore and criticised its own company. DPD disabled the AI component and attributed it to a recent system update.

The least consequential of the four and the most reported, because it is funny. Worth including only because it is the failure most vendor demos are designed to preempt, which tells you something about what gets optimised.

5. The fluent wrong answer nobody reports

No incident, no headline, no tribunal. A customer asks something, receives a confident and incorrect answer, acts on it, and either comes back annoyed or does not come back at all.

This is the failure mode that dominates by volume and it is nearly invisible to every metric on a support dashboard. It resolves the ticket, so resolution rate rises. Nobody reopens, so first contact resolution rises. It usually does not generate a survey response, so CSAT is unaffected.

Every number on the wall improves while the outcome gets worse. That is not a subtle measurement flaw — it is the default behaviour of the standard metric set applied to automated resolution, and it is why splitting CSAT by handler and plotting it against deflection is the single most useful thing you can add to a dashboard.

There is no incident report for this one because there is nothing to report. It looks like success from inside the building.

6. The escalation that does not happen

The other unreported one. A customer wants a person, the system keeps offering articles, and the loop continues until they give up.

It is measured as a deflection success. It is the mechanism by which deflection rate rises without any improvement in service — make humans harder to reach and the number moves — and the churn arrives later with no ticket attached to explain it.

Two configurations matter here and both are usually left at the vendor default. The number of failed attempts before an automatic handover, and whether asking for a human works as a direct command rather than a phrase the classifier has to recognise. Test the second one yourself during any trial. Type it in anger, phrased badly, and see what happens.

What the six have in common

None are model quality problems. A better model does not fix an unbounded action surface, an undefined escalation path, or a metric set that scores a fluent wrong answer as a success.

Five of the six are configuration and measurement. That is the useful conclusion, and it is more demanding than it sounds — the work sits with the buyer, in decisions taken before the contract, using questions most evaluations do not ask.

What to ask a vendor about this

• What does the system do when it does not know? Ask for the actual behaviour, not the confidence-threshold setting.
• Show me the escalation path when a customer types get me a human. In the product, not in slides.
• Which topics can it speak about, and where is that list? If the answer is anything in the knowledge base, the policy problem is unsolved.
• How would I detect a confidently wrong answer that the customer accepted? If there is no answer, you are buying failure mode five.
• Who is liable for what it says? The tribunal answered this in 2024. Ask anyway and note the reaction.

The four incidents are documented and dated from published reporting, cited below. The six-mode taxonomy and the reading that five of the six are buyer-side configuration are ours. We do not run tickets through these platforms and this is not a test report.

WRITTEN BY AR · UPDATED 2026-08-05

I read the fine print. Vendor pricing pages, billing definitions, terms, funding filings and acquisition notices — then I do the arithmetic nobody publishes: what a platform actually costs at your volume, what its headline metric is really counting, and who owns it now. I do not run benchmarks, and no page here pretends otherwise.

How we work · About the author