The Bot Did It

There is a particular kind of corporate miracle that happens the moment an AI chatbot says something useful.

When the chatbot answers a customer's question, it is an innovation. When it handles a complaint, it is a service improvement. When it summarizes complicated information, it is a productivity breakthrough. When it books, recommends, explains, drafts, nudges, and reassures, it is the friendly new face of the business.

Then the chatbot says the wrong thing.

Suddenly, the friendly new face of the business becomes a suspiciously independent creature.

It was only an informational tool. It was only experimental. It was only summarizing. It was not authorized. It was not official. It was not speaking for the company, even though the company placed it on its website, under its brand, within its customer journey, with the apparent purpose of answering questions on its behalf.

The chatbot goes from digital employee of the month to raccoon in the server room.

This is the absurdity Bruce Schneier and Nathan Sanders attacked in The Guardian. Their argument is direct enough to make many AI deployment strategies visibly uncomfortable. If a company deploys an AI agent to act for it, the company should be responsible for what the agent says and does.

That sounds almost boring until one notices how much of the current AI economy depends on pretending otherwise.

The pitch is that AI agents will reduce labor, accelerate service, answer customers, summarize reality, resolve friction, and eventually act on our behalf. The escape hatch is that when the system causes harm, the company would like everyone to remember that artificial intelligence is complicated, probabilistic, difficult to control, and best approached with a certain philosophical generosity.

That is a convenient little arrangement. The company gets the savings. The user gets the hallucination. The machine gets the blame.

The airline chatbot that apparently acted alone

The Air Canada chatbot case remains one of the purest examples of modern AI absurdity because the facts are almost too clean.

A man was trying to travel after his grandmother died. He asked Air Canada’s chatbot about bereavement fares. The chatbot told him he could book a flight and apply for the bereavement discount afterward. He followed that guidance. Air Canada later refused the refund and argued that its actual policy said something else.

There is a serious consumer-protection issue here, but the comedy sits in the company’s posture. Air Canada effectively tried to treat its chatbot as if it had wandered into the transaction under its own mysterious authority.

The chatbot was not some teenager with a fake badge loitering near the check-in counter. It was not a cousin of the website. It was not a spiritual adviser with aviation-themed hobbies. It was Air Canada’s chatbot, deployed by Air Canada, on Air Canada’s website, answering questions about Air Canada’s fares.

That should not require a philosophy department.

The tribunal did not accept the company’s attempt to separate itself from the system it had chosen to deploy. The customer had relied on information presented through the airline’s own digital channel. The fact that the answer came from software did not make it less connected to the company.

The most revealing part is not that the chatbot made a mistake. Humans make mistakes. Websites contain errors. Call-center employees misstate policies. Brochures become outdated. The revealing part is the attempted escape from responsibility once the mistake came through an AI interface.

The chatbot was useful enough to replace a human answer, but apparently not official enough to bind the company to the answer.

That is the trick. AI is presented as competent when it lowers cost, but demoted to unreliable machinery when accountability arrives.

The search engine that started writing answers

The German Google AI Overviews ruling creates a different version of the same problem.

Traditional search engines have long benefited from a relatively simple story. They index information. They organize information. They point users to information. They are not necessarily the original speaker of everything users find.

AI summaries complicate that story. An AI Overview does not merely hand the user a map. It produces a new answer. It compresses sources, rewrites language, decides what to include and what to omit, and presents the result at the top of the user’s experience, backed by the platform's authority.

That is no longer just a directory. It is a machine standing behind a desk and saying, “Here is the answer.”

According to the Guardian commentary, the German court treated Google’s AI search summaries as expressions of Google’s business activity. Wired and The Next Web also reported on the ruling and its implications for Google’s responsibility when AI-generated summaries produce false statements.

That shift is uncomfortable for companies that have grown used to the old internet posture. They want the power of publishing when it helps them capture attention, shape answers, and keep users inside the interface. They want the innocence of mere carriage when the answer injures someone.

AI Overviews make that posture harder to maintain. When a system rewrites information into an authoritative-looking answer, the platform is doing more than pointing. It is participating in the creation of the statement the user sees.

The machine may be generating the sentence, but the business built the room in which that sentence appears.

Disclaimers are not industrial-strength accountability remover

The modern AI interface often comes with a disclaimer that performs the emotional labor of corporate anxiety.

The output may be wrong. The system may make mistakes. Users should verify important information. The tool is for informational purposes only. The answer is not professional advice. The company is not responsible if the chatbot confidently improvises its way into disaster while using the tone of a patient hotel concierge.

Disclaimers have a role. No serious person should pretend that every AI output carries the same status as a signed contract or sworn affidavit. Context, design, reliance, user expectations, and the role of the system all matter.

But the disclaimer cannot become a magic napkin waved over a commercial product.

If a company designs an interface to appear helpful, fluent, authoritative, immediate, and official, it cannot be shocked when people treat it as helpful, fluent, authoritative, immediate, and official. If the chatbot is embedded in the customer-service flow, users will treat it as customer service. If an AI summary is placed above search results, users will treat it as the platform’s answer. If an agent is allowed to book, buy, approve, deny, recommend, or escalate, users will treat it as part of the organization that deployed it.

Companies know this. The whole point of the interface is to reduce friction.

The customer is not supposed to read six policy documents after asking the chatbot. The traveler is not supposed to interpret the chatbot as experimental theater. The search user is not supposed to hire a research assistant after reading the AI Overview. The patient, student, employee, applicant, client, or customer is not supposed to reconstruct the full evidentiary chain behind every answer.

If that were the expectation, the product would be a very expensive way to create more work.

The ugly little joke is that AI systems are sold as a way to save users time until the output is wrong. Then the user is told the real responsibility was to distrust the system from the beginning.

Every company wants an agent until agency law shows up

The word “agent” is now everywhere in AI marketing. It sounds active, competent, and inevitable. Agents will manage workflows. Agents will book travel. Agents will negotiate purchases. Agents will monitor accounts. Agents will talk to other agents, which is a sentence that should make every compliance officer reach quietly for a stronger coffee.

The word also comes with baggage. An agent acts for someone. In ordinary life, the concept is not mysterious. If a company sends a salesperson into a meeting, the company cannot always escape the consequences by saying the salesperson used words generated by a biological neural network. If a customer-service representative gives a policy answer, the company cannot treat that person as a separate civilization.

If a human employee publishes a company statement, the company does not usually get to say the mouth was merely a tool.

AI makes this familiar issue stranger, faster, and more scalable. A bad human answer may affect one customer. A bad automated answer can be replicated across millions of interactions. A human representative may improvise. An AI system can improvise with the confidence of a machine that has never had to sit through a deposition.

That is why Schneier and Sanders’ argument lands. If companies want AI agents to act for them, those agents should not become legal ghosts when something goes wrong.

The alternative creates a perverse incentive. Replace the human who creates liability with the machine that supposedly dissolves it. Let the chatbot answer the question, let the agent make the recommendation, let the AI summary shape belief, and then tell the injured party that the system was never truly speaking for the organization.

That is not innovation. That is accountability laundering.

The economics look different when the apology has a price tag

The most interesting part of this debate is not whether companies will stop using AI. They will not. The more interesting question is which uses survive once responsibility is priced honestly.

A chatbot that answers low-risk questions and escalates sensitive issues may still be valuable. A system that invents refund rules is less attractive. An AI summary product that is carefully constrained, logged, corrected, and governed may be commercially viable. One that produces false claims and expects a disclaimer to absorb the blast radius may become expensive. An AI agent that books routine appointments within clear rules may work. One that negotiates, promises, denies, purchases, or advises without strong controls may turn into a litigation subscription with a user interface.

That is how accountability disciplines hype.

The first wave of AI deployment often treats governance as an unpleasant decorative expense. The demo is fast. The savings are obvious. The interface is slick. The operational risk is quietly sent to the future, where it waits with invoices.

Liability drags that future into the present.

If the company is responsible for the chatbot’s answer, then testing is not optional. Monitoring is not optional. Escalation paths are not optional. Records are not optional. Policy alignment is not optional. Human review is not a sentimental tribute to the old economy. It is part of the control system that makes automation survivable.

That does not make AI useless. It makes unserious AI expensive.

The companies that complain loudest may be the ones whose business case depended on the customer absorbing the risk. Companies that build properly may still deploy powerful systems, but their pitch will have to mature. “Our AI can do the work” is no longer enough. The serious question is whether the company will stand behind the work after the machine does it.

The bot did it, and that is the problem

The failure is not only technical. It is behavioral. It reveals how quickly institutions try to split themselves in half when automation becomes inconvenient. The company wants to be present when the chatbot works and absent when it fails. It wants the machine to feel official to the customer but unofficial to the court. It wants the benefits of agency without the burden of agency.

That contradiction is not sustainable. Customers do not interact with model weights.

They interact with businesses. They see the airline, the bank, the insurer, the retailer, the platform, the publisher, the hospital, the school, or the government agency that placed the system in front of them. The interface may be artificial, but the relationship is not.

That is the part companies keep trying to blur.

AI does not remove responsibility from the organization that deploys it. In many cases, it makes the responsibility clearer because the organization chose the system, designed the workflow, controlled the context, benefited from the interaction, and invited the user to rely on the result.

The bot did it is not a strategy. It is an admission that the company gave the bot something to do.