AI Agents Can Do Anything. Your Chatbot Can't.

Why ChatGPT feels effortless

Three things are true of a conversation with Claude or ChatGPT that are not true of a conversation with your support bot.

People choose it. They open it out of curiosity, with time to spare and patience for a second attempt. Nobody opens your support bot because they want to. They open it because a package is missing, a payment failed, or something they expected to work didn't. The user arrives annoyed before the first message. That's not a quality difference — it's a tolerance difference, and it's enormous.

It has no obligation to be right about you. A general assistant can be interestingly approximate. It can hedge, explore, offer three possibilities. A bot answering a question about your refund, on your account, under your contract cannot. One wrong number is a complaint, and in a regulated industry it can be a reportable event.

Its failures disappear. When ChatGPT misses, you rephrase and move on. You don't file a ticket, call anyone, or tell your colleagues. The same miss from a bank's assistant becomes an escalation, a CSAT score, sometimes a tweet. We are comparing a system whose failures are silently absorbed with one whose failures are counted, logged and reported upward.

Add to that the freedom to be expansive. A general assistant can say "it depends, here's how to think about it." Your bot operates inside policy, legal review and brand guidelines — constraints that push it toward hedging, qualifying and over-explaining. The same caution that keeps you compliant is what makes the bot feel evasive.

Why enterprise bots can't simply be ChatGPT

Then there's the part everyone underestimates: what it actually takes to answer a simple question.

"Where is my order?" requires order management. "Why was I charged twice?" requires billing. "Can you change my address?" requires the CRM, write permissions, and an audit trail. Each connection is an integration to build, secure, maintain and version. Permissions are rarely clean. Exposing internal data to a model raises security, privacy and compliance questions that take months to resolve, and sometimes the answer is no.

Even when the plumbing works, the bot often has partial information, insufficient permissions, or no ability to perform the action the customer is asking for. It can tell you where your order is. It cannot reroute it, because nobody was willing to give an LLM write access to logistics.

A general assistant has none of these problems, because it has none of these responsibilities.

But the constraints are not the excuse

Here's where the argument usually stops, and where it shouldn't.

Two organisations with comparable systems, comparable regulation and comparable integration budgets produce chatbots of very different quality. The constraints explain the ceiling. They don't explain the distance between teams operating under the same ceiling.

Building a good conversational experience is still hard, and LLMs did not make it easy. They made one part easier — understanding what people say — and quietly made another part harder.

A rule-based bot failed visibly. It returned a fallback message, dropped into a menu, or said "I didn't understand that." The failure announced itself.

An LLM-powered bot fails beautifully. It produces a fluent, confident, well-structured answer to a question the customer didn't ask. The conversation looks successful. The transcript reads well. The containment metric goes up. And the customer leaves without what they came for.

That's the real shift. Bad design used to be loud. Now it's articulate.

One sentence in, a page back

A customer asks: "Can I change my delivery address?"

The bot replies with four paragraphs covering the delivery policy, account settings, shipping restrictions and possible exceptions. Somewhere inside is the answer.

Technically it responded. The information may even be correct. But the customer asked one sentence and received a page, and they now have to read all of it to find out whether the answer is yes.

This is verbosity without value, and it's one of the most common failure modes I see. It raises cognitive load, buries the answer, invites follow-up questions the bot could have pre-empted, and — worst of all — looks impressive in a demo. Long, polished responses photograph well. They perform badly.

The fix isn't shorter answers. Some questions genuinely need detail. The fix is answers proportionate to the question.

I'd add that this is easy to miss in your own work. When I ran 166 of my own research conversations through our own analysis, one of the things that came back was that I had written nearly twice as many words as the people I was interviewing — 27,400 to their 14,681. I had read every one of those conversations myself and never noticed. It took a machine counting to show me.

Which is why you need to see where it actually fails

If the failures that matter are the ones that look fine, then you cannot find them by looking at whether the bot responded.

Response time, message volume, intent accuracy and containment rate will all look healthy while a customer reads four paragraphs, gives up, and calls you instead. These metrics aren't wrong — they're accurately reporting things that are not the problem.

What you need to know is narrower and harder: did the customer get what they came for, did the conversation make that easy, and when the bot couldn't help, did it say so and hand over cleanly. That means reading conversations — which nobody can do at volume by hand, which is the whole reason this is difficult.

We've written about how we break that down into resolution accuracy, containment quality and dialog fluency, and how those combine into a single score.

The comparison you're actually losing

Your customers aren't really comparing your bot to ChatGPT's capabilities. They're comparing it to the feeling of being understood on the first try.

You can't give them an unconstrained assistant. You can give them one that answers the question they asked, admits what it can't do, and doesn't make them work for it.

That's achievable inside every constraint described above. It just isn't achievable without knowing where you're currently failing.