Field Notes

Fluency Used to Be Evidence. Now It's Free.

Every system that talks to people can sound competent now. Almost none of them are measured on whether the person got anywhere.

I wrote a piece recently arguing that bad chatbot design used to be loud and is now articulate. Two readers independently told me I'd drawn the boundary too narrowly.

They were right, and the wider version is more uncomfortable.

What fluency used to tell you

For most of the history of software, a clear sentence was expensive.

If an error message explained what had gone wrong and what to do next, somebody had thought about the failure. If a help article answered the question you actually had, somebody had spoken to people who had it. If a flow walked you through a task in a sensible order, somebody had watched a person attempt it.

Fluency wasn't the goal. It was the residue of someone understanding the problem well enough to be clear about it.

That made it a reliable proxy. Badly written software was usually badly thought-through software, and you could judge one by the other without ever reading the code.

That correlation is now broken. Producing fluent, well-structured, confident text costs nothing and requires no understanding of the user's situation whatsoever. The output looks exactly like the output of a system that was designed carefully. The signal we spent thirty years learning to read has stopped carrying information.

It was never only about chatbots

Dani O'Singh, a managing editor at Clarity Studio, put it better than I had:

"The same trouble runs through help articles and setup screens, where a sentence can be perfectly grammatical and still leave the person with a failed payment no idea what to press next."

Once you're looking for it, it's everywhere.

The error message that is a complete sentence and tells you nothing. The confirmation email that acknowledges your request without resolving it. The status page reporting all systems operational while support fills with people who can't log in. The onboarding flow where every individual screen is clear and the sequence makes no sense. The dashboard accurately reporting things that aren't the problem.

Each one produces output that would pass any review you could run on it. Grammatical, on-brand, internally consistent, factually defensible. And each one leaves a person stuck.

Why our instrumentation can't see it

Florent Balkoulga, who runs IT projects in banking and insurance, named the mechanism in a comment that I've thought about more than my own article:

"In many business systems, we measure what is easy to capture: response time, availability, error rates, transaction volumes. But we don't always measure what happens between those metrics and the user's actual goal. How many steps did they have to go through? How many corrections? How many times did they have to switch between systems or ask for help?"

And then the line that reframed it for me:

"The system can report success while the user experiences friction."

Look at what's on a standard operational dashboard and you'll notice almost every metric is an observation about what the system did. It responded in 240ms. It returned a 200. It stayed up. It completed the transaction.

Very little of it observes what the person had to do.

That's not an oversight, it's an artifact of where instrumentation is easy. The system emits its own events. The user's experience doesn't emit anything — it has to be inferred from behaviour, which is harder and which most teams never get to.

So we built an entire discipline of observability around one side of the interaction, and then automated the other side with systems that can produce plausible output indefinitely without succeeding at anything.

Three tests that work

Dani's is the one I haven't been able to improve on:

"Read the answer aloud to someone who has the problem and ask what they'd do now. An articulate answer that earns a shrug has failed as surely as the old menu did."

What makes it work is who you read it to. Not a colleague, not a stakeholder — someone who has the problem and doesn't have your system knowledge. Everyone who built the thing already knows what to press next. That's precisely why they're the wrong people to judge whether the words are any good.

It defeats builder bias by construction, which no internal review process manages.

Florent's is the quantitative sibling: count the steps, the corrections, the switches between systems, the requests for help. Not was the output correct, but what did the person have to do.

Our own version predates both, and it's the one we used for years before any of this was measurable. We'd take a real transcript — or a design proposal, before it shipped — and lay it out as a theatre script. Two characters, lines, stage directions if we wanted them. Then read it as a scene.

If it worked as a scene, the conversation was good. If it didn't, we rewrote it until it did.

The criterion we used was whether it was entertaining, which sounds frivolous and isn't. A scene holds attention when both characters want something and the exchange moves. A scene where one character delivers a well-formed paragraph and the other says "ok" and leaves isn't a scene — it's a monologue with a witness. Which is a fair description of most bad bot conversations.

Staging it also does something the other two can't: it makes the user's lines visible as lines. In a log you read what the system said. In a script you watch a character ask the same thing three times in slightly different words, and on a page that's unmistakable in a way it never is in a dashboard.

And unlike the others, it runs before anything ships. You can stage a design proposal. You can't run analytics on a conversation that hasn't happened yet.

Three methods, three trades, one instinct: judge the conversation as a conversation rather than as a system output.

The limit of all three

None of them scale, and I don't want to pretend otherwise.

You can read ten answers aloud. You cannot read four thousand, and the ten you pick will be the ten someone already suspected. Counting steps and corrections by hand across a month of traffic is a research project, not an operating practice. And staging transcripts as scenes is wonderful for a design review and hopeless as a weekly routine — we know, because we tried to make it one.

What you can do at scale is count the residue. People who rephrase the same request three times. People who backtrack. People who come back two days later with the same question nobody logged as unresolved. People who simply stop, without complaining, which registers on most dashboards as a clean exit.

That's an approximation. A behavioural proxy for a thing you'd rather observe directly, and it will miss cases where someone was confused but persistent, or satisfied but chatty. I'd still take it over fluency, because fluency now tells you nothing at all and effort signals at least point in the right direction.

The question worth asking

I've stopped asking whether a system responded correctly. It's the wrong question, and increasingly every system passes it.

The question I'd put to any interface that talks to people — a bot, an error state, a help centre, a form — is narrower and harder to fake:

What did the person do next?

If the answer is "the thing they came to do," it worked. If the answer is "asked again, differently" or "went to find a human" or "closed the tab," then it doesn't matter how well it was written.

An articulate answer that earns a shrug has failed as surely as the old menu did. The difference is that the old menu had the decency to look like a failure.

See how Inquio can help your business case

Curious about Inquio?

Never miss a product update, expert advice, and more.

By clicking “Subscribe” you agree to our Terms of Use and Privacy Policy.

Curious about Inquio?

Never miss a product update, expert advice, and more.

By clicking “Subscribe” you agree to our Terms of Use and Privacy Policy.

Curious about Inquio?

Never miss a product update, expert advice, and more.

By clicking “Subscribe” you agree to our Terms of Use and Privacy Policy.

Ready to see your bot report card?

Notebook with Bot Report Card

Ready to see your bot report card?

Notebook with Bot Report Card

Ready to see your bot report card?

Notebook with Bot Report Card

© 2026 Inquio LLC. All rights reserved

8 The Green, Suite A, Dover, DE 19901, USA

© 2026 Inquio LLC All rights reserved

8 The Green, Suite A Dover, DE 19901, USA

© 2026 Inquio LLC. All rights reserved

8 The Green, Suite A, Dover, DE 19901, USA