Product
Reporting Bot Quality to Management
"How's the bot doing?" deserves an answer that doesn't start with a tutorial on containment rate.

Every conversational AI team has a version of this meeting.
Someone senior asks how the bot is doing. You open a dashboard. Within ninety seconds you are explaining what containment rate measures, why it went up, and why that isn't necessarily good news. By the time you've finished the caveat, the question has moved on.
Next quarter, you explain it again. To some of the same people.
This isn't a presentation problem. It's a structural one, and it has a cost: a function nobody can summarise is a function nobody can fund.
The three answers teams give, and why each one fails
The metrics answer. Containment at 68%, fallback at 4%, CSAT at 3.9. Accurate, and almost entirely unreadable to anyone outside the team. Worse, the numbers move for reasons that need explaining every time — containment went up because the bot got better, or because it got harder to reach a human, and only you know which.
The invented answer. Someone builds a "bot health score" in a spreadsheet, weighted by whatever felt right at the time. It's readable. It's also unfalsifiable, and everyone in the room knows the team that's being measured built the measurement.
The anecdote. "A customer complained that it couldn't handle refunds." Vivid, memorable, and statistically meaningless. One story moves a budget conversation further than a dashboard, which is exactly the problem — you're being judged on which anecdote reached which executive.
All three fail the same test. None of them lets a non-specialist understand the state of the bot without a specialist standing next to them.
What a single number has to do to be worth having
The instinct — give them one number — is right. Most attempts at it fail for a predictable reason: the number arrives with nothing behind it.
A score that can't be decomposed is just a vibe with a decimal point. When someone asks why it moved, "it's a composite" is not an answer, and the next meeting goes back to raw metrics.
A number that works has to survive two questions in sequence.
Why did it move? → the Inquio Score breaks into Resolution Accuracy, Containment Quality, Dialog Fluency and Safety & Compliance, so you can say "accuracy dropped, everything else held."
Show me. → each dimension traces to the actual conversations behind it.
That structure is what lets one artifact serve two audiences. The executive reads the top number and stops. The conversation designer reads the same report and goes three levels down. Nobody is translating for anybody.
One card is a snapshot. A series is a story.
A single quality score answers "how are we doing." It cannot answer "are we getting better," which is the question that actually determines whether your work gets resourced.
For that you need the same measurement, taken the same way, repeatedly. Not a one-off audit. Not a different consultant with a different methodology each year. The same score produced by the same process, on a regular cadence, so the comparison between two points is valid.
That sounds obvious and it's where most quality reporting falls apart. If you change how you measure between March and June, the difference between March and June tells you nothing. The comparability is the value — more than any individual number.
It's why Inquio produces a Bot Report Card on a repeating schedule, each generated identically, rather than a single assessment. A series gives you something no individual report can: the shape. You can see that the score dipped the week of a release and recovered. You can see that a dimension has been drifting down for two months while the headline stayed flat. You can point at a specific card and say "this is where it broke."
And when someone asks whether the work was worth it, you have a before and an after that were produced identically.
Three honest caveats
A single number can be gamed like any other. The difference is that four dimensions are harder to game simultaneously than one, and the conversations behind them are harder still. But if the score becomes a target in isolation, it will go the way of containment rate. Report it with its components, always.
Composites hide compensating movements. Accuracy can fall while fluency rises and the headline barely twitches. This is a real limitation, and the answer is the same as for any metric: the score tells you when to look, not what you'll find. If you expect it to render a verdict, you'll eventually stop trusting it.
Cadence matters more than people expect. A report card over a single low-volume day is mostly noise — scores will swing wildly between adjacent days and you'll read meaning into randomness. For reporting upward, weekly or monthly windows give you something stable enough to draw a line through. Keep the short windows for debugging a specific release.
What this actually changes
Not the quality of your bot. The score doesn't fix anything.
What it changes is whether anyone outside your team can see the quality of your bot without you in the room — and whether, six months from now, you can prove the work you did made a difference.
Both of those are the difference between a programme that gets invested in and one that quietly gets questioned.





