Most production dashboards were built for a world where “working” meant “responding.” A conversational system powered by a large language model breaks that assumption: it can respond instantly, at full uptime, and still be wrong.
The failure modes that matter — a fabricated fact, a tone mismatch, an answer that ignores what the user actually asked — don’t show up as errors. They show up as a slow erosion of trust, usually noticed by a user before it’s noticed by a dashboard.
A useful quality signal has three properties. It’s dimensional: a single pass/fail score collapses too much information, so correctness, groundedness, and task completion need to be tracked separately. It’s continuous: quality needs to be sampled from live traffic, not just a fixed test set that goes stale the moment the model or the product changes. And it’s actionable: a score with no attached example is a number nobody trusts — every metric should trace back to the conversation that produced it.
None of this requires exotic tooling. It requires treating evaluation as a first-class part of the system, built alongside the feature rather than bolted on after a user complains.