Evaluate the outcome, not the eloquence.

Conversation quality is not operational quality

A model may explain the correct process and still call the wrong tool. It may select the right tool with invalid arguments. It may complete a task but violate an approval policy, duplicate an action during retry, or report success before the underlying system confirms it.

That is why a single “good answer” score is too weak. The evaluation must observe the plan, tool calls, state changes, final outcome, and evidence returned to the user.

An operational scorecard

• CompletionDid the requested business outcome actually happen?
• CorrectnessWere decisions consistent with domain data and constraints?
• Tool useWere the right tools called with valid, authorized arguments and safe retry behavior?
• PolicyWere approval gates, permissions, limits, and prohibited actions respected?
• EvidenceCan a person audit what the system observed, decided, changed, and verified?
• RecoveryDoes a partial failure lead to a known, recoverable state?
• EfficiencyAre latency and total cost appropriate for the value and urgency of the task?

Build the evaluation set from real work

Start with representative objectives, not generic prompts. Include the common path, missing data, ambiguous intent, contradictory records, unavailable tools, permission boundaries, timeouts, duplicated requests, and adversarial input.

The best evaluation case describes the initial business state, the permitted actions, the expected evidence, and the acceptable end state.

Some criteria can be checked deterministically: record creation, totals, identifiers, permissions, and tool arguments. Judgment-based criteria can use carefully specified rubrics and human review. The two approaches should complement each other.

Production traces should improve the test suite

Offline evaluation finds known problems. Production reveals unknown ones. Traces should capture the objective, relevant context, decisions, tool calls, results, latency, cost, approvals, and final verification while protecting sensitive data.

When a user corrects the system, an operation fails, or a new edge case appears, that trace can become a regression case. This turns evaluation into an engineering loop rather than a launch checklist.

Where should a team start?

1. 01

   Choose one workflow

   Prefer a valuable task with measurable end state and clear tool boundaries.

2. 02

   Define failure

   List unsafe, incomplete, expensive, slow, and misleading outcomes before implementation.

3. 03

   Instrument the path

   Preserve enough structured evidence to reproduce and inspect every decision.

4. 04

   Gate releases

   Run the evaluation set on model, prompt, tool, policy, and data-contract changes.