Evaluating AI pipelines before customers do

An AI system without evaluation is a system you cannot change. The same pattern held at Mentally AI and at Qoves: build the evaluation harness first, then move fast on models.

An evaluation set is a product asset#

Collect real inputs, including the ugly ones, and label the outcomes you expect. It should be versioned next to the code, small enough to run on every change, and boring: no flaky captures, no data that leaks customer information. It runs in CI and fails builds when scores drop past tolerance.

def score(run, baseline, tolerance=0.02):
    delta = run.mean_score - baseline.mean_score
    if delta < -tolerance:
        raise Regression(
            f"{run.name}: {delta:+.3f} vs baseline {baseline.name}"
        )
    return delta

Structured prompts, validated outputs#

At Qoves we constrain the output shape, validate it against a schema, reject and retry on violation, and measure the retry rate as a health signal. It's not exciting, but it keeps regressions from reaching customers.

Monitor distribution, not just averages#

Production inputs drift. Log score distributions and input statistics per version, and alert on shift, not only on errors. A drop in average score is a late signal; a change in input length or image size distribution is an early one.

The habit that matters#

Evaluation is not a launch gate, it is a loop: change, run, compare, decide. Teams that keep the loop tight can adopt new models weekly. Teams that evaluate by feel adopt them never.