Eval-driven development
The defining methodology of serious AI engineering: build the eval first, then iterate against it. Because LLM systems are non-deterministic and "looks good" doesn't scale, an eval set is the only honest signal that a change helped. Evals are to AI what unit tests are to software.
Why "try it and see" fails
A single good output says nothing about the distribution of inputs. Tuning a prompt, swapping a model, or changing retrieval by eyeballing one example means you fix one case and silently break others. Without a measurement, "better" is a feeling.
The loop
- Collect real examples — from logs, users, and known failure cases.
- Define success — exact checks where possible, a rubric + LLM-as-judge where not, human review for the rest.
- Establish a baseline — score the current system.
- Change one thing, run the whole eval set, compare to baseline.
- Error-analyze the failures (systematically), which feeds the next change and new eval cases.
- Gate releases on the eval in CI — no regressions ship.
It compounds
Every production failure becomes a new eval case, so the suite keeps getting stricter as the system meets reality — the flywheel applied to quality. The eval set becomes your most valuable, hardest-to-copy asset.
Where it plugs into the stack
Eval-driven development is the practice; the targeted notes are the techniques: building eval sets, LLM-as-judge, regression testing, RAG eval, agent eval.
Pitfall
Building the product first and bolting on evals "later" — by then you have no baseline, no regression safety net, and changes are pure guesswork. Start the eval set on day one, even tiny; a 20-example eval beats zero by an enormous margin.
Connects to: designing eval sets · iterating prompts · the data flywheel