AI CLI
AI Output Evaluation — know if the model is actually right
Build a golden set, score with exact-match + rubrics + LLM-as-judge, track accuracy/precision/recall alongside cost and latency, and gate every prompt or model change on a regression run so silent quality drops get caught before users do.
Quick reference — reach for these first
labeled.jsonl # {input, expected} pairsFreeze a representative, labeled set BEFORE tuning. This is your source of truth.assert out.strip() == expectedCheapest, deterministic check for structured/classification outputs. Run it first.score = sum(criteria_met)/len(criteria)Deterministic 0-1 per-criterion scoring for open-ended text (has citation? on-topic? correct format?).judge(prompt, output) -> {score, reason}Grade subjective quality with a model — but validate the judge against human labels first.eval baseline vs candidate; diff metricsRe-run the full suite on every prompt/model change. Block the merge on any drop.The evaluation loop: labeled set in, a gated ship/block decision out
Text equivalent: A left-to-right pipeline. A frozen golden set of labeled examples feeds a run of the prompt and model. Outputs are scored three ways — exact-match, rubric, and a calibrated LLM-as-judge. Scores roll up into metrics: accuracy, precision, recall, pass-rate, plus cost and latency. A regression gate compares candidate against baseline and either ships the change or blocks it, and blocked or escaped failures are fed back into the golden set.
Common tasks
Build the set · Score it · Gate changes
| Build the set | Score it | Gate changes |
|---|---|---|
| Define success criteria and the metric that maps to it | # write down: task, target metric (accuracy? pass-rate?), threshold | One sentence states what 'good' means and the single number you will gate on. |
| Collect 50-200 real, representative examples | sample from prod logs + known edge/failure cases | Class balance and input distribution roughly match production traffic. |
| Label expected outputs (human or trusted source) | two labelers; reconcile disagreements | Inter-labeler agreement is high; disagreements are documented, not guessed. |
| Pin determinism for reproducible runs | temperature=0, fixed seed, pinned model id | Same input twice gives the same output and the same score. |
| Implement exact-match / structural scoring | normalize() then ==; validate JSON schema | Scorer passes on a hand-checked correct output and fails on a wrong one. |
| Write a rubric for open-ended outputs | list binary criteria -> fraction met = score | Two people scoring the same output with the rubric get the same number. |
| Build an LLM-as-judge for subjective quality | judge with a rubric prompt; return score + reason | Judge outputs parse cleanly and include a justification for each score. |
| Calibrate the judge against human labels | judge the labeled set; measure agreement vs humans | Judge/human agreement is high (e.g. >0.8) BEFORE you trust it unsupervised. |
| Compute aggregate + per-class metrics | accuracy, precision, recall, pass-rate per class | You see per-class numbers, not just one average that hides a weak class. |
| Record cost and latency per run | sum input+output tokens * price; p50/p95 latency | Quality, $/1k calls, and p95 latency all appear in one results table. |
| Snapshot a baseline result | eval run --save baseline.json | baseline.json holds the current prompt/model scores for later diffing. |
| Regression-test every change against baseline | eval run candidate; diff vs baseline # gate in CI | Merge is blocked automatically if any gated metric regresses. |
| Feed real prod failures back into the set | add escaped bugs as new labeled cases | Each incident becomes a permanent regression case so it cannot silently return. |
Evaluate a 3-class sentiment classifier prompt and gate a prompt change
classification eval
# labeled.jsonl
{"text":"Shipping was fast and the case feels premium.","expected":"positive"}
{"text":"It works but the app is clunky and slow.","expected":"neutral"}
{"text":"Battery died in two days. Returning it.","expected":"negative"}
{"text":"Exactly what I needed, five stars.","expected":"positive"}
{"text":"Arrived on time. Nothing special.","expected":"neutral"}
{"text":"Terrible support, never replied.","expected":"negative"}
{"text":"Love it, would buy again.","expected":"positive"}
{"text":"Does the job, a bit overpriced.","expected":"neutral"}
# 8 labeled cases, balanced across the 3 classesSYSTEM = (
"Classify the sentiment of the user text as exactly one word: "
"positive, neutral, or negative. Output only that word, lowercase."
)
# call with temperature=0, pinned model id (e.g. a current Claude Sonnet),
# so runs are reproducible
# model returns a single token like: positiveimport json, collections
rows=[json.loads(l) for l in open("labeled.jsonl")]
cls=["positive","neutral","negative"]
tp=collections.Counter(); fp=collections.Counter(); fn=collections.Counter()
correct=0
for r in rows:
pred=classify(r["text"]).strip().lower() # your model call
exp=r["expected"]
if pred==exp: correct+=1; tp[exp]+=1
else: fp[pred]+=1; fn[exp]+=1
acc=correct/len(rows)
print(f"accuracy={acc:.2f} ({correct}/{len(rows)})")
for c in cls:
p=tp[c]/(tp[c]+fp[c]) if tp[c]+fp[c] else 0
rec=tp[c]/(tp[c]+fn[c]) if tp[c]+fn[c] else 0
print(f"{c:8} precision={p:.2f} recall={rec:.2f}")
# deterministic scorer; no LLM-judge needed for closed-set labelspython eval.py
# accuracy=0.88 (7/8)
# positive precision=1.00 recall=1.00
# neutral precision=0.67 recall=0.67
# negative precision=1.00 recall=1.00
# tokens/call~45 p95 latency~0.6s cost~flatpython eval.py --prompt candidate > cand.txt
diff baseline.txt cand.txt
# candidate: accuracy=0.88->0.94, neutral recall 0.67->0.83, cost/latency unchanged
# DECISION: ship — pass-rate up, no class regressed, cost flat.
# If any class recall had dropped or cost spiked: BLOCK the change.Troubleshooting — symptom → first check → next check
| Symptom | First check | Establishes | Next check |
|---|---|---|---|
| Eval set is tiny or unrepresentative (passes everything) | Compare set's class balance and input length distribution to a week of prod logs. | Whether the set exercises the cases users actually send, not just easy ones. | Grow to 50-200 examples, oversample known edge/failure cases, rebalance classes. |
| LLM-judge scores disagree with human reviewers | Run the judge over your human-labeled set and compute agreement rate. | Whether the judge is trustworthy or just confidently wrong. | Tighten the rubric, add few-shot graded examples, re-calibrate until agreement is high; never ship an unvalidated judge. |
| Metric looks great but users complain | Trace 5 complaints to their eval cases — are they even in the set, and does the metric reward them? | Whether you are measuring the wrong thing (e.g. accuracy when task-success is what matters). | Switch to a metric tied to the real task (task-success, recall on the costly class) and add the complaint cases. |
| Eval scores flap between identical runs | Check temperature, seed, and whether the model id is pinned (not an auto-updating alias). | Whether nondeterminism is in sampling vs the scorer itself. | Set temperature=0, pin seed and exact model id; make the scorer deterministic; median over N runs if still noisy. |
| Eval passes but production fails (distribution shift) | Diff recent prod inputs against the eval set — new topics, formats, or languages? | That prod has drifted away from what the frozen set covers. | Continuously sample fresh prod traffic into the set and re-label; add every escaped failure as a case. |
| A change ships, aggregate score flat, one class collapses | Open the per-class precision/recall table, not the single average. | Whether an average is masking a regression in a minority class. | Gate on per-class floors too, not just the overall number; block the change until the class recovers. |
| Quality improved but cost/latency exploded | Compare tokens-per-call and p95 latency in the candidate vs baseline row. | Whether the gain is worth the spend — a bigger model or longer prompt may not pay off. | Report quality AND cost AND latency together; decide against the budget, or try a smaller model / shorter prompt. |
References
- Anthropic — Create strong empirical evaluationsdocs.anthropic.com
- Anthropic — Define your success criteriadocs.anthropic.com
- Anthropic — Using the Evaluation tool (Console)docs.anthropic.com
- OpenAI — Evaluating model performance (Evals)platform.openai.com
- Hugging Face — Evaluate: metrics for MLhuggingface.co
Scoring examples use Python with a closed-label set, so exact-match suffices and no judge is needed; model calls run at temperature=0 with a pinned current Claude Sonnet/Opus 4.x id for reproducibility. Model ids and pricing are referenced generically — confirm the exact id and token prices in the Anthropic docs before wiring cost math. The LLM-as-judge pattern is only for open-ended/subjective outputs and must be calibrated against human labels before it gates anything.