A Hands-On Course · 12 lessons

Verifying Agent Work

Prove the agent did it right — deterministic guardrails, the verification ledger, golden journeys, outcome grading, and evals.

Short lessons (~5–8 min each), each with one tangible win and a retrieval-practice quiz. Built for engineers who already use AI coding tools and want the non-obvious mechanics.

Grounded in the agentpatterns.ai corpus (CC BY 4.0). Keep the Glossary open as you go.

Part 1 · The Trust Problem

1 The Confident Liar "Build passed. Tests green. No issues found." The agent is fluent, confident, and — often enough to matter — wrong. This course is about never taking its word for it. 2 Guardrails Beat Guidance "Don't break any links" is a prompt — sometimes ignored. A link checker on every URL is a guardrail — it runs every time, and the model gets no vote. 3 The Verification Ledger Prose claims are unfalsifiable. Turn every check into a row — tool, command, exit code — and the evidence becomes a query, not a sentence you have to trust.

Part 2 · Verifying As You Build

4 Check at Each Step An agent writes 500 lines, then verifies. The wrong assumption was at line 10. Everything after it is built on a mistake — and unwinding that cascade is the expensive part. 5 The Pre-Completion Checklist Agents optimize for completion, not correctness. Left alone, one declares "done" after a partial build and a test run it chose not to investigate. So block the completion signal until a fixed sequence has passed. 6 Red-Green for Agents Tests are a more precise spec than prose. Write them failing first, pass them with minimal code, refactor against green — but only if each phase is a separate invocation. 7 Chain-of-Verification When no test or type checker reaches a claim, the model can verify itself — but only one variant works, and used naively it overturns correct code 1-in-4 times.

Part 3 · Evaluating Behavior

8 Testing What It Decides The same task admits many valid paths. Assert on the exact output and you get false negatives on correct work and false positives on lucky runs. Test the decision and the end-state instead. 9 Golden Journeys Unit tests pass. Integration tests pass. The agent left a corrupt cache and a stuck worker behind. One rule catches it: no feature is complete if the system can't restart cleanly afterward. 10 Grade the Outcome Check that the agent called tool X before tool Y, and you penalize every valid solution its author didn't anticipate. Ask instead: is the system in the correct state? 11 Evals at Scale Some failures live in no single run. The agent answers every turn plausibly, yet a pattern across a thousand traces is broken. And the thing you edit most — the skill — usually ships with no eval at all.

Capstone

12 The Verification Decision Table Eleven lessons, one reflex: never accept a self-reported "done" — reach for the check that fits the claim. This is the whole course as a lookup table, and a mixed review to prove it stuck.