Improvement & Flywheel Loops

The inner loop converges one run. The outer loop makes the next run start smarter — if you wire the human's verdicts back into the rules.

Why this, for you: you ship a loop, babysit it for a week, and it never gets easier — you are approving the same kind of bad step on Friday that you rejected on Monday. The fix is not a better prompt mid-run. It is treating every checkpoint decision as signal and closing an outer loop that improves the system between runs, not just inside one.

Everything so far in this course tunes a single run: name the loop, pick its context, give it a body, detect convergence, gate its cost, cap its blast radius. This lesson zooms out one level. There are two loops, not one — and the slow one is where compounding lives.

1 Two loops: the run, and the system around it

The inner loop is the agent grinding toward an objective in a single run. The outer loop is you-and-the-system across runs: the rules, rubrics, verifiers, and prompts that every future run inherits. A flywheel loop closes the outer one — outcomes and human checkpoints from this run become the constraints that shape the next.

A human checkpoint inside an agent loop is not a safety gate around the loop — it is a control primitive whose verdict substitutes back into the loop and changes what the next iteration does. Read it as exhaust, and you throw away the highest-quality training signal the system will ever get.

The corpus codifies four decision verbs a checkpoint exposes — and the two middle ones are pure improvement signal:

Verb	Effect on this run	Signal for the outer loop
`approve`	Continue unchanged	The rules were right — low signal
`edit`	Replace the arguments, continue	Right step, wrong specifics — a rule is too loose
`reject`	Skip; model re-plans on the rejection	Wrong branch — a guardrail is missing
`respond`	Synthesize a tool result from the reply	The human is the tool — automate it later

A recurring edit at the same checkpoint says your rule under-specifies the action. A recurring reject says a deterministic guardrail is missing. That is the flywheel input: promote the human verb into a rule, and next run the loop handles the case itself.

2 Feed outcomes back into the rules, not the run

The temptation is to fix things in-flight — patch the prompt this turn, nudge the agent this run. That helps one trajectory and evaporates. The flywheel pushes the lesson down a level: into the completion audit, the rubric, or a separate-context grader that every future run passes through.

Anthropic's Managed Agents outcomes attaches a rubric and runs the grader in a separate context window "to avoid being influenced by the main agent's implementation choices" — and reports task success improved by up to 10 points over a standard prompting loop. The grader is a rule the outer loop wrote down.

The mechanism that makes a goal-driven loop converge is the completion audit — the injected demand to map every requirement to concrete evidence before declaring done, and to reject proxy signals ("tests pass", "I remember doing this") as proof. Each human reject at the finish line is telling you which requirement the audit failed to enforce. Add it.

The worker must not grade its own homework

When the agent that did the work is also the auditor — the same-session design — long transcripts produce false-positive completion, the documented LLM-as-judge self-enhancement bias. A separate-context grader is the structural defense. A flywheel built on self-audit just compounds the bias faster.

3 Place checkpoints where they earn their cost

A flywheel that wakes a human on every iteration does not improve — it saturates the reviewer at machine speed until they approve by reflex. The corpus is blunt: per-iteration checkpoints are the failure mode. Three placements pay back the per-call cost.

On low confidence — trip only when an in-loop score (classifier, planning uncertainty, verifier divergence) falls below threshold. Fails when "confidence" is vibes, because then it fires every turn.
On budget threshold — trip when the loop has burned a configured fraction of its token/dollar/tool-call budget without a stop signal; ask the human to extend, narrow scope, or abort. This is the dual of the goal-driven loop's budget_limit wind-down — same trigger, wakes a person instead of the agent.
On irreversible action — trip before anything the loop cannot undo from its own state: merge, deploy, publish, paid API call.

Pair every interrupt() with a timeout default. interrupt() waits forever by design — a checkpoint that fires at 2am holds state and produces nothing until someone resumes it. An unbounded wait is a throughput loss disguised as a safety win.

And reach for HITL only as the residual. If a test, lint, schema check, or convergence detector can grade the step mechanically, a human checkpoint is theatre — the verifier is already the stop condition, and the human only adds latency. The empirical baseline keeps you honest: 45.1% of merged Claude Code PRs still required human revision, and at that rate per-action gating would saturate any reviewer before it ever saved time.

# Budget-threshold checkpoint — the wake-the-human dual of budget_limit def budget_check(state): if state.tokens_used > 0.8 * state.budget and not state.warned: decision = interrupt({"reason": "80% spent, no convergence"}) # reject/edit here is not just control — log it as outer-loop signal record_checkpoint_verdict(decision) # feeds the next run's rules

↪ Your win: close the slow loop

Name both loops — the inner run converges; the outer system improves between runs.
Treat checkpoint verdicts as signal — a recurring edit means a loose rule; a recurring reject means a missing guardrail.
Push lessons into rules, not runs — completion audits, rubrics, and separate-context graders persist; in-flight patches evaporate.
Never let the worker grade itself — a separate-context grader defeats self-enhancement bias; outcomes lifted task success up to 10 points.
Place checkpoints sparsely — low confidence, budget threshold, irreversible action — and pair every interrupt with a timeout default.

Retrieval practice — recall, don't peek

Question 1A human checkpoint inside an agent loop is best understood as…

Question 2A recurring reject at the same checkpoint most directly tells you that…

Question 3Anthropic's outcomes runs its grader in a separate context window mainly to…

Question 4Per-iteration human checkpoints in a high-throughput loop fail because they…

Question 5 · spaced recall from an earlier lessonFrom the convergence lesson: a loop should stop when…

Ask me anything. Want help turning a week of checkpoint rejections into a concrete rubric line, or deciding which interventions deserve a separate-context grader versus a deterministic check? Next, the capstone: The Loop Engineering Decision Table — read the pathology you see and reach for the lever that fixes it.