The Feedback-to-Rule Loop

The same review comment, written for the third time, is not a comment — it's an unencoded rule asking to be promoted into the harness.

Why this, for you: you keep typing the same nit — "this catch block swallows the error," "don't import the UI layer from services" — and every PR re-pays the cost of you remembering it. The fix isn't typing faster. It's draining the comment into a mechanical check so it never reaches review again.

A recurring review comment is evidence of an invariant that lives in one reviewer's head. This lesson turns that head-knowledge into a deterministic check — and, just as importantly, teaches you to retire the check before your rule library rots.

1 Three strikes is the trigger

One comment is noise. Two is coincidence. The promotion threshold is load-bearing: the same comment across three or more PRs in a window is the signal that an unencoded invariant is costing you on every review.

One or two occurrences is a hypothesis; three or more is a pattern. Encode after one or two and you freeze a guess into a rule — suppression comments proliferate and the rule's signal degrades.

So the loop is simple: a comment recurs → categorize it → encode the smallest enforceable check → ship it with remediation text → track its hit count → retire it when the count trends to zero.

2 Categorize before you encode

Match the rule's placement to the comment's category. Promoting a semantic check into a regex linter is a category error: it fires on every legitimate exception and erodes trust in the whole lint stack — the rule becomes the new recurring noise source.

Comment category	Encoding layer
Style or formatting	Linter rule (ESLint, Ruff, …)
Architectural boundary	AST / import check, dependency-graph rule
Safety or correctness invariant	Pre-completion checklist entry, type or runtime check
Spec or contract violation	Evaluator rubric line, integration test

Then pick the cheapest mechanism that fires deterministically. A one-line ESLint rule beats a multi-file AST plugin when both would work — over-engineering adds maintenance cost the retirement step can't recover.

Why mechanical beats a sharp eye

Anthropic draws the line explicitly: "Unlike CLAUDE.md instructions which are advisory, hooks are deterministic and guarantee the action happens." A reviewer's eye is probabilistic; a mechanical check fires every time. Promotion converts an advisory comment into an enforced invariant.

3 Remediation text is non-optional

A rule that says no fs in renderer without saying what to do instead doesn't remove the bottleneck — it relocates it from review to comprehension. The shape that works states what's wrong, what to do instead, and where the rationale lives:

ERROR: Service layer cannot import from UI layer. Move shared logic to a Provider in src/providers/, or keep UI-specific code in src/ui/. See docs/architecture/layer-rules.md for the dependency diagram.

A semantic invariant — "this handler swallows the database error" — lands as a checklist line, not an AST check, because whether the catch block "swallows" depends on what it does with the error:

# .claude/checklists/pre-merge.json "id": "ERR01", "severity": "HIGH", "check": "Every catch in src/handlers/ re-throws, logs at error level, or wraps with context. Empty catches fail.", "remediation": "Re-throw, wrap via new HandlerError(msg, {cause: err}), or ctx.logger.error before a 5xx."

4 Every rule has a shelf life — retire it

Refactors obviate boundaries, model upgrades eliminate failure modes, conventions solidify until no one would write the violation. Annotate each rule with its obsolescence condition — the observable signal it has done its job — and delete it when the hit count has stayed at zero.

Without retirement, the lint stack accumulates dead weight and priority saturation kicks in: when every rule carries equal weight, nothing signals priority and adherence degrades across the whole set — context rot means later rules get recalled less accurately as the context fills.

This is not the same as Bugbot's learned rules

The learned-rules pattern (Cursor Bugbot) tunes the reviewer's defaults from accept/reject signals. This loop promotes the invariant out of the reviewer entirely — into the lint stack, checklist, or rubric. They compose: Bugbot sharpens what review catches; this loop drains high-frequency comments before they ever reach review.

↪ Your win: the same nit, never written twice

Three strikes is the trigger — same comment across 3+ PRs; fewer is a hypothesis, not a pattern.
Categorize before encoding — style→linter, boundary→AST check, safety→checklist, spec→rubric.
Pick the cheapest deterministic mechanism — a one-line rule beats a plugin the retirement step can't recover.
Ship remediation text — what's wrong, what to do instead, where the rationale lives.
Retire on zero hits — delete decayed rules before priority saturation degrades the rest.

Retrieval practice — recall, don't peek

Question 1You should promote a recurring review comment into a harness rule once it has fired across…

Question 2Forcing a semantic, correctness-style comment into a regex linter mainly causes…

Question 3A rule shipped without remediation text mostly…

Question 4Letting promoted rules accumulate without retirement leads to…

Question 5 · spaced recall from an earlier lessonTo make an AI-authored PR reviewable, the author should…

Ask me anything. Want help picking the right encoding layer for a comment you keep writing, or a retirement annotation for a rule that's gone quiet? Next, the capstone: The Review Decision Table — match the review pathology in front of you to the playbook move that resolves it.