Signal Over Volume

More comments don't mean a better review. The win is an AI reviewer that stays silent unless it has something high-confidence to say — and a hard rule for the one place silence is a trap.

Why this, for you: you've already learned to tune out the bot that flags every any and every missing comma. That habit is the real cost — when you discount all of its output, the one critical finding gets discounted too. This lesson is about earning attention back: fewer comments, and a deliberate carve-out for security, where a quiet AND a clean review look identical.

An AI reviewer that always produces output, whatever its value, trains you to ignore it. The fix is to treat silence as a valid review outcome — and then to notice the one class of bug where silence lies to you.

1 Silence is a valid review outcome

The signal-over-volume principle is simple: when the agent has nothing high-confidence to add, it says nothing. GitHub measured this across 60M+ Copilot code reviews — in 71% of reviews Copilot surfaces actionable feedback; in the remaining 29% the agent says nothing at all. GitHub explicitly rejected maximizing comment frequency, stating "more comments don't necessarily mean a better review."

Reviewers have a fixed budget of attention per PR. A tool that emits many low-value comments gets all of its output discounted — including the high-value findings. That discounting is a learned response to repeated false positives, not a deliberate choice.

So the lever is a confidence floor. Each potential finding must clear a minimum bar (the reference prompt uses ≥90%) before it surfaces; everything below is suppressed, not queued. Severity routing reinforces it — CRITICAL and HIGH become PR comments; MEDIUM and LOW are omitted unless you ask for a full review.

2 Shape the comments that do survive

Suppression is half the design. The findings that clear the bar still need to be cheap to read:

Tactic	What it replaces
Multi-line contextual comments	Single-line pointers you must reconstruct context around
Clustered feedback	One comment per instance of the same pattern error
Batch autofixes	Applying each fix to a class of issues one at a time

Validate the result with two loops, not comment count: reactions (a declining thumbs-up ratio shows signal degradation) and resolution tracking (findings you consistently dismiss are false positives to suppress). GitHub's architecture redesign earned an 8.1% lift in positive feedback by improving signal; a later move to a stronger reasoning model added a further 6% despite review latency rising 16% — fewer, better comments beat faster, noisier ones.

3 The one place silence is a trap

A strict confidence floor has a dangerous edge: a clean review and a missed bug look identical. Nowhere is that worse than security. Rabbi et al. (EASE 2026) filtered 33,000+ agent-authored PRs down to 675 security-related ones, and found AI weaknesses are not diverse — six CWEs account for ~80% of all findings:

CWE	Weakness	Share
CWE-1333	ReDoS (catastrophic regex backtracking)	36.2%
CWE-78	OS command injection	13.0%
CWE-22	Path traversal	10.3%
CWE-134	Format-string	8.2%
CWE-79	Cross-site scripting	7.1%
CWE-798	Hard-coded credentials	5.7%

52.4% of these security PRs merged — and only 1.8% of rejections cited distrust in AI code. Reviewers reject on inactivity, test gaps, and style, not on SAST-detectable CWEs. Serious flaws pass review while minor issues cause rejection.

Worse, the human-PR heuristics you lean on stop working. Commit-message quality — a strong predictor for human PRs — has no predictive value for agent PRs (high-quality messages: 45.6% acceptance; low-quality: 58.0%). A CI-passing patch with a careful message still ships a polynomial regex. The proxy "a careful author writes careful code" is exactly what an agent breaks.

# A human-PR heuristic approves this — passing tests, clean CI, detailed message. EMAIL_RE = re.compile(r"^([a-zA-Z0-9_\.\-])+@(([a-zA-Z0-9\-])+\.)+([a-zA-Z0-9]{2,4})+$") # Nested quantifiers overlapping on '-' → exponential backtracking. Textbook CWE-1333.

Signal-over-volume needs a security override

Don't let the confidence floor silence the six CWEs. Run SAST aimed at them — ReDoS analyzers, shell-command taint tracking, path-traversal linters, format-string and XSS checkers, secret scanners — as blocking CI gates on agent PRs, and require an adversarial pass for any diff touching re.compile, exec/subprocess, path construction, format strings, HTML output, or credentials. A No high-confidence findings. response must never be read as a security all-clear.

↪ Your win: a reviewer worth reading, with a security floor under it

Treat silence as output — 29% of Copilot reviews intentionally say nothing; a confidence floor (≥90%) suppresses low-signal findings rather than queuing them.
Route by severity — CRITICAL/HIGH surface as comments; MEDIUM/LOW only on request.
Shape survivors — multi-line context, cluster repeated patterns, batch autofixes.
Measure signal, not volume — reactions and resolution rate, not comment count.
Override for security — six CWEs cover ~80% of agent flaws; gate them with SAST and adversarial review, because a clean review and a missed CVE look the same.

Retrieval practice — recall, don't peek

Question 1In GitHub's 60M-review data, the signal-over-volume design means that in 29% of reviews Copilot…

Question 2The mechanism that makes noisy AI review backfire is that reviewers…

Question 3You should validate an AI reviewer's signal quality primarily through…

Question 4The security-review gap study found that across agent-authored PRs, security flaws…

Question 5 · spaced recall from an earlier lessonFrom Lesson 1: as agents author more PRs, the bottleneck shifts to…

Ask me anything. Want the full signal-over-volume review prompt with the severity routing wired in, or a SAST gate config aimed at the six CWEs? Next in Part 1: Anatomy of Agentic Review — why an agentic review is a pipeline over the diff, not a single "find bugs" prompt.