Right Token — Lean Context, Lean Output

The cheapest token is the one you never generate. Cut waste at three sources — tool output, generated code, and the codebase the agent reads.

Why this, for you: you picked the right model last lesson, but it's drowning. One tool returns a 10,000-token API dump, the agent writes a verbose loop where a comprehension would do, and it re-reads the same messy file four times. None of that is the model's fault — it's tokens leaking at the source, and you can plug each leak.

Right Model cuts the price per token. Right Token cuts the count. They multiply: a cheaper model that also handles half the tokens is a 4× win. The leverage lives in three places — what tools return, what the agent writes, and what it has to read.

1 Tool output is a context injection

Every tool call's output enters the context window. A tool that returns a 10,000-token API response when 200 tokens would do consumes 10% of a 100k window on a single call. The fix is one question: what does the agent need to know to decide its next action? Return that, and nothing else.

A CI-status tool should return "3 checks passed, 1 failed: lint" — about 20 tokens — not the full Actions run object with 40+ fields at ~400 tokens. Same decision, 20× fewer tokens, and no parsing for the agent.

The mechanism that punishes bloat is attention dilution. Self-attention scores every token against every other, so noise competes with signal for the model's focus. Liu et al. (2023) found accuracy drops sharply when the relevant document sits in the middle of a long context — the "lost in the middle" effect. Oversized output doesn't just cost money; it buries the field that matters.

Two more levers compound the savings:

Eliminate functional overlap — two tools that both "search" force the agent to reason about which to use before acting. That reasoning burns tokens and invites error. Merge them or scope them apart.
Cap the toolset — the agent reviews every available tool before each call, so more tools means more selection tokens per decision. A multi-server MCP setup can spend ~55,000 tokens on tool definitions alone (Anthropic) before any work.

Over-filtering backfires

A summary that drops "unimportant" fields will eventually drop the one a rare-but-valid path needs — and the agent cannot ask for data it doesn't know exists. A useful heuristic: tool output should fit in a paragraph. If it can't, filter, structure it carefully, or write it to a file — but validate that you haven't silently amputated an edge case.

2 Lean output: structural beats "be concise"

The obvious move — telling the agent "write concise code" — is the wrong one. It creates a competing objective: the agent does less work, not better work. Cursor reported that a token-preservation instruction made GPT-5-Codex refuse a task: "I'm not supposed to waste tokens, and I don't think it's worth continuing." Prompt-level conciseness is fragile.

Cut tokens structurally, not by directive. ShortCoder (Liu et al., 2026) shows AST-preserving syntax transforms achieve 18–38% token reduction on HumanEval with no loss of correctness — same behavior, fewer tokens.

These are ordinary Python idioms (PEP 8) applied systematically: a comprehension instead of a loop-plus-append, an f-string instead of concatenation, dict.get(k, default) instead of an if key in dict check. And the way to get them is to show, not tell — agents pattern-match from examples far more reliably than they follow abstract directives.

# Verbose: 38 tokens def get_valid_users(users): results = [] for user in users: if user.is_active: results.append(user.name) return results # Idiomatic: 24 tokens (37% reduction) — identical behavior def get_valid_users(users): return [user.name for user in users if user.is_active]

The win compounds: every time generated code re-enters the context window on a later turn, the shorter version costs less again. The saving repeats for the life of the session.

Don't over-compress

ShortCoder found that reductions past ~30% correlated with an 18.7% drop in unit test pass rates in DeepSeek Code experiments — aggressive compression collapses multi-step logic into single expressions that fail edge cases. Frontier models already write idiomatically, so measure the real saving before bolting on transforms, and always validate against a test suite.

3 Lean input: clean code is cheaper to read

The third source is the codebase itself. In a controlled minimal-pair study, Trivedi & Schmitt (SonarSource, 2026) built six pairs of Java repos — identical architecture and behavior, differing only on rule violations and cognitive complexity — and ran Claude Code across 660 trials, cleaning messy code and degrading clean code in both directions.

Metric	Clean vs messy
Task pass rate	Statistically indistinguishable
Tokens per task	7–8% lower on cleaner code
File revisitations	34% lower on cleaner code

Cleanliness is a cost lever, not a capability fix. Agents don't fail more on messy code — they just spend more finding the right file. Lower complexity lets the agent locate logic from names on the first try; the 34% revisit drop is the mechanism, and the 7–8% token saving follows from it.

Don't refactor for the agent

7–8% on a $1,200/month spend is ~$1,080/year — an 80-hour refactoring sprint pays back in ~11 years. Worse, agents re-introduce mess: a longitudinal study of 806 Cursor-adopting repos (He et al., MSR 2026) found warnings rise ~30% and complexity ~42% after adoption. Capture the saving from maintainability work you'd do anyway — linters, complexity budgets, CI quality gates — not a sprint aimed at the agent.

↪ Your win: plug leaks at the source

Tools return the next decision's inputs — size for the decision, not for completeness; output fits a paragraph.
Scope tools apart and cap the set — overlap and bloat both tax selection tokens per call.
Cut generated tokens structurally — idiomatic transforms give 18–38% free; never prompt "be concise."
Show, don't tell — code examples in the prompt beat abstract directives, and the saving compounds across turns.
Treat clean code as a side-effect cost lever — 7–8% from quality gates you'd run anyway, not a refactor sprint.

Retrieval practice — recall, don't peek

Question 1The right rule for sizing a tool's output is to return…

Question 2Adding "write concise code" to the agent prompt tends to…

Question 3In the minimal-pair study, cleaner code left pass rate unchanged but cut…

Question 4Oversized tool output hurts quality, not just cost, because of…

Question 5 · spaced recall from an earlier lessonCost-aware routing says default to…

Ask me anything. Want a before/after on one of your real tool outputs, or a "show-don't-tell" idiom block for your AGENTS.md? Next in Part 2: Right Cache — Caching Discipline — order context stable-first so the cache prefix hits, and why a tokenizer swap silently voids every cached token.