Token Engineering · ~8 min
A finance dashboard reports every cut as "lower spend." The Pareto frontier is the one frame that tells you which cuts were free and which quietly sold off your quality.
The whole premise of token engineering is "fewer, cheaper tokens without losing quality." Everything so far has been the fewer-and-cheaper half. Measurement is what enforces the without — and it has two layers: a loop that attributes spend on a live workflow, and a frame that compares whole configurations on cost and quality at once.
Recurring agentic workflows — per-PR review, CI triage, daily reports — accumulate three costs that are invisible inside any single run and only emerge against aggregated history. Treat the workflow as a measurable cost surface and run an instrument → attribute → fix → verify loop over it.
| Structural cost | What it looks like | Lever |
|---|---|---|
| Tool-definition payload re-sent every turn | 5 MCP servers × 30 tools ≈ 30–60K tokens of metadata per turn — 25–30% of a 200K context | Prune the manifest; load tools lazily |
| Deterministic data-gathering inside the LLM loop | Each gh issue view or diff read is a decide-call-receive round-trip | Move to a pre-agentic CLI step writing a workspace artifact |
| Frequency-multiplied small inefficiencies | 5% waste on 100 runs/day is 5 runs/day of pure overhead | Cost-weight every metric by runs/day first |
Instrument by capturing every API call into one normalized log — GitHub routes all traffic through a proxy that writes a
token-usage.jsonl per run (input, output, cache-read, cache-write, model, timestamp); where teams already run OpenTelemetry,
the same data lands on gen_ai.usage.* spans. Attribute with a single scalar, because raw token counts mislead —
output tokens cost roughly 4× input and tiers differ in price:
ET/run × runs/day, not ET/run. In every published case the headline
cut won because the workflow ran many times a day — frequency is the multiplier, not per-run cost. A 62% reduction on a 6.8-runs/day
job dwarfs a 19% reduction on a once-a-day job.A fix that lowers input tokens but raises output tokens has saved nothing. In a pre-registered trial, aggressive prompt compression cut input tokens 62% yet raised total cost 1.8% because the model answered at greater length; lighter compression raised cost 14.1% from output expansion alone. Re-run the workflow set after every change and confirm ET trends down — without an output-side metric the regression is invisible.
Profiling tells you where a workflow leaks tokens. It does not tell you whether plugging the leak cost you quality. For that you need to compare whole configurations — and a configuration is the joint product of every knob that moves spend or quality on the same task: model and tier, reasoning effort, context discipline, cache regime, temporal (sync vs. batch) tier, and scaffold. Change one knob, get a new point on the plot.
Plot each configuration's mean cost against its mean quality, then keep only the non-dominated set — the Pareto frontier. Now the two changes the bill conflates pull apart:
Two axes, two rules. The cost axis is USD per successful task — divide spend by validated outputs, not attempts:
if 100 requests need 112 tries to produce 100 good answers, cost is spend / 100. Retries and failures belong in the numerator,
or a flaky-but-cheap config flatters itself. The quality axis is one outcome metric per plot — test pass rate for code,
task pass rate for multi-step agents, field accuracy for extraction. Mixing surfaces collapses configs that win on one task and lose on
another.
Two empirical findings recur across published sweeps, and both cut against intuition.
The frontier is sparse — and the most expensive model is rarely on it. Across 21,730 rollouts over nine models and nine benchmarks, HAL found fewer than one-third of tested models sit on the frontier for a given benchmark on average, and the costliest models are seldom Pareto-optimal. A separate Q2-2026 study of 20 frontier models found only six on the joint cost/quality/speed frontier — fourteen were dominated by a cheaper or faster sibling, yet most still carried production traffic. Choosing the biggest model on reflex overspends.
HAL found that increasing the reasoning-token budget reduced accuracy in 21 of 36 model × benchmark settings. Effort is a knob with its own Pareto sweep — the cheapest effort tier that holds quality is the default, never the highest. (This is exactly why the previous lesson said to scale effort to difficulty rather than max it.)
Pareto plots built on a single generation per configuration suffer the optimizer's curse — taking maxima over noisy samples inflates accuracy estimates up to 8.7% and cost up to 88% at ≤10 generations. Average over multiple runs per cell (or apply explicit bias correction) before the verdict is load-bearing.
A worked shape: a team sweeping their PR-review agent finds Opus-medium-effort is dominated — Sonnet with caching matches its quality at one fifth the cost — a free win. But the Sonnet→Haiku step is not horizontal: it drops 14 points of pass rate, which the bill alone would have reported as a 75% cost cut. Either move may be right depending on what the agent is for. The point is that the plot makes the choice explicit instead of letting a finance dashboard make it for you.
ET/run × runs/day; the multiplier is how often it runs.Retrieval practice — recall, don't peek
Question 1When prioritising always-on workflow fixes, you should rank by…
Question 2A fix cuts input tokens 62% but total cost rises 1.8%. The most likely cause is…
Question 3On the cost/quality plot, a model downgrade that loses quality shows up as a…
Question 4The cost axis of the frontier should be denominated in…
Question 5 · spaced recall from an earlier lessonFrom the effort-scaling lesson: reasoning effort should be set to…
token-usage.jsonl proxy, or sketching a Pareto sweep grid for your own
agent? Next, the capstone: Cut Token Spend Without Losing Quality — a symptom-to-move table that routes whatever you're staring at
to the right of the four rights.