The $112 night
Hedge is a daily calibration trivia game I build solo; the architecture lives in the case study. This follows the agent-fleet entry: what one night of un-tiered subagents cost, and the controlled experiment that fixed how the models get allocated.
Back in the quality-campaign entry I described how Hedge gets built now: every task goes to a fresh AI implementer agent, and nothing lands without two reviews, one against the spec and one against the code. That machinery was humming along this week, rebuilding my question-review tooling as a web app, and one night I let a whole phase of it run while I slept. I woke up to good code, a working special-days editor, and a bill: roughly thirty agent dispatches, every single one quietly running on the most expensive model available, implementers and reviewers alike. $112 of a $200 promotional credit, gone in one night.
The bill
The galling part wasn't the number, it was what the money bought. Most of those dispatches were doing careful, honest, completely mechanical work: porting screens I already had to a new surface, wiring forms to API calls that already existed. I had tiered models to task complexity in earlier sessions, but that habit lived in my head, and a new session doesn't inherit my head. It ran everything on the flagship because nobody told it not to. So this time the rule got written down where the sessions live, and the next phase became a controlled experiment.
The experiment
Same pipeline, same kind of work: the scheduling screen, the question bank, import, and PWA support. But this time the plan itself carried a model assignment per task. Mid-tier models did all seven implementation tasks and all the spec reviews. The flagship was allowed exactly three appearances, as a reviewer only: once each on the two paths that write to the production schedule, and once for a final pass over everything.
What the cheap models caught
The mid-tier agents were better than I'd been pricing in. Five of the seven implementation tasks landed clean on the first pass, and the spec reviewers caught real gaps in the other two. My favorite moment of the whole phase: one of the cheap implementers caught a bug in the plan it had been handed, a query limit of 5000 that the server would have rejected outright, because the server caps explicit limits at 1000. The expensive model wrote that plan. The cheap one refused to implement the mistake. Cheap does not mean credulous.
What the expensive one caught
And the flagship earned its three appearances. Its reviews found a race where a stale duplicate-check could paint its results over a newer preview, a hole where table rows you had selected and then filtered out of sight would still ride along into a schedule write, and an error message that told a comforting lie about why a day was full. Notice the shape of that list. Every real bug was in code that writes. None were in code that only reads. The whole phase cost about $18 against the first night's $112, and the bugs that mattered still got caught, because the expensive judgment was parked exactly where mistakes are irreversible.
Blast radius
So the lesson I'm keeping is not "use cheap models," it's an allocation rule. Tier by blast radius, not by difficulty. A fiddly porting job is still cheap-model work, because the review net catches whatever it fumbles and a fumble costs one revision. A schedule write that silently carries the wrong five questions into players' mornings is flagship work even when the code looks boring, because nothing downstream will ever flag it. The two-review structure is what makes this safe at all: implementers can be cheap precisely because every line they produce gets read twice, by a skeptic, on the way in. The reviewer is the last one holding the work before it ships. Pay for the skeptic.