Choosing a Model for the Task: When Cheap and Fast Wins

You have three models in the picker and you reach for the strongest one every time, because choosing is more effort than paying. That default is costing you money on the tasks where it is overkill and, more annoyingly, costing you minutes of dead waiting on the tasks where you are in a tight loop.

The useful question is not which model is smarter. It is how much irreducible reasoning a given task actually contains, and what happens when the answer comes back wrong.

The axis that matters

Most of what we hand to an agent is not hard. Renaming a symbol across forty files is not hard. Writing a describe block from a spec you have already written is not hard. Generating a Zod schema from a Prisma model is not hard. These tasks are long and fiddly, which feels like difficulty, but the reasoning depth is close to zero. A model that is fast and cheap does them at the same quality as a model that is slow and expensive, because there is nothing to be clever about.

The tasks with genuine reasoning depth are the ones where the shape of the answer is not determined by the input. Why does this test fail only in CI. What should the boundary between these two services be. Where is the race condition. Here the strong model is not a luxury, and trying to save four cents is not a serious plan.

A working taxonomy

Mechanical refactors across many files. Cheap and fast. The change is specified precisely, the diff is reviewable, and the type checker catches the misses. Give it the full list of call sites up front rather than making it search.

Writing tests from an existing spec. Cheap. You already did the thinking when you wrote the spec. Verification is running the suite. If the tests pass and cover what you asked for, the model was right, and you know within thirty seconds.

Boilerplate and scaffolding. Cheap, and use the fastest thing you have. A new route handler that follows the pattern of the four existing route handlers is pattern matching, not design.

Commit messages and changelogs. Cheapest available, always. You read the output immediately and fix it in five seconds if it is off. There is no world in which this justifies a frontier model.

Code review. Strong model. This is the clearest case in the list, and it is worth being blunt about why: the failure mode is silence. A cheap model reviewing a diff will confidently say "looks good" and miss the off-by-one in the pagination logic. You do not find out. There is no test that fails, no type error, no compile break. The whole value of review is catching what you did not already know, so a reviewer that misses things quietly has negative value.

Greenfield architecture. Strong model, and use it as a thinking partner rather than a code generator. The output you want here is a plan and a set of trade-offs, not files.

Debugging a bug you cannot reproduce. Strong model. This is the highest-reasoning task most of us encounter. It needs to hold several competing hypotheses, weigh them against sparse evidence, and propose the diagnostic that discriminates between them. Cheap models tend to grab the first plausible story and start editing.

Migrating a framework version. Split it. The strong model reads the changelog, inspects your usage, and writes the migration plan as a numbered list of concrete steps. The cheap model then executes each step. This is the pattern that generalises furthest, and it is worth naming properly.

Plan strong, execute cheap

The reason this works is that a good plan removes the reasoning from each step. "Step 4: in apps/api/src/routes/*.ts, replace every res.json(x) with return c.json(x) and change the handler signature from (req, res) to (c)" contains no decisions. Any competent model executes it.

The failure mode to watch for is a plan that is really a wish. "Step 4: update the route handlers to the new API" pushes the reasoning back into execution, where you have deliberately put your weakest model. If a step could be argued about, it belongs in the plan, not in the run.

Latency is a cost you pay in attention

Here is the case people underrate. Assume you are in a tight loop, twenty small tasks in a session, each one verifiable by glancing at the diff or running one test.

The fast model returns in about 5 seconds and is right 85% of the time. Twenty runs is 100 seconds. Three of them are wrong, and each wrong one costs a retry at 5 seconds plus roughly 30 seconds of you reading and re-prompting, so 3 × 35 = 105 seconds. Total: 205 seconds, about three and a half minutes.

The strong model returns in about 45 seconds and is right 95% of the time. Twenty runs is 900 seconds. One is wrong, costing 45 + 30 = 75 seconds. Total: 975 seconds, over sixteen minutes.

Those latency figures are illustrative rather than measured, and the accuracy rates are made up for the sake of the sum. The structure holds anyway: when you catch errors in seconds, being right 85% of the time is fine, and the extra ten points of accuracy costs you five times the wall-clock. You also stop context-switching, which does not show up in either total but is the part you actually feel.

The reverse case

Now change one thing. The task is a framework migration across sixty files, and you are not going to review every hunk closely.

The cheap model gets through forty files cleanly and then, on the ones with an unusual pattern, does something subtly wrong. It half-migrates. Some handlers use the new signature, some use the old, a few use a hybrid that happens to type-check. You do not notice until something breaks in staging, and then you spend an hour reconstructing which files it touched and what it did to each of them.

The difference is not the model's average quality. It is that the verification loop is long and the failure is quiet. When you cannot check the work in under a minute, the expected cost of an error swamps the price difference. That is the whole rule.

Do not route by model where you cannot verify cheaply

Automatic routing by task type is tempting and mostly a trap, because routers classify on surface features. "Refactor" gets routed cheap whether it is a rename or a redesign of the caching layer. Route on verifiability instead, which you can usually assess in a second: is there a test, a type check, a diff I will read, or an obvious visual result. If yes, go cheap and escalate on failure. If no, do not.

Test it on your own work

Leaderboards are measured on problems that are not yours. Build a tiny private eval instead: pull ten real tasks from your last month of git history, ones where you know what the right answer was. Run each against your cheap and strong models from a clean session. Record whether the output was correct, how long it took, and how long you spent fixing it.

Ten tasks is enough to be informative and small enough that you will actually do it. You will usually find the cheap model is fine for more of your work than you assumed, and catastrophically not fine for one specific category you would not have guessed. That category is the one worth knowing.

The models will all get better, and the ones you are calling cheap today will handle next year what only the expensive tier handles now. What will not change is that the cost of a wrong answer is set by how long it takes you to notice.