The real lever isn't the model. It's how small you make the task.

The default move in applied AI is to reach for the most capable model you can afford and worry about cost later. Bigger model, better output. That's the mental model, and most of the time nobody bothers to test it.

We tested it. The result ran the opposite direction, and the reason had less to do with the models than with how the task around them was built.

The experiment

We took one retrieval task — given an incoming request for quote, find and rank the most relevant historical request/offer pairs — and ran it through the exact same agent, the same dataset (~1,000 cases), the same configuration. The only variable we changed was the model underneath: a standard, full-size model (GPT-4o) versus GPT-5.4 nano, a model a fraction of its size.

The agent itself is deliberately narrow. It does one job, on a fixed procedure, with a fixed output structure. That design choice turns out to be the whole story.

Worth noting up front: the search step itself is model-independent. Both runs hit the same index and surfaced essentially the same candidate cases (~90.5% first-hit, ~95.5% recall either way). So this isn't a test of search. It's a test of what each model does with what search hands it: how faithfully it ranks, renders, and returns the right cases inside the lane the architecture gives it.

The results

The small model won on accuracy by about 7 points on both first-hit and recall. It used 27% fewer input tokens and roughly half the output tokens, and because GPT-5.4 nano pricing is far lower per token, the total cost came out around 17x cheaper — about $21 per thousand runs versus $350. It also failed one-twentieth as often: a 0.2% failure rate against 4.35%.

The bigger model won exactly one category: raw speed, finishing about 20 to 40% faster per case.

Why scope is the real lever

"Smaller is better" isn't a general law. What's true is narrower and more useful: when you scope a task tightly enough, a small model has everything it needs and nowhere to wander.

That scoping is an architecture decision, not a model decision. Instead of handing a model a broad goal and hoping its capability covers the gaps, the agent decomposes the goal into one well-defined job: follow a strict matching procedure, fill a fixed set of fields, emit a fixed structure. The hard thinking is encoded in the design of the task, not left to the model to improvise. Inside that lane, there is very little a small model can get wrong.

And the same constraint that helps the small model is what exposes the bigger one. Extra capability, on a narrow task, shows up as expansiveness, and expansiveness is a liability here. Look at where GPT-4o's failures came from: 17 runs looped past the turn cap, 9 produced invalid JSON, 3 blew through the context limit. That's a model with more headroom than the task needs, using it to over-elaborate, over-run, and step outside the structure the architecture defined. The small model simply stayed in the lane.

You can see it in the accuracy gap too. Since search fed both models the same candidates, the only thing left to differ was how faithfully each carried that signal into the final answer. The small model preserved almost all of it; its 89.7% nearly matches the 90.5% search ceiling. The bigger model lost about eight points by abbreviating and dropping cases on the way out. It wasn't finding worse answers; the loose rein let it deliver them worse.

The honest caveat

It wasn't a clean sweep on raw speed. The small model was slower per case. In practice that matters far less than the table suggests, because this runs as a background agent, not in a loop someone is sitting and waiting on. When the work executes asynchronously and no one is watching a spinner, whether a step takes three seconds or four is invisible; a few seconds of per-case latency changes nothing downstream. The speed gap would only bite on a genuinely real-time, user-facing path, and this isn't one. The one run that came in as a significant outlier is still worth chasing down, but that's a reliability fix to make, not a reason to pick the more expensive model.

The takeaway

The instinct to throw a bigger model at a problem is often a substitute for the harder work of scoping the problem down. When a task is broad and underspecified, raw capability papers over the gaps, and you pay for it in tokens, latency, and unpredictability. When the architecture narrows the task to a single, well-defined job with a clear contract, that capability has nothing left to do, and a small, cheap, disciplined model can match or beat the expensive one.

So the real lever isn't the model. It's the architecture that decides how big the task in front of the model is. Get the scope small enough, and the model choice gets easy, and often it gets cheap.