KitCutThe cheaper model wasn't half the price
Every KitCut film is written, drawn and scored by Claude Opus, and Opus is the most expensive model we could use. So we tested the obvious saving: the same films made by a cheaper model, graded blind. The cheaper model saved a quarter, not half, and wasn't faster. The saving we found in the end came from somewhere else.
A KitCut film is one long piece of work by one model. It writes the narration, draws every scene in code, records the voice, looks at stills of its own work and fixes them, and writes the music. The model's bill is most of what a film costs us, so the choice of model is most of our margin. We don't make that choice on a price list. We make it with a bake-off: the same prompts, made by each candidate on the same studio code, then graded by a judge who sees the frames but not which model made them.
Opus against Sonnet
Four real prompts, each a film of a minute or less: a product launch for a finance app, an explainer on Cloudflare's 1.1.1.1, a collage history of ice cream, and a painted bedtime story about a lighthouse. Both models ran at the same effort setting.
| Opus 5.5 | Sonnet 5.5 | |
|---|---|---|
| Claude's list price per film | $3.51 | $2.59 |
| Time to make a film | 14.6 min | 15.7 min |
| Blind grade, "professional" (out of 5) | 4.0 | 3.75 |
| Films won, by eye | 2 | 1 (plus one tie) |
Sonnet won one film outright, the Cloudflare explainer. It's a clear, well-built film, and you can see why it won:
It lost two. One was on visible defects: the collage film had a blank panel and an empty frame. The other was on story. The fourth was a tie.
So Sonnet came out a little worse and no faster. And it saved only $0.92 a film, 26%, when its price per token is far lower than Opus's.
Why a cheaper model wasn't half the price
The token counts explain it. At the same effort, Sonnet wrote 45% more output tokens than Opus, and 58% more of its own thinking. It took more words to reach a slightly worse film. A smaller second reason is that both models read a lot of cached context, and that part of the bill doesn't shrink as much as the headline price.
A model's price per token is only half the sum. The other half is how many tokens it needs for your job, and you only find that out by running your job.
A GPT model, and a number we got wrong
The same day we ran OpenAI's GPT-6.1 Sol through the same agent. Its films graded level with Opus by the blind judge, and by eye it won one, lost two and tied one, like Sonnet. It was slower: 18.8 minutes a film.
We first recorded it as 42% cheaper. That was wrong. The agent that runs our films didn't recognise the model, so it priced its tokens at Opus's rates, and the "42%" was Opus's price applied to GPT's token count. We found the error a week later while checking facts for another film. What the provider actually charged was never recorded, so we quote no price for that run.
One practical lesson from it: a model from another family filled in every optional argument of our tools with placeholder values, and a tool that reads a placeholder as a real answer stalls. If you swap the model behind your agent, test your tools, not just the output.
Cheap models for the small steps?
A week later we tried the other popular idea: keep Opus for the film itself, and hand the small steps to cheaper models. The candidates were the second reviewer that checks a finished film for glitches, the short words for a film's share page, and a series' episode log.
It didn't pay. Those small steps are 5.8% of the studio's Claude usage. The author, the pass that writes the film, is about three quarters of the time. Moving the small steps can't move the total much. And the cheaper models were worse at the steps that matter:
- The glitch reviewer: Opus found 20 of 27 known glitches. GPT-6 Luna found 2 of 16.
- Share-page words and episode logs, on 14 films, read blind: Haiku, Luna and Gemini Flash were worse; Sonnet and Sol were level with Opus but no faster.
Why no faster? Each question ran through a full agent process, which took 12 to 19 seconds. The same Opus question as one plain API request took 5. The overhead was the process, not the model.
The saving that was there all along
The lever turned out to be the setting we hadn't questioned: how hard the author thinks. We made four films with Opus at its highest effort and again one level down:
- Claude's own time per film: 12.4 minutes down to 5.8;
- output tokens: 74,000 down to 34,000;
- list price per film: $3.74 down to $2.37;
- the blind grade, and our own read of the frames: level.
That's more saving than Sonnet gave, with the better model kept. We haven't switched it on for everyone yet. Four films is a small set, and this measurement didn't check motion glitches (things that only show when the film plays). It changes how every customer's film is made, so it gets one more round: short films made at both settings, each checked by the glitch reviewer and watched by a person.
We'd already seen the effect in one place: at the highest effort, Opus once thought for over ten minutes before writing the first line of a 40-second scene. That's part of why long films are written scene by scene.
If you're choosing a model for an agent
- Run your own job, not a benchmark. Price per token times tokens per job is the only price that counts, and the second number depends on the model.
- Grade blind, and keep a control. We judged by the frames, without names.
- Check what your metering knows. An unrecognised model can be priced at someone else's rates without any warning.
- Measure where the usage is before moving steps to cheaper models. Ours was three quarters in one step.
- Try the effort setting before the model. It was the biggest lever we found, and it kept the best model.
KitCut makes every film with Claude Opus. How KitCut compares explains what that adds over running Opus yourself, and Plans and credits shows what a film costs you. For more of what the studio's own logs taught us, read what we learned mining our agent's transcripts.