GoKawiilAnthropic's Claude Opus 4.7 migration guide says the new tokenizer uses "roughly 1.0 to 1.35x as many tokens" as 4.6. I measured 1.47x on technical docs. 1.45x on a real CLAUDE.md file. The top of Anthropic's range is where most Claude Code content actually sits, not the middle.
Same sticker price. Same quota. More tokens per prompt. Your Max window burns through faster. Your cached prefix costs more per turn. Your rate limit hits sooner.
So Anthropic must be trading this for something. What? And is it worth it?
I ran two experiments. The first measured the cost. The second measured what Anthropic claimed you'd get back. Here's where it nets out.
What does it cost?
To measure the cost, I used POST /v1/messages/count_tokens — Anthropic's free, no-inference token counter. Same content, both models, one number each per model. The difference is purely the tokenizer.
Two batches of samples.
First: seven samples of real content a Claude Code user actually sends — a CLAUDE.md file, a user prompt, a blog post, a git log, terminal output, a stack trace, a code diff.
Second: twelve synthetic samples spanning content types — English prose, code, structured data, CJK, emoji, math symbols — to see how the ratio varies by kind.
The core loop is three lines of Python:
... continue reading