GoKawiilInput and output tokens
A request goes through the GPU in two phases, and they cost different amounts.
First, during prefill, the model reads your request and context: the system prompt, your CLAUDE.md , your message, and everything that's been added to the conversation since (the files Claude has read and the output of the commands it ran). Those are your input tokens.
Then, during decode, it writes output tokens: its thinking, the tool calls it makes, and the text you see. This happens one token at a time; a 200-token response is 200 runs of the model, one after the other. Per token, decode keeps the GPU busy for a lot longer, which is why output is priced at roughly 5x input.
A lot of the output tokens in a session are thinking tokens, and how much thinking the model does per turn is what the effort level controls. Like the model, the level you pick with /effort sticks around as your default for the next session too.
Tip: run /model and /effort once in a fresh session to see what you're actually on. Both remember whatever you picked last time, and you want that decision to be deliberate.
Tip: if you already know a session is going to be grunt work, MAX_THINKING_TOKENS=0 claude turns thinking off for that one session (except on Fable 5), which is the step below /effort low.
Prompt caching
If a request starts with exactly the same tokens as a request the server just saw, the state for that shared beginning comes out the same, so the server can keep it around from last time and only prefill whatever comes after it. This is called prompt caching.
Reading from the cache costs 0.1x the input price, because the server loads the state instead of computing it. Writing tokens into the cache costs a bit more than normal input, up to 2x, since the server also has to hold on to the state afterwards. But the write happens once per token, and the 0.1x reads happen on every turn after it.
... continue reading