AI engineering · LLM cost · Tokens · Prompt caching
Every AI provider charges by the token, but the bill is not just “tokens × price”. Output costs several times more than input, saved (cached) text costs a fraction, and a long conversation re-sends everything on every turn. This post builds the formula step by step, works through it with real numbers, and ends with a real task from my own AI coding session: 12 million tokens that cost about six dollars.
The method works for any AI tool. Worked examples use Claude’s published prices; OpenAI and Google are compared later. All prices are US dollars per million tokens, checked on 24 October 2026.
Step 1What a token is
A model doesn’t read words or letters. It reads tokens: chunks of text, often a whole short word, sometimes part of a longer one, plus spaces and punctuation. Here is roughly how a sentence breaks up:
The split above is illustrative. Every provider uses its own tokeniser, so the same text gives different counts on different models. Even within one provider it changes: Anthropic says its newer tokeniser (Claude 4.7 and later) produces about 30% more tokens for the same text than older models. So you can’t compare prices per token across models. Compare the cost of the same task instead.
Step 2The four meters
A request is billed on up to four separate counts. Prices below are for Claude Opus 5.5 at the Standard rate:
Everything you send: instructions, tool definitions, documents, the conversation so far. The base rate everything else is measured against.
Saving part of the prompt so later calls can reuse it. 1.25× input for a 5-minute cache, 2× for a 1-hour cache.
Reusing a saved prompt. 0.05× input on Opus 5.5 and Sonnet 5.5; 0.1× on most other Claude models.
Every token the model writes, including hidden “thinking” tokens. 5× the input price.
Step 3The formula
Add the four meters. Counts are in tokens, prices are per million tokens, so divide by one million:
Because every price is a fixed multiple of the input price, you can also turn any request into effective tokens: what it would cost as plain input. For Opus 5.5 with a 1-hour cache:
Worked example: one simple request
You send a 2,000-token prompt to Claude Sonnet 5.5 ($2 input, $10 output) and get a 500-token answer, with no caching:
Step 4Why long conversations get expensive
Models have no memory between calls. To continue a chat, the app sends the whole conversation again every time: the instructions, every earlier message, every tool result. So each turn’s input is bigger than the last.
Say the instructions and tool definitions are S tokens, and each turn adds t tokens. Turn k sends S + k·t tokens. Over N turns, the total input is:
Take an AI agent with S = 20,000 tokens of instructions and tools, adding t = 3,000 tokens per turn (your message, its reply, a file it read), for 50 turns, writing 800 output tokens each turn. By turn 50 a single call sends 170,000 tokens. Over the whole conversation:
| Opus 5.5, 50 turns | Tokens | Rate | Cost |
|---|---|---|---|
| Input, no caching | 4,825,000 | $4.00 | $19.30 |
| Output | 40,000 | $20.00 | $0.80 |
| Total without caching | $20.10 | ||
| Cache reads | 4,655,000 | $0.20 | $0.93 |
| Cache writes (1 hour) | 170,000 | $8.00 | $1.36 |
| Output | 40,000 | $20.00 | $0.80 |
| Total with caching | $3.09 |
Running total across the 50 turns. Without caching the curve bends upward, because every turn re-sends a longer history at full price. With caching, the history is read back at 5% of the price, so the line is almost straight.
Step 5When caching pays, and when it doesn’t
Writing to the cache costs more than plain input, so caching only pays if the saved text is read back enough times. Measure everything in units of “the prompt sent once at full price” and count n later reuses:
5-minute cache: 1.25 + 0.1n
1-hour cache (Opus 5.5): 2 + 0.05n
| Reuses (n) | No cache | 5-minute cache | 1-hour cache |
|---|---|---|---|
| 0 | 1.00 | 1.25 | 2.00 |
| 1 | 2.00 | 1.35 | 2.05 |
| 2 | 3.00 | 1.45 | 2.10 |
| 10 | 11.00 | 2.25 | 2.50 |
| 50 | 51.00 | 6.25 | 4.50 |
The catch is expiry. A cache only lives for its time limit, and each read restarts the clock. If you step away from an agent for longer than that, the next call has to write the whole history to the cache again, at the write price. With a 400,000-token history on Opus 5.5, that one re-write costs about $3.20. The 1-hour cache costs more to write than the 5-minute one, but it survives a coffee break.
Real example12 million tokens for six dollars
This is a real task from my own work. I asked Claude Code (running Claude Opus 5.5) to change the footer on 17 PowerPoint files from a long-running session. It took 26 model calls. Because the session’s history was already about 450,000 tokens, every call re-sent a lot of text:
| Meter | Tokens | Rate (per 1M) | Cost | Share |
|---|---|---|---|---|
| Fresh input | 52 | $4.00 | $0.0002 | 0% |
| Cache reads | 11,437,878 | $0.20 | $2.29 | 37% |
| Cache writes (1 hour) | 430,652 | $8.00 | $3.45 | 56% |
| Output | 18,890 | $20.00 | $0.38 | 6% |
| Total | 11,887,472 | $6.11 | 100% |
Check it with the effective-tokens formula: 52 + 2 × 430,652 + 0.05 × 11,437,878 + 5 × 18,890 = 1,527,700 effective tokens. At $4 per million that is $6.11, the same answer.
What the numbers say
Three lessons from one small job:
- The work itself was cheap. Writing scripts, editing files and replying produced only 18,890 output tokens: 38 cents.
- The history was the cost. 99.8% of the tokens were the conversation being re-sent. A fresh session would have done the same job for a fraction of the price.
- Expiry was the biggest single item. The session sat idle overnight, the cache expired, and the first call next morning re-saved 430,000 tokens. That alone was $3.45, more than half the total.
Across providersSame method, different numbers
Every major provider uses the same building blocks: separate input and output prices, a discount for cached input, and higher output prices. The details differ. Standard rates per million tokens:
| Model | Input | Cached input | Saving to cache | Output |
|---|---|---|---|---|
| Claude Opus 5.5 | $4.00 | $0.20 | $5 (5 min) or $8 (1 hour) | $20.00 |
| Claude Sonnet 5.5 | $2.00 | $0.10 | $2.50 or $4 | $10.00 |
| Claude Haiku 5.5 | $0.10 | $0.01 | $0.125 or $0.20 | $0.50 |
| OpenAI gpt-6.1-sol | $2.00 | $0.10 | no separate price listed | $10.00 |
| OpenAI gpt-6-luna | $0.10 | $0.01 | no separate price listed | $0.50 |
| Gemini 3.1 Pro (preview) | $2.00 | $0.20 | storage $4.50 per 1M tokens per hour | $12.00 |
| Gemini 3.8 Flash | $0.75 | $0.075 | storage $0.50 per 1M tokens per hour | $3.75 |
Gemini 3.1 Pro prices are for prompts up to 200,000 tokens. Gemini 3.8 Flash prices are scheduled to double on 1 January 2027. Check each provider’s page before you rely on a number: these change often.
The modifiers that change the bill
| Modifier | What it does | Where |
|---|---|---|
| Batch | About 50% off input and output if you can wait for results instead of getting them live. | Claude, OpenAI, Gemini |
| Long prompts | Higher prices once a prompt passes a size limit. | Gemini Pro over 200K, OpenAI flagship models over 272K, Claude Haiku 5.5 over 100K; most other Claude models are flat up to 1M |
| Thinking tokens | A model’s hidden reasoning is billed as output, at the highest rate. | All providers that offer reasoning models |
| Cache storage | An hourly fee to keep a cache you created yourself. | Gemini (explicit caching) |
| Data residency | A premium for keeping processing in one country. | Claude (US-only inference) and others |
Putting it to workHow to cut your token bill
Going furtherWays to use fewer tokens
Cheaper prices help, but sending fewer tokens helps more. Each method below has a worked saving, using the same prices as the rest of this post.
Replace a long conversation with a short summary and carry on from there. In Claude Code, /compact summarises the session so far and /clear starts fresh. In the 50-turn agent from Step 4, compacting at turn 26 into a 5,000-token summary:
The re-sent history shrinks at full price.
Re-reads were already cheap, and the summary has to be written and cached.
A cache re-save after a break costs $0.80, not $1.36.
Everything a tool returns joins the history and is re-sent on every later turn. Read the lines you need, not the whole file: search instead of opening everything, show the last 50 log lines instead of the full log, limit search results. A log read at turn 10 and kept for 40 more turns:
$0.32 with caching, $3.28 without.
$0.016 with caching, $0.16 without.
And more room left in the context window.
In a retrieval (RAG) app, each question adds the retrieved chunks to the prompt. They change with every question, so caching rarely helps. Better ranking lets you send fewer. An internal chatbot on Claude Sonnet 5.5 answering 10,000 questions a day:
800 tokens each: 8,000 tokens per question.
3,200 tokens per question, after re-ranking.
About $2,880 a month.
Output is the expensive meter. Set a maximum length (max_tokens), ask for a specific shape (“three bullet points”, a fixed JSON format), and limit thinking on simple tasks. The same chatbot, 10,000 answers a day on Sonnet 5.5:
Long, chatty answers.
Short answers in a fixed shape.
About $2,400 a month.
Most requests in a busy app are easy. Send those to a small model and only the hard ones to a large one. One million requests a month, each 2,000 tokens in and 500 out:
$0.018 per request.
Hard 20% stays on Opus.
Check quality on a sample of routed answers first.
- Count all four meters separately: input, cache writes, cache reads and output.
- Use the multipliers for your exact model; they differ even within one provider.
- Add the conversation history to every turn’s input, not just the new message.
- Include thinking tokens in output.
- Compare models by cost per finished task, not price per token.
- Check the provider’s pricing page on the day you publish or budget.
Sources (checked 24 October 2026): Anthropic Claude pricing, OpenAI API pricing, Google Gemini API pricing. Token counts in the real example come from the session’s own usage records.
About The Author
DrJha