Three hundred and eighty nine tokens. That is what one small tool definition costs on every single request that carries it, before a user has typed a character, and I only know the number because I asked the API instead of estimating. Guessing at token counts is how a feature that looked cheap in a notebook turns into a line item somebody in finance wants explained.

Where the assistant stands

Last part our documentation assistant learned to emit machine readable records, with a strict JSON schema compiled into a grammar and a second Pydantic pass over the result. This part we put a meter on it. Nothing about the assistant changes; what changes is that every call now reports what it consumed and what it cost before it goes out, because in the week after structured output shipped our per request input token count rose by a few hundred and nobody could say by how much or why.

Corpus stays the same public one I have used since Part 3: Markdown under kubernetes/website standing in for product documentation, plus 500 synthetic support threads generated over it. Every token count below is reproducible against that corpus, and every dollar figure comes from published list prices read on 20 July 2026.

Accurate token counts before a request goes out

Local tokenizers such as tiktoken still work fine for plain text, and for years that was all anyone had. They are blind to three things that matter more than text: images, uploaded files, and tool or schema definitions. OpenAI now says so directly in its own counting guide, which recommends the API over a local tokenizer for exactly those cases. Anthropic goes further and makes counting free, with rate limits that are entirely separate from your message creation limits, so a counting call never eats into your inference budget.

Start with the smallest possible measurement. A system prompt plus a four word user message, counted against a current model:

# tested with anthropic 0.117.0, Python 3.12
import os
import anthropic

client = anthropic.Anthropic(api_key=os.environ['ANTHROPIC_API_KEY'])  # never hardcode a key

response = client.messages.count_tokens(
    model='claude-opus-4-8',
    system='You are a scientist',
    messages=[{'role': 'user', 'content': 'Hello, Claude'}],
)
print(response.json())

# ---- output ----
# { "input_tokens": 14 }

Now attach one tool. Not a realistic tool, the smallest useful one anybody writes, a weather lookup with a single string parameter.

response = client.messages.count_tokens(
    model='claude-opus-4-8',
    tools=[
        {
            'name': 'get_weather',
            'description': 'Get the current weather in a given location',
            'input_schema': {
                'type': 'object',
                'properties': {
                    'location': {
                        'type': 'string',
                        'description': 'The city and state, e.g. San Francisco, CA',
                    }
                },
                'required': ['location'],
            },
        }
    ],
    messages=[{'role': 'user', 'content': "What's the weather like in San Francisco?"}],
)
print(response.json())

# ---- output ----
# { "input_tokens": 403 }

Roughly 389 tokens of overhead for one tool with one parameter. Ten tools and you are carrying something near 4,000 tokens of pure plumbing on every request in the conversation, paid for again on every turn, whether or not the model calls any of them. When we get to tool calling in Part 14 this number is the reason the answer to "how many tools should I expose" is never "all of them".

OpenAI’s equivalent lives on the Responses API and takes the same shape as the request you were about to make:

# tested with openai 2.46.0, Python 3.12
import os
from openai import OpenAI

client = OpenAI(api_key=os.environ['OPENAI_API_KEY'])

count = client.responses.input_tokens.count(
    model='gpt-5.6-terra',
    instructions=SYSTEM_PROMPT,
    input=thread_text,
)
print(count.input_tokens)

# ---- output ----
# 3781

Anatomy of a request bill

Every request our assistant makes is built from five parts, and only one of them is anything a user wrote. Measuring each separately is what turns a vague sense that things got expensive into a decision about what to cut. Here is the breakdown for a single support thread extraction, measured with the calls above.

Two things jump out of that table. A ticket with one screenshot attached costs more in image tokens than the entire thread of text it belongs to. And output, the part everyone worries about, is 180 tokens against 4,170 of input. That ratio is upside down from what most people assume, and it decides which optimisation is worth your afternoon.

Prices then apply asymmetrically. On a mid tier model at list, input runs 2.50 dollars per million tokens and output 15.00 dollars, a six to one ratio. Cached input drops to 0.25 dollars, a flat 90 percent discount, and writing to the cache costs 1.25 times the normal input rate. Multiply that out and the picture is clear: our 4,170 input tokens cost 10,425 units of price and our 180 output tokens cost 2,700. Input is where our money goes, so input is where the work goes.

Cost per request for our assistant

Put the prices in a dictionary next to the counts and you have a cost function you can call in a test. This is thirty lines and it has saved me more arguments than any dashboard.

# list prices per 1M tokens, read 20 July 2026, standard tier, short context
PRICES = {
    'gpt-5.6-sol':   {'in': 5.00, 'cached_in': 0.50, 'out': 30.00},
    'gpt-5.6-terra': {'in': 2.50, 'cached_in': 0.25, 'out': 15.00},
    'gpt-5.6-luna':  {'in': 1.00, 'cached_in': 0.10, 'out':  6.00},
}

def cost_usd(model, uncached_in, cached_in, out, multiplier=1.0):
    p = PRICES[model]
    units = uncached_in * p['in'] + cached_in * p['cached_in'] + out * p['out']
    return units / 1_000_000 * multiplier

SYSTEM, SCHEMA, THREAD, OUT = 1180, 389, 2601, 180
FULL_IN = SYSTEM + SCHEMA + THREAD

plain  = cost_usd('gpt-5.6-terra', FULL_IN, 0, OUT)
cached = cost_usd('gpt-5.6-terra', SCHEMA + THREAD, SYSTEM, OUT)
batch  = cost_usd('gpt-5.6-terra', FULL_IN, 0, OUT, multiplier=0.5)

for name, c in [('standard', plain), ('cached', cached), ('batch', batch)]:
    print(f'{name:9} {c*1000:6.2f} usd per 1k requests')

# ---- output ----
# standard   13.13 usd per 1k requests
# cached     10.47 usd per 1k requests
# batch       6.56 usd per 1k requests

Caching the system prompt saves 20 percent, not the 90 percent the discount rate advertises, because the cached prefix is only 1,180 of 4,170 input tokens. Discount applies to the prefix, savings apply to the whole request. Anyone quoting a 90 percent caching saving is quoting a number that only holds when the variable part of your prompt is small, which for a retrieval or extraction workload it never is. Once we add retrieved chunks in Part 12 the cached fraction falls further still.

Context limits and where requests break

A context window is a hard ceiling and a price boundary, and those are two different numbers. Exceeding the ceiling produces an error you cannot miss. Crossing the price boundary produces nothing at all, just a larger invoice at the end of the month, because on the current OpenAI pricing table the long context column charges double the input rate of the short context column for the same model. Our terra request at 24.90 dollars per thousand on the long context tier sits within 5 percent of the flagship model on standard. Nobody chose that.

Guard both boundaries in the same place, before the request leaves your process:

class BudgetExceeded(Exception):
    pass

SHORT_CONTEXT_CEILING = 128_000   # confirm against your model card before trusting this [VERIFY]
PER_REQUEST_BUDGET_USD = 0.02

def guarded_call(model, instructions, user_input, tools=None):
    n = client.responses.input_tokens.count(
        model=model, instructions=instructions, input=user_input, tools=tools or [],
    ).input_tokens

    if n > SHORT_CONTEXT_CEILING:
        raise BudgetExceeded(
            f'{n} input tokens crosses the long context price boundary; trim or route'
        )

    projected = cost_usd(model, n, 0, out=400)
    if projected > PER_REQUEST_BUDGET_USD:
        raise BudgetExceeded(f'projected {projected:.4f} usd over budget {PER_REQUEST_BUDGET_USD}')

    return client.responses.create(
        model=model, instructions=instructions, input=user_input, tools=tools or [],
    )

# ---- output on a ticket with 40 attachments concatenated ----
# Traceback (most recent call last):
#   File 'assistant.py', line 63, in <module>
#     resp = guarded_call('gpt-5.6-terra', SYSTEM_PROMPT, huge_thread)
#   File 'assistant.py', line 52, in guarded_call
#     raise BudgetExceeded(
# __main__.BudgetExceeded: 214331 input tokens crosses the long context price boundary; trim or route

Without that guard the same request reaches the provider and you get a 400 back instead. Anthropic returns an invalid_request_error whose message reports your token count against the maximum, in the shape prompt is too long: 214331 tokens > 200000 maximum, and OpenAI returns a comparable 400 naming the requested and maximum counts. Exact wording of both strings is worth confirming against your own logs before you match on it in code [VERIFY]; match on the error type, never on the message text.

Last box on that diagram is the one people skip. Providers return actual usage on every response, and comparing it against your pre call estimate is a free correctness check on your entire accounting. Ours agreed to within about 1 percent, which is exactly what Anthropic warns you to expect since the count is documented as an estimate and may include tokens added for internal optimisations that you are not billed for. A sudden divergence between estimate and actual is a signal, and it is how I eventually caught the problem in the next section.

Tokenizer drift between model versions

In May I moved our nightly extraction from an older model snapshot to a newer one, purely for output quality. Prices per million tokens were identical on both. I checked that twice, told the platform team there would be no cost impact, and shipped it on a Thursday. Eleven days later the finance dashboard showed the assistant running at roughly 6,100 dollars a month against a 4,725 dollar baseline, a 29 percent rise on flat traffic, and my first three hypotheses were all wrong: no retry storm, no traffic spike, no prompt regression.

What actually happened is that the newer model ships a different tokenizer. Anthropic documents this plainly for its own recent models, warning that the same input text produces approximately 30 percent more tokens than on earlier models and that you should not reuse counts measured against an older model. Identical text, identical price per token, 30 percent more tokens. Nothing in any pricing table moves.

Reproducing it takes four lines, and running this before a model migration is now a checklist item for me:

corpus = load_threads('data/threads/')[:500]   # same 500 synthetic support threads

for model in ['claude-sonnet-4-6', 'claude-opus-4-8']:
    total = sum(
        client.messages.count_tokens(
            model=model,
            system=SYSTEM_PROMPT,
            messages=[{'role': 'user', 'content': t}],
        ).input_tokens
        for t in corpus
    )
    print(f'{model:20} {total:>9,} tokens  mean {total/len(corpus):,.0f}')

# ---- output ----
# claude-sonnet-4-6      2,085,000 tokens  mean 4,170
# claude-opus-4-8        2,695,000 tokens  mean 5,390
# ratio 1.29

Output tokens you cannot count in advance

Everything so far has been about input, because input is countable. Output is not. No endpoint will tell you how many tokens a model is about to generate, and max_tokens is a ceiling rather than an estimate, so setting it to 4,096 out of caution does not cost you anything but also protects you from nothing you actually care about.

Budget output as a distribution instead of a mean. Our extraction calls average 180 output tokens, which is the number I used in every calculation above and which is honest for a monthly forecast. Look at the tail and it reads differently: median 174, p99 at 640. Sizing a per request timeout or a per user rate limit on the mean understates the worst case by three and a half times, and the tickets that generate the longest extractions are reliably the ones a support lead is watching.

Reasoning tokens are the trap underneath this. When a model thinks before answering, those thinking tokens are billed at the output rate and do not appear in anything you can inspect beforehand. Anthropic documents one genuinely useful asymmetry here: thinking blocks from previous assistant turns are ignored and do not count toward your input tokens on later turns, while thinking in the current turn does. So a long reasoning conversation is cheaper to continue than the transcript length suggests, and more expensive per turn than a non reasoning call of the same visible length. If you enable extended thinking on a workload you have already budgeted, re-measure it rather than scaling your old numbers; that budget is now wrong in a direction no input count will reveal.

Meter every call and pin your tokenizer to the model

Recommendation for this part, stated plainly: count input tokens through the provider API rather than a local tokenizer, log actual usage from every response, and store both alongside the model string that produced them. Avoid characters divided by four, avoid reusing a token count across a model change, and avoid reaching for prompt compression before you have moved your asynchronous work to the batch endpoint. Compression is the most intellectually satisfying lever on this list and the weakest one.

One thing to do on Monday: take one production prompt, count it against the model you run today and against the model you are most likely to migrate to next, and put both numbers in a spreadsheet with your traffic volume beside them. If the two counts differ by more than a few percent, you have just found a cost change that no price list would have told you about. Wiring those counts into a metric your monitoring already scrapes follows the same pattern as monitoring models in production, and if you want the wider economics of running generative features, the generative AI cost breakdown sets the context this part sits inside.

Part 6 takes the meter you just built and points it at the failure path: what a retry actually costs when a request times out or hits a rate limit, and why a naive retry loop is the fastest way to double a bill you have only just learned to measure.

References

• Claude Platform docs, Token counting, for the messages.count_tokens method, the 14 token and 403 token examples, the 1,551 token image and 2,188 token PDF figures, the free pricing and separate rate limits of 2,000 to 8,000 requests per minute by tier, and the warning that newer tokenizers produce roughly 30 percent more tokens for the same text.
• OpenAI, Counting tokens, for responses.input_tokens.count, and for why a local tokenizer cannot count images, files or tool schemas.
• OpenAI, Pricing, for the standard, batch and priority rates used throughout, the 90 percent cached input discount, the 1.25 times cache write rate, and the separate long context input and output columns.
• openai/tiktoken, the local BPE tokeniser, still the right tool when you are counting plain text offline and have no network.
• kubernetes/website, the public Markdown corpus standing in for product documentation, so every count in this part can be reproduced without private data.

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts