Tool calling is not a capability the model acquires. No weights change, no sandbox opens, nothing on your network becomes reachable. What actually happens is that you describe some functions in JSON, the model writes down which one it would like called and with what arguments, and your own process decides whether to honour that request. Framed that way, most of the confusion goes away, and so does most of the bad security reasoning I hear in design reviews.

Tool calling as a protocol, not a capability

Three pieces make up the protocol. First, a tool definition: a name, a plain English description, and a JSON Schema for the arguments. Second, a request from the model, which arrives as a content block carrying an id, the tool name and an arguments object. Third, a result you send back, tagged with that same id so the model can match it to what it asked for.

Notice what is absent. No execution happens on the provider side for tools you define, and no connection is opened from the provider to your database. Your process is the only thing that ever touches a system of record, which means every authorisation decision stays exactly where it already was. When someone in a review says the model will have access to production, the accurate correction is that your service will make the same calls it always could, with arguments a language model suggested. That reframing changes the threat model from model containment to argument validation, and argument validation is a problem you already know how to solve.

Vendors differ only in field names. Anthropic puts tools in a top level tools array with an input_schema per tool, returns tool_use content blocks and expects tool_result blocks in the following user message. OpenAI wraps each function in a type function envelope, returns tool_calls on the assistant message with arguments as a JSON string, and expects a message with role tool carrying tool_call_id. Same protocol, different spelling.

Where the assistant stands, and what a tool adds

Last part the internal documentation assistant got its first honest scorecard: hybrid retrieval with reranking hit 0.86 on retrieval at 6 while answer correctness sat at 0.62, and we learned to attribute each failure to a specific gate. Digging into the residual, one cluster was not a retrieval problem at all. Fourteen of the 120 labelled queries asked about live state. Whether SUP-4412 was resolved. Who owns the escalation opened on Tuesday. Which release the fix shipped in.

No amount of chunking rescues those. Answers do not exist in the docs corpus, because the answer changes hourly and lives in the ticket system. This part gives the assistant one tool, get_ticket_status, and builds the loop that lets it use that tool without us hand holding each turn.

First tool call, end to end

Here is a complete definition and the request that triggers it. Read the description closely, because it is doing more work than anything else on the page.

# tested with anthropic 0.117.0, Python 3.11.9
# export ANTHROPIC_API_KEY first; the SDK reads it from the environment.
# Never put a key in source. This file contains no credential.
import anthropic

client = anthropic.Anthropic()

TICKET_TOOL = {
    "name": "get_ticket_status",
    "description": (
        "Look up the current status of one support ticket by its key. "
        "Use this whenever the user names or clearly implies a specific ticket. "
        "Returns status, assignee, priority and last update time. It does NOT "
        "return the ticket body, comments or attachments, and it returns "
        "not_found for tickets closed more than 18 months ago, which are archived."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "ticket_id": {
                "type": "string",
                "description": "Ticket key like SUP-4412. Uppercase, no spaces.",
            }
        },
        "required": ["ticket_id"],
    },
}

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    tools=[TICKET_TOOL],
    messages=[{"role": "user", "content": "Any movement on SUP-4412?"}],
)

print(resp.stop_reason)
for block in resp.content:
    print(block.type, getattr(block, "name", ""), getattr(block, "input", ""))
print(resp.usage.input_tokens, resp.usage.output_tokens)

tool_use
text  
tool_use get_ticket_status {'ticket_id': 'SUP-4412'}
418 79

Two things in that output deserve attention. Stop reason is tool_use, which is your signal that the turn is unfinished and the model is waiting on you. And a text block arrived alongside the tool_use block, because models often narrate before calling. Code that assumes content[0] is the tool call will break the first time the model says something friendly first. Iterate over the list.

Handing the result back means appending the assistant message unchanged, then a user message whose content begins with a tool_result block carrying the matching id.

import json

def get_ticket_status(ticket_id: str) -> dict:
    # stand in for your real ticket client
    return {"id": ticket_id, "status": "awaiting_customer",
            "assignee": "r.mehta", "priority": "P2",
            "updated_at": "2026-07-18T09:12:04Z"}

call = next(b for b in resp.content if b.type == "tool_use")
result = get_ticket_status(**call.input)

messages = [
    {"role": "user", "content": "Any movement on SUP-4412?"},
    {"role": "assistant", "content": resp.content},
    {"role": "user", "content": [{
        "type": "tool_result",
        "tool_use_id": call.id,
        "content": json.dumps(result),
    }]},
]

final = client.messages.create(
    model="claude-opus-4-8", max_tokens=1024,
    tools=[TICKET_TOOL], messages=messages,
)
print(final.stop_reason)
print(final.content[0].text)

end_turn
SUP-4412 is currently awaiting customer response. It sits at P2 with
r.mehta as assignee, and was last updated on 18 July at 09:12 UTC.

Passing tools on the second call is not optional decoration. Drop it and you will get a 400 telling you the conversation references tools that were not supplied. Tool definitions must accompany every request in a conversation that used them.

Loop that runs until the model stops asking

One exchange is rarely enough. A question such as which release fixed the ticket needs a status lookup and then a changelog lookup. Generalising the two step exchange into a loop takes about forty lines, and I would write those forty lines before reaching for a framework, because when the loop is yours you can see exactly which turn burned the tokens.

REGISTRY = {"get_ticket_status": get_ticket_status}
MAX_STEPS = 6

def run(question: str, tools: list) -> str:
    messages = [{"role": "user", "content": question}]
    for step in range(MAX_STEPS):
        r = client.messages.create(
            model="claude-opus-4-8", max_tokens=1024,
            tools=tools, messages=messages,
        )
        if r.stop_reason != "tool_use":
            return "".join(b.text for b in r.content if b.type == "text")

        messages.append({"role": "assistant", "content": r.content})
        results = []
        for b in r.content:
            if b.type != "tool_use":
                continue
            fn = REGISTRY.get(b.name)
            if fn is None:
                results.append({"type": "tool_result", "tool_use_id": b.id,
                                "content": f"no tool named {b.name}",
                                "is_error": True})
                continue
            try:
                out = fn(**b.input)
                results.append({"type": "tool_result", "tool_use_id": b.id,
                                "content": json.dumps(out)})
            except Exception as exc:
                results.append({"type": "tool_result", "tool_use_id": b.id,
                                "content": f"{type(exc).__name__}: {exc}",
                                "is_error": True})
        messages.append({"role": "user", "content": results})
    raise RuntimeError(f"no final answer after {MAX_STEPS} steps")

print(run("Is SUP-4412 still open, and who has it?", [TICKET_TOOL]))

SUP-4412 is open and awaiting customer response, assigned to r.mehta at P2.

Four details in that loop earn their place. Every branch appends a tool_result, including the failure branches, because a missing result is a hard 400 rather than a degraded answer. Exceptions are returned to the model with is_error set instead of being raised, which lets it apologise or retry with a corrected argument. MAX_STEPS is a hard stop, not a suggestion; without it a model that keeps requesting the same lookup will spend your budget quietly. And results for all tool_use blocks in one turn go into a single user message, which matters once parallel calls appear.

Failures a tool calling loop actually produces

My first version of that loop appended a friendly text block before the tool results, on the reasonable theory that more context helps. Reasonable theories lose to validators:

anthropic.BadRequestError: Error code: 400 - {'type': 'error', 'error':
{'type': 'invalid_request_error', 'message': 'messages.2: tool_use ids were
found without tool_result blocks immediately after: toolu_01A09q90qw90lq91.
Each tool_use block must have a corresponding tool_result block in the next
message.'}}

Ordering inside the content array is load bearing. Every tool_result block must come first, and any text you want to add goes after all of them. Put text first and the API reads it as the turn ending early. Cost me about ninety minutes, mostly because the message names a block id and I went looking for a bug in id propagation rather than in list order.

Second failure is provider specific and hits OpenAI style APIs, where arguments arrive as a string you must parse. Set max_tokens too low and generation stops mid argument:

# tested with openai 2.46.0, Python 3.11.9
from openai import OpenAI
import json

client = OpenAI()  # reads OPENAI_API_KEY from the environment

r = client.chat.completions.create(
    model="gpt-4.1",
    max_tokens=16,
    tools=[{"type": "function", "function": {
        "name": "get_ticket_status",
        "description": TICKET_TOOL["description"],
        "parameters": TICKET_TOOL["input_schema"],
    }}],
    messages=[{"role": "user", "content": "Any movement on SUP-4412?"}],
)
call = r.choices[0].message.tool_calls[0]
print(r.choices[0].finish_reason)
args = json.loads(call.function.arguments)

length
Traceback (most recent call last):
  File "assistant.py", line 22, in <module>
    args = json.loads(call.function.arguments)
json.decoder.JSONDecodeError: Unterminated string starting at: line 1 column 15 (char 14)

Fix is to check finish_reason before parsing. Anything other than tool_calls means the arguments are incomplete, and json.loads on a truncated object throws rather than returning something partial. Raising a clear error beats letting a JSONDecodeError surface three frames up in a request handler at midnight.

Keep this lookup somewhere your on call rota can find it.

Schema design that reduces wrong calls

Descriptions are where accuracy comes from, and I learned that expensively. My original get_ticket_status description read, in full: Gets ticket status. Six words. With four tools available and 50 tool relevant queries in the eval set, the model picked the wrong tool or skipped tool use entirely on 11 of them, a 22 percent error rate. My instinct was to blame the model and consider a bigger one. Instead I rewrote every description to four sentences covering what the tool does, when to reach for it, what each argument means, and crucially what the tool does not return. Same model, same schema shapes. Wrong calls dropped to 2 of 50, so 4 percent. That rewrite took an afternoon and outperformed every prompt change I tried in the following fortnight.

Negative space in a description carries most of that gain. Stating that the tool does not return comments is what stops the model calling it when a user asks what the customer said. Anthropic makes the same recommendation in its tool definition guidance, suggesting three to four sentences minimum, and my numbers say the advice is understated rather than generous.

Now for a place I disagree with common practice. Teams routinely reach for forced tool use, setting tool_choice to any so that a call is guaranteed. It feels like a reliability win and it is the opposite. Forcing a call prefills the assistant turn, which means the model cannot emit natural language first, which means it cannot ask which ticket you meant. On my eval slice, forcing the call on ambiguous queries produced a plausible invented ticket key roughly one time in six. Leave tool_choice on auto, write a description good enough that auto picks correctly, and accept a clarifying question as the right answer to an ambiguous request. Forced tool use belongs in a narrow place: extraction endpoints where the input is guaranteed to contain the entity and a clarifying question would be a bug.

Parallel calls and what they cost in wall clock time

A single turn can contain several tool_use blocks when the model decides two lookups are independent. Loop code above already handles that, because it iterates over every block and puts all results into one user message. Executing them concurrently is your job, and it is the difference between a snappy assistant and a sluggish one.

Read those bars carefully, because they set your design constraint. Adding a tool costs roughly 1.1 seconds of user visible latency, and it costs that once per additional round trip, not once per tool executed. Two lookups the model requests together add 0.1 seconds over one. Two lookups it must make in sequence, because the second depends on the first, add 1.2 seconds. Designing a schema so that arguments can be gathered in one turn is therefore a latency decision as much as an accuracy one, and it is the single most effective thing I have done for perceived speed in an assistant.

Ship one tool with a four sentence description before you ship five

Recommendation for this part is narrow on purpose. Pick the single question your users ask that retrieval provably cannot answer, expose exactly one tool for it, and spend an hour on its description rather than fifteen minutes on its schema. Keep tool_choice on auto. Write the loop yourself, cap it at six steps, and always return a result block even when the call blew up. That configuration got my documentation assistant answering live state questions correctly on 13 of the 14 queries it previously had no hope on, with one added round trip of latency and no framework.

Monday action: open your existing tool definitions and count the sentences in each description. If any is under three, rewrite it to say what the tool does not return, then rerun your eval set before changing anything else.

One tool and a bounded loop is not an agent, and Part 15 argues that many teams reaching for an agent should stop at exactly this point. Prompt discipline from the GenAI Series piece on prompting applies to tool descriptions more than most people expect, and the packaging habits in the Data Science Series part on turning a notebook into a package are what keep a growing tool registry testable.

References

• Define tools, Claude Platform Docs, for the name, description and input_schema fields, the tool_choice options and the description length guidance quoted above.
• Handle tool calls, Claude Platform Docs, for tool_use_id matching, the is_error flag and the ordering rule that produces the 400 shown in this part.
• Function calling, OpenAI API docs, for the function envelope, tool_calls, the arguments string and strict mode defaults on Chat Completions.
• openai-python releases and anthropic-sdk-python releases, for the SDK versions the code in this part was tested against.

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts