One line from a log review last April: {'ticket_id': 'N/A', 'severity': 'high', 'customer_impact': 'unknown'}. Schema valid. Types correct. Every required field present. Our extraction dashboard had shown zero validation failures for nineteen consecutive days, and that record was wrong in every way that mattered to the people reading it. Constrained decoding had guaranteed the shape of the answer and absolutely nothing about its truth.

Where the assistant stands

Last part our documentation assistant answered support questions from a layered, cached prompt with refusal examples baked in. Prose answers, read by humans. This part we make it emit machine readable records, because within a fortnight the product team stopped asking for better answers and started asking for a field: which product area is this ticket about, how severe is it, and can we count them by week. Prose cannot be counted. That single request is what turns a chat feature into a data producer, and it is where most teams meet their first silent failure.

My corpus for this part is the same public one I have used since Part 3: the Markdown under kubernetes/website, standing in for product documentation, plus 500 synthetic support threads I generated over it so the numbers below can be reproduced without anyone’s private data.

Three ways to get JSON, and what each guarantees

Asking politely is where everyone starts. You write "respond only with JSON matching this shape" and most of the time it works. JSON object mode is a step up: the provider constrains the decoder so that whatever comes out is syntactically valid JSON, though its keys and types are still a suggestion. Strict JSON schema output is the third rung, where the provider compiles your schema into a grammar and the sampler is physically unable to emit a token that would violate it.

I ran all three over the same 500 threads with the same prompt text, and classified every failure into one of three buckets: unparseable output, output that parsed but broke the schema, and output that parsed and matched the schema while being factually wrong about the thread it was reading.

Look at the black bars. That is the whole argument of this part. Vendor documentation frames structured outputs as the answer to unreliable model output, and it is genuinely excellent at what it claims: no more parse errors, no more missing fields, no retries for schema violations. But a hallucinated value that fits your type annotation sails through untouched, and it is more dangerous than malformed JSON precisely because malformed JSON fails loudly at 3pm while 'N/A' in a string field poisons a dashboard quietly for a month. If you want the mechanics of why models fabricate confident values in the first place, hallucination and temperature covers it.

Constrained decoding on OpenAI and Claude

Both major providers now let you hand over a Pydantic model directly and get a typed object back. Here is the OpenAI side, using the extraction schema our assistant needs.

# tested with openai 2.46.0, pydantic 2.13.4, Python 3.12
import os
from openai import OpenAI
from pydantic import BaseModel, Field

client = OpenAI(api_key=os.environ['OPENAI_API_KEY'])  # never hardcode a key

class TicketExtraction(BaseModel):
    ticket_id: str = Field(description='Support ticket id, format SUP-00000')
    product_area: str
    severity: str
    customer_impact: str

completion = client.chat.completions.parse(
    model='gpt-4o-2024-08-06',
    messages=[
        {'role': 'system', 'content': 'Extract ticket fields from the support thread.'},
        {'role': 'user', 'content': thread_text},
    ],
    response_format=TicketExtraction,
)

choice = completion.choices[0]
print(choice.finish_reason, choice.message.refusal)
print(choice.message.parsed)

stop None
ticket_id='SUP-04812' product_area='ingress-controller' severity='high'
customer_impact='TLS termination failing across three production namespaces'

Claude reached general availability with the same idea and a different parameter name. On the wire the field is output_config.format with type: json_schema; the Python SDK also accepts output_format as a convenience and translates it internally. Earlier code you may find in the wild uses a top level output_format plus the beta header structured-outputs-2025-11-13, which still works during a transition period but is no longer needed.

# tested with anthropic 0.117.0, same TicketExtraction model as above
import os
from anthropic import Anthropic

client = Anthropic(api_key=os.environ['ANTHROPIC_API_KEY'])

response = client.messages.parse(
    model='claude-opus-4-8',
    max_tokens=1024,
    messages=[{'role': 'user', 'content': thread_text}],
    output_format=TicketExtraction,
)

print(response.stop_reason)
print(response.parsed_output)

# ---- output ----
# end_turn
# ticket_id='SUP-04812' product_area='ingress-controller' severity='high'
# customer_impact='TLS termination failing across three production namespaces'

Google’s Gemini API exposes the same capability through a response schema on the generation config, and Azure OpenAI mirrors the OpenAI shape with a deployment name instead of a model name. Vendor specifics live in the cloud series on this site; what matters here is that the concept is portable and only the parameter names are not.

Schema rules that surprise people

Hand-written schemas fail immediately, and the failures are worth seeing because they teach the rules faster than any prose. Here is a schema written the way a normal API designer would write one, with an optional notes field.

completion = client.chat.completions.create(
    model='gpt-4o-2024-08-06',
    messages=[{'role': 'user', 'content': thread_text}],
    response_format={
        'type': 'json_schema',
        'json_schema': {
            'name': 'ticket_extraction',
            'strict': True,
            'schema': {
                'type': 'object',
                'properties': {
                    'ticket_id': {'type': 'string'},
                    'severity': {'type': 'string'},
                    'notes': {'type': 'string'},
                },
                'required': ['ticket_id', 'severity'],
            },
        },
    },
)

# ---- what actually happens ----
# Traceback (most recent call last):
#   File 'extract.py', line 14, in <module>
#     completion = client.chat.completions.create(
# openai.BadRequestError: Error code: 400 - {'error': {'message':
#   "Invalid schema for response_format 'ticket_extraction': In context=(),
#    'additionalProperties' is required to be supplied and to be false.",
#   'type': 'invalid_request_error', 'param': 'response_format'}}

Add 'additionalProperties': False, re-run, and you get a second rejection that is more interesting:

openai.BadRequestError: Error code: 400 - {'error': {'message':
  "Invalid schema for response_format 'ticket_extraction': In context=(),
   'required' is required to be supplied and to be an array including
   every key in properties. Missing 'notes'.",
  'type': 'invalid_request_error', 'param': 'response_format'}}

Fix is 'type': ['string', 'null'] on notes and its name added to required. Every key present, absence expressed as an explicit null. This feels backwards to anyone who has designed a REST payload, and it is one of the places where instinct actively hurts you. Optional parameters are also the single most expensive thing in a compiled grammar: Anthropic caps a request at 24 optional parameters across all strict schemas and 16 parameters using union types, because each one roughly doubles a portion of the grammar state space. There is also a limit of 20 strict tools per request and a hard 180 second compilation timeout. Exceed any of it and you get a 400 reading Schema is too complex for compilation.

Validation layer that catches what the grammar cannot

Now the story behind that log line. We shipped ticket extraction in March with ticket_id typed as a plain string, because the schema rules above had already forced me to drop the regex pattern I wanted and I told myself we would add it back later. Extraction ran nightly over the support archive. Dashboard showed a 100 percent success rate for nineteen days, which should have been my first suspicion rather than my comfort. Then a support lead asked why the weekly rollup showed 61 tickets when her own queue had over 400, and the join finally made sense: 11 percent of extracted records carried ticket_id values like 'N/A', 'unknown' or 'SUP-XXXXX', every one of them a perfectly valid string, every one of them joining to nothing. Three weeks of nightly runs, roughly 40 hours of recomputation to repair, and the actual fix was 30 lines of Pydantic.

Architecture I settled on splits the schema in two. A wire model, flat and permissive, is what the grammar compiles, so it stays cheap and never hits a complexity limit. A validated model, with real business rules, runs immediately afterwards in your own process where you can express anything Python can express.

import re
from pydantic import BaseModel, ValidationError, field_validator, model_validator

TICKET_RE = re.compile(r'^SUP-d{5}$')
SEVERITIES = {'low', 'medium', 'high', 'critical'}

class ValidatedTicket(BaseModel):
    ticket_id: str
    severity: str
    customer_impact: str
    source_quote: str          # verbatim sentence the model relied on

    @field_validator('ticket_id')
    @classmethod
    def real_identifier(cls, v: str) -> str:
        if not TICKET_RE.match(v):
            raise ValueError(f'ticket_id {v!r} is not a real identifier')
        return v

    @field_validator('severity')
    @classmethod
    def known_severity(cls, v: str) -> str:
        low = v.strip().lower()          # capitalisation is not guaranteed
        if low not in SEVERITIES:
            raise ValueError(f'severity {v!r} outside the allowed set')
        return low

    @model_validator(mode='after')
    def quote_is_grounded(self, info):
        thread = (info.context or {}).get('thread', '')
        if self.source_quote not in thread:
            raise ValueError('source_quote not present verbatim in the thread')
        return self

record = ValidatedTicket.model_validate(
    response.parsed_output.model_dump(), context={'thread': thread_text}
)

pydantic_core._pydantic_core.ValidationError: 1 validation error for ValidatedTicket
ticket_id
  Value error, ticket_id 'N/A' is not a real identifier
  [type=value_error, input_value='N/A', input_type=str]

That source_quote field is the cheapest quality lever in this entire part. Forcing the model to return a verbatim sentence from the thread, then checking membership with a Python in, caught 31 of those 54 records on its own. It costs perhaps 40 output tokens per call and needs no judge model, no embedding comparison and no second request. Failed records go to a quarantine table with the raw response attached rather than being silently dropped, because a quarantine you can query is how you find out that your prompt regressed.

Failures you will actually hit

Most damaging one is not an exception at all, it is a None. Both providers document that a truncated or refused response will not match your schema, and their SDK helpers respond by handing you nothing.

response = client.messages.parse(
    model='claude-opus-4-8',
    max_tokens=64,                      # deliberately too small
    messages=[{'role': 'user', 'content': long_thread}],
    output_format=TicketExtraction,
)
print(response.stop_reason)
print(response.parsed_output.ticket_id)

# ---- output ----
# max_tokens
# Traceback (most recent call last):
#   File 'extract.py', line 21, in <module>
#     print(response.parsed_output.ticket_id)
#           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# AttributeError: 'NoneType' object has no attribute 'ticket_id'

Keep this table near your extraction code. It is the lookup I wish someone had handed me in March.

Cost and latency of a schema

Structured output is not free, and the bill arrives in three places. First, a schema the platform has not seen recently must be compiled into a grammar, and that compilation lands on your first request. On my extraction schema the cold call ran a shade over 2 seconds against roughly 0.7 seconds warm, which is invisible in a batch job and very visible on a user facing endpoint. Compiled grammars are cached for 24 hours from last use, so a low traffic endpoint pays this repeatedly.

Second, enabling structured output injects an additional system prompt describing the expected format, so your input token count rises and you pay for those tokens like any others. Third, and this is the one that actually cost me money, changing the schema invalidates the prompt cache for that conversation thread, alongside invalidating the grammar cache. If you followed the caching layout from Part 3 and then start generating schemas dynamically per request, you will quietly undo the saving. Treat a schema as a versioned artifact that ships with a release, the same way you would treat a database migration.

Make every field required and validate meaning separately

Recommendation for this part, stated plainly: turn on strict JSON schema output everywhere you consume model output programmatically, make every field required with nullable unions for genuine absence, and never treat the parsed object as validated data. Avoid plain JSON object mode; it buys you syntax and leaves 12 percent of records broken in ways that look fine. Avoid hand-written raw schemas too, unless you enjoy 400 errors; generate them from Pydantic and let the SDK apply the transformations.

One thing to do on Monday: open your extraction code, find the field that downstream systems join on, and write a validator that rejects anything failing its real format. Then add a source_quote field and a membership check. If your dashboard has been reporting a 100 percent success rate, those two changes will tell you what it has actually been measuring. Structuring your project so validators and schemas live in a proper module rather than a notebook cell is covered in notebook to Python package.

Part 5 picks up the meter that has been running underneath all of this: token budgets, context limits and what a single request genuinely costs, including the extra input tokens that structured output quietly added in this part.

References

• Claude Platform docs, Structured outputs, for output_config.format, the messages.parse helper, the 20 strict tool, 24 optional parameter and 16 union parameter limits, the 180 second compilation timeout, the 24 hour grammar cache, property ordering and the enum capitalisation caveat.
• OpenAI, Structured model outputs, for the json_schema response format, the strict flag, chat.completions.parse and responses.parse, the refusal field, and the requirement that every key appear in required.
• Pydantic docs, Validators, for field_validator and model_validator modes and validation context.
• kubernetes/website, the public Markdown corpus standing in for product documentation, so the extraction runs in this part can be reproduced without private data.

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts