Why Are Your AI Outputs Inconsistent in the First Place?
If your AI workflow keeps breaking because the output format drifts between runs, you are not doing anything wrong with your prompt. You are using the wrong tool for a structured-data problem. Large language models generate text token by token, and "format" is just a pattern they imitate. Without a hard constraint, the same prompt produces JSON one day and prose the next.
Practitioners notice this most when they try to chain steps. The first AI step generates a list of items. The second step is supposed to ingest those items and act on them. Then one day a stray comma, a renamed key, or a hallucinated field breaks the chain. According to a 2026 DEV Community technical report on LLM structured output, parsing AI text with regex is the root cause of nearly every reliability incident in production AI workflows.
The fix is not better prompting. It is constraining the output at the model layer. This is what JSON Schema mode, also called Structured Outputs, was built to do. Once you switch it on, the model is mathematically prevented from generating tokens that would break the schema.
What Are Structured Outputs and How Do They Actually Work?
Structured Outputs is a feature that ensures the model always generates responses that exactly match a JSON Schema you supply. The model cannot omit a required field. It cannot invent a key not in the schema. It cannot return a string when you asked for a number. The format is enforced at inference time through constrained decoding.
Constrained decoding is the technical mechanism. At each token-generation step, the runtime checks which tokens are valid given the schema. Invalid tokens are masked out. The model then samples only from the valid set. This is why structured outputs are not "the model trying harder." The model literally cannot produce schema-violating output.
According to OpenAI's August 2024 announcement, GPT-4o-2024-08-06 with Structured Outputs scores a perfect 100% on complex JSON schema following evaluations, compared to roughly 40% without the feature. The same approach is now standard across providers, with each implementation having a different name but the same underlying mechanism.
The headline benefit is that you can now treat AI output as a typed object, not a string to parse. Your downstream code receives a JSON object whose shape you control. This is the difference between a workflow that runs reliably for six months and one that breaks every Tuesday.
How Does Each Major AI Provider Support Structured Outputs?
The three major providers each support structured outputs but call them slightly different things. The mechanism is similar enough that a single mental model covers all three. Pick your provider based on which model you already use, not on this feature alone.
OpenAI GPT-4o and later. Activated by setting the response_format parameter to json_schema with your schema attached. Available on GPT-4o, GPT-4o-mini, and all subsequent models. The Python SDK has first-class Pydantic support, which means you write a Pydantic class and pass it directly. According to the OpenAI cookbook published in 2024, this approach replaces both function calling and JSON mode for most extraction use cases.
Google Gemini 1.5 Pro and later. Activated through responseSchema on the generation config. Google announced full JSON Schema support and implicit property ordering across all Gemini models in early 2026. The same Pydantic and Zod definitions used for OpenAI translate directly. According to Google's developer blog, ordered properties are particularly useful for extraction tasks where field sequence matters.
Anthropic Claude. Implemented through tool calling rather than a dedicated parameter. You define a tool whose input_schema is the JSON Schema you want the response to follow, then force the model to call that tool. The output of the tool call is your structured data. The mechanism is different but the reliability outcome is the same.
Try This Schema: A Structured Outputs Template That Actually Works
The single best way to learn structured outputs is to take a real text-extraction task and write the schema for it. Here is a complete schema you can copy-paste into OpenAI's playground or any tool that calls the API. It extracts structured contact information from messy text such as an email signature or a LinkedIn bio.
Try this schema:
{ "name": "extract_contact", "schema": { "type": "object", "properties": { "full_name": { "type": "string", "description": "Full name as written" }, "job_title": { "type": "string", "description": "Current job title" }, "company": { "type": "string", "description": "Current employer name" }, "email": { "type": "string", "description": "Primary email address, lowercase" }, "phone_country_code": { "type": "string", "description": "Country dial code with plus prefix, e.g. +852" }, "phone_number": { "type": "string", "description": "Local number without country code" }, "linkedin_url": { "type": "string", "description": "Full LinkedIn URL or empty string if not present" } }, "required": ["full_name","job_title","company","email","phone_country_code","phone_number","linkedin_url"], "additionalProperties": false }, "strict": true }
Pair it with a prompt as simple as "Extract structured contact data from this signature: [paste signature]." Run it 50 times across different signatures. Every output will be a valid JSON object with exactly those seven fields. Empty fields return an empty string rather than being omitted, which is what required with additionalProperties: false guarantees.
When Should You Use Structured Outputs Instead of Regular Prompting?
Structured outputs are the right tool when AI output feeds directly into automated downstream code, when you need type guarantees, or when multiple systems consume the same response. They are the wrong tool for free-form writing, brainstorming, or any task where the value is in the prose itself.
Use structured outputs when you are extracting data from documents, emails, transcripts, or web pages. The classic example is invoice extraction. Vendor name, total amount, due date, line items, tax breakdown. According to the OpenAI cookbook on structured outputs, this single use case accounts for the majority of production deployments.
Use structured outputs when you are routing or classifying. Customer support tickets being labelled with category, priority, and required action. Lead inquiries being sorted by industry, deal size, and follow-up timeline. The output feeds straight into a CRM or queue, so consistency matters more than expression.
Use structured outputs when you are building AI agents that take actions. Each tool call needs a strict input shape. Without schema enforcement, an agent might pass a string where a number was expected, or omit a field that the next tool needs, and the whole chain stalls.
Do not use structured outputs for first-draft writing, creative ideation, summarisation that humans will read, or any conversational interaction. Forcing JSON onto these tasks ruins the natural language quality and adds friction without a corresponding reliability gain.
What Are the Common Mistakes With JSON Schema Mode?
Three mistakes appear in nearly every team's first attempt at structured outputs. Each one masquerades as a different problem, which is why they keep recurring. Knowing them in advance saves a week of debugging.
The first mistake is making fields optional when they should be required with empty fallbacks. Optional fields can be omitted entirely, which means your downstream code has to handle missing keys. Required fields with documented empty-state values, such as an empty string or zero, are simpler to consume. Use required on every field and define what "no value" looks like in the description.
The second mistake is over-engineering the schema. Deeply nested objects, conditional schemas, and union types all stress the model. The schema should be as flat as your downstream code allows. According to the 2026 DEV Community report, schemas with three or more levels of nesting see noticeably higher failure rates compared to flat designs.
The third mistake is treating schema validity as semantic correctness. The model can return a perfectly valid JSON object that contains the wrong order ID or a hallucinated confidence score. Schema enforcement guarantees structure, not truth. Always add a semantic validation layer afterwards, especially for numeric fields and identifiers.
How Do You Roll Structured Outputs Into Your Real Workflow?
The fastest way to integrate structured outputs is to pick one existing AI workflow that breaks regularly and rebuild it. Most practitioners have at least one of these. The email-to-CRM step that drops phone numbers. The receipt parser that misses tax codes. The meeting notes splitter that loses action items.
Start by writing the schema in plain JSON Schema, before touching any prompt. List every field you actually need downstream, mark them all required, and write a one-line description for each. The discipline of defining the schema first usually reveals that the original prompt was asking for too much.
Run the new structured prompt against ten real inputs. Compare against your old approach by counting how many outputs need manual fixing. The drop is usually dramatic. According to multiple 2026 industry reports, teams that move from prompt-only extraction to JSON Schema mode report failure rates dropping from 15 to 25 percent down to under 1 percent on the same dataset.
Then add the semantic validation. Check that the extracted email is a real email pattern. Check that the date is parseable. Check that the amount is a positive number. The combination of structural and semantic validation is what production AI looks like.
For Hong Kong practitioners, structured outputs is one of the highest-leverage techniques to learn this year. It is the difference between an AI workflow that demos well and one that survives a quarter of real-world use. Once you start thinking in schemas, you stop fighting your prompts. 懂AI，更懂你 UD相伴，AI不冷。
📐 Build AI Workflows That Actually Survive Production
Structured outputs is one piece of the reliability stack. UD's AI engineering team helps Hong Kong organisations design AI workflows that hold up under real-world load, with schema-first thinking, validation layers, and human-in-the-loop checkpoints. We'll walk you through every step, from your first schema to your first agent in production.
