Download App
Ready to become a better product manager?
Build your product skills with short, practical lessons you can fit around your day.
Start with our free courses and upgrade anytime.
Structured outputs are schema-constrained model responses that downstream software can treat as an application contract: the provider or runtime constrains generation to a declared JSON Schema (shape, types, enums, required fields) and the application validates, handles refusal and no-data paths, and versions the contract. A schema guarantees structure, never truth, a perfectly valid object can still carry wrong values. Use native response-schema or strict tool-argument features for machine-readable shape; use validation, explicit unknown and refusal states, bounded retries, fallbacks, field-level evals, and compatibility handling for everything the schema cannot prove.
The PM owns the output contract: the minimum schema that supports the Product decision and UI, semantic meaning and null semantics per field, closed versus open value sets, refusal and no-data behavior, validation layers, bounded retry and fallback policy, observability (parse failures, refusal and null rates, enum drift, retry and fallback rates), field-level eval ownership, and version and compatibility discipline. Engineering owns schema implementation, parser and business-rule validation, retry and fallback mechanics, and telemetry; trust and safety own refusal and policy requirements; design owns how unknown, refused, and partial states render. The PM decides when free text is the right answer instead, and refuses the comforting fiction that a valid schema means correct content.
Start from the Product decision and UI states, then declare the smallest schema that supports them: named fields with semantic definitions, required versus nullable status, enums only for real closed sets, bounded arrays, stable machine keys, an explicit unknown or not-applicable state, and a refusal path that is not the normal schema. Constrain generation with the provider's native response-schema feature (OpenAI strict JSON Schema via text.format, Anthropic output_config.format JSON Schema, Gemini response_format JSON Schema, exact subset, streaming, and refusal behavior are provider and model specific and must be verified on the deployed combination), then validate in layers: parse and type check, semantic check, business-rule check. Define a reason-aware retry contract (never blindly retry refusals), a fallback per failure mode, per-field accuracy evals owned by the evaluation harness, structural telemetry without sensitive values, and explicit versioning with additive-first changes, consumer compatibility, migration, and rollback.
Synthetic example, hypothetical scenario, not measured results. A support-triage flow returns issue_type, urgency, account_action, evidence, and needs_human_review. One response is schema-valid but urgency is wrong for the cited evidence; a second has no usable evidence, so evidence carries an explicit unknown state and needs_human_review routes to a person instead of hallucinating a confident urgency. Validation catches the shape; a field-level eval catches the wrong urgency; the null and review rates tell the PM the schema or prompt needs work, none of which the schema alone could prove.
This is the operating worksheet for the page: ten steps that turn “ask for JSON” into an inspectable Product contract. Work it top to bottom for one use case; statuses use text labels, never color alone.
Deterministic by design: no model call is needed to run the lab. Execution and grading of field-level quality live with the evaluation harness, not inside this worksheet.
Run any draft schema through these eight flags before it ships. Each flag names the smell, the Product risk, and the fix, in text.
| Flag | What it smells like | Fix |
|---|---|---|
| Ambiguous field name | A field like confidence with no stated meaning, source, or allowed interpretation. | Write the semantic contract per field, or split the field. |
| Optional-field overload | Most fields optional with no missing-data behavior, so consumers guess. | Mark required vs nullable vs optional explicitly; define what each absence means. |
| Missing null and no-answer path | Non-null everything, forcing hallucinated filler when evidence is thin. | Add explicit unknown / not-applicable / refused states. |
| Free-form blob hidden inside structure | An analysis string presented as if the schema made it structured. | Separate machine-controlled, evidence, and human-readable fields explicitly. |
| Unbounded array | A dump-everything list with no empty, max-items, order, duplicate, or partial-result semantics. | Bound arrays: emptiness, limits, ordering, duplicates, item IDs, partial acceptance. |
| Policy encoded as enum | An enum that bakes in Product policy with no other path and no version plan. | Keep enums to real closed sets; define other semantics; version deliberately. |
| Missing version | No schema version, so renames and enum changes break consumers silently. | Add an explicit version with compatibility, migration, and rollback. |
| Unsafe UI assumption | Frontend renders schema-valid URLs, markdown, amounts, or recommendations without checks. | Apply normal escaping, authorization, and business validation after parsing. |
Rows are failure modes; columns are the Product response. Fill one row per mode your use case faces, a blank cell is an unowned failure.
| Failure mode | Detect | User behavior | Retry | Fallback | Telemetry |
|---|---|---|---|---|---|
| Invalid syntax | Parser rejects output. | Safe error or retry state; never render raw. | Once on transient cause only. | Simpler schema or ask user. | Parse-failure rate. |
| Schema mismatch | Type or required-field check fails. | Safe error state. | Once after prompt or schema fix, not blindly. | Human review or deferred action. | Validation-failure rate by field. |
| Refusal | Refusal field, finish reason, or safety block. | Safe explanation plus next step. | Never blindly retry. | Human review or safe free-text path. | Refusal rate. |
| Safety block | Content-filter or policy signal. | Safe error state. | Never retry to evade. | Escalate per policy. | Block rate by category. |
| Missing data | Explicit unknown state. | Ask user or show unknown honestly. | No, retry cannot invent evidence. | Human review or partial render. | Null rate by field. |
| Semantically wrong value | Field-level eval or business rule flags it. | Correct or escalate before acting. | Only with changed input or model. | Human review; block writes. | Field accuracy + business impact. |
| Unsupported enum value | Enum check fails or other overused. | Safe error or review state. | Once on transient cause. | Route to review; revisit enum design. | Other rate + distribution drift. |
| Truncated output | Length or completeness check fails. | Loading-to-error transition. | Once with adjusted limits. | Simpler schema or deferred action. | Truncation rate. |
| Version mismatch | Consumer rejects unknown version or key. | Safe error state. | No, retry cannot fix contract drift. | Pin prior version; migrate. | Version adoption + breakage. |
The matrix is a menu, not a mandate: keep the rows your use case faces, drop the rest, and record the decisions on the contract, not as tribal knowledge.
Hypothetical scenario with illustrative structure, not measured results. Schema: issue_type, urgency, account_action, evidence, needs_human_review. One response marks urgency high while its own evidence describes a cosmetic question, shape passes, meaning fails. A second response has no usable evidence, so evidence carries an explicit unknown state and needs_human_review routes to a person instead of inventing a confident urgency. The lesson: validation owns the shape, a field-level eval owns the urgency judgment, and the human-review guardrail owns the consequence, three different contracts, not one.
Hypothetical scenario with illustrative structure, not measured results. Schema v1: audience, change_type (enum: feature, fix, docs), breaking_change (boolean), user_action (nullable, null means no action required), summary. The enum stays a real closed set instead of absorbing policy; user_action nullability distinguishes “no action” from “unknown”; adding a nullable migration_guide_url later is backward-compatible because existing consumers ignore it. A field-level eval checks breaking_change accuracy separately from summary quality, one score never covers both.
Hypothetical scenario with illustrative structure, not measured results. The normal schema expects answer, evidence, risk. An unsafe request triggers the provider's refusal or safety behavior instead, a refusal field, finish reason, safety block, or error, depending on provider and runtime. The application handles the non-normal path explicitly: a safe user-facing state, no hallucinated answer object, no blind retry, and a logged refusal event. Expected Product lesson: every schema ships with two shapes, the normal result and the refusal, and only one of them carries the business data.
Hypothetical scenario with illustrative structure, not measured results. v1 carries priority: high | medium | low; v2 carries severity: critical | major | minor. This is not a rename, downstream triage, SLAs, and analytics keyed on the old semantics change meaning. The migration plan pins v1 consumers, maps old values to new semantics explicitly (including what has no mapping), dual-validates during transition, and rolls back to v1 on breakage. Provider enforcement of the new schema is only one layer; consumer compatibility is the rest.
Hypothetical scenario with illustrative structure, not measured results. Output: eligible: true with reason: 'Account is older than 30 days', schema passes, but the source account is 12 days old. Expected lesson, stated verbatim on the contract: structured validity does not prove factual correctness. Only a semantic check against the source record (or a field-level eval with ground truth) catches this class, never the parser.
A schema should encode what downstream Product behavior actually needs, the minimum contract that supports the decision and UI. More fields mean more generation complexity, more null and placeholder behavior, more versioning burden, more downstream coupling, and more eval burden. Smaller is not always better; minimal-for-the-decision is.
Type alone never resolves meaning: a field named confidence could be model self-confidence, evidence strength, classifier probability, business confidence, or a user-entered value. For important fields record meaning, source, allowed interpretation, and null semantics, alongside the type, not inside it.
Required means present; optional means possibly absent; nullable means explicitly no value. Because providers implement optionality differently, some require every property listed as required with null unions expressing absence, verify the semantics on your deployed combination and document the provider-specific mapping as implementation detail, never as the concept itself.
Unknown (looked, found nothing), not applicable (field does not apply here), not observed (source never consulted), refused by policy, and generation failure each drive different fallbacks. Do not collapse them into one empty string when the Product depends on the distinction, and do not force hallucinated values merely to satisfy a non-null schema.
Use enums for real closed sets, define unknown and other semantics where the Product needs them, and version enums deliberately, never hundreds of values encoding volatile policy. Bound arrays (empty? max items? order meaningful? duplicates? item IDs? partial results acceptable?) and verify minItems and maxItems support per provider. Model domain structure with shallow nesting, deep nesting multiplies schema complexity, UI coupling, migration burden, and debugging cost. And when code joins on labels, use stable machine keys with display text rendered separately, never generated natural-language labels as durable database IDs.
A schema may legitimately carry an analysis string, but that does not make the analysis structured. Distinguish machine-controlled fields from evidence and source fields and from human-readable explanation fields, and evaluate each class on its own terms.
Provider or generation constraint (schema-constrained decoding); parser and type validation (can the app parse it?); semantic validation (do the values make sense?); business-rule validation (is it safe to act on?). Never collapse these, each catches a failure class the others cannot see.
Where native schema enforcement guarantees syntax and shape, repair for those failure modes is unnecessary. Repair formatting only when safe and observable, never silently rewrite business-critical values, log every recovery path, and prefer explicit retry or error where the value, not the whitespace, is in doubt.
Retry decisions depend on failure type: transient API errors and provably transient invalid outputs may retry once; refusals, unsupported requests, schema-version mismatches, and semantic errors must not loop identically. A safety refusal retried until it disappears is an incident, not resilience, and a semantically wrong value rarely fixes itself on an identical second roll.
Candidate fallbacks: safe error state, ask the user for missing data, simpler schema, human review, free-text result, another model or provider, deferred action. A fallback model is not always the answer, the fallback must preserve the Product semantics and safety properties the primary path promised, especially before any durable write.
Never assume fields are valid because the schema parsed: URLs may be untrusted, recommendations unsupported, amounts wrong, markdown unsafe. Apply normal escaping, authorization, and business validation, structured outputs are not a security boundary, and input-side attacks such as prompt injection need their own controls. And never let typed output write directly to durable systems on shape alone: validate, authorize, confirm identity, enforce business rules, and require human approval for high-impact writes. Action safety detail lives with tool and agent owners.
Monitor beyond parse success: parse and validation failure rate, refusal rate, null rate by field, unknown and other rates, enum distribution, retry and fallback rates, schema-version adoption, latency and cost, and semantic accuracy by field. Never send sensitive field values to analytics, structural counts and generic warning codes only.
Evaluate each critical field separately: a response can be valid JSON while its category is wrong, its evidence unsupported, and its review flag false under high uncertainty. High null, unknown, or other rates are Product signal, insufficient source data, schema mismatch, weak prompt, bad enum design, or population mismatch, not noise to suppress. Route execution and grading to the evaluation harness; deciding what each field must prove lives here.
Providers may enforce schemas through constrained decoding, grammar constraints, or native schema validation, explain only what documented sources support, at PM decision depth, never provider internals as fact. Streaming support is provider and model specific: design for incomplete partial objects, deferred final validation, and mid-stream refusal. Schema complexity can move prompt tokens, latency, output length, and retry rates, state quantitative claims only with current provider evidence, never invented cost numbers.
These adjacent concepts overlap with output contracts. The table states the boundary so this page does not absorb their jobs, each links to its specialist owner.
| Concept | Its job | Boundary |
|---|---|---|
| Prompt template | Reusable input and instruction rendering with variables | Templates own input structure, variables, and escaping; this page owns the output contract, schema, validation, and compatibility. |
| Few-shot examples | Demonstrations that teach field semantics and content quality | Examples teach meaning inside the shape; schemas enforce the shape. Never use prose demonstrations as a shape guarantee. |
| System prompt | Persistent instruction and policy layer | System text states constraints; response-layer enforcement plus application validation implements them. Keep giant schemas out of system text when native features exist. |
| Tool instructions | When and how the model may call tools | Typed data for the app to consume lives here; selecting or invoking an action lives there. Shared schema syntax does not make them one decision. |
| Tool calling | Model-driven invocation of functions and APIs | Output contracts return data; tool calls request actions with side effects, permissions, and approvals. See also function calling for the interface form. |
| Model Context Protocol (MCP) | Standard interface for exposing tools and resources to models | MCP governs how tools and resources are connected and exposed; the typed output contract for what the model returns lives here. |
| Evaluation harness | Field-level correctness, semantic quality, regression detection | The harness runs and grades field accuracy, refusal correctness, and business metrics; deciding what each field must prove lives here. |
| Guardrails | Policy enforcement, safety checks, and escalation | A schema is not a safety control: refusal handling, human-review routing, and write authorization live with guardrails and policy owners. |
| Prompt injection | Input-side manipulation of model behavior through untrusted content | Injection defenses protect what goes into the model; a schema constrains only what comes back. Structured validity is not an input-trust or security boundary. |
Function calling has its own glossary owner alongside tool calling: both describe action interfaces, neither replaces the output-data contract owned here. Prompt engineering is the parent discipline, method and technique selection live there; machine contracts live here.
Structured outputs are schema-constrained model responses that downstream software can treat as an application contract: generation is constrained to a declared JSON Schema (shape, types, enums, required fields), and the application validates the result, handles refusal and no-data paths explicitly, and versions the contract. The schema guarantees structure, never truth, so validation, fallbacks, field-level evals, and compatibility handling are part of the concept, not optional extras.
No. JSON mode typically guarantees syntactically valid JSON without enforcing a full declared schema; structured-output features constrain generation to the schema you declare, required keys, types, enums. Exact capabilities, supported schema subsets, and model compatibility are provider and model specific, so verify on your deployed combination rather than treating one provider's distinction as universal.
A structured output returns typed data for the application to consume, render, or store; function or tool calling asks the model to select or request an action with typed arguments. They can share JSON Schema-like contracts, but they are different Product decisions: output contracts live here, action semantics, permissions, authorization, side effects, live with tool instructions and agent tooling.
No, this is the central rule of the page. A schema enforces shape, types, enums, and required fields; it cannot prove factual correctness, semantic correctness, evidence support, correct classification, or safe business action. A schema-valid refund_eligible flag can still be wrong, which is why semantic validation, business-rule checks, and field-level evals exist.
Depending on provider and runtime, you may receive a structured refusal field, a finish reason, a safety block, a content-filter signal, an exception, or simply no normal schema output. Application code must handle a non-normal result path explicitly, render a safe state, route to review, or ask the user, and must never blindly retry a safety refusal until it disappears.
A required field must be present; an optional field may be absent; a nullable field can be present with an explicit no-value state. The distinction matters because absence and explicit unknown drive different Product behavior, and because providers implement optionality differently (some require every property listed as required with null unions for absence), so treat any single provider's workaround as implementation detail, not the concept.
Decide by what the consumer needs to distinguish: explicit null (or an unknown enum value) says the model looked and found nothing, while an omitted optional field says the field does not apply to this response. Never use an empty string as a universal absence state, and never collapse unknown, not applicable, refused, and generation failure into one value when the fallback logic depends on telling them apart.
Treat the schema as an API-like contract: additive changes (a new nullable or optional field consumers ignore) are usually safe; renames, enum value changes, meaning changes without key changes, nullability tightening, and array-ordering changes can break UI, storage, analytics, automations, and evals. Ship an explicit version, check consumer compatibility, plan migration and rollback, provider enforcement is only one layer.
Support exists on current providers but is provider and model specific, verify on your deployed combination. Product issues to design for: partial objects are incomplete until the stream finishes, final validation happens at completion, incremental UI rendering must tolerate missing fields, and refusal or termination can arrive mid-stream.
Yes, at three layers beyond the generation constraint: parser and type validation (can the app parse it?), semantic validation (do the values make sense?), and business-rule validation (is it safe to act on?). Structured outputs shrink the first failure class; they do not remove the other two, and auto-repair should touch formatting only when safe and observable, never silently rewrite business-critical values.
Yes. A valid schema is not a passing quality eval: each critical field needs its own accuracy measurement, plus refusal correctness, null and other-rate monitoring, enum distribution, and regression detection. Route execution and grading infrastructure to the evaluation harness; deciding what each field must prove lives in the schema contract.
When the Product value is open-ended explanation, exploration, or persuasion rather than a downstream decision, storage write, or automation trigger, or when forcing structure would manufacture false precision (hallucinated values to satisfy non-null fields). A schema can still carry a human-readable explanation field alongside machine-controlled fields; do not pretend that makes the prose itself structured.
For reusable input structure, see prompt template; for teaching field meaning with demonstrations, see few-shot examples; and for proving field quality, see evaluation harness. For hands-on prompting workflows, continue to Prompt engineering for PMs , that guide owns tutorial workflows; this page owns the entity reference. For the broader AI product picture, see the AI Product Management hub.
Build your product skills with short, practical lessons you can fit around your day.
Start with our free courses and upgrade anytime.