Structured outputs

• You need deterministic fields for UI rendering, routing, or scoring.
• Multiple services consume the model output and contract stability matters.
• You’re measuring quality over time and need consistent keys for metrics.

Structured outputs reduce QA toil and make evals cheaper, but can increase refusal rate and latency if the schema is complex. PMs trade expressiveness for reliability: the stricter the schema, the more you must handle fallbacks and validation. It also affects how quickly engineers can integrate new use cases.

Define a minimal schema with strong typing, examples, and short field descriptions. Use function calling or response format features when available. Add runtime validators that auto-repair minor issues and log failures to your eval harness. In 2026, keep per-field quality dashboards (accuracy, null rate) and roll back schema changes that hurt downstream metrics.

A release-notes summarizer returns JSON with fields {audience, tone, risk_callout, CTA}. Validation catches 3% malformed responses; auto-repair trims it to 0.4%, keeping latency at 850 ms and support tickets flat after launch.

• Designing verbose schemas that exceed token budgets and slow responses.
• Skipping runtime validation and silently accepting malformed payloads.
• Changing field names without versioning, breaking dependent services.