Download App
Ready to become a better product manager?
Build your product skills with short, practical lessons you can fit around your day.
Start with our free courses and upgrade anytime.
Few-shot examples are input-output demonstrations supplied in model context so the model can infer desired task, style, and format patterns without changing model weights. Each exemplar demonstrates one behavior or failure-mode response, a common case, an ambiguous input, missing data, a refusal, a formatting rule, or a tool-use decision, with an explicit input/output boundary and a coverage role. They are runtime behavior controls with token, latency, maintenance, and privacy costs, proven by eval comparison and ablation rather than by count: they demonstrate how to answer, they do not supply fresh facts (retrieval), enforce machine contracts (structured outputs), retrain the model (fine-tuning), or substitute for system-level safety controls.
The PM owns the exemplar-set contract: the target behavior each example teaches, the failure mode it covers, its input/output boundary, its token contribution, its eval contribution, its owner and version, its privacy review, and when it is removed. Engineering owns prompt implementation in code, schema and tool integration, and eval infrastructure; trust and safety own policy and refusal requirements. The PM decides the smallest set that measurably covers needed behaviors, and routes missing-facts problems to retrieval, machine-contract problems to schemas, capability gaps to model or architecture choices, and governance of the resulting assets to the prompt library.
Define the target behavior and the failure modes that matter, then choose the smallest set of candidate exemplars that covers them, preferring one distinct coverage role per example over near-duplicate happy paths. Label each example with its coverage role, keep inputs realistic but anonymized or synthetic, keep outputs correct and format-consistent, and check for contradictions across examples before shipping. Compare quality with and without each example on representative eval cases, ablate exemplars that no longer move results, track token and latency cost per example, and revalidate the set on event-driven triggers, policy, schema, tool, model, or tone changes, rather than on a calendar schedule.
Synthetic example, hypothetical scenario, not measured results. A support copilot answers billing questions without inventing policy. The set carries four exemplars, each with a coverage role: a common refund question (common case), an ambiguous account state (ask-or-escalate instead of guessing), a prohibited promise such as a guaranteed refund (refusal plus redirection), and an incomplete request missing the account identifier (missing-data behavior). A later eval comparison shows the missing-data exemplar changes missing-data handling while a fifth near-duplicate happy-path exemplar changes nothing, so the team keeps the former and removes the latter.
This is the operating worksheet for the page: nine steps that turn “add a few examples” into an inspectable coverage contract. Work it top to bottom for one behavior; statuses use text labels, never color alone.
Deterministic by design: no model call is needed to run the lab. The eval-comparison step uses whatever harness the team already has, execution and grading live with the evaluation harness, not inside this worksheet.
Rows are exemplars; columns are the behaviors a Product team usually needs to cover. Mark each cell teaches, reinforces, or out of scope, in text. A healthy set covers distinct columns with few exemplars; ten checks in the happy-path column and none in refusal or missing-data is the classic failure this matrix exists to expose.
| Exemplar | Common intent | Ambiguous input | Missing data | Refusal / out-of-scope | Formatting | Tool use |
|---|---|---|---|---|---|---|
| Common refund question | teaches | out of scope | out of scope | out of scope | reinforces | out of scope |
| Ambiguous account state | out of scope | teaches | reinforces | out of scope | out of scope | out of scope |
| Prohibited promise | out of scope | out of scope | out of scope | teaches | reinforces | out of scope |
| Incomplete request | out of scope | reinforces | teaches | out of scope | out of scope | out of scope |
Coverage categories are a menu, not a mandate: locale or tone, localization, unsafe requests, long inputs, and multi-turn context join the matrix only when the task faces them. Small models and strict formats often need more formatting coverage; support flows need more refusal and missing-data coverage. Let the failure modes choose the columns.
One card per exemplar, reviewed whenever the set changes. If a reviewer cannot answer each row, the example's place in the set is unproven.
Ablation is practical discipline, not causal proof: it answers “is this example still earning its place on current traffic?”, not a universal claim about the example's truth across all distributions.
Hypothetical scenario with illustrative structure, not measured results. Goal: answer billing questions without inventing policy. Four exemplars, each owning one coverage role:
| Exemplar | Coverage role | What it teaches |
|---|---|---|
| Common refund question | Common case | Answer shape: short grounded answer citing the supplied policy excerpt, no invented details. |
| Ambiguous account state | Ambiguity handling | Ask for the missing distinguishing fact or escalate instead of guessing between two plan states. |
| Prohibited promise | Refusal + redirection | Decline the guaranteed-refund promise, state what is possible, offer a human review path. |
| Incomplete request | Missing-data behavior | Request the account identifier explicitly rather than answering generically. |
Why four near-duplicate happy-path examples would be weaker: they would all land in the first row's column, adding tokens without teaching ambiguity, refusal, or missing-data behavior, the exact failures billing support faces. Testing the set means one eval case per row that fails when its exemplar is removed, plus a leakage check for verbatim overlap with example phrasing and entity reuse.
Hypothetical scenario with illustrative structure, not measured results. Goal: extract problem, evidence, non-goals, and open questions from PRD prose into four fields. The boundary is clean: the schema enforces the fields, names, types, required status, while examples clarify what counts as evidence versus assumption. One exemplar shows a metrics claim with a cited source mapped to evidence; a second shows an identical-sounding claim with no source mapped to assumption plus an open question. Without the schema, examples alone cannot guarantee machine-readable shape; without the examples, the schema cannot teach semantic judgment. Route shape guarantees to structured outputs; keep meaning calibration here.
Hypothetical scenario with illustrative structure, not measured results. An old exemplar contains a deprecated pricing tier. After a pricing change, the new model follows the stale demonstration too literally, quoting the retired tier confidently, even though the current instructions state the new tiers. The lesson: examples are live Product assets that can override current intent, so they need ownership, versions, eval cases that would catch the drift, and event-driven review triggers (policy, schema, tool, model, or tone changes). Deprecation with a replacement belongs to the prompt library; detecting that the exemplar went stale belongs here, in the ablation card's maintenance row.
Hypothetical scenario with illustrative structure, not measured results. Three exemplars, common case, missing data, refusal, each linked to eval cases. Removing the common-case example changes nothing: the instructions already carry the pattern, so the team removes it and reclaims its tokens. Removing the missing-data example brings the missing-data failure straight back, so the team keeps it and records the ablation result on its card. Expected Product decision: keep the missing-data and refusal exemplars, drop the redundant common case, and spend the saved budget on the next uncovered failure mode. No benchmark numbers are claimed; the decision logic is the asset.
No aggregate score is computed: descriptive checks beat an arbitrary number that invites gaming.
Negative examples, demonstrations of what not to do, can help when paired with an explicit labeled contrast showing what to do instead. Never ship a bare bad output and expect the model to infer the correction, and never assume all providers and models interpret negative demonstrations identically: verify on your deployed combination.
Conflicting exemplars, divergent tone, formatting, field names, refusal behavior, or policy, do not average out; they resolve by accident. More examples never fix inconsistent Product policy: reconcile the conflict first, then encode one answer in the set.
Order can matter in some contexts and models, and similar-versus-diverse selection serves different mechanisms (fit vs coverage vs resemblance to the current request), but neither direction is a universal law. Anthropic's current guidance suggests relevant, diverse examples wrapped in <example> tags inside <examples>, with a starting tip of 3–5 that must be validated rather than obeyed; OpenAI's current guide shows diverse input/output pairs placed in the developer message with XML or Markdown boundaries. Treat every such recommendation as attributed and model-dependent, pin what your evals confirm, and record it on the card.
A static set is predictable, reviewable, versionable, and stable in token cost. Dynamically retrieving the most similar exemplars per request can improve fit but adds retrieval complexity, per-request behavior variance, and provenance requirements. Do not build a retrieval system inside this page: when dynamic selection is on the table, route its architecture to context engineering and retrieval-augmented generation.
Every exemplar consumes context on every request: more tokens, more latency, higher provider cost, and less room for user data and retrieved context. Long-context placement also matters, current provider guidance favors stable reusable content early (which can interact with prompt-caching behavior depending on provider and runtime) and query-specific material late. Prefer decision logic over price tables: no vendor pricing is frozen here because it would be stale on arrival. Size the set against the token budget and the workflow latency SLO, per exemplar.
Exemplars drift when UI labels, policies, schemas, tools, model behavior, regulatory requirements, or brand tone change. Ownership plus event-driven review triggers, not calendar rotation, keeps the set current: each change class names which cards get revalidated, and governance of versions and deprecation lives with the prompt library. Re-evaluate the whole set on every model migration.
Exemplars can cause phrase copying, entity reuse, stale-name carryover, accidental disclosure, and style overfitting. Test for verbatim overlap, invented reuse of example details, and unsafe carryover, without claiming exact memorization probabilities the evidence does not support.
Keep sensitive customer conversations out of prompts; use anonymized or synthetic-but-realistic exemplars, minimize data, and check logging and provider data-handling boundaries, this page makes no legal-compliance claims. Synthetic examples cover edge cases and privacy needs but can encode unrealistic assumptions and create circular evals when the same system generates and grades them: every synthetic exemplar gets human review. Route redaction mechanics to PII redaction and leak containment to data leakage.
Exemplars can demonstrate allowed answers, refusals, redirections, and escalation, but safety policy lives in the system layer, permissions, and guardrails, never in examples alone. Translated exemplars do not preserve behavior automatically: evaluate per locale. Tool-use demonstrations (when to call, how to fill arguments, when not to call) stay illustrative here; interface truth lives with tool instructions.
These adjacent concepts overlap with demonstrations. The table states the boundary so this page does not absorb their jobs, each links to its specialist owner.
| Concept | Its job | Boundary |
|---|---|---|
| Prompt engineering | The discipline of designing and testing instructions | Few-shot is one technique inside that discipline; method, iteration, and technique selection live there. |
| Prompt template | Reusable parameterized structure with variables and rendering | Examples may live inside templates as versioned fixtures; structure, variables, escaping, and rendering live with the template owner. |
| Prompt library | Governance of reusable prompt assets | A library may store exemplar sets; discovery, ownership, versioning, and deprecation live there, design of the set itself lives here. |
| Structured outputs | Schema-constrained, machine-readable response contracts | Schemas enforce shape; examples teach field semantics and content quality. Never use prose demonstrations as a shape guarantee. |
| Tool instructions | When and how the model may call tools | Examples may demonstrate tool use, but interface design, permissions, and authorization live outside any demonstration. |
| Retrieval-augmented generation | Supplying current, task-specific facts | Retrieval answers with fresh facts; examples demonstrate how to answer. Never smuggle stale knowledge into exemplars. |
| Evaluation harness | Test execution, datasets, rubrics, regression comparison | The harness runs and grades with-versus-without and ablation comparisons; deciding what each exemplar must prove lives here. |
Fine-tuning has no glossary owner in the current inventory: the boundary is stated in the definition and FAQ (runtime context vs training) without inventing an owner link. Zero-shot and one-shot are modes of this same technique, defined on this page, not separate owners.
Few-shot prompting supplies input-output demonstrations in model context so the model can infer desired task, style, and format patterns without changing model weights. Each exemplar demonstrates one behavior or failure-mode response with an explicit input/output boundary and a coverage role, it is a runtime behavior control, not training.
There is no universal count: use the smallest set that measurably covers the behaviors and failure modes you care about, then verify with evals. Ten near-duplicate happy-path examples usually add less value than a smaller set covering distinct cases, common, ambiguous, missing-data, refusal, formatting, and tool-use, each proven by a with-versus-without comparison and an ablation check.
One-shot is the single-example special case; few-shot uses several demonstrations; zero-shot uses instructions alone with no examples. None is categorically better: the right mode depends on model capability, task ambiguity, format strictness, edge-case coverage, context budget, cost and latency, maintenance burden, and eval evidence.
No. Unhelpful, contradictory, or stale exemplars can narrow outputs, leak example details into answers, or override current intent, while always consuming tokens. Every example should earn its place through measured improvement on its target cases without regressions elsewhere.
Order can matter in some contexts and models, recency and position effects are reported in the literature, but no single ordering rule holds universally. Treat ordering as an eval-backed Product choice on your deployed model combination: if reordering moves results, pin the order and record it; if it does not, do not enshrine a placement ritual.
Both pressures are real: similar examples improve task fit, diverse examples improve coverage. The Product question is which mechanism each exemplar serves, common-case fit, failure-mode coverage, or resemblance to the current request, and a set chosen only for similarity usually under-covers failures while a set chosen only for diversity can dilute the core pattern.
Few-shot examples are runtime context: cheap to change, easy to version, paid for on every request in tokens and latency, and they do not change model weights or guarantee generalization. Fine-tuning changes model behavior through training: stronger for stable high-volume patterns but heavier to build, evaluate, and maintain. Route stable, high-volume, latency-sensitive patterns toward fine-tuning only with current provider guidance and eval evidence, never assume one always replaces the other.
Ask whether the model needs an example of how to answer or fresh facts to answer with. Demonstrations of desired behavior belong to few-shot examples; current or task-specific knowledge belongs to retrieval. Do not smuggle stale facts into exemplars that should come from a permissioned, fresh corpus.
No. When downstream systems parse the output, a native schema mechanism plus validation is the contract that guarantees machine-readable shape; examples teach field semantics and content quality inside that shape, for instance, a schema enforces a risk_level enum while examples teach what evidence maps to each label.
Do not paste real sensitive customer conversations into prompts for realism. Prefer anonymized or synthetic-but-realistic exemplars with a privacy review, minimize data, and check logging and provider data-handling boundaries. Synthetic examples still need human review: they can encode unrealistic assumptions and create circular evals when generated by the same system they test.
Compare quality with and without it on representative eval cases, inspect target-case improvement and regressions elsewhere, and measure token and latency cost. Then ablate: remove one exemplar, rerun the relevant evals, and keep only examples whose removal would bring the failure back. One offline metric never captures full Product quality.
No. A set that works on one model may be unnecessary, behave differently, cost too much, or conflict with new native capabilities on another. Treat every model upgrade or migration as a re-evaluation trigger for the whole exemplar set, examples are not model-agnostic product policy.
For the broader instruction-design discipline, see prompt engineering; for reusable parameterized structure, see prompt template; and for governance of exemplar sets as assets, see prompt library. For hands-on prompting workflows, continue to Prompt engineering for PMs , that guide owns tutorial workflows; this page owns the entity reference. For the broader AI product picture, see the AI Product Management hub.
Build your product skills with short, practical lessons you can fit around your day.
Start with our free courses and upgrade anytime.