Prompt engineering

Prompt engineering is the discipline of designing, testing, and maintaining the instructions, examples, and constraints a product sends to a language model to elicit a desired behavior for a specific task. It covers what the prompt asks the model to do and how that request is framed, not the broader runtime information environment those instructions run in (context engineering), the schema mechanism that enforces machine-readable outputs (structured outputs), or the test infrastructure that proves the prompt works (evaluation harness). Prompt behavior is empirical: it is measured against representative cases, not assumed from wording alone.

When to use it

A product behavior can be changed by clearer task framing, sharper instructions, better examples, tighter constraints, or a clearer output request.
A new AI-powered flow needs a reliable baseline before investing in heavier levers such as retrieval pipelines or model adaptation.
Outputs need consistent structure, tone, refusal behavior, or fallback handling that the current instruction set does not produce.
Prompt changes can be paired with representative eval cases and versioned rollout, so improvements are proven rather than assumed.
The failure diagnosis points at instruction ambiguity, missing constraints, unhelpful examples, or a weak output request, not at missing facts, stale retrieval, tool schemas, or model capability.

PM decision impact

The PM owns the prompt contract: the intended behavior for this task, what the model must do and must never do, which examples justify their token cost, what the output contract and failure behavior are, and what eval evidence gates a rollout. Engineering owns prompt implementation in code, schema and tool integration, and eval infrastructure; design owns tone and UX surfacing; trust and safety own policy and refusal requirements. The PM does not need to author every production prompt personally. The PM sets the acceptance criteria they build against, and decides when prompting is no longer the right lever, routing missing-facts problems to retrieval, machine-contract problems to schemas, and capability gaps to model or architecture choices.

How to do it

Start from the smallest instruction that states the job, the audience, the constraints, and the output contract. Add examples only when they change measured behavior, prefer platform schema mechanisms over prose when downstream systems parse the output, and define failure behavior for unknown, missing, or conflicting inputs. Treat prompt changes like product behavior changes: version them, test them against normal, edge, and unsafe cases, and re-evaluate when models, context, tools, or policies change.

Example

Synthetic example, hypothetical scenario, not measured results. A support copilot drafts a reply from retrieved help-center articles. The prompt states the job (draft a response grounded only in the supplied articles), constraints (no invented policy details, no promises about refunds), the output contract (short answer plus cited article titles), and failure behavior (say the answer is unavailable and offer a human handoff when the articles do not cover the question). A later change that shortens rambling answers belongs to prompt engineering; a change that retrieves the wrong article belongs to retrieval and context engineering.

Common mistakes

Treating clever wording as a substitute for missing facts, no instruction compensates for context the model never received.
Assuming universal magic words: a phrase that helps one model on one task may do nothing on another, so every technique needs eval evidence.
Judging a prompt by a single good output instead of representative, edge, and regression cases.
Relying on prose instructions for machine-readable outputs instead of schema-constrained mechanisms.
Asking for hidden internal reasoning traces instead of verifiable artifacts such as citations, structured plans, or tool outputs.
Treating delimiters or formatting alone as a security boundary against untrusted content.

Learn it in CraftUp

AI Product Management hub Prompt engineering for PMs (blog)Start learning free in the CraftUp app

Last updated: September 23, 2026

The prompt contract anatomy

Treat every production prompt as a small behavior contract, not a paragraph. Fill each row before the prompt ships, a row you cannot fill is a Product decision you have not made yet.

Contract block	What it states	What happens without it
Job	The outcome the model must produce for this task.	The model optimizes for the wrong thing, fluent text instead of the decision the user needs.
Audience and context	Who the output serves and which supplied context it may use.	Generic answers that ignore the user's situation or leak in irrelevant material.
Instructions	What the model must do, stated plainly and in order of importance.	Inconsistent behavior across runs, models, and phrasings of the same request.
Constraints	What the model must avoid, preserve, or never invent.	Banned claims appear, tone drifts, and policy edges get improvised per response.
Inputs	The data supplied for this call, separated from the instructions themselves.	Trusted instructions and untrusted data blur together and retrieved text gets treated as orders.
Examples	Input-output pairs that demonstrate format, tone, or edge handling, only where they change measured behavior.	Token cost and latency rise while the model copies stale or irrelevant patterns.
Output contract	Structure, fields, and format, backed by schema mechanisms where software parses the result.	Downstream parsing breaks and every consumer writes its own fragile repair logic.
Failure behavior	What to do when inputs are unknown, missing, conflicting, or unsafe: decline, ask, or escalate.	The model guesses confidently instead of saying the information is unavailable.
Evaluation	How the team knows the prompt works: representative cases, edge cases, and regression sets.	One good demo becomes the quality bar and regressions ship silently.

Prompt failure diagnosis matrix

Not every bad response is a prompt problem. Read the symptom, inspect the likely layer first, and fix the layer that is actually broken, rewriting instructions rarely fixes missing facts or wrong schemas.

Symptom	Inspect this layer first	Why it is usually not the wording
Wrong or invented facts	Retrieval, context selection, freshness, see context engineering.	The model never received the correct facts, so no phrasing can produce them.
Right facts, wrong format or broken parsing	Output schema and validation, see structured outputs.	Prose instructions are a weak enforcement mechanism for machine contracts.
Inconsistent task behavior across runs	Instruction ambiguity, conflicting rules, or unhelpful examples in the prompt.	This is the core prompt-engineering failure: the contract is underspecified.
Model ignores policy, tone, or refusal rules	Instruction hierarchy: system and developer layers vs task prompt, see system prompt.	A task prompt cannot reliably override a missing or conflicting persistent layer.
Wrong tool called, or tool loops	Tool descriptions, schemas, and call budgets, see tool instructions.	Tool selection follows tool definitions more than task prose.
Worked last month, broken now	Version drift: model, provider behavior, context, tool schema, or policy changed.	The wording did not change, the system around it did. Re-run evals before rewriting.
Slow or expensive responses	Context size, example count, and token allocation, see token budget.	Each added example and context block has a latency and cost price.
Stale or copied example text in answers	Example quality and freshness in the prompt.	Examples teach by pattern-matching, including the patterns you no longer want.

Is prompting the right lever?

Before wordsmithing, route the problem to the lever that owns it. Prompting is one lever among several, using it for every failure is how teams get longer prompts and unchanged quality.

If the problem is primarily	Reach for	Specialist owner
Ambiguous task, weak constraints, unhelpful examples	Prompt wording and contract	This page, the contract above
Missing, stale, or conflicting knowledge	Retrieval, freshness, and precedence rules	Context engineering
Strict machine-readable output keeps breaking	Schema-constrained outputs and validation	Structured outputs
Wrong tool chosen or tool misuse	Tool descriptions, schemas, and budgets	Tool instructions
Repeated capability gap on measured cases	Model choice, architecture, or training adaptation	Model and architecture decision, not a wording fix
Unsafe, leaking, or permission-crossing behavior	Guardrails, access control, and red-teaming	Guardrails and prompt injection
Nobody can say what good looks like	Product policy decision plus measurement	Evaluation harness

What prompt engineering is not

These adjacent concepts overlap with prompting. The table states the boundary so this page does not absorb their jobs, each links to its specialist owner.

Concept	Its job	Boundary
Context engineering	The runtime information environment: sources, freshness, permissions, budgets	Prompt engineering designs the instructions; context engineering designs everything those instructions run in. Source and freshness problems live there.
System prompt	The persistent, high-priority instruction layer: persona, policy, escalation	The system prompt is one surface inside prompt engineering, not a synonym for it. Its governance lives on its own page.
Prompt template	Reusable parameterized prompt structure: variables, escaping, versioning	Templates own the reusable structure; this page owns designing and testing the behavior inside it.
Prompt library	Governed collection of reusable prompts across teams	The library owns distribution and governance; this page owns whether an individual prompt is well-designed.
Few-shot examples	Input-output pairs that demonstrate style, structure, or edge handling	Examples are one prompting technique with real costs. Selection and maintenance detail lives on the few-shot page.
Structured outputs	Schema-constrained, machine-readable response contracts	When software parses the answer, schemas do the enforcing and prompts supply the semantics. Schema detail lives there.
Evaluation harness	Systematic test sets, graders, metrics, and regression infrastructure	This page explains what behavior to test and why; the harness owns how to run, grade, and track it at scale.

No magic words

Prompt behavior is empirical and model-dependent. Claims such as “always act as an expert,” “always think step by step,” or “politeness always improves quality” are not universal laws, a technique that helps one model on one task may do nothing on another. Test each technique against representative cases on the models you actually ship, and keep only what measured behavior justifies.

Do not ask for hidden reasoning

Never instruct a model to reveal its hidden internal reasoning trace, and never treat a requested step-by-step as an auditable record. When reasoning quality matters, ask for verifiable artifacts instead, citations to supplied sources, structured plans, intermediate checks, or tool and evaluation outputs, and follow current provider guidance for the models you ship.

Instructions are not a security boundary

Separating instructions from user data and retrieved content reduces accidents, but delimiters and formatting alone do not stop prompt injection. Treat retrieved and tool content as data, never as orders, and route permission and adversarial problems to prompt injection and guardrails instead of cleverer wording.

Worked example: support copilot

Hypothetical scenario with illustrative structure, not measured results. A support copilot drafts a reply from retrieved help-center articles and customer context.

Product requirement: answers must stay within the supplied articles, cite them, and never invent refund policy.
Prompt instructions: draft from the supplied articles only, flag gaps explicitly, keep the brand tone, and end with either citations or a handoff offer.
Supplied elsewhere (not the prompt's job): which articles were retrieved, how fresh each is, and whose account data is eligible, owned by retrieval and context engineering.
Output contract: short answer plus cited article titles, enforced by a schema where the UI parses citations, owned with structured outputs.
Expected failure mode: confident refund promises when articles are silent. The fix is failure behavior in the prompt (decline and offer handoff), not an extra retrieved paragraph.
Eval cases: covered question, uncovered question, conflicting articles, hostile user content, and a model-version swap, each with an expected behavior written down first.
Ownership split: shorter rambling answers belong to prompt engineering; wrong-article retrieval belongs to retrieval; citation parsing belongs to schemas; refusal policy belongs to trust and safety.

PM prompt checklist: define, build, test, ship

Define

User, job, and the decision the output supports.
Accepted output behavior and explicitly unacceptable output.
Failure and fallback behavior: decline, ask, or escalate.
Evaluation acceptance criteria before any wording is written.

Build

Smallest instruction that states job, constraints, and output contract.
Examples only where measured behavior justifies their cost.
Schema mechanisms where software parses the output.
Instructions, data, and retrieved content kept visibly separate.

Test

Normal cases, edge cases, and known historical failures.
Missing, ambiguous, and conflicting inputs.
Adversarial and unsafe inputs where the feature is exposed to them.
Model and version changes re-run against the same regression set.

Ship

Prompt ID, version, intended behavior change, and owner recorded.
Eval evidence attached to the change, with known limitations stated.
Staged rollout with monitoring on quality, latency, cost, and escalations.
Rollback path when the underlying model, context, or policy changes.

Prompt engineering FAQ

What is prompt engineering?

Prompt engineering is the discipline of designing, testing, and maintaining the instructions, examples, and constraints a product sends to a language model to elicit a desired behavior for a specific task. It covers what the prompt asks and how that request is framed, not the broader runtime information environment (context engineering), the schema mechanism for machine-readable outputs (structured outputs), or the test infrastructure that proves the prompt works (evaluation harness).

How is prompt engineering different from context engineering?

Prompt engineering designs the instructions, task framing, examples, constraints, and output request. Context engineering is broader: it designs the whole runtime information environment those instructions run in, source selection, retrieval, memory, freshness, permissions, conflict precedence, and token allocation. A prompt is one component inside a context-engineered system.

Is prompt engineering still relevant with newer, more capable models?

The wording tricks matter less with each model generation, but the discipline matters more: intended behavior must still be defined, failure behavior for missing or conflicting inputs must still be specified, outputs consumed by software still need contracts, and behavior still needs evaluation when models change. Prompts regress when models, context, tools, or policies change, so re-evaluation is part of the job.

What is the difference between prompt engineering and fine-tuning?

Prompting and context changes alter the runtime inputs a model receives; fine-tuning alters model behavior through training or adaptation. Frame the choice around behavior stability, available examples and data, iteration speed, latency and cost, task consistency, and evaluation evidence. Neither universally replaces the other.

What is a system prompt, and how does it relate to prompt engineering?

A system prompt (or provider-equivalent developer instruction layer) is one persistent, high-priority instruction surface inside a product, not the whole discipline. Prompt engineering spans task instructions, examples, and constraints across the available message and context structure. Exact role semantics vary by platform, so persistent policy and persona rules belong to the system prompt while per-task behavior belongs to task prompts.

Do I need few-shot examples in every prompt?

No. Examples are one technique among several, and they cost tokens, latency, and maintenance. Add examples only when they change measured behavior on representative cases, keep them short and current, weigh quality against copying and overfitting behavior, and never put personal or customer data in them.

Should prompts be long or short?

As short as the behavior contract allows. Start from the smallest instruction that states the job, constraints, and output contract, then add length only when eval evidence shows it changes behavior. Longer prompts cost more, run slower, and create more surface for conflicts and staleness.

How do I test whether a prompt works?

A prompt is not good because one example worked. Test it against representative normal cases, known edge and failure cases, missing or ambiguous inputs, and unsafe inputs where relevant, then re-run those cases as regression tests whenever the prompt, model, context, tools, or policy changes. Full test-set execution and grading infrastructure lives with the evaluation harness.

Can prompt engineering stop hallucinations?

No, wording alone cannot guarantee factual correctness. Prompts can require the model to stay within supplied sources and to say when information is missing, but missing or stale facts are fixed by retrieval and context design, machine-readable reliability is fixed by schema mechanisms, and permission problems are fixed by guardrails and access control. A perfectly followed prompt can still produce a bad product outcome.

When should I use structured outputs instead of prompt wording?

Whenever a downstream system parses the response. Prefer platform or schema mechanisms for the machine contract (fields, types, required values) and use prompt instructions for the semantic behavior around that contract, what to say, what to refuse, how to handle gaps. Regex parsing of free prose should not be the default when stronger mechanisms exist.

What should a product manager actually own about prompts?

The PM owns the prompt contract: intended behavior, what the model must do and must never do, which examples justify their cost, the output contract, failure and fallback behavior, evaluation acceptance criteria, and versioned rollout. The PM does not need to author every production prompt personally, engineering, design, and trust and safety own implementation, surfacing, and policy, but the PM decides what counts as working.

For PM workflow depth, research synthesis, PRD drafts, user stories, and stakeholder updates, continue to Prompt engineering for PMs. That guide owns tutorial workflows; this page owns the entity reference. For the broader AI product picture, see the AI Product Management hub.