Context engineering

Context engineering is the system-level discipline of deciding what information a model may use at runtime, which sources are eligible, how they are selected, ordered, scoped, transformed, and refreshed, and what happens when sources conflict, go stale, exceed budget, or cross permission boundaries. Prompt engineering designs the instructions; context engineering designs the whole information environment those instructions run in.

When to use it

Answers depend on current user or account state, billing, entitlements, configuration, that training data cannot know.
Responses must reconcile current internal knowledge, such as policies, docs, or changelogs, with an explicit freshness requirement.
An assistant carries conversation history, memory, or tool results across turns or multi-step agent workflows.
Multiple sources can disagree, go stale, or carry different permissions, so precedence and eligibility must be explicit.
Token, latency, or cost budgets force real choices about what to include, summarize, compact, or leave out.

PM decision impact

The PM owns the context contract: which sources are allowed for this task, how fresh each must be, who may access it, what wins when sources conflict, what must never enter context, and what the fallback is when data is missing. Those choices move groundedness, retry and escalation rates, latency, cost per task, and privacy risk together, adding relevant context helps, adding irrelevant context hurts. Engineering, data, and security own retrieval architecture, storage, access-control implementation, and observability; the PM sets the acceptance criteria they build against.

How to do it

List candidate context sources for one task, then mark each with authority, freshness requirement, permission boundary, and relevance rule. Define precedence for conflicts, transformation and compaction rules, a token and latency budget, a fallback when sources are missing, and what gets logged for debugging. Test the pipeline against missing, stale, noisy, conflicting, and unauthorized context before ship, and re-check it whenever models, sources, or policies change.

Example

Synthetic example, hypothetical values, not measured results. A B2B support copilot answers an account-specific billing question. Candidate sources: system safety rules, the account's live subscription record, the last three tickets, the public pricing page, and long-term memory. The PM selects the live subscription record as authoritative for plan and balance, allows the recent tickets for background, excludes memory because it holds a stale plan from before an upgrade, and blocks the pricing page from overriding account data. If the subscription lookup fails, the copilot says so and offers a human handoff instead of guessing. Eval checks: correct plan cited, no stale facts, no other customer's data, and latency within the workflow budget.

Common mistakes

Stuffing every available fact into context instead of selecting for relevance, freshness, and authority.
Letting teams edit live context blocks without versioning, review, or regression checks.
Adding sources without measuring token, latency, and cost impact on the real workflow budget.
Shipping without a precedence rule, so stale docs can silently override live account state.
Logging or exposing assembled context, including other users' data, without a permission and retention policy.

Learn it in CraftUp

AI Product Management hub AI agents impact patterns (blog)Start learning free in the CraftUp app

Last updated: September 23, 2026

The context assembly pipeline

Use this pipeline as the working mental model. Every AI response your product produces passed through something like it, the PM job is to make each stage an explicit decision instead of an accident.

1. Task and user intent
What decision is the user trying to make? The task determines which sources are even candidates.
2. Candidate sources
Instructions, session state, history, retrieved knowledge, tool outputs, memory, examples, policies. List them; do not include them yet.
3. Permission and eligibility filter
Is the user authorized for this source? Is the data needed for this task? Anything failing this check never enters context.
4. Relevance selection
Which candidates actually change the answer for this task? Exclude the rest, irrelevance is a cost, not a safety margin.
5. Freshness and conflict handling
How fresh must each source be? When two sources disagree, which one wins? Write the precedence down.
6. Compression and transformation
Summarize, compact, dedupe, or restructure into the shape the task needs, prose, structured state, or tool schemas.
7. Ordering and formatting
Place authoritative and load-bearing blocks where they survive truncation; keep untrusted content clearly separated from instructions.
8. Token and latency budget
Assemble against a per-workflow budget. Truncate low-value context first, never safety blocks first.
9. Model and tool execution
Run with explicit stop conditions, tool-call limits, and fallback behavior when sources or tools fail.
10. Evaluation and logging
Record which sources entered context, their versions and freshness, what was truncated, and whether the outcome was correct, without retaining sensitive payloads by default.

Context source decision matrix

For each candidate source, fill one row before you ship. If you cannot answer a cell, that is the review to schedule, not a detail to defer.

Source	Purpose	Authority	Freshness	Permissions	Fallback
System and product instructions	Non-negotiable behavior, tone, refusal rules	Highest for policy; never overridden by retrieved content	Versioned; changes go through review	Same for all users of the feature	Refuse or escalate, never improvise policy
Live account and application state	Facts about this user: plan, balance, configuration	Highest for account facts; overrides docs and memory	Read at request time for decisions	Current user only; enforce tenancy	Say data is unavailable; offer human handoff
Retrieved documents	Current knowledge: policies, docs, changelog	Below live state; above memory for general facts	Per-collection refresh contract	Only sources the user may access	Answer from model knowledge with a caveat, or decline
Conversation history	What was already established in this session	High for intent; latest explicit instruction wins over older turns	Session-scoped; compact as it grows	Current session only	Ask a clarifying question instead of assuming
Tool and API results	Real-world data and actions	Authoritative within the tool's domain only	Fresh at call time; never cached across trust boundaries without a rule	Per-tool allowlists; approvals for writes	Retry within budget, then degrade to a safe partial answer
Long-term memory	Reuse across turns and sessions	Lowest when it conflicts with live state or latest instruction	Retention policy with expiry and user deletion	User-visible and controllable where required	Ignore memory and proceed from current sources
Examples	Format, tone, and edge-case guidance	Illustrative only; never a source of facts	Reviewed on a schedule; removed when stale	No personal or customer data in examples	Fall back to the minimal instruction skeleton

What context engineering is not

These adjacent concepts overlap with context engineering. The table states the boundary so this page does not absorb their jobs, each links to its specialist owner.

Concept	Its job	Boundary
Prompt engineering	Designing instructions, messages, and examples	A prompt is one component inside a context-engineered system. Prompt technique lives on the prompt engineering page.
RAG	Retrieving external knowledge at query time	Retrieval is one possible context source. Full retrieval pipelines, chunking, and ranking live on the RAG page.
Context window	The model's capacity constraint in tokens	The window is a constraint; context engineering is the design discipline inside it. Model limits live on the context window page.
Agent memory	Persisting and reusing facts across turns and sessions	Memory architectures live on the agent memory page. Here, memory is treated as one governed source among several.
Token budget	Per-request token allocation for cost and latency control	Budget mechanics live on the token budget page. Here, budget is the constraint that forces selection.
Fine-tuning	Changing model weights through training	Fine-tuning changes the model; context engineering changes the runtime information it receives. Neither universally replaces the other.

When to use context engineering

The task needs user or account state the model was never trained on.
Answers depend on current internal knowledge with a freshness contract.
An agent must carry tools, state, and memory across steps.
Policy or personalization varies by user, account, or region.
Conversation history materially changes what the right answer is.
Multiple sources must be reconciled under one precedence rule.

When not to over-engineer context

A stable static instruction already solves the task reliably.
No external or user state is needed for a correct answer.
Retrieval adds latency, cost, and failure modes without a measurable quality gain.
Structured code or deterministic rules solve the task more reliably than model reasoning.
The task is simple and single-turn with no history, tools, or policy variance.

Permissions, privacy, and untrusted content

Assembling context widens the privacy and security surface: the system may gather sensitive information the user never typed. Treat these as product requirements, not implementation details. This page states the questions; it is not legal advice, and deep security architecture belongs with its specialist owners.

Is the user authorized to access each source, and is the data actually needed for this task?
Can a retrieved chunk expose another tenant's or another user's data?
Is assembled context logged, where is it sent, and how long is it retained?
Is memory user-visible and deletable where required, and can sharing or export leak hidden context?
Is retrieved and tool-provided content treated as data, never as instructions? Retrieved text can contain injected instructions, source trust and instruction-versus-data boundaries are part of the design. See prompt injection and guardrails.

Trade-offs: quality, latency, cost, privacy, maintainability

Quality: missing relevant context causes wrong or generic answers; irrelevant context causes noise, conflicts, and stale facts. Relevance beats volume.
Latency: database reads, retrieval, reranking, tool calls, and compaction each add time, and larger inputs take longer to process. Define a UX latency budget per workflow and evaluate each pipeline stage against it.
Cost: input-token processing, retrieval infrastructure, reranking and extra model calls, memory storage and lookup, and additional tool calls all contribute. No model prices are quoted here, verify current first-party pricing when costing a feature.
Privacy: every added source is a new exposure path. Prefer the minimum context that answers the task, with tenancy and retention enforced.
Maintainability: version context blocks independently so policy or copy updates do not require model changes, but every editable block needs ownership, review, and regression checks. For compaction mechanics, see context compaction.

Failure diagnostic

Not every bad model response is a context problem, task ambiguity, model capability limits, and eval design also matter. When the outcome is wrong, use this table to isolate the context layer first.

Symptom	Likely context cause	What to check
Generic answer missing account facts	Relevant source never selected	Was the live record eligible and selected for this task?
Recently changed facts reported wrong	Stale context	Freshness contract per source; recrawl, cache, and memory TTLs
Rambling or self-contradictory answer	Irrelevant or noisy context overload	Relevance rules; what can be excluded or compacted
Right facts, wrong conclusion	Conflicting sources, no precedence	Authority ordering; which source should have won
Answer cites a doc instead of live state	Wrong source authority	Precedence rule between docs, state, and memory
Follows instructions found in a document	Untrusted content treated as instructions	Instruction-versus-data separation; injection tests
User sees another account's data	Permission or tenancy failure	Eligibility filter, namespaces, logging and retention
Answer cuts off or ignores system rules	Truncation removed critical blocks	Budget policy; protected blocks reserved before truncation
Summary drops dates, numbers, or names	Lossy compaction	Compaction rules; source references preserved for re-expansion
Correct answer exists but was not used	Retrieval miss or poor ranking	Recall, filters, and ranking on real queries, not synthetic ones
Contradicts what the user just said	Memory overriding latest instruction	Recency rule: latest explicit instruction wins

Evaluation and observability

Answer three questions: did we assemble the right context, did the model use it correctly, and did the product outcome improve? Measure offline regressions and live product metrics, never a generic benchmark target invented for the occasion. Full eval methodology lives on evaluation harness.

Assembly quality: relevant source present, stale or unauthorized source excluded, correct precedence applied, trace complete.
Task quality: factually correct, grounded in the cited sources, policy-compliant, correct tool use.
Product quality: user success rate, correction and retry rates, latency against budget, cost per task, escalation and support volume, trust signals.
Observability minimum: which sources were considered, which entered context, their versions and freshness, retrieval and tool results, what was truncated or compacted, and final context metadata, logged without retaining sensitive payloads by default.

Who owns what

Area	Typical owner
User job, source and business requirements, freshness expectations	PM
Permission and product policy, cost-versus-latency trade-off, fallback UX	PM, with security and legal input
Quality and evaluation acceptance criteria	PM
Retrieval architecture, storage, indexing, caching, runtime assembly	Engineering and data
Access-control implementation, observability infrastructure	Engineering and security

Boundaries vary by company, treat this as a starting split to negotiate explicitly, not a universal org chart.

Worked examples

Multi-step agent: procurement-review assistant (synthetic)

Synthetic scenario with hypothetical structure, not measured results. A procurement agent reviews a vendor renewal across three steps: it pulls the current contract record, retrieves the latest usage report, then drafts a recommendation. Context evolves at each step, step one needs contract terms, step two adds usage data, step three needs both plus the draft history. The PM's rules: the live contract record outranks retrieved playbook docs, usage data must be from the current quarter, the agent may read but never send email, and each step logs which sources it used so a wrong recommendation can be traced to the step and source that caused it.

Bad example: stuff everything into context

The same copilot, done badly: every ticket in account history, the full help center, all memory entries, and three years of changelog are included in every call. The result is predictable, a stale doc contradicts the live subscription record with no precedence rule, an old memory reintroduces a cancelled plan, token and latency budgets blow past the workflow SLO, and a support macro from another account leaks into the trace. The fix is never a bigger window; it is selection, authority, freshness, permissions, and budget.

Implementation checklist

Define the task and the user decision it serves.
List candidate context sources, then justify each inclusion.
Mark the authority of every source.
Define the permission boundary per source.
Define the freshness requirement per source.
Define the relevance and selection rule.
Define the precedence and conflict rule.
Define transformation and compaction behavior, with protected blocks.
Define the token, latency, and cost budget.
Define the fallback when a source is missing or a tool fails.
Define privacy-safe observability: what is traced, what is never retained.
Define assembly, task, and product evals before ship.
Test missing, stale, noisy, conflicting, unauthorized, and truncated context.
Schedule re-review for model, source, and policy changes.

Frequently asked questions

What is context engineering?

Context engineering is the system-level discipline of deciding what information a model may use at runtime: which sources are eligible, how they are selected, ordered, scoped, transformed, and refreshed, and what happens when sources conflict, go stale, exceed budget, or cross permission boundaries.

How is context engineering different from prompt engineering?

Prompt engineering designs the instructions, messages, and examples presented to the model. Context engineering is broader: it designs the whole information environment those instructions run in. A prompt is one component of a context-engineered system, not a synonym for it.

Is RAG part of context engineering?

Retrieval is one possible source of context, not the whole discipline. Context engineering decides whether to retrieve at all, what to retrieve, how to filter and rank it, where to place it, how fresh it must be, what permissions apply, and how it combines with instructions, state, memory, and tool results.

Does a larger context window remove the need for context engineering?

No. A larger window raises the capacity ceiling but does not solve relevance, freshness, authority, conflict, cost, latency, or permission problems. Irrelevant or stale context can still degrade answers, slow responses, cost more, and expose data that should never have been assembled.

Is memory part of context engineering?

Memory is one way to persist and reuse information across turns and sessions. Context engineering decides whether memory should be used for this task, which memories are relevant, how they are summarized or retrieved, what permissions and freshness rules apply, and when the latest explicit user instruction overrides a stored memory.

What does a product manager actually own in context engineering?

The PM owns the context contract: allowed sources, freshness expectations, permission and product policy, precedence when sources conflict, token and latency budgets, fallback UX, observability requirements, and quality and evaluation acceptance criteria. Retrieval architecture, storage, caching, and access-control implementation typically sit with engineering, data, and security.

How do you evaluate whether context engineering improved the product?

Check three layers: assembly quality (right sources present, stale or unauthorized sources excluded, correct precedence, complete traces), task quality (correct, grounded, policy-compliant answers and tool use), and product outcomes (user success, correction and retry rates, latency, cost per task, escalations, and trust signals), measured on offline regression sets and live product metrics.

Can too much context hurt an LLM product?

Yes. Missing relevant context reduces quality, but excessive irrelevant context adds cost and latency and can introduce noise, conflicting instructions, stale facts, and privacy exposure. There is no universal token percentage that marks the boundary, it depends on the task, sources, and budget.

What are the most common context-engineering failure modes?

Missing relevant context, stale context, irrelevant or noisy context overload, contradictory sources with no precedence rule, wrong source authority, untrusted retrieved content treated as instructions, permission leakage, truncation of critical blocks, compaction that drops key facts, retrieval misses, and memory that contradicts the latest user instruction.

How does context engineering change for AI agents?

It becomes more important, not different in kind. Multi-step workflows add available tools and schemas, previous tool results, plan and execution state, and evolving memory to the context at each step, so selection, freshness, permission, budget, and conflict rules must hold across steps, not just a single call.

What to do next

Understand context engineering here, then map your candidate sources, define authority, freshness, and permissions, choose selection and transformation rules, set budgets and fallback, and instrument evaluation. When one layer needs depth, go to its specialist: prompt engineering, RAG, agent memory, evaluation harness, or the AI Product Management hub for building and managing AI products end to end.