Agent orchestration

• Multiple agents or models must collaborate (e.g., planner + executor).
• You need reliability under load and clear ownership of failures.
• Latency or cost needs optimization across many calls.

Orchestration determines reliability and cost ceilings. PMs set retry/backoff policies, caching, parallelism, and human approval points. It shapes UX (progress, fallbacks) and SLOs. Poor orchestration can create loops, cost blowups, or brittle dependencies.

Define a graph of steps with clear inputs/outputs. Add timeouts, budgets, and circuit breakers. Cache deterministic steps. In 2026, use event-driven orchestrators with observability hooks so you can replay failures and measure per-step quality.

A roadmap assistant orchestrates intent detection → retrieval → draft → critique → final. Adding timeouts and caching cut p95 time from 11 s to 6 s and reduced cost per task 28% while keeping quality scores flat.

• Running all steps sequentially when some can be parallelized.
• No circuit breakers, leading to runaway retries on third-party outages.
• Mixing concerns (business logic and orchestration) in prompts instead of code.