Agentic AI Design Patterns That Work in Production

The agentic AI design patterns that survive production are mostly simple ones: a model calling well-defined tools, a router that sends each request to the right path, a checker that reviews output, and a human who approves anything risky. Fully autonomous, open-ended agents are the exception, not the default. This guide walks through the patterns we reach for, when each fits, and when you should not build an agent at all.

Workflows vs agents: get the vocabulary right

It helps to separate two ideas, a distinction Anthropic also draws in its widely cited article Building effective agents:

• Workflows — the LLM runs inside steps you define in code. The path is predictable.
• Agents — the LLM decides which steps to take and which tools to call, in a loop, until it judges the task complete.

Most business systems we see need workflows with a few agentic steps. Workflows are cheaper, faster, easier to test and easier to explain to an auditor. Reach for full agency only when the path genuinely cannot be known in advance.

The core agentic AI design patterns

1. Tool use (the augmented LLM)

The building block of everything else. The model gets a list of tools — search, database query, send email, create ticket — with clear names, descriptions and typed parameters. It decides when to call them and uses the results.

What makes it work in production:

• Few, well-named tools. Ten precise tools beat forty overlapping ones.
• Strict schemas. Validate every argument before execution; never pass model output straight to a shell or SQL.
• Helpful errors. Return messages the model can act on (“date must be ISO 8601”) instead of stack traces.
• Standard interfaces. The Model Context Protocol lets you expose tools once and reuse them across clients. See our MCP servers guide.

2. Prompt chaining

Break a task into fixed steps, where each LLM call works on the output of the previous one: extract → validate → draft → format. Add plain-code checks (“gates”) between steps.

Use it when the task decomposes cleanly and you want each step to be simple and testable.

3. Routing

A classifier (a small model or even rules) looks at the input and sends it to a specialised path: billing questions to one prompt and toolset, technical issues to another, anything unclear to a human.

Routing is also the main cost lever: send easy requests to a fast, cheap model and hard ones to a frontier model.

4. Parallelisation

Run independent sub-tasks at the same time and combine the results. Two flavours:

• Sectioning — split work into parts (e.g. review security, style and performance separately).
• Voting — run the same check several times or with different models and take the majority for higher confidence.

5. Planning

The model writes an explicit plan before acting, then executes it step by step and updates it as it learns. Plans make long tasks more reliable and give humans something to review before anything happens. In coding agents this is the “plan mode” you approve before edits begin.

6. Orchestrator-workers

A lead model breaks a task into sub-tasks that are not known in advance, hands each to a worker (often with its own clean context), and merges the results. This is the pattern behind multi-file coding changes and multi-source research.

Watch the costs: every worker is another set of model calls. Give workers narrow instructions and limited tools.

7. Reflection (evaluator-optimiser)

One call produces output; another critiques it against explicit criteria; the first revises. It works well when you can state what “good” looks like — a style guide, a rubric, a failing test. Cap the number of loops, because models can polish forever.

The strongest version uses external feedback rather than self-judgement: run the tests, validate the JSON, check the citation actually exists.

8. Human-in-the-loop

The agent pauses for approval at defined checkpoints: before sending an email, issuing a refund, merging code or deleting data. Good human-in-the-loop design means:

• the agent presents a short summary and the exact action it wants to take,
• approval is one click, rejection comes with a reason the agent can use,
• the risk level decides the checkpoint, not a blanket “approve everything”.

9. Guardrails

Guardrails are the controls around the model, not inside the prompt:

• Input checks — detect prompt injection, off-topic or abusive requests.
• Permission boundaries — the agent acts with the user’s permissions, never an admin key.
• Output checks — schema validation, PII filters, policy rules.
• Budgets — maximum steps, tokens, time and spend per task.
• Sandboxing — code execution and browsing in isolated environments.

Treat any content the agent reads (web pages, emails, documents) as untrusted. Indirect prompt injection is the most common real-world attack on agents.

Pattern cheat sheet

Pattern	Best for	Main risk	Control
Tool use	Any action or data lookup	Wrong or unsafe calls	Strict schemas, least privilege
Prompt chaining	Fixed multi-step tasks	Errors carried forward	Code gates between steps
Routing	Mixed request types, cost control	Misclassification	Fallback to human or default path
Parallelisation	Independent checks, confidence	Cost multiplies	Limit fan-out
Planning	Long, multi-step tasks	Plans drift from reality	Re-plan after each step, human review
Orchestrator-workers	Unpredictable sub-tasks	Cost and coordination overhead	Narrow workers, budgets
Reflection	Output with clear quality criteria	Endless loops	Max iterations, external checks
Human-in-the-loop	Irreversible or costly actions	Approval fatigue	Risk-based checkpoints
Guardrails	Everything in production	False sense of security	Layered checks, red teaming

When NOT to use agents

We regularly advise clients to build something simpler. Skip the agent when:

• The steps are known. If you can draw the flowchart, write it as code with a few LLM calls.
• A single well-prompted call works. Retrieval plus one good prompt solves most Q&A. Our RAG best practices guide covers that route.
• Latency matters. Agent loops add seconds or minutes.
• Errors are expensive and hard to detect. Autonomy without verification is a liability.
• You cannot measure success. If there is no evaluation set, you cannot tell whether the agent got better or worse after a change.
• Rules or regulation require explainability. Deterministic workflows are far easier to audit. See our GDPR and AI Act checklist.

Making agents production-ready

The pattern is only half the job. In our projects we’ve found the same few practices separate demos from systems people rely on:

1. Evaluate before and after every change. Keep a set of real tasks with expected outcomes and run it in CI. Our LLM evaluation guide shows how.
2. Trace everything. Log each model call, tool call, input, output, cost and latency. Open-source tools such as Langfuse or Arize Phoenix help.
3. Keep context small and relevant. Long, noisy context is a common cause of agents losing track. Summarise, retrieve, and give sub-agents fresh context.
4. Make actions idempotent and reversible. Drafts before sends, soft deletes, dry-run modes.
5. Pin model versions and re-run your evaluation before upgrading.
6. Choose a framework last. Decide the pattern first; then pick the lightest framework that supports it. We compare options in AI agent frameworks compared and top GitHub repos for AI agents.

Key takeaways

• Start with a workflow; add agentic steps only where the path is truly unpredictable.
• Tool use, routing and chaining solve most business problems.
• Use planning, orchestrator-workers and reflection for long or open-ended tasks, with budgets and iteration limits.
• Put humans at risk-based checkpoints, not everywhere.
• Guardrails live outside the prompt: permissions, validation, sandboxing and spend limits.
• No evaluation set, no agent.

Building an agent that earns its keep

The right pattern depends on your process, your data and how much risk you can accept. We help teams choose the simplest design that works, build it with proper guardrails and tracing, and measure it against real tasks. Have a look at our AI automation service, or tell us about the process you want to automate and we’ll tell you honestly whether it needs an agent.