Building Effective Agents: from composable workflows to autonomous agents

Original source

Anthropic, 2024-12-19: Building effective agents.

This is an original learning guide, not a reproduction. Copyright and the complete article remain with the original source.

What you should be able to do after this guide

• Distinguish an augmented LLM, a workflow, and an agent instead of calling every multi-step system an agent.
• Explain the control flow and fit conditions of Prompt Chaining, Routing, Parallelization, Orchestrator–Workers, and Evaluator–Optimizer.
• Choose the smallest sufficient structure for a real product task and explain why a more complex design is not yet justified.
• Specify environment feedback, permission boundaries, maximum steps, success criteria, and human checkpoints for an autonomous agent.
• Evaluate quality, cost, latency, diagnosability, and error accumulation together rather than ranking systems only by “degree of automation.”

The idea in one sentence

The article offers a practical complexity ladder for agent engineering: make a single augmented LLM work first, compose fixed workflows according to task structure, and give process control to an autonomous agent only when the steps truly cannot be hard-coded and the environment can keep supplying ground truth.

First distinguish three layers

Augmented LLM: the foundation beneath every pattern

A plain LLM call receives input and generates output. An augmented LLM can also use retrieval, tools, and memory. It might retrieve a product specification, call an inventory API, consult a brand glossary, and decide what information to retain.

An augmented LLM is not necessarily an agent. A developer can still hard-code “retrieve, call the model once, validate the JSON.” Augmentation describes available capabilities; workflow and agent describe who controls the execution path.

Workflow: code determines most of the path in advance

In a workflow, models and tools run along code paths defined by the developer. A model may generate or classify at a particular step, but code largely decides what runs next, where results go, and what a failure branch does. Workflows are predictable, testable, and make it easier to validate each boundary.

Agent: the model chooses the path from runtime state

An agent reads the task and environment feedback, then dynamically selects the next step, tool, and subgoal. The loop continues until external success, a need for human judgment, a blocker, or a stop limit. Its value comes from an open path, which also creates higher latency, cost, and compounding-error risk.

Dimension	Single augmented LLM	Workflow	Agent
Who chooses the steps?	Developer arranges one main call	Developer predefines a control graph	Model selects dynamically at runtime
Best fit	Clear input and output; one inference can finish	Stable decomposition into steps or categories	Number of steps and required tools cannot be predicted
Where feedback appears	Usually after the final output	Every fixed gate can validate	Every turn must obtain environmental ground truth
How it stops	Call returns or validation fails	Path endpoint, branch failure, or approval	Success criterion, maximum turns, budget, blocker, or takeover
Main risk	Insufficient output quality	Upstream errors propagate; routing can be wrong	Cost and errors compound; behavior is harder to reproduce

More autonomous does not mean more mature

Autonomy exchanges predictability, cost, and latency for flexibility. If a fixed problem is already solved reliably, turning it into an agent usually adds failure surfaces rather than value.

One product task across every structure

We will use one running example: an ecommerce team needs to publish a new product page. Inputs include supplier specifications, brand terminology, target markets, and a launch date. Outputs include compliant product copy, English and Chinese versions, a CMS draft, and a verified preview.

The same business label can hide a simple or open-ended task. What matters is how the objective is constrained:

• if the only need is a short summary from a specification, one call may suffice;
• if the system must extract facts, check compliance, draft copy, and translate in a fixed sequence, Prompt Chaining fits;
• if different product categories follow different regulations, Routing fits;
• if independent checks can run together, Parallelization fits;
• if the required assets for each market cannot be known in advance, Orchestrator–Workers may fit;
• if copy can improve against an explicit rubric, Evaluator–Optimizer fits;
• only when the objective becomes “complete the launch, recover from unknown CMS errors, and repair problems found in the preview” does it begin to require an agent.

flowchart TD
    T["New-product launch goal"] --> B["Augmented LLM: retrieval + tools + memory"]
    B --> Q{"One call + validation is enough?"}
    Q -->|"Yes"| O["Return output"]
    Q -->|"No"| K{"Can the path be predefined?"}
    K -->|"Yes"| W["Choose a fixed workflow"]
    W --> C["Chaining"]
    W --> R["Routing"]
    W --> P["Parallelization"]
    W --> H["Orchestrator–Workers"]
    W --> E["Evaluator–Optimizer"]
    K -->|"No"| G{"Reliable feedback and controlled risk?"}
    G -->|"No"| M["Narrow the task / add human decisions"]
    G -->|"Yes"| A["Autonomous agent loop"]

Five workflows: the real difference is the control graph

1. Prompt Chaining: a fixed pipeline

Mechanism: decompose the task into a known sequence in which each model call processes the previous output. Programmatic gates can stop or return work when an intermediate result is invalid.

The product task might become:

extract supplier facts
    → schema gate: are required fields complete?
    → create factual summary
    → compliance gate: any prohibited claims?
    → draft marketing copy
    → translate and check glossary
    → create CMS draft

Each step is easier, at the cost of more calls and latency, while omissions from an early step can propagate. Good chaining is not splitting one prompt into ten arbitrary prompts. It creates checkable boundaries—for example, stopping on an incomplete specification instead of asking downstream models to invent missing values.

Fit condition: stable steps, fixed order, and clear input-output contracts at each boundary.

2. Routing: classify, then enter a specialized path

Mechanism: a router reads the input and produces a category; code sends the task to the corresponding prompt, model, or toolchain.

Products might first be classified as ordinary consumer goods, food, medically adjacent devices, or high-risk unknowns. Each category receives different compliance rules, while low-confidence and high-risk cases go directly to human review.

route = classify(product_spec)
if route.confidence < 0.85 or route.label == "high_risk_unknown":
    return request_human_review(route)
return run_specialized_workflow(route.label, product_spec)

Routing creates separation of concerns: each path handles relatively homogeneous inputs. Its main risk is misclassification. A downstream workflow can execute perfectly under the wrong rules. The router therefore needs its own evaluation, confidence handling, and fallback path.

Fit condition: inputs fall into distinct categories and classification can reach the required reliability.

3. Parallelization: independent sections or independent judgments

The article identifies two forms:

1. Sectioning runs different subtasks at the same time. For a product, fact verification, brand-tone review, SEO suggestions, and preliminary regulatory review can run concurrently if they do not depend on one another’s unfinished results.
2. Voting runs several independent attempts on the same question and aggregates them. Multiple reviewers might flag prohibited claims, using a threshold to balance false negatives and false positives.

Parallel execution does not automatically improve correctness. Shared bad input can make every branch fail together, and contradictory outputs require an aggregation policy. Stopping means more than “all model calls returned”: define timeouts, partial-failure rules, a minimum response count, and conflict escalation.

Fit condition: subtasks are genuinely independent, or the value of multiple perspectives exceeds the additional call cost.

4. Orchestrator–Workers: subtasks are discovered at runtime

Mechanism: a central orchestrator interprets the specific input, dynamically decomposes the work, delegates to worker LLMs, and synthesizes their results. It resembles parallelization, but the essential difference is that the developer did not predefine the subtasks; the orchestrator creates them for this input.

Suppose the product unexpectedly launches in three markets. The orchestrator discovers that market A needs battery-shipping language, market B needs a localized size table, and market C lacks image alt text, so it creates different workers. A different product may require only one worker.

Explicit state should include task IDs, worker briefs, dependencies, worker status, evidence sources, completed outputs, and unresolved merge conflicts. Without it, typical failures are duplicate investigation, missed subtasks, and lost evidence during synthesis.

Fit condition: the overall goal is clear, but the number and shape of subtasks become known only after inspecting the input.

5. Evaluator–Optimizer: a feedback loop with a rubric

Mechanism: a generator produces a candidate, an evaluator applies explicit criteria and provides feedback, and an optimizer revises. The loop ends at a quality threshold or an iteration limit.

Product copy can be scored for factual consistency, absence of prohibited claims, glossary consistency, title length, and required-information coverage. Iteration is useful only when feedback can be made actionable and the revised output can be checked for improvement.

generate draft → evaluator returns per-criterion scores and actionable feedback
       ↑                                      ↓
       └──── below threshold with budget: targeted revision ────┘
                         pass / iteration limit: stop

If “good copy” has no operational criteria, the evaluator may simply restate a preference. If generator and evaluator share the same blind spot, the loop reinforces the wrong direction. Limit iterations and store scores and revision rationales so you can test whether iteration actually adds value.

Fit condition: success criteria are clear, articulated human feedback improves results, and a model can provide similarly useful feedback.

Autonomous agent: what changes when control moves to the model?

Now redefine the goal: “Within the granted permissions, complete the product-page launch. Recover from missing information, CMS errors, and preview problems; ask a person only when business judgment is required.” The developer cannot enumerate every step. The agent may inspect the specification, discover a missing field, query a supplier system, create a draft, open the preview, notice layout overflow, revise the content, and rerun acceptance.

A minimal loop looks like this:

state = initialize(goal, permissions, budget)
 
while not externally_verified(state):
    if state.steps >= MAX_STEPS or state.cost >= MAX_COST:
        return stop_with_checkpoint(state)
 
    action = llm_choose_action(state.current_context)
 
    if action.requires_ungranted_permission:
        return ask_human_for_authorization(action, state)
 
    observation = execute_in_environment(action)
    state.record(action, observation)
 
    if observation.is_blocked or action.requires_product_judgment:
        return ask_human_for_judgment(state)
 
return publish_verified_result(state)

State an agent must maintain

• Goal state: what “launched” means and which acceptance checks remain.
• World state: current CMS draft, selected assets, preview URL, tool results, and errors.
• Plan state: current subgoal, pending tasks, and abandoned approaches with reasons.
• Permission state: readable and writable resources plus actions requiring approval.
• Resource state: turns, tokens, cost, and deadline.
• Evidence state: the environment result supporting each completion claim.

A stop condition is not the sentence “task complete”

Reliable termination combines several conditions:

1. External success: a CMS draft exists, required fields are complete, and preview checks pass—not merely a verbal completion claim.
2. Safety boundary: publication, payment, deletion, or external communication requires authorization.
3. Resource limit: maximum turns, cost, duration, or repeated actions.
4. Blocker escalation: missing business facts, missing permission, or repeated tool failures go to a person.
5. Failure checkpoint: preserve current state, verified facts, and a recoverable next step instead of silently ending.

A decision table for choosing the structure

Observed task structure	Prefer	Signal not to escalate
One call with retrieval passes acceptance	Single augmented LLM	The only goal is to “look agentic”
Fixed steps make each inference easier	Prompt Chaining	Decomposition creates no checkable boundary
Stable categories need specialized handling	Routing	Categories are ambiguous and there is no low-confidence fallback
Subtasks are independent or need independent perspectives	Parallelization	Strong dependencies or no aggregation rule
Subtask count varies with each input	Orchestrator–Workers	The task list is actually known in advance
A clear rubric supports measurable refinement	Evaluator–Optimizer	Evaluation merely rewrites subjective preferences
The path is open and the environment can verify progress	Agent	Success is unobservable, permissions are excessive, or errors are irreversible

Real systems can combine these structures. A router might send high-risk products to people and ordinary products into a chain; compliance and SEO checks may run in parallel; copy may receive at most two evaluator-optimizer rounds. Every edge still needs an input-output contract. Framework support does not make a connection correct by default.

What evidence does the article provide?

The source is an engineering article distilled from Anthropic’s own building experience and work with teams in multiple industries. It is not a randomized controlled study, and it does not benchmark the six structures under a single protocol. Its evidence contribution is different:

• a shared vocabulary from augmented LLMs through workflows to agents;
• recurring control-flow patterns observed in production practice;
• fit conditions, examples, and engineering cautions for each pattern;
• reasons domains such as customer support and coding can fit agents: goals unfold through conversation, tools can take action, results can return feedback, and human oversight can remain meaningful.

Use the evidence correctly

Treat the article as a source of design hypotheses and review checklists, then validate them on your own tasks. “An official engineering article recommends it” is not evidence that a pattern will be better under your data, model, and risk conditions.

Common failure modes

1. Framework first: choose a multi-agent framework, then search for a problem to fill it. Abstraction hides control flow and leaves only surface logs for debugging.
2. Overlong chains: every small step becomes another model call without adding a validation boundary. Latency rises while errors still propagate.
3. Wrong route, flawless execution: the specialized path passes all its checks under the wrong category. Evaluate routing separately and preserve an abstention path.
4. Fake parallelism: subtasks actually depend on the same unfinished intermediate result and must later be rerun.
5. Vague orchestration: workers lack delivery formats, evidence requirements, and boundaries, so the synthesizer receives incomparable outputs.
6. Self-congratulatory evaluator loop: scores move without an external-user or rule improvement. Require an iteration cap and independent acceptance.
7. No environmental ground truth: the model can only read its own previous prose, creating a closed “plan → claim success → plan again” loop.
8. No approval for high-impact actions: the system treats draft generation and real publication as equivalent tool calls, magnifying the consequence of an error.

Historical contribution and today’s engineering interpretation

The 2024 article’s core contribution

It did not invent every pattern. It organized common production structures into a selection map and clarified the architectural distinction between workflows and agents. More importantly, it rejected framework complexity as a measure of sophistication and returned the evaluation target to task performance and engineering cost.

Principles worth retaining today

• Begin with the smallest measurable system; only real failure evidence justifies more complexity.
• Draw control flow and feedback edges before choosing a framework. A framework cannot define success for you.
• Every agent turn needs environmental ground truth, while long loops need resource limits and human checkpoints.
• Tool descriptions, parameters, and error feedback deserve the same iterative testing as prompts, consistent with the ACI lesson in SWE-agent.
• Evaluate quality, cost, latency, recovery time, and human-intervention rate together.

What should not be copied mechanically

Models, SDKs, and frameworks change, as does the best implementation of each pattern. Customer experience in the article is not a universal causal law. “Prefer simplicity” does not mean rejecting complexity forever; it means requiring observable benefit before adopting it.

Connections to other landmarks

ReAct supplies the basic thought-action-observation loop, while AutoGen demonstrates multi-agent conversation orchestration. This article offers the engineering map for deciding when those structures are justified. Context Engineering and Harness Engineering then explain how a multi-turn system maintains attention and a verifiable environment.

Check your understanding—do not merely memorize pattern names

1. Why is an LLM with retrieval and tools not necessarily an agent?
2. For the product-launch task, what is the fundamental difference between Prompt Chaining and Orchestrator–Workers?
3. What conditions must hold before Evaluator–Optimizer is worthwhile?
4. Give three external success conditions, two stop limits, and one action requiring human approval for an autonomous launch agent.
5. If a fixed workflow already passes 96% of acceptance cases, what evidence would you require before approving a migration to an agent?

Answers and hints

1. Retrieval and tools augment capability. An agent is defined here by model-directed runtime control. If code fixes the call order, the system can remain a workflow.
2. A chain’s steps and order are known before execution. Orchestrator–Workers discovers the required subtasks from the specific input, delegates them, and synthesizes their results.
3. You need an explicit rubric, feedback the generator can act on, measurable gains from iteration, and both a pass threshold and iteration cap.
4. Example: required fields complete, compliance checks pass, and preview layout passes; no more than 20 steps or a fixed cost; clicking the real public-publish action requires approval.
5. Demand a representative class of real tasks that fixed paths genuinely cannot handle, a measured net benefit from agent behavior, and acceptable cost, latency, regression, permission, and recovery risk.

Previous: SWE-agent · Next: Effective Context Engineering · Back to the landmark reading index