Effective Context Engineering: managing an agent’s limited attention

Original source

Anthropic’s Applied AI team, 2025-09-29: Effective context engineering for AI agents.

This is an original learning guide, not a reproduction. Copyright and the complete article remain with the original source.

What you should be able to do after this guide

• Distinguish prompt engineering from context engineering and list the main components that enter a single inference.
• Treat context as a finite attention budget and use “the smallest sufficient set of high-signal tokens” as a decision rule, rather than optimizing for either shortest or longest.
• Design a full lifecycle for a long task: pinned core, just-in-time retrieval, deduplication and packing, execution feedback, external notes, compaction, and recovery.
• Decide what must remain verbatim, what needs only a pointer, what can be summarized, and what should be removed.
• Choose correctly among compaction, structured note-taking, and multi-agent work rather than treating all three as the same kind of “memory.”

The idea in one sentence

Context engineering does not stuff every potentially useful item into the model window. It selects the smallest sufficient set of high-signal tokens from a growing information universe before every inference, then continuously updates, externalizes, and cleans that working state after each action.

Why a well-written prompt is no longer enough

Prompt engineering primarily concerns the wording and organization of instructions. That may dominate a one-shot classification or generation task. A long-running agent, however, can receive all of the following in one inference:

• system instructions and permission boundaries;
• available tools and parameter documentation;
• a small number of examples;
• user messages and conversation history;
• retrieved documents, code, and web excerpts;
• tool results, logs, and test output;
• the current plan, to-do list, and external memory;
• new evidence produced by earlier actions.

Every action adds more candidate information. The problem is no longer just “How do I write the perfect instruction at the beginning?” It becomes “What should the model see on this turn? What survives into the next turn? How do we preserve continuity when the window approaches its limit?” That is the scope of context engineering.

Define four objects first

Object	Meaning	Example
Information universe	All historical and external information the task might access	Entire repository, all logs, documentation store, message history
Candidate context	Material that may help this turn and awaits selection	Search hits, latest test, relevant rule, current plan
Packed context	Tokens actually provided to this model inference	System prompt, tool definitions, selected code, and summaries
External state	Persistent information outside this turn’s window that can be reread through a pointer	Files, databases, structured notes, artifact URLs

A larger window does not mean every token will be used with equal reliability. The article uses a finite “attention budget” and diminishing returns as its mental model: irrelevant, duplicated, conflicting, and low-signal material competes with what matters. “Smallest” therefore does not mean shortest. It means no unnecessary burden and no deletion of constraints, evidence, or state required to finish the task.

A long window does not remove the need for context engineering

Capacity answers “Can this material fit?” It does not automatically answer “Can the model reliably find and apply the correct item among large amounts of similar, conflicting, or stale information?”

The context lifecycle: a loop, not a one-time assembly step

flowchart TD
    U["Information universe: repository, docs, history, tool results"] --> C["Candidate inventory"]
    C --> H["Pinned priority: goal, permissions, hard constraints"]
    C --> R["Just-in-time retrieval: paths, queries, relevant excerpts"]
    C --> N["Recent state: plan, open questions, latest evidence"]
    H --> F["Deduplicate, rank, and pack to budget"]
    R --> F
    N --> F
    F --> M["Model reasons and chooses an action"]
    M --> O["Environment observation: result, error, test, log"]
    O --> J{"Same phase and healthy window?"}
    J -->|"Yes"| C
    J -->|"Near limit / phase boundary"| X["External notes + compaction"]
    X --> D["Discard redundant raw output; preserve evidence pointers"]
    D --> C

Input, output, state, feedback, and stopping for one cycle

Part	What must be explicit
Input	Current goal, hard constraints, available tools, candidate material, and token budget
State	Current subgoal, completed work, open questions, latest evidence, provenance locations, and remaining budget
Control flow	Pack pinned items first; retrieve by relevance and freshness; deduplicate; rank; reduce overflow to summaries or pointers
Output	Context for this inference plus recoverable references to material not selected
Feedback	New facts, errors, tool output, and state changes from the action, returned to the candidate pool
Phase stop	When acceptance passes, human judgment is required, or the window nears a threshold, write a checkpoint and compact before starting a new context phase

“Stop” does not necessarily terminate the entire agent. It may terminate one context phase and resume the next from a summary, external notes, and evidence pointers.

How to design five core context components

1. System prompt: choose the right altitude

The article identifies two extremes. One hard-codes every branch into a brittle pseudo-program; the other says little more than “do the right thing” and assumes shared background. A useful altitude should:

• define role, goal, permissions, and inviolable boundaries clearly;
• provide strong heuristics without enumerating every path;
• separate background, instructions, tool guidance, and output contracts into clear sections;
• begin with a small complete version and add rules in response to observed failures.

“Minimal” is not synonymous with “short.” If safety, data format, or approval constraints are necessary, they must remain complete.

2. Tool definitions: actions that also control incoming context

Tools change the environment and return new material to the window. Good tools have a focused responsibility, little overlap, clear parameters, recoverable errors, and token-efficient output. If a human cannot confidently choose between two tools, the model will also waste attention at that decision.

Useful result controls include filtering, pagination, limits, summaries, provenance, and on-demand expansion. Returning one hundred thousand log lines is rarely “more informative”; it pushes retrieval work into the finite window.

3. Examples: a few canonical cases, not an edge-case inventory

Few-shot examples turn abstract rules into shapes the model can imitate. Stuffing every edge case into the prompt creates competing patterns. Choose a small, diverse, canonical set of positive and negative examples, with each example illustrating an important decision boundary.

4. Retrieval: move from preloading to just in time

A just-in-time strategy first exposes lightweight identifiers—file paths, saved queries, web links, or directory indexes—and lets the agent load local data for the current question. Filenames, directories, timestamps, and sizes also carry purpose signals, enabling progressive disclosure:

inspect directory map → choose relevant package → search symbol → open local file
                      → run targeted test → expand only code tied to the failure

This keeps the window focused and avoids preloading every object. The tradeoff is slower runtime exploration and a need for usable navigation tools and heuristics. A hybrid is often practical: preload stable material that must always apply, and retrieve large or dynamic data on demand.

5. Message history and working state: preserve causality, not every byte

A long-running agent must know why it reached the current state, but does not need every log forever. The most valuable history includes goals, key decisions, verified facts, abandoned paths with reasons, unresolved risks, and source locations. Raw tool output superseded by a newer result can leave the active context while its conclusion and pointer remain.

A complete example: a multi-hour Python configuration migration

This is a teaching example. The goal is to migrate legacy YAML configuration to a new schema while retaining CLI compatibility and updating tests. We use “100 attention units” as a relative budget; these are not tokens from any real model.

Phase 1: build a task map

Material	Treatment	Reason
User goal, preserve old CLI, change only target directory	Pin	Every action must obey it
Repository entry guide, test command, directory map	Keep summary and paths	Supports navigation without preloading all documentation
All 800 configuration files	Keep only directory and query entry points	Full text is unnecessary now
Retrieved schema definition and two representative samples	Expand verbatim	Current diagnosis directly depends on them

The agent uses perhaps 35 relative units to establish the map instead of loading all 800 files.

Phase 2: localize and design

The agent searches the parser entry point, reads old and new schemas, runs one failing fixture, and writes a structured note:

## Goal
Migrate YAML to schema v2 without breaking legacy CLI input.
 
## Verified
- Parser entry: `src/config/load.py`
- Current failing fixture: `tests/fixtures/legacy_timeout.yml`
- `timeout` changed from scalar to `{value, unit}`.
 
## Decision
- Normalize legacy scalar at the parser boundary.
 
## Open
- Does export need to preserve legacy spelling?
 
## Evidence pointers
- Test run: artifact `runs/184/pytest.txt`
- Schema: `src/config/schema_v2.py:40`

The note does not copy the entire test output. It preserves traceable locations and keeps Verified, Decision, and Open distinct.

Phase 3: expand on demand during implementation

Implementation context contains only the parser, related types, target tests, and latest failure summary. Old search results have served their localization purpose and collapse to one line. Unrelated READMEs, successful installation logs, and duplicate traces are removed. When a tool returns a new test failure, the prior failure does not remain at equal priority; it is marked superseded.

Phase 4: compact before the window is exhausted

Before the threshold, the system writes a checkpoint and compacts:

• must preserve: goal, hard constraints, architecture decision, changed files, unresolved export question, and latest test status;
• may summarize: three explored approaches and reasons for rejection;
• pointer only: long test log, complete diff, and original schema;
• may discard: repeated success output, the same resolved syntax error, and file snippets replaced by newer state.

The new window resumes from the compacted summary, structured note, and recently relevant files—not from a vague sentence such as “We made many changes and things look good.”

Phase 5: isolate an independent exploration

If export compatibility requires deep investigation across several modules, a sub-agent can receive that bounded task in a clean context. It reads substantial material but returns only a conclusion, evidence paths, confidence, open questions, and a recommended action. The lead does not inherit the full exploration trace, isolating its detail cost.

Finally, the lead runs full acceptance, marks the note complete, and preserves the final report and test artifact. Temporary search output no longer enters future tasks.

Three long-task strategies solve different problems

Strategy	Core operation	Best fit	Main risk
Compaction	Summarize a nearly full message history with high fidelity and continue in a fresh window	Continuous conversation with a recent causal chain	Aggressive compression removes details whose importance emerges later
Structured note-taking	Persist regular structured state outside the window and reread it later	Milestones, cross-session work, or long iteration	Notes go stale or an inference is recorded as fact
Multi-agent	Give focused subtasks independent windows and return compressed deliverables	Deep independent exploration where parallelism pays	Lost evidence, duplicate work, or unclear subtask boundaries

The article advises optimizing compaction for recall first and precision second: preserve every potentially critical item before trimming redundancy. Clearing obsolete raw tool results deep in the history is one relatively safe starting point.

An implementable context selector

def build_context(task, candidates, budget):
    pinned = select_hard_requirements(task)  # goal, safety, permissions
    recent = select_latest_state(candidates)
    retrieved = retrieve_for_current_subgoal(task.subgoal, candidates)
 
    items = deduplicate(pinned + recent + retrieved)
    items = mark_provenance_and_freshness(items)
    items = rank_by(
        items,
        requiredness=True,
        relevance=True,
        recency=True,
        evidence_quality=True,
    )
 
    packed = []
    for item in items:
        view = item.full_text
        if not fits(packed, view, budget):
            view = item.summary_with_pointer
        if fits(packed, view, budget):
            packed.append(view)
 
    assert contains_goal_constraints_and_open_state(packed)
    return packed

A production system will be more complicated, but the important rules are visible: pin hard constraints; retrieve around the current subgoal; preserve only the latest trustworthy version of duplicates; attach provenance and freshness to summaries; and, under pressure, degrade to “summary + pointer” rather than arbitrary truncation.

What should stay, and what should leave?

Content type	Default policy
Goal, permissions, safety, and data contracts	Pin; changes must be explicit
Latest failure, current code, and open blocker	High-priority retention
Verified facts	Preserve claim, verification method, and evidence pointer
Design decisions	Preserve decision, rationale, alternatives, and date
Long raw logs	Keep the latest relevant excerpt; store the rest externally with a pointer
Observation superseded by newer evidence	Mark stale and remove from active context
Repeated explanations and social filler	Usually discard
Unverified inference	May remain, but label it hypothesis rather than compacting it into fact

Removal is task-dependent. “The code is long” is not a reason to remove it when that code is the current edit target. “It was generated recently” is not a reason to preserve an unrelated new log.

Evidence and boundaries of the source

The source is an official engineering article for agent builders. It combines Anthropic product experience, customer practice, and references to long-context research to propose the attention-budget model, the smallest high-signal token set, just-in-time retrieval, and three long-task strategies. It is not a randomized comparison of every strategy under one task protocol, and it does not provide a universal token recipe for all models.

Separate evidence levels

“The article reports that the team uses a strategy” is a verified source fact. “The same strategy will improve your system” remains a hypothesis to test. Evaluate representative-task success, error types, cost, latency, and post-compaction continuity—not token count alone.

Common failure modes

1. Context dumping: preload an entire repository or every log because each item “might help,” leaving no priority structure.
2. Instruction conflict: old rules, current rules, and examples coexist without versions or an authority hierarchy.
3. Retrieval without provenance: a summary enters context without a path or timestamp, so it cannot be checked and may be stale.
4. Over-compression: only conclusions remain; constraints, rejected alternatives, and open questions disappear, causing repeated mistakes.
5. Fact promotion during compaction: “the model suspects a cache problem” becomes “the root cause is the cache.”
6. Rotting structured notes: code changes while a note remains the assumed source of truth; notes need ownership, freshness, and executable checks.
7. Under-specified sub-agent return: one sentence arrives without evidence, scope, or confidence, leaving the lead unable to verify it.
8. Optimizing only token count: context gets shorter while task success declines. Desired behavior—not the smallest bill—is the actual objective.

Historical contribution and today’s engineering interpretation

Contribution of the original article

It expands the core work of a long-running agent from “write a good prompt” to “continuously maintain the full inference state,” defining context as the tokens actually present at sampling time. System prompts, tools, examples, retrieval, history, compaction, notes, and multi-agent designs can therefore be reasoned about under one attention budget.

Engineering principles you can use now

• Keep raw evidence in externally addressable storage; place only the current view and pointers into context.
• Record provenance, time, authority, and supersession status for context items.
• Treat compaction as a tested state transition: afterward, the system must still answer the goal, current state, next step, blockers, and key decisions.
• Build task-level evaluations for retrieval and compaction that catch missed constraints, stale information, and repeated exploration.
• Begin with a minimum viable tool set whose results expand on demand. Tool count is not a capability metric.

How this differs from memory

Generative Agents focuses on which experiences deserve long-term storage, retrieval, and reflection. Context here is every token entering one inference; memory is one possible source. The context window, external storage, and memory policy are not interchangeable.

Check your understanding: can you make the tradeoffs?

1. Why does “the smallest high-signal token set” not mean “make the system prompt as short as possible”?
2. During a migration, when should complete test output enter context, and when should only a pointer remain?
3. What continuity problem does compaction solve, and what different problem does structured note-taking solve?
4. What fields must a sub-agent return to be both compressed and verifiable?
5. How can you detect that compaction promoted a model inference into a fact?

Answers and hints

1. “Smallest” is constrained by task sufficiency. Necessary permission, safety, format, and acceptance rules remain even when they are long; deleting them destroys sufficiency.
2. Keep the relevant verbatim excerpt while diagnosing the current failure. Once resolved or needed only for audit, externalize the full log and retain the result, run ID, and path.
3. Compaction compresses a growing conversational trace while preserving recent causality. Structured notes persist milestone state outside the window for cross-window or cross-session recovery.
4. At minimum: task scope, conclusion, evidence location, verification method, confidence or uncertainty, open questions, and recommended next action.
5. Compare fact tables before and after compaction; every verified claim should retain an evidence pointer, while unsupported material must retain a hypothesis or inference label.

Previous: Building Effective Agents · Next: Harness Engineering · Back to the landmark reading index