Agent architecture · Context engineering
Keeping the global agent lean: memory compression and sub-agents
An LLM agent re-reads its whole context at every step: the system prompt, the task, and every thought, tool call and tool result so far 1. On long tasks this window, which is the global agent's working memory, fills with stale observations. Two families of techniques keep it bounded. Compression rewrites what is already in the window into fewer tokens. Sub-agents stop the noise from entering it at all: they run exploration in separate contexts and return only a condensed result 7.
The problem
Why the global context fills up
Result derived in this sheet
The first identity is bookkeeping for a loop that only appends, such as a ReAct-style loop. The second assumes a constant amount added per step (, ). Under that assumption the context grows linearly in , and the total number of prompt tokens read over the run grows quadratically. Tool results (file contents, web pages, logs) usually make much larger than . Prompt caching lowers the price of re-reading a stable prefix. It does not shrink the window, and it does not reduce the number of tokens the model must attend over.
| Symbol | Meaning | Unit |
|---|---|---|
| Tokens in the agent's context at the start of step t + 1 | tokens | |
| Initial context: system prompt, tool definitions, task statement | tokens | |
| Tokens generated at step i (reasoning and the tool call) | tokens | |
| Tokens of the tool result (observation) returned at step i | tokens | |
| Constant per-step averages of aᵢ and oᵢ (simplifying assumption) | tokens | |
| Number of steps in the run |
Sources:1
Sources:2
Architecture
Two levers: compress the window, or keep noise out of it
Orchestrator with a memory layer and isolated sub-agents
- Global agent context
- Compression and external memory
- Delegation: spec out, condensed report back
- Raw tool traffic, never enters the global context
- Task arrives — The orchestrator loads the goal into its working context and pins it there.
- Delegate — Separable subtasks go out as self-contained specs, in parallel when they are independent.
- Explore in isolation — Each sub-agent runs its own loop. Bulky tool output fills its own window, not the orchestrator's.
- Return condensed reports — Only findings, evidence pointers and open questions come back.
- Compact and persist — Near the threshold, old turns are summarised and old tool results cleared. Decisions are written to external notes and retrieved when needed.
- Synthesise — The orchestrator merges the reports and produces the answer.
Lever 1
Compression: rewrite the window into fewer tokens
| Technique | Keeps in the window | Drops | Main risk |
|---|---|---|---|
| Rolling summarisation (compaction) | A model-written summary of older turns, plus the most recent turns verbatim 7 | Verbatim history | Details the summariser judged unimportant are lost for good; summaries of summaries drift |
| Tool-result clearing | The tool call and a placeholder or pointer to the stored output 7 | Bulky old observations | The agent must re-run or re-read the tool if it needs the data again |
| External memory with paging (MemGPT-style) | A small main context; the agent itself moves information in and out of external storage 3 | Nothing permanently: data lives outside the window | Retrieval misses; extra calls spent managing memory |
| Structured notes / scratchpad files | Agent-written plan, decisions and open questions, re-read when needed 7 | The raw trace | Notes are only as good as the agent's discipline in writing them |
| Memory stream with reflection | Memories retrieved by recency, importance and relevance, plus higher-level reflections 4 | Low-scoring memories (they stay in the store) | Reflections can encode wrong inferences that later steps trust |
| Token-level prompt compression (LLMLingua) | Tokens that a small language model judges informative 5 | Low-information tokens | Lossy for exact strings such as code, identifiers and numbers |
Result derived in this sheet
Between compactions the context grows as in the append-only model. After each compaction it falls back to about , so the context follows a sawtooth bounded by . Choosing leaves room for the summarisation call itself. A larger compression ratio frees more space and loses more detail. Clearing tool results first is the cheapest reduction, and it can be undone if the raw output is stored.
| Symbol | Meaning | Unit |
|---|---|---|
| Context window of the model | tokens | |
| Trigger fraction of the window, 0 < θ < 1 | ||
| Length of the summary that replaces older turns | tokens | |
| Recent turns kept verbatim after compaction | tokens | |
| Compression ratio of the summarised segment |
Lever 2
Sub-agents: context firewalls for exploration
Result derived in this sheet
The orchestrator pays tokens per subtask, however many steps the sub-agent took. The cost of exploration lives in a context that is discarded once the report is returned. The total spend of the system is the sum over all contexts. Delegation therefore lowers the peak load on any single window, but it usually raises the total number of tokens processed.
| Symbol | Meaning | Unit |
|---|---|---|
| Orchestrator context after K delegated subtasks (its own reasoning turns omitted) | tokens | |
| Context of sub-agent k at the end of its subtask | tokens | |
| Number of delegated subtasks | ||
| Task specification sent to sub-agent k | tokens | |
| Condensed report returned by sub-agent k | tokens | |
| Sub-agent system prompt and tool definitions | tokens | |
| Steps taken by sub-agent k | ||
| Generated tokens and observation tokens at step i of sub-agent k | tokens |
Decompose
Split the goal into subtasks that share little state, and run them in parallel when they are independent.
Write a self-contained spec
State the objective, scope boundaries, allowed tools, output format and a stopping criterion. The sub-agent sees none of the orchestrator's history, so must carry everything it needs.
Explore in isolation
The sub-agent runs its own loop. Its of raw output stays in its own window.
Return a condensed report
Return findings, evidence pointers (paths, URLs, line numbers), confidence and open questions, within a budget of tokens.
Integrate and verify
Merge the reports, resolve conflicts, spot-check the pointers, and record decisions in external notes.
Global context size under the three strategies
View the data
| Series | Tool calls performed by the system | Tokens in the global agent's context (tokens) |
|---|---|---|
| Single agent, append-only | 0 | 2000 |
| Single agent, append-only | 1 | 3500 |
| Single agent, append-only | 2 | 5000 |
| Single agent, append-only | 3 | 6500 |
| Single agent, append-only | 4 | 8000 |
| Single agent, append-only | 5 | 9500 |
| Single agent, append-only | 6 | 11,000 |
| Single agent, append-only | 7 | 12,500 |
| Single agent, append-only | 8 | 14,000 |
| Single agent, append-only | 9 | 15,500 |
| Single agent, append-only | 10 | 17,000 |
| Single agent, append-only | 11 | 18,500 |
| Single agent, append-only | 12 | 20,000 |
| Single agent, append-only | 13 | 21,500 |
| Single agent, append-only | 14 | 23,000 |
| Single agent, append-only | 15 | 24,500 |
| Single agent, append-only | 16 | 26,000 |
| Single agent, append-only | 17 | 27,500 |
| Single agent, append-only | 18 | 29,000 |
| Single agent, append-only | 19 | 30,500 |
| Single agent, append-only | 20 | 32,000 |
| Single agent with compaction | 0 | 2000 |
| Single agent with compaction | 1 | 3500 |
| Single agent with compaction | 2 | 5000 |
| Single agent with compaction | 3 | 6500 |
| Single agent with compaction | 4 | 8000 |
| Single agent with compaction | 5 | 9500 |
| Single agent with compaction | 6 | 11,000 |
| Single agent with compaction | 7 | 12,500 |
| Single agent with compaction | 8 | 14,000 |
| Single agent with compaction | 9 | 15,500 |
| Single agent with compaction | 10 | 4000 |
| Single agent with compaction | 11 | 5500 |
| Single agent with compaction | 12 | 7000 |
| Single agent with compaction | 13 | 8500 |
| Single agent with compaction | 14 | 10,000 |
| Single agent with compaction | 15 | 11,500 |
| Single agent with compaction | 16 | 13,000 |
| Single agent with compaction | 17 | 14,500 |
| Single agent with compaction | 18 | 16,000 |
| Single agent with compaction | 19 | 4000 |
| Single agent with compaction | 20 | 5500 |
| Orchestrator with sub-agents | 0 | 2000 |
| Orchestrator with sub-agents | 1 | 2000 |
| Orchestrator with sub-agents | 2 | 2000 |
| Orchestrator with sub-agents | 3 | 2000 |
| Orchestrator with sub-agents | 4 | 2000 |
| Orchestrator with sub-agents | 5 | 3000 |
| Orchestrator with sub-agents | 6 | 3000 |
| Orchestrator with sub-agents | 7 | 3000 |
| Orchestrator with sub-agents | 8 | 3000 |
| Orchestrator with sub-agents | 9 | 3000 |
| Orchestrator with sub-agents | 10 | 4000 |
| Orchestrator with sub-agents | 11 | 4000 |
| Orchestrator with sub-agents | 12 | 4000 |
| Orchestrator with sub-agents | 13 | 4000 |
| Orchestrator with sub-agents | 14 | 4000 |
| Orchestrator with sub-agents | 15 | 5000 |
| Orchestrator with sub-agents | 16 | 5000 |
| Orchestrator with sub-agents | 17 | 5000 |
| Orchestrator with sub-agents | 18 | 5000 |
| Orchestrator with sub-agents | 19 | 5000 |
| Orchestrator with sub-agents | 20 | 6000 |
| Criterion | Compression in one contextLever 1 | Delegation to sub-agentsLever 2 |
|---|---|---|
| Where the noise goes | Enters the window and is removed later | Never enters the global window |
| Global context growth | Sawtooth bounded by | per subtask, independent of |
| Total tokens | Lower: one context, plus the summarisation calls | Higher: Anthropic reports that its multi-agent research system used about 15× the tokens of a chat interaction 6 |
| Latency | Sequential; each compaction adds a model call | Independent subtasks can run in parallel |
| Who decides what is lost | The summariser | The sub-agent writing the report; the orchestrator never sees the rest |
| Coordination cost | None | Spec quality, duplicated work, conflicting findings |
| Best fit | Long sequential work on tightly coupled state | Breadth-first work with separable subtasks; less suited to tasks with heavy shared context and dependencies, such as most coding work 6 |
Compression and delegation attack the same problem at different points in the loop.
Compression in one contextLever 1
- Where the noise goes
- Enters the window and is removed later
- Global context growth
- Sawtooth bounded by
- Total tokens
- Lower: one context, plus the summarisation calls
- Latency
- Sequential; each compaction adds a model call
- Who decides what is lost
- The summariser
- Coordination cost
- None
- Best fit
- Long sequential work on tightly coupled state
Delegation to sub-agentsLever 2
- Where the noise goes
- Never enters the global window
- Global context growth
- per subtask, independent of
- Total tokens
- Higher: Anthropic reports that its multi-agent research system used about 15× the tokens of a chat interaction 6
- Latency
- Independent subtasks can run in parallel
- Who decides what is lost
- The sub-agent writing the report; the orchestrator never sees the rest
- Coordination cost
- Spec quality, duplicated work, conflicting findings
- Best fit
- Breadth-first work with separable subtasks; less suited to tasks with heavy shared context and dependencies, such as most coding work 6
VerdictThe two levers compose. An orchestrator can delegate exploration to sub-agents and still compact its own history on very long runs. Delegate when subtasks are separable; compress when the state is tightly shared.
Sources:6
Summary drift
Repeated compaction summarises summaries, so early constraints get paraphrased away. Mitigation: pin the goal and constraints in a section that compaction never touches, and keep a verbatim decision log in external notes.
Irrecoverable exact details
Identifiers, paths and numbers are the first things a summary loses. Mitigation: store raw tool outputs outside the window and keep pointers in context. Clearing with a pointer preserves recoverability; summarising does not.
Telephone effect in delegation
A report is a lossy compression, and it can be wrong. The orchestrator cannot check what it never saw. Mitigation: require evidence pointers in every report, and spot-check them before acting on a finding.
Under-specified delegation
Vague specs lead sub-agents to duplicate each other's work or misread the scope 6. Mitigation: give each spec an explicit objective, boundaries, output format and stopping criterion.
Cost explosion
Every sub-agent re-reads its own prompt and history, so total tokens scale with the number of sub-agents and their steps. Mitigation: scale the number of sub-agents and their step budgets to the difficulty of the task.
Sources:6
- Keep the goal, the constraints and the current plan in a pinned prefix that compaction never rewrites.
- Clear old tool results before summarising anything else, and keep a pointer to the stored raw output.
- Trigger compaction at well below the window, leaving room for the summarisation call.
- Give every sub-agent a self-contained spec and an explicit report budget .
- Ask for evidence pointers in every report, and verify a sample of them.
- Evaluate task success and total tokens, not only the peak size of the global context.
QuestionAn orchestrator delegates subtasks. Each sub-agent makes 30 tool calls averaging 2,000 observation tokens each. Specs are tokens and reports are tokens. By how many tokens does the orchestrator's context grow?
- A4,000 tokens
- B240,000 tokens
- C244,000 tokens
- D1,000 tokens
Show the answer
- A — Correct. . The sub-agents' observations never enter the orchestrator's window.
- B — Incorrect. That is , the observation tokens spread across the four sub-agent contexts, which are discarded after each report.
- C — Incorrect. This adds the sub-agents' observations to the specs and reports. That is the growth you would get without delegation, plus overhead.
- D — Incorrect. That is the cost of a single subtask; there are of them.
References
- (2023). ReAct: Synergizing Reasoning and Acting in Language Models. International Conference on Learning Representations (ICLR). arxiv.org/abs/2210.03629 unverified
- (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics. arxiv.org/abs/2307.03172 unverified
- (2023). MemGPT: Towards LLMs as Operating Systems. arXiv. arxiv.org/abs/2310.08560 unverified
- (2023). Generative Agents: Interactive Simulacra of Human Behavior. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST). arxiv.org/abs/2304.03442 unverified
- (2023). LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). arxiv.org/abs/2310.05736 unverified
- (2025). How we built our multi-agent research system. Anthropic Engineering. www.anthropic.com/engineering/built-multi-agent-research-… unverifiedEngineering blog post; check the token-usage figure and the remarks on task fit against the published text.
- (2025). Effective context engineering for AI agents. Anthropic Engineering. www.anthropic.com/engineering/effective-context-engineeri… unverifiedEngineering blog post describing compaction, tool-result clearing, structured note-taking and sub-agent architectures.