Aller au contenu

LLM agentsContext engineeringAgent memoryMulti-agent systemsArchitectureAvancé

Par · Publiée le · 12 min de lecture

Sommaire
  1. Why the global context fills up
  2. Two levers: compress the window, or keep noise out of it
  3. Compression: rewrite the window into fewer tokens
  4. Sub-agents: context firewalls for exploration

Après cette fiche, vous saurez

  • Explain why the context of an agent loop grows with every tool call, and why that hurts both cost and answer quality.
  • Distinguish compression techniques (summarisation, tool-result clearing, external memory, prompt compression) by what each one keeps and what it discards.
  • Describe how an orchestrator delegates to sub-agents with isolated contexts and receives condensed reports.
  • Estimate the orchestrator's context growth with and without delegation using a simple token model.
  • Choose between compaction and delegation for a given task and anticipate their failure modes.

Formulaire

Les équations de la fiche, d’un coup d’œil

  1. 01Append-only agent loopCt=C0+∑i=1t(ai+oi),∑t=1TCt=T C0+(a+o) T(T+1)2C_t = C_0 + \sum_{i=1}^{t} \left(a_i + o_i\right), \qquad \sum_{t=1}^{T} C_t = T\,C_0 + (a+o)\,\frac{T(T+1)}{2}
  2. 02Threshold-triggered compactionC←C0+s+rwhenever C>θW,ρ=C−C0−rsC \leftarrow C_0 + s + r \quad \text{whenever } C > \theta W, \qquad \rho = \frac{C - C_0 - r}{s}
  3. 03Orchestrator versus sub-agent contextCorch=C0+∑k=1K(dk+rk),Cksub=C0′+dk+∑i=1nk(ak,i+ok,i)C^{\text{orch}} = C_0 + \sum_{k=1}^{K} \left(d_k + r_k\right), \qquad C^{\text{sub}}_k = C'_0 + d_k + \sum_{i=1}^{n_k} \left(a_{k,i} + o_{k,i}\right)

Agent architecture · Context engineering

Keeping the global agent lean: memory compression and sub-agents

An LLM agent re-reads its whole context at every step: the system prompt, the task, and every thought, tool call and tool result so far 1. On long tasks this window, which is the global agent's working memory, fills with stale observations. Two families of techniques keep it bounded. Compression rewrites what is already in the window into fewer tokens. Sub-agents stop the noise from entering it at all: they run exploration in separate contexts and return only a condensed result 7.

Sources:17

The problem

Why the global context fills up

Append-only agent loop
Ct=C0+∑i=1t(ai+oi),∑t=1TCt=T C0+(a+o) T(T+1)2C_t = C_0 + \sum_{i=1}^{t} \left(a_i + o_i\right), \qquad \sum_{t=1}^{T} C_t = T\,C_0 + (a+o)\,\frac{T(T+1)}{2}

Result derived in this sheet

The first identity is bookkeeping for a loop that only appends, such as a ReAct-style loop. The second assumes a constant amount added per step (ai=aa_i = a, oi=oo_i = o). Under that assumption the context grows linearly in TT, and the total number of prompt tokens read over the run grows quadratically. Tool results (file contents, web pages, logs) usually make oo much larger than aa. Prompt caching lowers the price of re-reading a stable prefix. It does not shrink the window, and it does not reduce the number of tokens the model must attend over.

Variables
SymbolMeaningUnit
CtC_tTokens in the agent's context at the start of step t + 1tokens
C0C_0Initial context: system prompt, tool definitions, task statementtokens
aia_iTokens generated at step i (reasoning and the tool call)tokens
oio_iTokens of the tool result (observation) returned at step itokens
a,oa, oConstant per-step averages of aᵢ and oᵢ (simplifying assumption)tokens
TTNumber of steps in the run

Sources:1

Sources:2

Architecture

Two levers: compress the window, or keep noise out of it

Orchestrator with a memory layer and isolated sub-agents

Architecture diagram. A user task enters the orchestrator, which reads and appends to its working context. A compactor summarises the working context when it nears its limit and returns a summary. The orchestrator writes notes to an external memory store and retrieves from it on demand. The orchestrator sends task specs to three sub-agents (literature search, codebase reading, test running). Each sub-agent exchanges many calls and raw results with tools, then returns a condensed report. The orchestrator synthesises the final answer.GLOBAL AGENTMEMORY LAYERSUB-AGENTS (ISOLATED CONTEXTS)User taskOrchestratorglobal agent · plans anddecidesWorking contextpinned goal · plan · recentturnsCompactorsummarise · clear old toolresultsExternal memorynotes files · vector storeSub-agent Asearch the literatureSub-agent Bread the codebaseSub-agent Crun the testsTools and environmentweb, files, shell, APIsFinal answergoalread / appendnear limitsummarywrite notesretrieve on demandspecspecspecraw resultsraw resultsraw resultsreportreportreportsynthesis
  • Global agent context
  • Compression and external memory
  • Delegation: spec out, condensed report back
  • Raw tool traffic, never enters the global context
  1. Task arrives — The orchestrator loads the goal into its working context and pins it there.
  2. Delegate — Separable subtasks go out as self-contained specs, in parallel when they are independent.
  3. Explore in isolation — Each sub-agent runs its own loop. Bulky tool output fills its own window, not the orchestrator's.
  4. Return condensed reports — Only findings, evidence pointers and open questions come back.
  5. Compact and persist — Near the threshold, old turns are summarised and old tool results cleared. Decisions are written to external notes and retrieved when needed.
  6. Synthesise — The orchestrator merges the reports and produces the answer.
Raw tool traffic (dotted) stays inside the sub-agents' contexts. Only task specs and condensed reports (teal) cross into the global context, and the compactor and external memory (amber) bound what remains.

Lever 1

Compression: rewrite the window into fewer tokens

Compression techniques differ mainly in what they keep in the window and whether what they drop can be recovered.
TechniqueKeeps in the windowDropsMain risk
Rolling summarisation (compaction)A model-written summary of older turns, plus the most recent turns verbatim 7Verbatim historyDetails the summariser judged unimportant are lost for good; summaries of summaries drift
Tool-result clearingThe tool call and a placeholder or pointer to the stored output 7Bulky old observationsThe agent must re-run or re-read the tool if it needs the data again
External memory with paging (MemGPT-style)A small main context; the agent itself moves information in and out of external storage 3Nothing permanently: data lives outside the windowRetrieval misses; extra calls spent managing memory
Structured notes / scratchpad filesAgent-written plan, decisions and open questions, re-read when needed 7The raw traceNotes are only as good as the agent's discipline in writing them
Memory stream with reflectionMemories retrieved by recency, importance and relevance, plus higher-level reflections 4Low-scoring memories (they stay in the store)Reflections can encode wrong inferences that later steps trust
Token-level prompt compression (LLMLingua)Tokens that a small language model judges informative 5Low-information tokensLossy for exact strings such as code, identifiers and numbers

Sources:7345

Threshold-triggered compaction
C←C0+s+rwhenever C>θW,ρ=C−C0−rsC \leftarrow C_0 + s + r \quad \text{whenever } C > \theta W, \qquad \rho = \frac{C - C_0 - r}{s}

Result derived in this sheet

Between compactions the context grows as in the append-only model. After each compaction it falls back to about C0+s+rC_0 + s + r, so the context follows a sawtooth bounded by θW\theta W. Choosing θ<1\theta < 1 leaves room for the summarisation call itself. A larger compression ratio ρ\rho frees more space and loses more detail. Clearing tool results first is the cheapest reduction, and it can be undone if the raw output is stored.

Variables
SymbolMeaningUnit
WWContext window of the modeltokens
θ\thetaTrigger fraction of the window, 0 < θ < 1
ssLength of the summary that replaces older turnstokens
rrRecent turns kept verbatim after compactiontokens
ρ\rhoCompression ratio of the summarised segment

Lever 2

Sub-agents: context firewalls for exploration

Orchestrator versus sub-agent context
Corch=C0+∑k=1K(dk+rk),Cksub=C0′+dk+∑i=1nk(ak,i+ok,i)C^{\text{orch}} = C_0 + \sum_{k=1}^{K} \left(d_k + r_k\right), \qquad C^{\text{sub}}_k = C'_0 + d_k + \sum_{i=1}^{n_k} \left(a_{k,i} + o_{k,i}\right)

Result derived in this sheet

The orchestrator pays dk+rkd_k + r_k tokens per subtask, however many steps nkn_k the sub-agent took. The cost of exploration lives in a context that is discarded once the report is returned. The total spend of the system is the sum over all contexts. Delegation therefore lowers the peak load on any single window, but it usually raises the total number of tokens processed.

Variables
SymbolMeaningUnit
CorchC^{\text{orch}}Orchestrator context after K delegated subtasks (its own reasoning turns omitted)tokens
CksubC^{\text{sub}}_kContext of sub-agent k at the end of its subtasktokens
KKNumber of delegated subtasks
dkd_kTask specification sent to sub-agent ktokens
rkr_kCondensed report returned by sub-agent ktokens
C0′C'_0Sub-agent system prompt and tool definitionstokens
nkn_kSteps taken by sub-agent k
ak,i,ok,ia_{k,i}, o_{k,i}Generated tokens and observation tokens at step i of sub-agent ktokens
  1. Decompose

    Split the goal into subtasks that share little state, and run them in parallel when they are independent.

  2. Write a self-contained spec

    State the objective, scope boundaries, allowed tools, output format and a stopping criterion. The sub-agent sees none of the orchestrator's history, so dkd_k must carry everything it needs.

  3. Explore in isolation

    The sub-agent runs its own loop. Its ∑iok,i\sum_i o_{k,i} of raw output stays in its own window.

  4. Return a condensed report

    Return findings, evidence pointers (paths, URLs, line numbers), confidence and open questions, within a budget of rkr_k tokens.

  5. Integrate and verify

    Merge the reports, resolve conflicts, spot-check the pointers, and record decisions in external notes.

Sources:67

Global context size under the three strategies

Line chart of context tokens against tool calls, from 0 to 20. The append-only single agent rises linearly from 2,000 to 32,000 tokens. The single agent with compaction rises to 15,500, drops to 4,000 at call 10, rises to 16,000, and drops again at call 19, forming a sawtooth. The orchestrator with sub-agents rises in steps of 1,000 every 5 calls, from 2,000 to 6,000.0500010,00015,00020,00025,00030,00035,00002468101214161820Tool calls performed by the systemTokens in the global agent's context (tokens)Compaction threshold θW = 16kSingle agent, append-onlySingle agent with compactionOrchestrator with sub-agentsSYNTHETIC DATA
Line chart of context tokens against tool calls, from 0 to 20. The append-only single agent rises linearly from 2,000 to 32,000 tokens. The single agent with compaction rises to 15,500, drops to 4,000 at call 10, rises to 16,000, and drops again at call 19, forming a sawtooth. The orchestrator with sub-agents rises in steps of 1,000 every 5 calls, from 2,000 to 6,000.010,00020,00030,00040,00005101520Tool calls performed by the systemTokens in the global agent's context (tokens)Compaction thresholdθW = 16kSingle agent, append-onlySingle agent with compactionOrchestrator with sub-agentsSYNTHETIC DATA
Synthetic, computed from the token model above: C0=2000C_0 = 2000 and a+o=1500a + o = 1500 tokens per tool call. Compaction triggers at θW=16000\theta W = 16000 and falls back to 4,000 tokens. The orchestrator delegates blocks of 5 tool calls with dk+rk=1000d_k + r_k = 1000. Each sub-agent's own context peaks near 9,000 tokens before being discarded.
View the data
Line chart of context tokens against tool calls, from 0 to 20. The append-only single agent rises linearly from 2,000 to 32,000 tokens. The single agent with compaction rises to 15,500, drops to 4,000 at call 10, rises to 16,000, and drops again at call 19, forming a sawtooth. The orchestrator with sub-agents rises in steps of 1,000 every 5 calls, from 2,000 to 6,000. — Synthetic data
SeriesTool calls performed by the systemTokens in the global agent's context (tokens)
Single agent, append-only02000
Single agent, append-only13500
Single agent, append-only25000
Single agent, append-only36500
Single agent, append-only48000
Single agent, append-only59500
Single agent, append-only611,000
Single agent, append-only712,500
Single agent, append-only814,000
Single agent, append-only915,500
Single agent, append-only1017,000
Single agent, append-only1118,500
Single agent, append-only1220,000
Single agent, append-only1321,500
Single agent, append-only1423,000
Single agent, append-only1524,500
Single agent, append-only1626,000
Single agent, append-only1727,500
Single agent, append-only1829,000
Single agent, append-only1930,500
Single agent, append-only2032,000
Single agent with compaction02000
Single agent with compaction13500
Single agent with compaction25000
Single agent with compaction36500
Single agent with compaction48000
Single agent with compaction59500
Single agent with compaction611,000
Single agent with compaction712,500
Single agent with compaction814,000
Single agent with compaction915,500
Single agent with compaction104000
Single agent with compaction115500
Single agent with compaction127000
Single agent with compaction138500
Single agent with compaction1410,000
Single agent with compaction1511,500
Single agent with compaction1613,000
Single agent with compaction1714,500
Single agent with compaction1816,000
Single agent with compaction194000
Single agent with compaction205500
Orchestrator with sub-agents02000
Orchestrator with sub-agents12000
Orchestrator with sub-agents22000
Orchestrator with sub-agents32000
Orchestrator with sub-agents42000
Orchestrator with sub-agents53000
Orchestrator with sub-agents63000
Orchestrator with sub-agents73000
Orchestrator with sub-agents83000
Orchestrator with sub-agents93000
Orchestrator with sub-agents104000
Orchestrator with sub-agents114000
Orchestrator with sub-agents124000
Orchestrator with sub-agents134000
Orchestrator with sub-agents144000
Orchestrator with sub-agents155000
Orchestrator with sub-agents165000
Orchestrator with sub-agents175000
Orchestrator with sub-agents185000
Orchestrator with sub-agents195000
Orchestrator with sub-agents206000
Compression and delegation attack the same problem at different points in the loop.
CriterionCompression in one contextLever 1Delegation to sub-agentsLever 2
Where the noise goesEnters the window and is removed laterNever enters the global window
Global context growthSawtooth bounded by θW\theta Wdk+rkd_k + r_k per subtask, independent of nkn_k
Total tokensLower: one context, plus the summarisation callsHigher: Anthropic reports that its multi-agent research system used about 15× the tokens of a chat interaction 6
LatencySequential; each compaction adds a model callIndependent subtasks can run in parallel
Who decides what is lostThe summariserThe sub-agent writing the report; the orchestrator never sees the rest
Coordination costNoneSpec quality, duplicated work, conflicting findings
Best fitLong sequential work on tightly coupled stateBreadth-first work with separable subtasks; less suited to tasks with heavy shared context and dependencies, such as most coding work 6

Compression and delegation attack the same problem at different points in the loop.

Compression in one contextLever 1

Where the noise goes
Enters the window and is removed later
Global context growth
Sawtooth bounded by θW\theta W
Total tokens
Lower: one context, plus the summarisation calls
Latency
Sequential; each compaction adds a model call
Who decides what is lost
The summariser
Coordination cost
None
Best fit
Long sequential work on tightly coupled state

Delegation to sub-agentsLever 2

Where the noise goes
Never enters the global window
Global context growth
dk+rkd_k + r_k per subtask, independent of nkn_k
Total tokens
Higher: Anthropic reports that its multi-agent research system used about 15× the tokens of a chat interaction 6
Latency
Independent subtasks can run in parallel
Who decides what is lost
The sub-agent writing the report; the orchestrator never sees the rest
Coordination cost
Spec quality, duplicated work, conflicting findings
Best fit
Breadth-first work with separable subtasks; less suited to tasks with heavy shared context and dependencies, such as most coding work 6

VerdictThe two levers compose. An orchestrator can delegate exploration to sub-agents and still compact its own history on very long runs. Delegate when subtasks are separable; compress when the state is tightly shared.

Sources:6

Summary drift

Repeated compaction summarises summaries, so early constraints get paraphrased away. Mitigation: pin the goal and constraints in a section that compaction never touches, and keep a verbatim decision log in external notes.

Irrecoverable exact details

Identifiers, paths and numbers are the first things a summary loses. Mitigation: store raw tool outputs outside the window and keep pointers in context. Clearing with a pointer preserves recoverability; summarising does not.

Telephone effect in delegation

A report is a lossy compression, and it can be wrong. The orchestrator cannot check what it never saw. Mitigation: require evidence pointers in every report, and spot-check them before acting on a finding.

Under-specified delegation

Vague specs lead sub-agents to duplicate each other's work or misread the scope 6. Mitigation: give each spec an explicit objective, boundaries, output format and stopping criterion.

Cost explosion

Every sub-agent re-reads its own prompt and history, so total tokens scale with the number of sub-agents and their steps. Mitigation: scale the number of sub-agents and their step budgets to the difficulty of the task.

Sources:6

  • Keep the goal, the constraints and the current plan in a pinned prefix that compaction never rewrites.
  • Clear old tool results before summarising anything else, and keep a pointer to the stored raw output.
  • Trigger compaction at θW\theta W well below the window, leaving room for the summarisation call.
  • Give every sub-agent a self-contained spec dkd_k and an explicit report budget rkr_k.
  • Ask for evidence pointers in every report, and verify a sample of them.
  • Evaluate task success and total tokens, not only the peak size of the global context.

QuestionAn orchestrator delegates K=4K = 4 subtasks. Each sub-agent makes 30 tool calls averaging 2,000 observation tokens each. Specs are dk=300d_k = 300 tokens and reports are rk=700r_k = 700 tokens. By how many tokens does the orchestrator's context grow?

  1. A4,000 tokens
  2. B240,000 tokens
  3. C244,000 tokens
  4. D1,000 tokens
Show the answer
  • A — Correct. ∑k(dk+rk)=4×(300+700)=4000\sum_k (d_k + r_k) = 4 \times (300 + 700) = 4000. The sub-agents' observations never enter the orchestrator's window.
  • B — Incorrect. That is 4×30×20004 \times 30 \times 2000, the observation tokens spread across the four sub-agent contexts, which are discarded after each report.
  • C — Incorrect. This adds the sub-agents' observations to the specs and reports. That is the growth you would get without delegation, plus overhead.
  • D — Incorrect. That is the cost of a single subtask; there are K=4K = 4 of them.

References

  1. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao (2023). ReAct: Synergizing Reasoning and Acting in Language Models. International Conference on Learning Representations (ICLR). arxiv.org/abs/2210.03629 unverified
  2. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics. arxiv.org/abs/2307.03172 unverified
  3. Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez (2023). MemGPT: Towards LLMs as Operating Systems. arXiv. arxiv.org/abs/2310.08560 unverified
  4. Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein (2023). Generative Agents: Interactive Simulacra of Human Behavior. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST). arxiv.org/abs/2304.03442 unverified
  5. Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, Lili Qiu (2023). LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). arxiv.org/abs/2310.05736 unverified
  6. (2025). How we built our multi-agent research system. Anthropic Engineering. www.anthropic.com/engineering/built-multi-agent-research-… unverifiedEngineering blog post; check the token-usage figure and the remarks on task fit against the published text.
  7. (2025). Effective context engineering for AI agents. Anthropic Engineering. www.anthropic.com/engineering/effective-context-engineeri… unverifiedEngineering blog post describing compaction, tool-result clearing, structured note-taking and sub-agent architectures.

Partager cette fiche

Mots-clés

En vedetteIARecherche d’information

Le pipeline RAG, de l’indexation à la réponse

La génération augmentée par récupération donne à un modèle de langage des passages pertinents au moment de répondre. Sa qualité se joue d’abord à la récupération : ce qui n’est pas retrouvé ne peut pas être cité.

Intermédiaire · 11 min de lecture ·

En vedetteÉconométrieFinance

Le modèle GARCH(1,1) et la dynamique de la volatilité

Comment une récurrence à trois paramètres reproduit les grappes de volatilité des rendements financiers, à quelles conditions elle est stationnaire et comment on l’estime par quasi-maximum de vraisemblance.

Intermédiaire · 10 min de lecture ·

En vedetteIAApprentissage profond

Mélange d’experts et routage top-k

Un mélange d’experts confie chaque entrée à quelques sous-réseaux choisis par un routeur. Le nombre de paramètres augmente avec le nombre d’experts, mais le calcul par jeton ne dépend que de kk.

Avancé · 11 min de lecture ·