Context Engineering: Budgets, Compaction and Memory for Long-Running Agents

Context is the scarcest resource in an agentic system. How to budget it, compact it, retrieve into it and offload it so long-running agents stay accurate and affordable.

Ask an experienced agent developer where their production incidents come from and the answer is rarely "the model got it wrong." It is that step nineteen was reasoning over a transcript stuffed with step-three noise, a stale file version and two contradictory tool results. Large context windows did not solve this; they made it cheaper to create.

Context engineering is the practice of deciding, per iteration, exactly what the model should see — and what it must not. It is the highest-leverage lever in agentic systems after loop termination.

Why more context degrades quality

Attention dilution. Retrieval accuracy drops as irrelevant material grows, especially in the middle of a long window.
Stale-fact contamination. An early version of a file or record stays in the transcript and competes with the current one.
Instruction drift. System rules stated once at the top lose salience 40,000 tokens later.
Cost and latency. Context is re-sent every iteration; a bloated transcript taxes every remaining step.

Build a context budget

Treat the window as an allocated resource with named partitions, not a bucket you fill until it errors. A workable split for a coding or research agent:

Partition	Share	Contents
Instructions	~10%	Role, hard constraints, output contract, tool descriptions
Task state	~15%	Goal, progress ledger, remaining budget, open questions
Working set	~50%	Current artefacts and the most recent tool results
Retrieved knowledge	~15%	Top-k documents with provenance
Headroom	~10%	Reserved for the response and one large tool result

When a partition overflows, the fix belongs to that partition — compact the working set, re-rank retrieval, tighten instructions. Global truncation is the worst available option because it silently drops the newest, most relevant material or the oldest, most binding rules.

Compaction: summarise structure, not prose

Compaction replaces a span of transcript with a condensed representation. Freeform summaries lose the details agents actually need, so compact into a schema:

Established facts — verified findings with their source.
Artefacts touched — paths or record IDs with current state.
Decisions and rationale — including approaches already ruled out, which prevents re-exploration.
Open items — what remains, in priority order.
Verbatim carve-outs — error strings, IDs, credentials handles, exact user requirements. Never paraphrase these.

Trigger compaction on a threshold (say 60% of the window) rather than at failure, and always keep the last one or two raw turns so the model retains immediate continuity.

Offload instead of remembering

The most reliable context saving is not putting something in context at all. Write intermediate output to files or a database and pass references. An agent that stores a 40-page report on disk and keeps a one-line pointer can revisit it precisely when needed, and pays nothing for it in the meantime. The same logic applies to tool design: tools that return summaries with a follow-up "fetch detail" call beat tools that dump everything.

Retrieval hygiene

Retrieval-augmented context has its own failure modes. Four rules:

Re-rank before injecting. Vector top-k is a candidate set, not an answer set.
Attach provenance to every chunk — source, timestamp, version — so the model can prefer fresh material and cite it.
Deduplicate aggressively. Three near-identical chunks triple the confidence of one claim without adding evidence.
Prefer targeted queries over broad ones. The agent should ask a question, not download a corpus.

Memory tiers

Not all memory belongs in the transcript. Separate the tiers explicitly:

Scratch — this iteration only; discard immediately.
Task — the progress ledger; survives compaction, dies with the task.
Session — user preferences and corrections within a conversation.
Durable — stable facts, conventions and constraints worth reloading in future runs.

Write to durable memory only for rules and preferences, never for narration of what happened. The test is simple: would a fresh session make a worse decision without this? If not, it is noise.

Instrument it

Log per-iteration token counts by partition, compaction events with before/after sizes, retrieved-chunk usage, and the ratio of context tokens to output tokens. When quality degrades late in a run, these four numbers usually name the cause before you read a single transcript.

Checklist

Context is partitioned with explicit shares and headroom.
Compaction is schema-driven, threshold-triggered and preserves verbatim critical strings.
Large artefacts live outside context behind references.
Retrieved chunks carry provenance and are re-ranked and deduplicated.
Memory tiers are distinct; durable memory holds rules, not narration.
Per-partition token usage is observable.

Context engineering pairs directly with loop engineering for termination discipline and graph engineering for structuring multi-stage work. If you are exposing your own data to agents, our MCP server tester and AI search readiness checker are free places to start, and our AI app development team builds these systems end to end.