Assembling Context

Context Is Built Fresh Every Turn

Here's the part people miss: the context window isn't set up once and left alone. It's rebuilt on every turn of the agent's loop. Each step, a little pipeline decides what goes on the desk this time - what to add, what to keep, what to clear off. Context engineering is really the design of that pipeline.

The Four Questions

Every assembly decision boils down to four questions. Memorize these - they're the entire discipline in four lines:

1. WHAT do we fetch?      (which docs / facts / files)
2. WHEN do we fetch it?   (up front, or just in time)
3. HOW do we compress it? (summarize, truncate, re-rank)
4. WHEN do we throw away? (evict stale or finished context)

Questions 1-2 are the retrieval pillar. Questions 3-4 are budget management - the part beginners skip, and the part that separates a demo from a production agent.

The Assembly Pipeline

Put concretely, here's what runs each turn:

USER INPUT
   │
   ├─▶ run several retrievals in parallel
   │     (keyword search, semantic search, file reads, db lookups)
   │
   ├─▶ MERGE all the candidates into one big pile
   │     (this pile is almost always TOO BIG for the budget)
   │
   ├─▶ RE-RANK and keep only the top few (more on this below)
   │
   ├─▶ layer in: system instructions + memory + tool defs
   │
   ▼
FINAL CONTEXT  →  send to the model  →  it acts  →  repeat next turn

Re-Ranking: The Make-or-Break Step

The merge step produces way more candidate chunks than the budget allows. Re-rankingis the step that scores every candidate against the actual question and keeps only the best handful. This is where context pipelines quietly succeed or fail.

WITHOUT re-ranking          WITH re-ranking
  retrieve 50 chunks          retrieve 50 chunks (high recall)
  dump all 50 on the desk     score each vs. the question
  hope the answer's in there  keep the top 5 (high precision)
  → bloated, rot, lost-       → tight, high-signal desk
    in-the-middle

Token-Budget Management

This is the "compress" and "throw away" questions made practical. The key insight: cut low-signal content before it hits the desk, not after. Common moves:

• Truncate tool outputs        (keep the useful head/tail, drop the rest)
• Compact old conversation     (summarize, as in Lesson 69)
• Drop low-relevance chunks    (below a score threshold → don't include)
• Cap each retrieval           (no single search can flood the desk)
• Clear finished tool results  (once read, the raw output can go)

Track the Budget Like a Bill

Every retrieval, every tool call, every re-rank shows up in cost and latency. Production teams track tokens-used and tool-calls per task, set budgets, and get alerted when an agent blows past them. It's the same discipline as watching a cloud bill - an agent quietly using 5× the context it needs is a quietly 5× expensive agent.

Hands-On: Be the Re-Ranker