Has this happened to you? You’ve been chatting with Claude or ChatGPT for an hour, and something degrades: it “forgets” what you said at the beginning, mixes up instructions, goes in circles. Not a bug: a context window reaching saturation, and a tool handling it more or less gracefully.
In the tokens article, we saw that everything in the context is paid for on every turn. Today, one floor up: how tools compress, summarize and forget, the concrete patterns behind Claude, ChatGPT, Copilot and the others. Let’s take it apart together. You’ll see: it’s not rocket science.
The work desk (and its two diseases)
The image to keep: the context window is the model’s desk. Everything it knows about your conversation sits on it: your messages, its answers, open files, tool outputs. What’s not on the desk doesn’t exist for it.
That desk has two diseases:
- The wall. The window is finite: 128,000, 200,000, sometimes a million tokens depending on the model. Once full, that’s it: something must be thrown away to put something else down.
- Dilution, the sneaky disease. Well before the wall, quality drops. A model drowning in 150,000 tokens retrieves information less reliably than at 20,000. Research even showed the middle of the context is retained worst (lost in the middle). A big desk covered in papers is not an efficient desk.
The corollary that stings: the giant window is not the magic fix. Gemini and its million-token window push back the wall, not the dilution (nor the bill). Hence a whole craft: context engineering. So how do the tools cope? Six patterns, from bluntest to finest.
Pattern 1: the sliding window (throw away the oldest papers)
The primitive method: when the desk overflows, throw away the oldest documents. That’s the sliding window of early chatbots, and ChatGPT’s baseline behavior in a very long conversation: the beginning eventually falls out of the window, silently.
Simple, but brutal: the oldest papers are often the most important ones, like the initial brief and the constraints set at the start. Like on a building site: toss the first page of the folder, and it’s usually the blueprint that ends up in the skip. That’s exactly the “it forgot what I told it at the beginning”. Nobody does only this anymore, but it’s the base brick.
Pattern 2: compaction (summarize before throwing away)
The star pattern: compaction.
- Claude Code does auto-compact: approaching the limit, it writes a structured summary of the conversation (decisions made, files touched, task state, next steps), then replaces the detailed history with that summary. The conversation continues, lightened.
- Copilot CLI does the same at 95% of the gauge, and
/compacttriggers it on demand.
The image: before clearing the desk, you write a synthesis sheet and keep only that. The subtlety few people see: the model itself writes the sheet, and the whole art lies in what it preserves. Decisions and constraints: yes. The forty back-and-forths it took to get there: no.
The word of honesty: a summary is a chosen loss. The detail the sheet didn’t keep is gone for good. That’s why, after a compaction, an agent can “forget” a nuance mentioned three hours earlier. Compaction manages saturation; it doesn’t cancel it.
Pattern 3: selective forgetting (drop the big stale blocks)
More surgical: instead of summarizing everything, identify the bulky blocks that became useless and drop only those. Suspect number one: old tool outputs. The 3,000-line listing the agent read an hour ago served its purpose: the decision derived from it is recorded; the listing itself is now dead weight.
That’s the principle of the Claude API’s context editing: clearing old tool results while keeping the conversation thread. On the desk: throw away the draft printouts, keep the notes taken on them.
Pattern 4: externalization (the sticky note instead of memory)
The most elegant pattern, and the reflex of modern agents: write outside the context. Rather than keeping a 2,000-token plan on the desk for the whole session, the agent writes it to a file (TODO.md, a plan, working notes) and keeps only a pointer on the desk: “the plan is in docs/plan.md”. Needs the detail? It re-reads the file, uses it, and the context breathes again.
You’ve seen it at work without naming it: Claude Code maintains its memory files and task lists, autonomous agents keep their logbooks. The disk is infinite and free; the desk is small and expensive: what must last goes to disk.
Pattern 5: just-in-time loading (bring only the useful file)
The mirror of the previous one: never put everything on the desk upfront. That’s RAG and its librarian (the three right excerpts at the right time), but also the daily mechanics of coding agents: they don’t load your whole project, they run targeted searches and open only the relevant files. Just-in-time context: the right information, at the right time, in small quantity.
(Developers who organize their code into small self-contained folders win twice here; we explain why on our dev blog Sense of [Blog]: a feature that fits in one folder is a folder that fits on the desk.)
Pattern 6: separate desks (delegate to avoid clutter)
The team pattern: each specialist works at their own desk, and reports only their conclusion to the boss. Copilot CLI’s Explore agent digs through the code on its own and comes back with three paragraphs: the 50,000 tokens of exploration never touched your main context. Context isolation is a form of compression. See the idea? Nothing compresses better than what never landed on the desk.
What your tools do, at a glance
| Tool | Its visible strategy |
|---|---|
| Claude / Claude Code | auto-compact with structured summary + externalization (memory files) + context editing on the API side |
| ChatGPT | sliding window + condensation in long conversations + its separate “memory” (see our article on AI memory) |
| Copilot CLI | /context gauge, auto-compaction at 95%, /compact, subagents at separate desks |
| Gemini | the very big table (1M tokens): useful, but dilution and the bill remain |
In summary
- The context window has two diseases: the wall (hard limit) and dilution (quality dropping well before). The big window only treats the first.
- Six patterns, from bluntest to finest: slide (drop the old), compact (summarize before dropping), forget selectively (big stale blocks), externalize (the sticky note on disk), load just-in-time (the librarian), separate the desks (delegate).
- Compaction is a chosen loss: sometimes the right move is still a fresh conversation and a clean brief.
- And the best compression is not dirtying the desk: short contexts, sliced tasks.
A well-kept desk rather than a giant desk: that’s context engineering in a nutshell. And the natural sequel follows: if the context is working memory, disposable by nature… how do AIs remember you? That’s exactly the topic of our article on AI memory. See? It’s not rocket science.
