The keyword from Pi dominating engineering discussions this week is Compaction — not deleting chat history, but having the LLM summarize older exchanges, preserve the recent ones, then resume. Pi reserves roughly 20,000 tokens (about 5 to 20 turns) for "recent conversation"; the rest is summarized by the model itself.
What this is
Every LLM has a context window limit — essentially, how much text it can process at once. Coding agents produce particularly "heavy" conversations: beyond user messages, they include system prompts, tool definitions, loaded code files, and the results of every tool call — quickly exceeding the limit as turns stack up.
Once the window fills, there are usually two paths: open a new session (losing all prior decisions and progress), or compress the history. Compaction takes the second route: an extra LLM call that summarizes older dialogue into a shorter digest, keeps the most recent segment raw, and lets the new conversation continue from there.
Pi's specific approach: it auto-triggers when a session nears the window limit, or users can manually type /compact to compress on demand. After compression, the most recent raw messages are kept (measured by token budget, not message count), with older portions replaced by summaries.
Industry view
We've noticed that coding-agent products are converging — Claude Code, Codex, and Pi have all shipped similar mechanisms. The problem they solve is concrete: complex coding tasks routinely involve dozens of tool-call rounds, and without compression they simply can't run to completion.
But there's a caution worth flagging. Research has shown that longer contexts degrade model output quality — dubbed "Context Rot." This means compaction isn't just about fitting things in; it's about helping the model "think clearly." The cost is an extra LLM call, raising both spend and latency. Summarization can also drop critical details, and implementations vary across vendors with no industry standard yet.
Impact on regular people
For enterprise IT: When procuring AI coding tools, don't just look at subscription fees — factor in the extra token calls triggered by compaction. The more complex the task, the higher the hidden cost.
For working professionals: The bar for "knowing how to use AI to write code" will rise again. The scarce skill isn't prompt-writing — it's decomposing tasks and managing context boundaries.
For consumer markets: Everyday users won't feel it yet, but long-conversation AI assistants (ChatGPT, Claude, etc.) will almost certainly adopt similar mechanisms. Expect "I seem to have forgotten what you said" notices to become more common.