What Is Context Compression?

Context Compression in AI Explainer

Context compression shortens the information passed to a language model while trying to retain what matters for the current task. It can remove redundant sentences, summarize passages, rank tokens, or extract only relevant evidence.

Compression may happen before retrieval results enter the prompt or after a long conversation accumulates. Some methods use a smaller model to estimate which tokens are informative, while others create task-focused summaries or compact memories.

A shorter context can reduce latency, cost, and distraction. It may let an application include evidence from more documents within a fixed window and help the model focus on details directly related to the question.

Compression is lossy unless it merely removes exact duplicates. A subtle exception, negation, date, or attribution may disappear. Systems should preserve provenance, protect critical fields, compare against uncompressed baselines, and allow retrieval of the original text.

Context compression reduces the amount of text supplied to a model while trying to preserve the information most important for the task.

Key terms include context window, compression, prompt, token, summary, relevance, provenance, information loss and baseline.

Compression removes redundancy or selects the parts judged most relevant to the task.

The saved context budget can reduce cost or make room for additional evidence.

A compact prompt can lose qualifications, dates, negations, or source attribution.

Original passages and provenance should remain available for verification and recovery.

Explore more "Explainers"

Discover additional explainers across politics, science, business, technology, and other fields. Each explainer breaks down a complex idea into clear, everyday language—helping you better understand how major concepts, systems, and debates shape the world around us.