DeepSeek’s test results show that when a new model built on the same foundation reuses an old KV Cache, first-token latency for long-context requests can improve by as much as 5.08x. However, Qwen testing produced a 78.30% harmful flip rate, meaning this optimization cannot be enabled by default across the board.
What this is
KV Cache is the intermediate state saved after a model processes context. It avoids repeatedly recalculating long prompts, RAG documents—information retrieved first and then used by the model to answer—and fixed instructions. The paper “Can KV Cache Be Reused Across Model Updates?” tested four sets of model upgrades, comparing task accuracy, first-token latency, and output behavior after old caches were passed to new models.
The results do not follow one universal rule: the harmful flip rate was just 0.53% for DeepSeek-V4.1-Flash to Flash-1017 and 4.22% for GLM. It reached 26.53% for DeepSeek-V4-Flash and 78.30% for Qwen3-30B to Instruct-2507. In suitable combinations, the median first-token latency speedup on HotpotQA was 5.08x. When the prefix is short, the overhead of reading the cache may even reduce speed.
Industry view
We believe this research turns the question of whether to invalidate caches during a model upgrade from a matter of intuition into a measurable engineering decision. For long-context inference services, cross-version reuse can directly reduce waiting time and computing costs, while allowing the same application infrastructure to support rapidly evolving model versions.
The risks are equally clear. An old cache essentially preserves the old model’s internal representation of the text. Similar capabilities between old and new models do not mean their behavior is fully compatible. An increase in overall accuracy may also conceal previously correct answers being changed to incorrect ones. The paper introduces a “functional cache divergence” metric specifically to identify cases where the surface-level answer remains unchanged but the internal probability distribution has shifted. In practice, deployments should at least run replay tests by model pair, prefix length, and task type, and establish automatic invalidation thresholds. The 5.08x speedup cannot be treated as a universal benefit.
Impact on regular people
For enterprise IT: With this approach, a model upgrade may eliminate one round of long-context recomputation, but organizations must add cache compatibility testing and rollback mechanisms. Whether the savings cover the cost of validation remains the key question.
For individual professionals: This will not directly change how we work, but it may make enterprise knowledge bases, customer support, and document analysis systems faster and cheaper to operate. At the same time, companies need to reassess whether answers remain stable after model upgrades.
For the consumer market: Cache reuse may lower the cost and accessibility threshold for AI applications, but incorrect answers may still increase silently. Whether the user experience improves ultimately depends on whether vendors properly validate model versions.