Prefix cache
A technique that reuses the already-computed attention state (KV cache) for a shared prompt prefix so later turns or agents avoid recomputing the same tokens from scratch.
Learn
When to use it
Use a prefix cache when recomputing the same prompt prefix for each model call becomes inefficient. Prefix caches store the already-computed attention state for shared prefixes, enabling faster processing for tasks like multi-turn dialogues or coordinated agent interactions.
Quick example
In DeepSeek-Reasonix, AI coding agents often need to maintain context across multiple turns of conversation. By implementing a prefix cache, the system reuses the attention state of the initial prompt prefix, avoiding redundant computation in subsequent turns. DeepSeek-Reasonix includes this feature to enhance efficiency and responsiveness in continuous coding sessions.
Ecosystem
Prefix caches are part of the optimization techniques around model inference, often paired with caching strategies and context management.
prompt → prefix cache → model → response
Misconceptions
| Misconception | Rebuttal |
|---|---|
| It stores full responses | It only caches the attention state of the prefix |
| It's a general-purpose cache | It's specific to shared prompt prefixes |
Trade-offs
- Efficiency — requires careful management of cache state
- Speed — gains depend on prefix reuse frequency
- Memory usage — additional storage for cached states