- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
UniCache asks whether different parts of a multimodal KV cache should be compressed in different ways. Its main argument is that instruction tokens, semantic image features, and source-image latents play different roles, so applying one compression policy to all of them can lose important information. The framework first separates compressible segments from protected ones. A small offline calibration set determines whether each task–type pair should use eviction or quantization. During inference, attention statistics guide the allocation of storage across types and the reduction of the total budget over time. The paper evaluates understanding, generation, and editing on BAGEL and SenseNova-U1. It also provides an optimized GPU implementation with physical KV compaction and low-bit storage. The strongest results are the balanced editing quality on BAGEL and the reported throughput improvements at long context lengths.
Critical strengths
-
The distinction between cache types is well motivated. Source-image latents preserve spatial details, while instruction tokens carry information needed to perform the requested change. It makes sense that aggressively removing tokens could damage the former, while quantization could affect the latter differently. Treating these as separate compression decisions is a useful direction for unified models.
-
The single-type compression experiment helps support the main idea. Figure 5 compares eviction and quantization while compressing only one cache type and leaving the others unchanged. This is more informative than comparing complete systems alone. It provides some evidence that the preferred compression policy depends on the type of information being stored.
-
The paper goes beyond simulated compression. The GPU implementation physically removes evicted entries and packs quantized ones. Handling heterogeneous segments under a shared softmax, including the log-sum-exp merging described in Appendix E, is an important practical detail. The reported long-context throughput gains suggest that the approach can offer more than a reduction in nominal cache size.
Critical weaknesses
-
How the allocation rule can remain dynamic when the total budget is fixed. The rule appears to become a one-time allocation after the first compression step, even if attention later shifts between instruction and visual tokens. If the implementation allows redistribution through another mechanism, that mechanism needs to be described. Otherwise, a simple allocation made once at prefill seems like an important baseline.
-
A stronger controlled hybrid baseline would help isolate the contribution. Most comparisons apply a single existing compression method, whereas UniCache combines eviction and quantization with additional protection and scheduling rules. This demonstrates that the complete package works better in some settings, but does not tell us how much comes from the adaptive controller rather than a straightforward fixed hybrid policy.
-
The compression figures need a clearer accounting convention. Protected segments are excluded from the shared compression budget, and Appendix B says that the minimum retention ratio is calibrated to match an average compression target. It is not clear exactly what is averaged, whether the initial full-KV step is included, or how protected storage and quantization metadata enter the reported ratios.
Questions for authors
-
Can you compare conditioning-attention trajectories from FullKV and compressed runs, and test whether the feedback mechanism pushes budgets down when FullKV conditioning importance remains stable?
-
Please define the averaging over steps, the treatment of the initial full-KV step and protected segments, and whether quantization metadata and residual buffers are counted. Are Figure 8 and the main quality tables using the same configurations?
3 comments
Response to Reviewer 9HDt (2/2)
Q1: Can you compare conditioning-attention trajectories from FullKV and compressed runs, and test whether the feedback mechanism pushes budgets down when FullKV conditioning importance remains stable?
Reply: We thank the reviewer for this question.
Following the reviewer's suggestion, we run FullKV and UniCache with the same input and seed on three GenEval prompts and three PIE-Bench examples (49 denoising steps, 512px) and conduct two comparisons:
First, we compare their conditioning-attention trajectories, averaged over all layers of the conditional branch. As shown in Figure 12 in the revised manuscript, the trajectories of FullKV and UniCache are highly similar and our UniCache does not cause the conditioning-attention mass lower than FullKV.
Second, to test whether the feedback pushes budgets down when the FullKV conditioning importance remains stable, we recompute the retention ratio of the temporal budget schedule from the conditioning mass measured in the FullKV run, instead of the compressed run, and compare it with the retention ratio actually used by UniCache:
| Example | Attention-trajectory MAE (%) | Mean retention-ratio difference (%) |
|---|---|---|
| GenEval 152 | 0.004 | 0.007 |
| GenEval 437 | 0.009 | 0.010 |
| GenEval 501 | 0.008 | 0.017 |
| PIE random editing | 0.108 | 0.176 |
| PIE change object | 0.660 | 0.375 |
| PIE add object | 0.442 | 0.594 |
The results show that the feedback does not push budgets down when conditioning importance remains stable. In all six examples, the compressed attention trajectory closely follows the FullKV trajectory, and replaying the schedule with the FullKV statistics changes the mean retention ratio by at most 0.594%, so the budget decline comes from denoising itself. We include this comparison in the appendix of the revised manuscript.
W3 & Q2: Protected segments are excluded from the shared compression budget, and Appendix B says that the minimum retention ratio is calibrated to match an average compression target. It is not clear exactly what is averaged, whether the initial full-KV step is included, or how protected storage and quantization metadata enter the reported ratios. Please define the averaging over steps, the treatment of the initial full-KV step and protected segments, and whether quantization metadata and residual buffers are counted. Are Figure 8 and the main quality tables using the same configurations?
Reply: We thank the reviewer and clarify each point below.
Averaging over steps. The average compression ratio is the mean of the per-step compression ratios, where a step is one forward pass of the model during inference: a decoding step that generates one text token in understanding, or a denoising step of the image latent in generation and editing.
Initial full-KV step. It is included in the reported ratios as uncompressed.
Protected segments. Boundary and current-latent KVs are kept uncompressed and excluded from the compression ratio.
Whether quantization metadata and residual buffers are counted? Both are counted in the compressed storage, including per-group scales and minima, the BF16 residual tail, and the indices of retained tokens, for UniCache and all baselines.
Are Figure 8 and the main quality tables using the same configurations? Yes, both use the configuration described in Section 4.1 and Appendix B. We have added these definitions to the revised manuscript and highlighted them in blue.
Official Comment by Authors
Dear Reviewer 9HDt,
Thanks for your positive initial review. If our response effectively resolves your issue, we'd be grateful if you consider further improving your score.
Thank you!
Authors
Response to Reviewer 9HDt (1/2)
We thank the reviewer for recognizing our motivation, the single-policy compression experiments, and our GPU implementation. We will provide detailed responses to your questions.
W1: How can the allocation rule remain dynamic when the total budget is fixed? The rule appears to become a one-time allocation after the first compression step, even if attention later shifts between instruction and visual tokens. If the implementation allows redistribution through another mechanism, that mechanism needs to be described. Otherwise, a simple allocation made once at prefill seems like an important baseline.
Reply: We thank the reviewer for the question. We clarify how the allocation remains dynamic.
Clarification of dynamic allocation. First, we use each type's previous allocation as an upper limit: the cap reflects irreversible eviction, so a type whose attention decreases continues to be compressed, with a small portion of the budget potentially remaining unused. Second, in understanding, the KVs of newly generated tokens are also included in compression and can consume this unused budget, so the budget released is not wasted. Third, in generation and editing, the attention mass assigned to conditioning KVs decreases as denoising progresses. Empirically, candidate importance decreases largely monotonically in generation and editing, so evicted KVs rarely need to be recalled and irreversible eviction has little effect on the final result.
Comparison with a one-time allocation at prefill. Following the reviewer's suggestion, we compare UniCache with a variant that allocates the type-level budgets once from the prefill statistics and keeps them fixed. Both use the same policy map, protection, and compression ratios as the ablations in Table 3 (approximately 80% on MME and 60% on GenEval and PIE-Bench):
The results show that our per-chunk dynamic allocation outperforms the one-time allocation on all metrics, with the largest gains on editing. As analyzed in Section 3.3 and Figure 6, the model's reliance on conditioning KVs changes across inference steps, and dynamic allocation follows this shift by releasing the conditioning KVs that are no longer needed. A one-time allocation fixed at prefill cannot track this shift.
W2: A stronger controlled hybrid baseline would help isolate the contribution. Most comparisons apply a single existing compression method, whereas UniCache combines eviction and quantization with additional protection and scheduling rules. This demonstrates that the complete package works better in some settings, but does not tell us how much comes from the adaptive controller rather than a straightforward fixed hybrid policy.
Reply: We thank the reviewer for this suggestion. The component ablation in Table 3 already reflects the contribution of the hybrid policy from the opposite direction: the "w/o Task/Type Awareness" variant keeps the adaptive controller but replaces the hybrid policy with a uniform policy, and it degrades generation and editing.
The controlled hybrid baseline suggested by the reviewer complements this ablation by keeping the hybrid policy and removing the adaptive controller. We conduct this experiment by sharing the calibrated policy map and protection rules of UniCache, but applying the same retention ratio to all candidate types and keeping it constant over time. Following the ablation setting of the paper, we compare the two complementary ablations with UniCache:
The results show that neither component alone is sufficient. The hybrid policy alone degrades editing in both structure and semantics, while the adaptive controller alone preserves them but degrades generation and editing detail. Only their combination stays close to the original model on all three tasks.