acceptodds
Under review as a conference paper at ICLR 2027

UniCache: Task- and Type-Aware KV Cache Compression for Unified Multimodal Models

Abstract

Unified multimodal models combine understanding, generation, and editing within a single network, offering a promising foundation for versatile multimodal applications. However, growing multimodal contexts make KV cache storage and access increasingly costly. Existing KV cache compression methods can reduce these costs, but applying a single policy uniformly can degrade quality across tasks, as shared attention brings together heterogeneous cache types with different attention distributions and temporal dynamics. Based on these findings, we propose UniCache, a training-free framework for task- and type-aware KV cache compression. UniCache identifies the cache segments activated by each task and assigns suitable compression policies to different segment types. It coordinates their parallel execution under a shared storage budget through attention-guided allocation and task-aware temporal scheduling. Experiments show that UniCache maintains balanced quality at high compression ratios: compresses KV cache 5 for understanding and editing and 2.5 for generation with negligible loss, and increases throughput up to 1.78, significantly improving the practicality of scaling unfied multimodal model to longer context length.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
unified multimodal models, kv cache compression, mixture of transformers

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_9HDtOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

UniCache asks whether different parts of a multimodal KV cache should be compressed in different ways. Its main argument is that instruction tokens, semantic image features, and source-image latents play different roles, so applying one compression policy to all of them can lose important information. The framework first separates compressible segments from protected ones. A small offline calibration set determines whether each task–type pair should use eviction or quantization. During inference, attention statistics guide the allocation of storage across types and the reduction of the total budget over time. The paper evaluates understanding, generation, and editing on BAGEL and SenseNova-U1. It also provides an optimized GPU implementation with physical KV compaction and low-bit storage. The strongest results are the balanced editing quality on BAGEL and the reported throughput improvements at long context lengths.

Critical strengths
  1. The distinction between cache types is well motivated. Source-image latents preserve spatial details, while instruction tokens carry information needed to perform the requested change. It makes sense that aggressively removing tokens could damage the former, while quantization could affect the latter differently. Treating these as separate compression decisions is a useful direction for unified models.

  2. The single-type compression experiment helps support the main idea. Figure 5 compares eviction and quantization while compressing only one cache type and leaving the others unchanged. This is more informative than comparing complete systems alone. It provides some evidence that the preferred compression policy depends on the type of information being stored.

  3. The paper goes beyond simulated compression. The GPU implementation physically removes evicted entries and packs quantized ones. Handling heterogeneous segments under a shared softmax, including the log-sum-exp merging described in Appendix E, is an important practical detail. The reported long-context throughput gains suggest that the approach can offer more than a reduction in nominal cache size.

Critical weaknesses
  1. How the allocation rule can remain dynamic when the total budget is fixed. The rule appears to become a one-time allocation after the first compression step, even if attention later shifts between instruction and visual tokens. If the implementation allows redistribution through another mechanism, that mechanism needs to be described. Otherwise, a simple allocation made once at prefill seems like an important baseline.

  2. A stronger controlled hybrid baseline would help isolate the contribution. Most comparisons apply a single existing compression method, whereas UniCache combines eviction and quantization with additional protection and scheduling rules. This demonstrates that the complete package works better in some settings, but does not tell us how much comes from the adaptive controller rather than a straightforward fixed hybrid policy.

  3. The compression figures need a clearer accounting convention. Protected segments are excluded from the shared compression budget, and Appendix B says that the minimum retention ratio is calibrated to match an average compression target. It is not clear exactly what is averaged, whether the initial full-KV step is included, or how protected storage and quantization metadata enter the reported ratios.

Questions for authors
  1. Can you compare conditioning-attention trajectories from FullKV and compressed runs, and test whether the feedback mechanism pushes budgets down when FullKV conditioning importance remains stable?

  2. Please define the averaging over steps, the treatment of the initial full-KV step and protected segments, and whether quantization metadata and residual buffers are counted. Are Figure 8 and the main quality tables using the same configurations?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 9HDt (1/2)

We thank the reviewer for recognizing our motivation, the single-policy compression experiments, and our GPU implementation. We will provide detailed responses to your questions.

W1: How can the allocation rule remain dynamic when the total budget is fixed? The rule appears to become a one-time allocation after the first compression step, even if attention later shifts between instruction and visual tokens. If the implementation allows redistribution through another mechanism, that mechanism needs to be described. Otherwise, a simple allocation made once at prefill seems like an important baseline.

Reply: We thank the reviewer for the question. We clarify how the allocation remains dynamic.

Clarification of dynamic allocation. First, we use each type's previous allocation as an upper limit: the cap reflects irreversible eviction, so a type whose attention decreases continues to be compressed, with a small portion of the budget potentially remaining unused. Second, in understanding, the KVs of newly generated tokens are also included in compression and can consume this unused budget, so the budget released is not wasted. Third, in generation and editing, the attention mass assigned to conditioning KVs decreases as denoising progresses. Empirically, candidate importance decreases largely monotonically in generation and editing, so evicted KVs rarely need to be recalled and irreversible eviction has little effect on the final result.

Comparison with a one-time allocation at prefill. Following the reviewer's suggestion, we compare UniCache with a variant that allocates the type-level budgets once from the prefill statistics and keeps them fixed. Both use the same policy map, protection, and compression ratios as the ablations in Table 3 (approximately 80% on MME and 60% on GenEval and PIE-Bench):

AllocationMME Total ↑GenEval ↑PIE Structure ↓PIE PSNR ↑PIE CLIP Target ↑
One-time allocation at prefill2374.530.7660.12117.69127.957
UniCache2377.360.7800.10119.10028.171

The results show that our per-chunk dynamic allocation outperforms the one-time allocation on all metrics, with the largest gains on editing. As analyzed in Section 3.3 and Figure 6, the model's reliance on conditioning KVs changes across inference steps, and dynamic allocation follows this shift by releasing the conditioning KVs that are no longer needed. A one-time allocation fixed at prefill cannot track this shift.

W2: A stronger controlled hybrid baseline would help isolate the contribution. Most comparisons apply a single existing compression method, whereas UniCache combines eviction and quantization with additional protection and scheduling rules. This demonstrates that the complete package works better in some settings, but does not tell us how much comes from the adaptive controller rather than a straightforward fixed hybrid policy.

Reply: We thank the reviewer for this suggestion. The component ablation in Table 3 already reflects the contribution of the hybrid policy from the opposite direction: the "w/o Task/Type Awareness" variant keeps the adaptive controller but replaces the hybrid policy with a uniform policy, and it degrades generation and editing.

The controlled hybrid baseline suggested by the reviewer complements this ablation by keeping the hybrid policy and removing the adaptive controller. We conduct this experiment by sharing the calibrated policy map and protection rules of UniCache, but applying the same retention ratio to all candidate types and keeping it constant over time. Following the ablation setting of the paper, we compare the two complementary ablations with UniCache:

SettingMME Total ↑GenEval ↑PIE Structure ↓PIE PSNR ↑PIE CLIP Target ↑
Original BAGEL2373.210.7810.10118.80528.174
Hybrid policy only2368.330.7660.12617.21226.957
Adaptive controller only2372.600.7670.10317.87528.221
UniCache2377.360.7800.10119.10028.171

The results show that neither component alone is sufficient. The hybrid policy alone degrades editing in both structure and semantics, while the adaptive controller alone preserves them but degrades generation and editing detail. Only their combination stays close to the original model on all three tasks.

AuthorsOpenReview ↗

Response to Reviewer 9HDt (2/2)

Q1: Can you compare conditioning-attention trajectories from FullKV and compressed runs, and test whether the feedback mechanism pushes budgets down when FullKV conditioning importance remains stable?

Reply: We thank the reviewer for this question.

Following the reviewer's suggestion, we run FullKV and UniCache with the same input and seed on three GenEval prompts and three PIE-Bench examples (49 denoising steps, 512px) and conduct two comparisons:

First, we compare their conditioning-attention trajectories, averaged over all layers of the conditional branch. As shown in Figure 12 in the revised manuscript, the trajectories of FullKV and UniCache are highly similar and our UniCache does not cause the conditioning-attention mass lower than FullKV.

Second, to test whether the feedback pushes budgets down when the FullKV conditioning importance remains stable, we recompute the retention ratio of the temporal budget schedule from the conditioning mass measured in the FullKV run, instead of the compressed run, and compare it with the retention ratio actually used by UniCache:

ExampleAttention-trajectory MAE (%)Mean retention-ratio difference (%)
GenEval 1520.0040.007
GenEval 4370.0090.010
GenEval 5010.0080.017
PIE random editing0.1080.176
PIE change object0.6600.375
PIE add object0.4420.594

The results show that the feedback does not push budgets down when conditioning importance remains stable. In all six examples, the compressed attention trajectory closely follows the FullKV trajectory, and replaying the schedule with the FullKV statistics changes the mean retention ratio by at most 0.594%, so the budget decline comes from denoising itself. We include this comparison in the appendix of the revised manuscript.

W3 & Q2: Protected segments are excluded from the shared compression budget, and Appendix B says that the minimum retention ratio is calibrated to match an average compression target. It is not clear exactly what is averaged, whether the initial full-KV step is included, or how protected storage and quantization metadata enter the reported ratios. Please define the averaging over steps, the treatment of the initial full-KV step and protected segments, and whether quantization metadata and residual buffers are counted. Are Figure 8 and the main quality tables using the same configurations?

Reply: We thank the reviewer and clarify each point below.

Averaging over steps. The average compression ratio is the mean of the per-step compression ratios, where a step is one forward pass of the model during inference: a decoding step that generates one text token in understanding, or a denoising step of the image latent in generation and editing.

Initial full-KV step. It is included in the reported ratios as uncompressed.

Protected segments. Boundary and current-latent KVs are kept uncompressed and excluded from the compression ratio.

Whether quantization metadata and residual buffers are counted? Both are counted in the compressed storage, including per-group scales and minima, the BF16 residual tail, and the indices of retained tokens, for UniCache and all baselines.

Are Figure 8 and the main quality tables using the same configurations? Yes, both use the configuration described in Section 4.1 and Appendix B. We have added these definitions to the revised manuscript and highlighted them in blue.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer 9HDt,

Thanks for your positive initial review. If our response effectively resolves your issue, we'd be grateful if you consider further improving your score.

Thank you!

Authors

Reviewer_Yh85OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
3(+1)
Soundness
2
Read review
Paper summary

UniCache is a training-free, task- and type-aware KV-cache compression framework for unified multimodal models spanning visual understanding, text-to-image generation, and image editing. It splits conditioning KVs into semantic types (e.g., instruction, source-ViT, source-VAE), assigns eviction or quantization per task–type through offline attention-concentration calibration, and distributes a shared budget at runtime by chunk-level attention mass; understanding uses a fixed retention ratio while generation/editing use a decreasing schedule, with boundary and current-latent KVs protected. On BAGEL and SenseNova-U1, with compression sweeps, component ablations, and A100 runtime measurements, the paper reports roughly 5× managed-cache compression for understanding/editing and 2.5× for generation with largely preserved benchmark quality, and up to 1.78× throughput in long-context settings.

Critical strengths

S1. A useful compression unit for unified multimodal inference The paper identifies semantic KV type as a practically relevant unit of compression, beyond treating the cache as a homogeneous token collection. The source-ViT/source-VAE distinction in BAGEL editing is particularly useful: source-image structure and edit semantics can respond differently to eviction and quantization. Figure 5 and Appendix A provide controlled evidence by compressing one type at a time while leaving the others unchanged. This is more informative than reporting only aggregate end-task improvements and motivates a credible application-specific contribution. S2. Evaluation connects quality preservation to physical implementation The evaluation covers two unified models and three task families, rather than only understanding on one backbone. The editing results are encouraging: in Table 1, BAGEL’s PIE-Bench PSNR is 18.969 with UniCache versus 18.805 with full KV and 14.663 with H2O, while CLIP Target remains close to full KV. The work also goes beyond mask-based theoretical compression: Appendix E describes physical compaction and packed low-bit storage, Figure 8 reports throughput and persistent managed-cache memory, and Appendix E reports allocation overhead of 0.45%–1.7%. These results support practical promise, even though they do not yet establish complete deployment-level benefits.

Critical weaknesses

W1. The incremental contribution over closely related adaptive methods is not sufficiently isolated The claimed novelty—task-dependent semantic segmentation, type-specific eviction/quantization, and denoising-aware scheduling—builds on ingredients with close antecedents: FastGen (profiling-based policy selection), Ada-KV (non-uniform head budgets), VL-Cache (modality-aware scoring and sparsity-aware layer budgets), MadaKV (modality-attention-proportional budgets, a close precedent to Eq. (2)), and EAKV (attention-statistics-driven allocation with temporal integration). The paper does not discuss EAKV, MadaKV, Ada-KV, or FastGen, and cites VL-Cache without substantive comparison. The revision should cite and distinguish these works along allocation unit, importance signal, operators, timing, and tasks, and add matched-budget comparisons with EAKV and MadaKV where applicable—or clearly labeled component adaptations where porting is infeasible. Current baselines and ablations do not isolate the gain: Table 3 removes task/type awareness jointly, protection removal drives GenEval 0.780→0.567 so its rules must be matched across baselines, and Table 4 shows attention-guided and linear decay tie at 0.780 without uncertainty estimates. A fixed type-policy map with a simple schedule, under identical protection and storage, would directly test the controller’s contribution.

Questions for authors
  1. Please expand the related-work discussion to include EAKV, MadaKV, Ada-KV, FastGen, MiKV, RazorAttention, and MEDA, and clarify the distinction from the already-cited VL-Cache. These works address attention-guided allocation, heterogeneous cache policies, importance-aware mixed precision, and multimodal compression,please explain what UniCache adds beyond these mechanisms.

  2. For Tables 1, 3, and 4, what is the compression denominator, and is the matched quantity final, time-averaged, or peak allocated bytes? How are protected KVs and generated-text KVs counted? Table 3 explicitly uses 60% compression for PIE-Bench whereas the main editing setting targets 80%; please clarify the intended relationship. Also explain the GenEval difference between “w/o Task-Aware Temporal Schedule” in Table 3 (0.766) and “Constant” in Table 4 (0.776), including any differing parameters or seeds.

  3. Does the BF16 baseline share the same attention batching, metadata reuse, and CFG optimizations? Please report peak allocated memory and full-request latency including prefill and temporary buffers.

If the issues mentioned above are resolved, I will increase the score.

5 comments
AuthorsOpenReview ↗

Response to Reviewer Yh85 (1/4)

Thank you for your constructive feedback. We will provide detailed responses to your questions.

W1.a (related work): The incremental contribution over closely related adaptive methods is not sufficiently isolated. The claimed novelty—task-dependent semantic segmentation, type-specific eviction/quantization, and denoising-aware scheduling—builds on ingredients with close antecedents: FastGen (profiling-based policy selection), Ada-KV (non-uniform head budgets), VL-Cache (modality-aware scoring and sparsity-aware layer budgets), MadaKV (modality-attention-proportional budgets, a close precedent to Eq. (2)), and EAKV (attention-statistics-driven allocation with temporal integration). The paper does not discuss EAKV, MadaKV, Ada-KV, or FastGen, and cites VL-Cache without substantive comparison. The revision should cite and distinguish these works along allocation unit, importance signal, operators, timing, and tasks, and add matched-budget comparisons with EAKV and MadaKV where applicable—or clearly labeled component adaptations where porting is infeasible.

Q1: Please expand the related-work discussion to include EAKV, MadaKV, Ada-KV, FastGen, MiKV, RazorAttention, and MEDA, and clarify the distinction from the already-cited VL-Cache. These works address attention-guided allocation, heterogeneous cache policies, importance-aware mixed precision, and multimodal compression, please explain what UniCache adds beyond these mechanisms.

Reply: Thank you for pointing out these relevant works. We have expanded the related-work discussion in the revised manuscript and highlighted the additions in blue.

Comparison along five dimensions. The table below compares these works along allocation unit, importance signal, operator, timing, and tasks.

MethodTasksAllocation unitImportance signalOperatorTiming
Ada-KV [1]Text generationHeadAttention in observation windowEvictionOnce after prefill
MEDA [2]Multimodal understandingLayerCross-modal attention entropyEviction + mergingOnce after prefill
EAKV [3]Long-video understandingLayerAttention entropyTop-K retention + anchor mergingPrefill, temporal integration over frames
FastGen [4]Text generationHeadProfiled attention structurePer-head eviction ruleProfiled at prefill, then fixed
RazorAttention [5]Text generationHead (retrieval vs. others)Offline retrieval-head identificationFull cache vs. truncation + compensation tokenOffline
MiKV [6]Text generationTokenAttention importanceMixed-precision quantizationDuring decoding
MadaKV [7]Multimodal understandingHead × modalityModality attention from proxy tokensEvictionOnce after prefill
VL-Cache [8]Multimodal understandingLayer × modalitySparsity + modality-aware scoringEvictionOnce after prefill
UniCacheUnderstanding, T2I, editingTask–KV-type pair, per layer chunkOffline calibrated concentration (policy) + online conditioning attention mass (budget)Eviction or quantization per typeEvery chunk across decoding / denoising steps

Distinction from VL-Cache. VL-Cache distinguishes visual and text tokens for layer-adaptive budget allocation and modality-aware token scoring, but applies the same eviction policy to all visual tokens. In BAGEL, source-ViT and source-VAE KVs both represent the source image, yet favor eviction and quantization, respectively. UniCache therefore assigns policies to task–type pairs and schedules their budgets across decoding and denoising.

Matched-budget comparison with MadaKV and EAKV. We reimplement both methods on BAGEL following the original papers, with their own scoring and allocation, at approximately 80% compression for understanding and editing and 60% for text-to-image generation:

MethodMME ↑GenEval ↑PIE Structure ↓PIE PSNR ↑PIE CLIP Target ↑
Full KV2373.210.7810.10118.80528.174
MadaKV2373.180.7080.13115.38128.631
EAKV2372.570.7560.11817.01428.189
UniCache2377.360.7800.10018.96928.211

The results show that head- or layer-level budget allocation alone does not resolve the type-dependent compression preferences in unified models. Both new baselines apply a single token-retention policy to all KV types and lose source-image details on editing, whereas UniCache assigns policies per task–type pair and updates their budgets throughout decoding and denoising.

AuthorsOpenReview ↗

Response to Reviewer Yh85 (2/4)

Q1 (continued)

Contributions of UniCache beyond these mechanisms. Comparing with these works further clarifies the contributions of UniCache: an analysis of task-dependent KV-cache behavior within models that unify understanding and generation, together with a compression framework derived from these findings:

(i) Beyond head- or modality-level differences, we identify task-dependent cache compositions and management requirements within unified models, guiding task-aware segmentation into compression candidates and protected objects.

(ii) We reveal different compression preferences even among cache types within the same modality, explain the complementary failure modes of eviction and quantization, and use offline calibration to assign policies to task–type pairs.

(iii) Beyond distributing a given budget, we identify contrasting conditioning-attention dynamics across decoding and denoising and develop task-aware total-budget scheduling coordinated with type-level allocation.

To our knowledge, UniCache is the first task- and type-aware KV-cache compression framework for unified understanding and generation models.

We include this discussion in the revised manuscript.

W1.b (ablation isolation): Current baselines and ablations do not isolate the gain: Table 3 removes task/type awareness jointly, protection removal drives GenEval 0.780→0.567 so its rules must be matched across baselines, and Table 4 shows attention-guided and linear decay tie at 0.780 without uncertainty estimates. A fixed type-policy map with a simple schedule, under identical protection and storage, would directly test the controller's contribution.

Reply: We would first like to clarify potential misunderstandings, regarding the joint ablation of task and type awareness and the role of protection, before addressing the remaining suggestions.

Clarification on task and type awareness. In unified models, task and KV type are inherently coupled: the task determines which KV types exist. Task and type are therefore not independent factors that can be freely combined, which is why Table 3 ablates them jointly.

To further separate their contributions, we split the "w/o Task/Type Awareness" row into two variants. "w/o Type Awareness" applies a single compression policy(H2O) to all candidate types within each task with a global budget. "w/o Task Awareness" applies the same policy map and same budget controller to all tasks:

SettingMME Total ↑GenEval ↑PIE Structure ↓PIE PSNR ↑PIE CLIP Target ↑
Full UniCache2377.360.7800.10119.10028.171
w/o Task/Type Awareness2372.600.7670.10317.87528.221
w/o Type Awareness2372.600.7800.10318.31928.205
w/o Task Awareness2377.360.7670.10218.48728.021

Removing type awareness degrades understanding and editing, where multiple candidate types coexist and benefit from type-specific policies and budgets. Removing task awareness degrades text-to-image generation and editing, where the budget must follow the denoising-specific decline of conditioning attention.

Clarification on protection. Identifying which KV segments to protect is part of our task-aware segmentation (Design Principle 1), which is one of our contributions, and the baselines in Tables 1–4 follow their original designs.

To verify that the gain of UniCache does not come from protection alone, we apply our protection rules to the baselines. Since Table 3 shows that protection mainly affects tasks with iterative denoising, we evaluate text-to-image generation and image editing:

MethodGenEval ↑PIE Structure ↓PIE PSNR ↑PIE CLIP Target ↑
H2O0.7770.12514.66328.567
H2O + protection0.7800.12515.52128.418
KIVI0.7770.09818.57627.812
KIVI + protection0.7780.09818.49227.853
SnapKV0.5580.14912.93228.942
SnapKV + protection0.7310.13115.99228.138
UniCache0.7800.10018.96928.211

The results show that our protection rules also benefit existing methods, showing that the task-aware identification of protected segments generalizes beyond UniCache. However, single-policy baselines still degrade on image editing even with protection. The gain of UniCache therefore comes from task- and type-aware policy assignment and budget scheduling in addition to protection.

AuthorsOpenReview ↗

Response to Reviewer Yh85 (3/4)

W1.b (continued)

Uncertainty estimates for Table 4. We repeat linear decay and attention-guided decay on GenEval with three random seeds and report the mean and standard deviation:

ScheduleGenEval ↑
Linear decay0.778 ± 0.004
Attention-guided decay0.780 ± 0.003

Across seeds, attention-guided decay matches linear decay on GenEval within seed variance, and achieves the best results across all PIE-Bench metrics in Table 4, showing that conditioning attention provides a more effective temporal signal for budget allocation across steps.

Additional Ablation Study of Budget Controller. Table 3 ablates attention-guided allocation and the task-aware temporal schedule separately, from which the contribution of the budget controller can be understood.

The suggested experiment further clarifies its overall contribution. We apply the same retention ratio to all candidate types, so that each type receives a budget proportional to its original KV size, and keep this ratio constant across steps. We then add the two components back in turn. The results are as follows:

SettingMME Total ↑GenEval ↑PIE Structure ↓PIE PSNR ↑PIE CLIP Target ↑
Fixed hybrid (proportional allocation + constant schedule)2368.330.7660.12617.21226.957
+ Attention-guided allocation2377.360.7660.11018.47728.010
+ Task-aware temporal schedule (full UniCache)2377.360.7800.10119.10028.171

The results show that attention-guided allocation brings the largest gain on tasks with multiple cache types, namely understanding and editing. The task-aware temporal schedule further improves generation and editing. Both components of the budget controller therefore contribute to UniCache, with complementary effects across tasks.

Q2: For Tables 1, 3, and 4, what is the compression denominator, and is the matched quantity final, time-averaged, or peak allocated bytes? How are protected KVs and generated-text KVs counted? Table 3 explicitly uses 60% compression for PIE-Bench whereas the main editing setting targets 80%; please clarify the intended relationship. Also explain the GenEval difference between "w/o Task-Aware Temporal Schedule" in Table 3 (0.766) and "Constant" in Table 4 (0.776), including any differing parameters or seeds.

Reply: We thank the reviewer for the question. Tables 1, 3, and 4 use the same accounting method:

  • Denominator. The per-step ratio is $ 1 − C/C_0 $ , where the denominator is the original BF16 storage of the managed conditioning KVs, namely instruction, source-ViT, and source-VAE KVs, together with the generated-text KVs in understanding.
  • Matched quantity. We match the average compression ratio, namely the mean of the per-step ratios over all decoding or denoising steps, with the initial full-KV step included. At each step, $ C_0 $ and $ C $ are summed over layers and CFG branches.
  • Protected and generated-text KVs. Protected KVs are excluded from $ C_0 $ and $ C $. In understanding, generated-text KVs are also compression candidates and are included in both $ C_0 $ and $ C $.

All baselines are measured with the same method, and their realized ratios are reported in Table 5. We have added these definitions to the revised manuscript and highlighted them in blue.

Relationship between the 60% and 80% settings. Since both generation and editing involve denoising-time scheduling, Table 3 targets approximately 60% compression for both tasks so that components are analyzed under a consistent setting. The main editing results in Table 1 use 80% compression, where the differences between compression policies are more pronounced.

Difference between Table 3 (0.766) and Table 4 (0.776). Both runs use the same seeds and the same 60% target for GenEval. The difference comes from the scope of the ablation. In Table 3, "w/o Task-Aware Temporal Schedule" removes the entire temporal mechanism, including the initial full-KV denoising step that provides the reference conditioning mass M(0), so a constant budget is applied from the first step. In Table 4, all three schedules share the initial full-KV step and differ only in how the retention ratio evolves afterwards.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Response to Reviewer vNkG (1/2)

We thank the reviewer for recognizing our type-specific KV management and our practical evaluation. We will provide detailed responses to your questions.

W1: The calibration set is drawn from MME, GenEval, and PIE-Bench, which are also used for evaluation. The paper does not say whether those calibration examples are excluded from the reported test results. Nor does it show how much the policy changes with a different sample of 10 examples or a different 0.5 threshold. Without this information, it is hard to tell whether the reported gains are robust.

Q1: Are the calibration examples excluded from the evaluation sets? How much do the results vary with the calibration sample, the 0.5 threshold, or the number of calibration examples?

Reply: We thank the reviewer for the question.

UniCache uses task-level calibration, not benchmark-specific calibration. We randomly sample 10 examples from each of VQAv2 train, COCO Captions train, and MagicBrush train for image understanding, text-to-image generation, and image editing, respectively. These examples determine the compression policy for each task–type pair, which are fixed and reused across evaluation benchmarks for the corresponding task.

Calibration robustness to calibration samples. We repeat calibration with five random subsets of 10 examples per task at the default threshold of 0.5 and all six task–type pairs retain the same policy across all five draws:

TaskKV typeConcentration (mean ± std)Eviction / 5 runsQuantization / 5 runs
UnderstandingInstruction0.794 ± 0.02650
UnderstandingSource-ViT0.675 ± 0.05150
GenerationInstruction0.729 ± 0.10450
EditingInstruction0.758 ± 0.03750
EditingSource-ViT0.637 ± 0.04350
EditingSource-VAE0.286 ± 0.07905

Calibration robustness to the threshold and the number of calibration examples. To assess sensitivity beyond random sampling, we independently vary the calibration size and threshold. Agreement with the default policy map remains the same across six task–type pairs and five draws, totaling 30 decisions per setting:

Factor variedSettingPolicy agreementChanged assignment
Calibration size, threshold = 0.5530/30None
10 (default)30/30None
2030/30None
5030/30None
Threshold, 10 examples per task0.430/30None
0.5 (default)30/30None
0.630/30None

These results are consistent with our analysis in Sec. 3: task–KV-type pairs exhibit distinct attention concentrations and compression-policy preferences, making UniCache's calibration-based policy assignment robust to the choice of calibration samples, sample size, and threshold.

We have added the discussion into the revision and highlighted it with blue.

AuthorsOpenReview ↗

Response to Reviewer vNkG (2/2)

W2 (throughput) & Q2: The 1.78x throughput gain is reported against full BF16 KV. The paper does not provide an equivalent end-to-end comparison with the compressed baselines. Does the throughput measurement include attention-statistics collection, budget updates, quantization, and dequantization?

Reply: We thank the reviewer for the suggestion to strengthen the efficiency evaluation.

What the throughput measurement includes. All runtime numbers are measured end to end on the inference path, excluding only model loading, with one warm-up run followed by three measured runs. The measured path includes every operation that UniCache adds at inference time: attention-statistics collection, eviction, quantization, dequantization, and attention-guided budget updates.

End-to-end comparison with compressed baselines. Following the reviewer's suggestion, we implement H2O and KIVI in the same engine. The KIVI path uses the official KIVI CUDA kernels, and our H2O implementation matches the token selection of the official PyTorch implementation under the same budget. We compare them with UniCache under matched compression ratios. Results at approximately 20K KV tokens on a single A100 (40 GB):

MethodUnderstanding (TPS) ↑Text-to-image (step/s) ↑Editing (step/s) ↑Request latency (s) ↓, und. / edit.
BF169.31.611.6017.1 / 33.2
H2O16.32.062.0011.1 / 25.4
KIVI13.61.982.1211.9 / 26.2
UniCache16.22.032.0511.1 / 25.5

Under matched compression ratios, the different compression policies achieve similar throughput, since the speedup at long context mainly comes from the reduced KV storage and access. However, as shown in Tables 1 and 2 in paper, the single-policy baselines degrade quality on at least one task, whereas UniCache preserves quality across all tasks.

W2 (compression accounting) & Q3: The paper excludes protected segments from the compression budget, so the stated 5x figure may not describe the reduction in total cache memory. What is the total KV-cache reduction after accounting for protected segments, and how exactly is the compression ratio calculated?

Reply: We thank the reviewer for the question.

Compression ratio. The reported compression ratio uses managed conditioning KV storage as its denominator, and we compute the managed and total reductions with the following formulas:

Managed=1−CC0,Total=1−C+PC0+P\text{Managed} = 1 - \frac{C}{C_0}, \qquad \text{Total} = 1 - \frac{C + P}{C_0 + P}

Here $ C_0 $ and $ C $ are the original BF16 and compressed storage of the instruction, source-ViT, and source-VAE KVs, together with the generated-text KVs in understanding, and $ P $ is the storage of protected segments. At each decoding or denoising step, all three are summed over layers and CFG branches, and $ C $ includes quantization metadata and residual tokens. The reported ratio is the mean of the per-step ratios over all steps. We have added this definition to the revised manuscript and highlighted it in blue.

Total reduction. The managed reduction is the realized compression ratio reported in Table 5. We further measure the protected share of the original KV cache on 100 representative inputs per task with our physical engine, and report the total reduction $ 1 − (C + P) / (C_0 + P) $ , averaged over steps in the same way:

TaskProtected share of original totalManaged reductionTotal reduction
Understanding2.51%80.00%77.99%
Text-to-image3.67%59.76%57.57%
Editing0.32%80.67%80.41%

The results show that protected KV occupies only a small share of the cache, so the total reduction stays close to the managed reduction, and editing retains about 5× compression after protection is counted.

With these responses, we hope that we have resolved your concerns and that you will consider adjusting your score upward. We have incorporated all relevant clarifications and results into the revised manuscript.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer vNkG,

For your queries, we carefully prepared the answers. We look forward to you considering raising your score if our response effectively addressed your question.

Thank you!

Authors

We sincerely thank the reviewers for their insightful feedback.

We are encouraged to see that the reviewers acknowledged the paper's contribution in identifying KV type as a well-motivated compression unit for unified multimodal models and assigning type-specific compression policies accordingly [Reviewers 9HDt, Yh85, vNkG]; supporting this idea with controlled single-type compression experiments [Reviewers 9HDt, Yh85]; and going beyond simulated compression with a physical GPU implementation that delivers real memory and throughput gains [Reviewers 9HDt, Yh85, vNkG]. We are also glad that Reviewer Yh85 confirmed after the rebuttal that all their questions have been well resolved.

We greatly appreciate the reviewers' efforts in evaluating our work, and the constructive suggestions are invaluable for further improving the paper, for example, adding a one-time allocation baseline and a fixed hybrid baseline to isolate the contribution of the budget controller [Reviewer 9HDt], expanding the related work with matched-budget comparisons against MadaKV and EAKV [Reviewer Yh85], analyzing the robustness of calibration and comparing end-to-end efficiency with compressed baselines [Reviewer vNkG], and clarifying the compression-ratio accounting [Reviewers 9HDt, Yh85, vNkG]. We have provided detailed point-by-point responses to each of the reviewers' comments in our individual replies.

The constructive suggestions allowed us to improve the paper, which we hope the reviewers will find to satisfactorily address the raised concerns. We uploaded a revised version of the paper incorporating the reviewers' suggestions. All changes are highlighted in blue color. We hope that based on the answers to their questions and the improvements made to the revised version, the reviewers will increase their scores.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.