- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes CATP, a training-free visual-context pruning framework for multi-image synthesis in unified multimodal models. Guidance-Consistent Pruning (GCP) synchronizes reference-token selections across CFG branches to avoid inconsistent visual conditioning. The Taylor-Calibrated Attention Metric (TCAM) addresses the bias introduced by query pooling through a second-order correction, while the Fast Scoring Operator (FSO) integrates this calibration into an efficient tiled computation. Experiments across different model architectures and benchmarks demonstrate favorable generation quality under aggressive context compression, together with meaningful attention acceleration.
Critical strengths
1. Well-motivated analysis of generation-specific pruning problems. The paper identifies how independently selected visual contexts across CFG branches can introduce unintended components into conditional guidance. GCP addresses this issue through a simple and effective synchronization strategy, yielding substantial improvements in generation quality, particularly under aggressive pruning. The connection between the identified problem, its mathematical formulation, and the proposed solution is clear and convincing.
2. An elegant connection between principled scoring and efficient implementation. TCAM derives a meaningful correction for pooling-induced attention bias from local query variation and the distribution of reference keys. FSO further reformulates this correction into statistics that can be efficiently accumulated during tiled attention computation. The components complement each other naturally, providing a favorable balance between scoring accuracy and computational efficiency. The consistent improvements across different models and benchmarks further demonstrate the practical value of the proposed framework.
Critical weaknesses
1. Standalone inference savings could be reported more directly. The main efficiency analysis focuses on attention-core latency, while the appendix provides whole-denoising speedups when CATP is combined with other acceleration methods. The standalone end-to-end denoising benefit of CATP, including the overhead of periodic scoring and token selection, is less explicit. Reporting this would provide a clearer picture of its practical acceleration.
2. The computational matching of adapted baselines could be clarified. The paper states that competing methods are adapted to equivalent computational budgets, but the specific configurations and matching criteria are not fully described. Since these methods reduce computation at different positions within the model, a concise explanation of their operating points would help readers interpret the comparisons more precisely.
Questions for authors
-
Could the authors report the standalone whole-denoising latency of CATP under the existing efficiency evaluation setting, including scoring and token-selection overhead? A brief addition to the current efficiency results would suffice.
-
Could the authors clarify how computational budgets were matched across the adapted baselines, particularly SDTM and ToPi? A compact description of the relevant configurations and budget calculation would be helpful.
4 comments
(2/2) W2&Q2: On Baseline Configurations.
(2/2) W2&Q2: On Baseline Configurations.
Thanks for your valuable suggestion. As described in Lines 376–377 (also Lines 374-376 of the original submission), we restrict all compared methods to reduce attention-core computation in image-conditioned branches, and adjust their configurations to ensure comparable computational budgets under each target retention ratio (i.e. 25% and 10%) by following common practices [1,2]. For Text-Aligned and VisionZip, we directly apply the same per-layer token retention ratio as our CATP. For ToPi and SDTM, whose official implementations retain full computation at certain layers or timesteps, we accordingly reduce the retention ratios in the compressible components to match overall budgets. In order to make fair comparisons, the details are specified as below:
1) ToPi. We keep the first of 28 layers unpruned in both settings. At 25% retention, we follow ToPi’s original schedule, performing full-context computation and updating pruning decisions every 10 timesteps. Ultimately, 5 timesteps retain full computation, while the remaining 44 timesteps retain 17% of reference tokens in 27 of 28 layers, yielding an average retention ratio of . At 10% retention, we extend the pruning interval to 25 timesteps, resulting in 2 full-computation steps and 47 steps retaining 7% of reference tokens in 27 of 28 layers. This yields an average retention ratio of . For clarity, the detailed configurations are summarized in Tab. R9 below.
Table R9: Detailed token retention ratios of the adapted ToPi baseline.
| Target Retention | Layer 1 (All steps) | Layers 2–28 (Full context) | Layers 2–28 (Pruned context) | Avg. Retention |
|---|---|---|---|---|
| 25% | 100% | 5 steps 100% | 44 steps 17% | 28% |
| 10% | 100% | 2 steps 100% | 47 steps 7% | 14% |
2) SDTM. Unlike context token pruning, SDTM reduces the denoising tokens, which also decreases projection computations. Therefore, we adjust its compression settings to match the overall attention-core GFLOPs, including both projection and attention matrix multiplications. Given its sensitivity to compression in the last several denoising steps, we follow the official configuration and retain full computation in the final 5 timesteps for both settings. At 25% retention, full computation is additionally performed every 8 timesteps, while the other steps retain 30% of denoising tokens per layer. At 10% retention, we perform full computation every 10 timesteps and retain 20% of denoising tokens otherwise. The resulting average attention-core GFLOPs for the Full Model, SDTM, and CATP are summarized in Tab. R10 below.
Table R10: Detailed configurations of the adapted SDTM baseline and average attention-core GFLOPs comparison with CATP.
| Target Retention | Full-computation | Reduced-computation | SDTM GFLOPs | CATP (Ours) GFLOPs |
|---|---|---|---|---|
| 25% | 4 + 6 steps | 39 steps (retain 30% tokens) | 391.8 | 311.4 |
| 10% | 4 + 5 steps | 40 steps (retain 20% tokens) | 315.9 | 198.0 |
Notably, across all the above settings, CATP consistently incurs no more computational cost than the compared baselines, while still delivering notable performance gains, as shown in Tabs. 1 and 2, further demonstrating the effectiveness of our method. These detailed configurations have been added to Appendix C of the revised manuscript.
[1] Visionzip: Longer is better but not necessary in vision language model. CVPR’25.
[2] Collaborative multi-mode pruning for vision-language models. CVPR’26.
Thank you for the detailed rebuttal. The additional standalone denoising results and clarification of baseline configurations adequately address my concerns. The response further strengthens my positive assessment of the paper, and I maintain my acceptance recommendation.
Dear Reviewer,
Thank you very much for your positive feedback and continued support of our work. We sincerely appreciate your time and thoughtful consideration throughout the review process.
Best regards,
The Authors
(1/2) W1&Q1: On Efficiency Results.
Dear Reviewer, we sincerely thank you for the insightful comments. We organize our responses into 2 sections to address your concerns regarding efficiency results (W1&Q1) and baseline configurations (W2&Q2).
(1/2) W1&Q1: On Efficiency Results.
Thanks for raising this concern. We would like to first clarify that the attention-core latency reported in Fig. 4(a) already includes all scoring, token selection, and pruning overhead, rather than only the reduced attention computation. While model inference involves multiple computational components, CATP specifically targets the costly attention induced by long visual contexts. Therefore, we initially follow prior practices [1,2] in primarily focusing on attention-core latency to directly assess the efficiency gains of the optimized component.
Furthermore, we fully agree with the reviewer’s suggestion and provide the end-to-end denoising speedup of standalone CATP, as shown in Tab. R8 below. Although the overall speedup is inevitably reduced when accounting for unoptimized components such as FFNs, CATP still achieves up to 1.44× acceleration over the entire denoising process. As similarly explored in other domains [3,4], this represents a meaningful practical efficiency gain for such a sequence-level optimization approach.
We have added these results to Fig. 4(b) and clearly discussed the scope of the reported acceleration in Lines 465-473 of the revised manuscript.
Table R8: End-to-end denoising latency (s) and speedup of CATP, across varying numbers of reference images.
[1] Flexprefill: A context-aware sparse attention mechanism for efficient long-sequence inference. ICLR’25.
[2] Xattention: Block sparse attention with antidiagonal scoring. ICML’25.
[3] LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference. EMNLP’24.
[4] Attend to not attended: Structure-then-detail token merging for post-training dit acceleration. CVPR’25.