- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Twilight is a training-free KV-cache compression framework for interleaved text–image generation. It identifies modality-specific retrieval heads using normalized attention standard deviation and applies retention policies tailored to head types and classifier-free guidance branches. Experiments on TwGI report substantial cache reductions while maintaining generation performance, supported by component ablations and decoding-efficiency measurements.
Critical strengths
- Meaningful efficiency gains. Twilight retains approximately 29–35% of the original KV cache on DPG-Bench while maintaining comparable scores. This is a practically useful result without retraining the underlying model.
- A coherent design for interleaved generation. The modality-specific head assignments and boundary- and CFG-aware retention policies address features that a uniform text-oriented policy would miss. The ablations support several of these distinctions.
Critical weaknesses
The incremental contribution is not sufficiently isolated. The proposed head-identification statistic measures attention dispersion, which is not necessarily equivalent to the need for long-range retrieval. The existing ablations establish the importance of head counts and modality-specific assignments, but do not directly compare selection criteria under the same retention policy and budget. Table 10 uses Twilight’s own identification method, so it does not resolve this question. Together with limited positioning against head-aware image-generation methods such as SSD and HeadKV, this leaves uncertainty about the contribution beyond adapting known head-aware compression principles to TwGI.
Questions for authors
- Clarification: When switching modalities, how is historical context supplied to heads that change from streaming to retrieval? Is the sequence re-prefilled, and which transition or prefill costs are included in Figure 6?
- Evidence that could change my assessment: With retention rules and cache budget fixed, how does normalized standard deviation compare with a simple alternative, such as attention mass outside the retained window? This would help establish the benefit of the proposed selector.