- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AnchorCache, an efficient inference method for in-context diffusion transformers. The key observation is that isolating reference tokens from the evolving target enables exact K/V caching, but also removes instruction conditioning from the reference representations. AnchorCache addresses this trade-off by introducing static instruction anchors that condition the reference branch during cache construction while preserving exact K/V reuse. Since this architectural modification initially degrades generation quality, the authors further propose a two-stage recovery procedure consisting of teacher-forced velocity distillation followed by short on-policy distillation on student-visited states. Experiments on image, speech, and video generation show quality close to full attention while achieving up to 6.40× DiT inference speedup.
Critical strengths
- AnchorCache directly addresses the tension between exact reference caching and instruction-aware conditioning with a simple static-anchor design, without introducing additional learnable modules. The method retains almost the same inference efficiency as isolated caching while substantially recovering generation quality.
- The method is evaluated across image, speech, and video generation, and consistently achieves quality close to the full-attention models with substantial speedups. The ablations also clearly separate the contribution of static anchors and on-policy recovery, and show that additional teacher-forced training cannot reproduce the gain from the on-policy stage.
Critical weaknesses
- Applying AnchorCache changes the attention topology and requires a non-trivial teacher-forced distillation stage followed by on-policy refinement. Therefore, the practical benefit depends on whether this one-time adaptation cost can be sufficiently amortized over inference, which is not analyzed in detail.
- Most detailed ablations are performed on Qwen-Image-Edit, while speech and video are each evaluated on a single model with more limited analysis. This makes the broad cross-modal generality claim somewhat less thoroughly supported than the image-generation results.
Questions for authors
see weakness