- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes RefAdapt-DiT, a training-free method for accelerating reference-conditioned diffusion transformers. The authors argue that reference K/V reuse should depend not only on how much the reference features change, but also on how strongly the target attends to them. Through empirical analysis, they observe that reference K/V generally evolves more slowly than target K/V in shallow-to-middle blocks, while this difference diminishes or even reverses in deeper blocks. They also find a correlation between target feature changes and reference K/V changes, and show that target-to-reference attention exposure helps predict the impact of stale reference K/V on target attention outputs. Based on these observations, RefAdapt-DiT combines current target-Q changes with historical attention exposure to dynamically determine when to refresh reference K/V. Specifically, it reuses cached reference K/V in earlier blocks and resumes reference computation once the combined score exceeds a threshold, recomputing the deeper blocks. Experiments on Qwen Image Edit and MiniMax H3 demonstrate speedups of 2.58× and 1.564×, respectively, while largely preserving generation quality.
Critical strengths
1. Practical inference acceleration with largely preserved generation quality. RefAdapt-DiT achieves substantial inference speedups on both Qwen Image Edit and MiniMax H3, while largely preserving generation quality. Compared with the evaluated acceleration baselines, the method provides higher speedups with competitive generation performance, demonstrating its practical effectiveness.
2. Reference-aware analysis and caching strategy. The motivation is straightforward and reasonable: whether reference K/V can be safely reused should depend not only on how much the reference features change, but also on how those changes affect the target attention output. The authors support this argument through controlled stale-K/V experiments, showing that exposure-conditioned reference drift correlates more strongly with target-attention perturbations than reference drift alone. These observations provide a useful motivation for the proposed caching strategy.
Critical weaknesses
1. The ablation study does not convincingly establish the effectiveness of the proposed controller. In Table 4, the Exposure Only baseline achieves a slightly lower speedup than RefAdapt-DiT (1.459× vs. 1.482×), but obtains higher PSNR (32.24 vs. 31.24) and SSIM (0.9958 vs. 0.9937). This raises questions about whether incorporating target-Q changes is actually necessary and whether they can serve as a reliable proxy for the target-side impact of stale reference K/V. Moreover, the controller signals have different numerical scales, yet the authors use the same threshold for all adaptive variants. A more comprehensive fidelity–speed trade-off comparison with independently tuned thresholds is needed to demonstrate the advantage of the proposed controller. In addition, this ablation is conducted on a separate 50-step trajectory rather than the actual few-step deployment settings.
2. The presentation is difficult to follow, and several important implementation details are insufficiently explained. The paper is unnecessarily complicated in its presentation, making it difficult to follow the motivation and the connections between different parts of the method. Some important quantities are not clearly defined. For example, Section 3.3 introduces the joint endpoint scalar without specifying how the individual K/V drifts are combined. The numerical values of the stability constant , controller threshold , and maximum cache age are also not provided. Furthermore, the figures and tables often lack sufficient discussion of their findings, such as the additional insights provided by the right-hand profiles in Figure 1. The implementation of ResComp, which is used to obtain the additional acceleration results, is also insufficiently described. These issues make the paper unnecessarily difficult to understand and reproduce.
Questions for authors
-
Speed–fidelity trade-off. Could the authors clarify how the shared threshold in Table 4 was selected? Since different controller signals have different numerical scales, would the advantage of the proposed controller remain consistent if the thresholds were independently tuned for each variant? In particular, does incorporating target-Q change provide additional benefits over using attention exposure alone?
-
Implementation details and clarity. Could the authors provide clearer explanations of the key formulas and figures? In particular, how exactly is computed, what values are used for the numerical stability constant , controller threshold , and maximum cache age , and how is ResComp implemented?
-
Few-step ablations. Could the authors further validate the proposed controller through ablation studies under the actual 4-step MiniMax H3 and 8-step Qwen Image Edit settings? In particular, does combining target-Q change and attention exposure consistently outperform the individual signals in these deployment settings?
I am open to raising my rating if the authors can satisfactorily address these concerns, particularly regarding the effectiveness of the proposed controller and the robustness of the ablation comparisons.