- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes MUG, a training-free, single-branch diffusion guidance method that constructs degraded internal references through token merging and performs CFG-style extrapolation in feature space to improve generation quality without additional model evaluations.
Critical strengths
The qualitative and quantitative results are promising.
The paper and method are easy to follow.
Critical weaknesses
MUG improves generation quality, but the paper does not sufficiently explain why extrapolating away from merged features leads to better predictions.
MUG is applied to cross-attention. Could it also be applied to self-attention and FFN layers?
Questions for authors
MUG can be viewed as enhancing attention outputs. Could directly scaling attention outputs achieve similar improvements, for example, by multiplying them by a constant factor or using token-wise attention entropy to identify important tokens and amplify their outputs with a factor greater than 1?
4 comments
Dear Reviewer,
We hope that our response has addressed your questions and comments, and would greatly appreciate that we have ample time to address any remaining questions or concerns, as fewer than two days remain in the discussion phase.
Thank you again for your time and effort in reviewing and making our manuscript better.
Authors of Submission 36194
Thank you for the authors' responses, which have addressed most of my concerns. However, the proposed method alters the structure of the originally generated images. Similarly, reducing the number of sampling steps in multi-step generation can also lead to structural changes. This raises the question of whether the proposed method offers any advantages over simply reducing the number of sampling steps.
Dear Reviewer icBe,
We are very glad that our additional evidence and discussions have addressed most of your concerns! We sincerely thank you for this insightful suggestion: using an earlier sampling prediction as the degraded reference is both natural and computationally attractive.
Initial one-shot test. We first conduct a rapid validation on 500 prompts using SDXL with 50-step sampling. We cache the earlier prediction at step 20 and use it for guidance once, at step 50:
=\epsilon(50)+\lambda\bigl[\epsilon(50)-\epsilon(20)\bigr].$$ | **Method** | **CLIP ↑** | **ImageReward ↑** | **PickScore ↑** | **$T_{\mathrm{E2E}}$ (s/img) ↓** | **$\Delta$Mem (MB) ↓** | |---|---:|---:|---:|---:|---:| | Baseline | 0.319 | 0.303 | 20.96 | 6.814 | 327.67 | | Early Exit as Degradation (step 50 vs. step 20; single update) | 0.319 | 0.307 | 20.98 | 6.928 | 327.80 | | Early Exit as Degradation (20-step lag; repeated) | 0.320 | 0.705 | 21.33 | 6.928 | 327.92 | | Merge–Unmerge as Degradation (ours) | **0.324** | **0.849** | **22.45** | 7.943 | 328.95 | The one-shot reference provides only a marginal improvement. We hypothesize that a single guidance update is too weak because it acts only after most of the trajectory has already been determined. The large state and noise-level gap between steps 20 and 50 may further mix the desired quality direction with ordinary denoising evolution. **Repeated 20-step-lag guidance.** We therefore design a stronger variant. Starting from step 21, each step uses the cached prediction from 20 sampling steps earlier: $$\epsilon_{\mathrm{guided}}(i) =\epsilon(i)+\lambda\bigl[\epsilon(i)-\epsilon(i-20)\bigr], \qquad i=21,\ldots,50.$$ This produces 30 guidance updates without any additional U-Net forward pass. It significantly improves over the one-shot variant while retaining near-baseline latency and memory. **Trade-offs.** Earlier-step degradation is highly efficient because it reuses existing predictions, but compares different latent states and noise levels. MUG instead constructs its clean and degraded retrievals at the **same $x_t$, timestep, and text condition**, yielding stronger improvements at the cost of additional cross-attention computation. **Few-step applicability.** Earlier-step guidance requires sampling history and is therefore unavailable for a 1-step trajectory. MUG remains applicable, as directly demonstrated by the 1-step DMD2 results in Table 2 and the “Compatibility with guidance-distilled models” discussion in Sec. 4: MUG improves CLIP from 0.333 to 0.336, ImageReward from 0.822 to 1.004, and PickScore from 22.27 to 22.31, but early exist is **not usable** in this case. Therefore, MUG has better generalizability as a universal guidance method. Based on our initial experiments, we also have good reasons to believe that earlier-step guidance **may not work well under a few-step (e.g., 2-8 steps) sampling set-up**, which are however an immensely important set-up in real applications and products. We sincerely thank you again for this inspiration. This temporal-reference idea deserves further development and systematic exploration, particularly regarding the temporal lag, activation window, and combination of multiple earlier predictions. Authors of Submission 36194
Dear Reviewer,
The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.
Best,
AC