- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a training-free guidance method for text-to-image diffusion generation that aims to separate structural alignment from local appearance refinement. To this end they extend Self-Swap Guidance (SSG, Zhang et al., 2026) with Spectral-Semantic Alignment (SSA), which sharpens the cross attention with temperature scaling and suppresses fine-scale variation in the difference of original and swapped branch predictions of SSG. Then the introduced Denoised-Content Recovery (DCR) amplifies the high-pass residual of the denoised estimate in order to improve the local details. The Trajectory Harmonization coordinates SSA and DCR during sampling: recovery activates halfway of the trajectory, while attention sharpening and structural guidance strength decrease over the final sampling steps. The proposed method demonstrates a significant reduction of FID over the baseline.
Critical strengths
-
The paper is well motivated by the goal of separately controlling structural alignment and local appearance refinement. Such control could allow the use of a more aggressive semantic guidance scale at the early sampling steps, in order to increase the prompt alignment, without compromising the local detail quality.
-
The proposed approach reports significant gains in FID and IS over the baseline, suggesting that the proposed approach may better handle the fidelity-alignment tradeoff.
Critical weaknesses
-
Captions of tables 1 and 2 say that baseline results are reported by Zhang et al (2026), while ARG is evaluated by the authors. I have a concern that the evaluation metrics may not be comparable because even slight differences in how the evaluation is performed can have a significant impact on the metrics. Therefore the authors should calculate the metrics for all methods themselves and compare with those.
-
Proposed ARG improves FID at the cost of a decrease in Aesthetic score. Therefore, it is unclear whether the gain in FID reflects better overall generation quality, particularly since the qualitative comparisons do not show a consistent improvement in the visual quality (e.g. see the laptop and cake examples in Fig. 3, in which ARG performance seems worse). A human evaluation would help to clarify this.
-
The proposed method is only evaluated on SDXL. Experiments on additional models such as SD1.5 and SD2.0 are required to justify its generality across different diffusion models.
Questions for authors
-
In section 4.3 the authors define the Backbone (ablation baseline) as SSG (Zhang et al., 2026), but the metrics in Table 3 for that baseline do not match the metrics reported in Table 2 for SSG. What is the reason of this mismatch?
-
The qualitative results in Figure 3 for ARG seem more blurry and low-quality in comparison to the baseline. Is this because of the Gaussian blur operation in Spectral-Semantic Alignment, which may guide the generation toward more blurred and washed out predictions?
-
Can the method be adapted for flow matching and applied to diffusion transformer architectures such as FLUX.2-klein?
-
Is this method compatible with CFG?
-
In Figure 1 (a) the low pass contrast is being multiplied by the guidance scale, but it is not clear what it is being added to. Even though it’s clear from the method section, the diagram should be fixed.
-
It is not clear what the dashed lines do in Figure 1 (b). It seems like in the early steps authors add the original estimate to itself. Please fix the diagram.
-
In Figure 1 (c) the vertical axes should explicitly identify the plotted quantities, for example , normalized attention-sharpening strength, and normalized recovery strength. The current labels “Structure”, “Attention”, and “Recovery” are ambiguous without consulting the caption.
5 comments
Q1: As summarized in Table R3.1 above and documented in Section 4.1 & Section 4.3 (Table 3), the difference arises because Table 2 uses SSG's native default setting () whereas Table 3 uses a strictly matched ablation setting ():
- Table 2 (
SSG: FID31.92published /31.75local): Evaluates SSG with its own default hyperparameters from Zhang et al. (2026) (ssg_scale = 3.0, nativeAttnProcessor2_0). - Table 3 (
Backbone: FID36.09): Because ARG operates at initial scale with our common cross-attention temperature processor and deterministic prompt-index seeds , evaluating the ablation backbone at would confound scale changes () with operator changes (). Thus, Row 0 of Table 3 runs raw unfiltered SSG under the exact same , processor loop (), and seeds. - Why raw SSG degrades from FID
31.75() to36.09(): This directly validates the core motivation of our paper (which you highlighted in Strength 1!). When raw, unfiltered SSG is pushed to an aggressive guidance scale (), its high-frequency token-swap noise causes severe oversharpening and mode narrowing (Precision drops from0.5022to0.4068and FID worsens to36.09). Once our low-pass filter (SSA-S, ) is applied at (Table 3), FID immediately drops from36.09to27.07(-9.02FID) and Precision jumps to0.5686. And Full ARG (FID 26.67, IR 0.5149) beats BOTH native SSG (FID 31.75, IR 0.2506) and matched SSG (FID 36.09, IR 0.3410). We have revised theTable 3caption andSection 4.3text to state this explicitly.
W2 & Q2: This is a central question that connects directly to our spectral decomposition of SSA vs. DCR (and shows an interesting contrast with Reviewer BJss, who noted in W1 that raw SSG outputs look unnaturally sharp and high-contrast compared to standard SDXL):
- Does SSA's Gaussian Blur Alone Soften High-Frequency Detail? Yes — and DCR (
Recover) Completely Restores It! To test your hypothesis rigorously, we measured the non-DC High-Frequency Power Spectral Density share (HF%), Haar wavelet finest-scale energy, Gradient RMS, Laplacian RMS, RMS Contrast, and Color Saturation across 500 paired decoded images (Section 4.5, Figure 4candAppendix D.1, Table 7 & Figure 7):
Table R3.2 (Revised Manuscript Appendix Table 7): Spectral & Perceptual Trajectory Across Stages (500 Paired Images + 5,000-Pair Preference Audits)
| Configuration | High-Freq. PSD Share (HF%) | Finest Haar Wavelet Energy | Color Saturation | Gradient RMS | Laplacian RMS | AES (5k) | ImageReward (5k) | PickScore (5k) | CLIP R@1 (500) |
|---|---|---|---|---|---|---|---|---|---|
| SSG Baseline | 7.44% | 8.5% | 0.2393 | 0.1096 | 0.2436 | 5.887 () / 5.949 () | 0.2506 / 0.3410 | 22.137 / 22.104 | 68.4% |
Low-Pass Only (SSA-S) | 3.19% | 4.1% | 0.2056 | 0.0588 | 0.1248 | 5.750 | 0.2473 | 22.091 | 68.6% |
| Full SSA | 3.20% | 4.1% | 0.2060 | 0.0584 | 0.1239 | 5.716 | 0.3990 | 22.216 | 73.8% |
| SSA + DCR () | 7.41% | 8.6% | 0.2510 | 0.0963 | 0.2228 | 5.845 | 0.5068 | 22.201 | 73.8% |
| Full ARG (Ours) | 7.44% | 8.6% | 0.2509 | 0.0961 | 0.2223 | 5.860 | 0.5149 | 22.195 | 75.0% |
- Look at what Table R3.2 proves:
- Your physical intuition about SSA alone (
SSA-S/Full SSA) is 100% correct: applying the Gaussian low-pass filter to the swap residual drops output high-frequency energy (HF%) from7.44%to3.19%–3.20%(4.1%finest Haar energy) and lowers saturation from0.2393to0.2056, which indeed looks softer and dropsAESto5.716. - However, once DCR (
Recover) is activated in ARG, output high-frequency energy is restored in all 500 out of 500 paired images (+4.201percentage points, 95% CI[+3.980, +4.433]), bringingHF%back to7.44%(matching SSG to within0.001%!), finest-scale Haar wavelet energy to8.6%(vs. SSG's8.5%), and color saturation to0.2509(higher than SSG's0.2393, i.e., NOT washed out!).
- Your physical intuition about SSA alone (
W2 & Q2:
- Why Is Aesthetic Score Slightly Lower (
5.860vs.5.887at , a % relative difference) While All Other 7 Metrics Improve?- As Reviewer
BJssnoted in W1, raw SSG produces artificially high micro-contrast and second-order edge ringing (Laplacian RMS = 0.2436,RMS Contrast = 0.2541). The LAION Aesthetic Predictor (an MLP on CLIP image embeddings without text input) is known to reward high micro-contrast and edge saturation even when prompt semantics fail or textures are "crunchy" (pushing SSG to inflatesAESto5.949while degradingFIDto36.09andPrecisionto0.4068!). - In contrast, human-preference-trained evaluators that jointly assess visual quality AND prompt faithfulness overwhelmingly prefer ARG across the entire 5,000-prompt dataset:
- ImageReward: more than doubles over default SSG (
0.25060.5149, %) and gains+0.1740over matched SSG (95% bootstrap CI[+0.1547, +0.1934], winning on % of all 5,000 prompt pairs). - PickScore: increases from
22.137/22.104to22.195( paired gain, 95% CI[+0.0705, +0.1125], winning on % of all 5,000 prompt pairs). - Independent Caption Retrieval (R@1 on 500 prompts): increases from
68.4%(SSG) to75.0%(ARG) ( percentage points).
- ImageReward: more than doubles over default SSG (
- As Reviewer
- Regarding the Laptop and Cake Examples in Figure 3:
In Figure 3, matched scale-6 SSG produces hyper-sharpened background edges that make natural depth-of-field bokeh in ARG appear softer at thumbnail scale, even when ARG restores missing prompt objects or avoids rim halos. To make fine-scale texture transparent, we added magnified detail zoom-ins in
Appendix D.9 (Figures 15–17)alongside rank-selected 5th/50th/95th percentile examples (Appendix D.7, Figures 11–12) and the complete 25-prompt archive (Appendix D.8, Figures 13–14).
W3 & Q3:
- Quantitative Results on
SD v1.5(), Flow-Matching Linear DiT (Sana-600M), and Additional Solvers (DDIM,DPM-Solver++,25-step Euler) on Full COCO-2017 5k (Section 4.7, Table 4&Appendix Figure 18): Table R3.3 reports full 5,000-image evaluations across Stable Diffusion v1.5, Sana-600M (Flow-Matching Linear DiT), and multiple solvers:
Table R3.3: Cross-Backbone (SD1.5, Full COCO-2017 5k) and Multi-Sampler (DDIM, DPM-Solver++, Euler-25) Results
| Backbone & Solver Setting () | Method | FID | CLIP | IS | AES | Prec. | Rec. | ImageReward | PickScore |
|---|---|---|---|---|---|---|---|---|---|
| SD1.5 () | Unguided | 45.13 | 0.2820 | 18.72 | 5.073 | 0.6206 | 0.4856 | -1.0921 | 19.417 |
| SD1.5 () | SSG () | 19.30 | 0.3027 | 33.36 | 5.235 | 0.6932 | 0.6418 | -0.3697 | 20.778 |
| SD1.5 () | ARG (Ours, ) | 19.53 | 0.3047 () | 33.58 () | 5.290 () | 0.6622 | 0.6744 () | -0.3241 () | 20.822 () |
| Sana-600M DiT (Flow-Match) | Unguided | 30.87 | 0.3086 | 25.98 | 5.580 | 0.6012 | 0.5380 | -0.0312 | 20.882 |
| Sana-600M DiT (Flow-Match) | SSG () | 35.38 ( worse) | 0.3186 | 26.63 | 5.779 | 0.4380 | 0.4890 | 0.3776 | 21.541 |
| Sana-600M DiT (Flow-Match) | ARG (Balanced / Align) | 29.97 / 34.34 | 0.3167 / 0.3188 | 28.40 / 27.50 | 5.661 / 5.764 | 0.5386 / 0.4400 | 0.5606 / 0.5102 | 0.2685 / 0.3847 | 21.205 / 21.521 |
| SDXL: Euler (25 steps) | SSG | 31.22 | 0.3123 | 33.31 | 5.901 | 0.5270 | 0.5108 | 0.2277 | 22.064 |
| SDXL: Euler (25 steps) | ARG (Ours) | 26.32 () | 0.3193 | 36.02 () | 5.836 | 0.5608 | 0.6230 | 0.4678 (%) | 22.128 |
| SDXL: DPM-Solver++ (25 steps) | SSG | 33.22 | 0.3118 | 32.72 | 5.926 | 0.4936 | 0.5202 | 0.2029 | 22.027 |
| SDXL: DPM-Solver++ (25 steps) | ARG (Ours) | 28.17 () | 0.3176 | 35.66 () | 5.832 | 0.5098 | 0.6464 | 0.4018 (%) | 22.136 |
| SDXL: DDIM (50 steps) | SSG | 29.62 | 0.3145 | 34.30 | 5.842 | 0.5520 | 0.5322 | 0.2757 | 22.151 |
| SDXL: DDIM (50 steps) | ARG (Ours) | 26.57 () | 0.3203 | 37.26 () | 5.781 | 0.5666 | 0.6050 | 0.5044 (%) | 22.203 |
On SD1.5 (5k = 5,000 images), ARG (Balanced) outperforms SSG on 6/8 metrics (CLIP, IS, AES, Recall, ImageReward, PickScore) at matched 5k FID (19.53 vs. 19.30, -25.60 FID vs. Unguided SD1.5); on Sana-600M DiT (5k), ARG slashes FID by -5.41 over SSG (35.38 29.97, even beating Unguided 30.87) with peak IS (28.40), Recall (0.5606), CLIP (0.3188), and ImageReward (0.3847); and across Euler-25, DPM-Solver++-25, and DDIM-50 (5k), ARG beats SSG on 7/8 metrics, reducing FID by -3.05 to -5.05 and boosting ImageReward by +82.9% to +105.5%!
We thank you for the thorough review and for highlighting the clear motivation of ARG (Align–Recover Guidance) — specifically, that separating structural alignment from local appearance refinement enables stronger early semantic guidance without sacrificing detail quality, yielding significant FID and IS gains over the baseline. Below, we address every weakness (W1–W3) and question (Q1–Q7) point by point.
W1: We completely share your commitment to strict metric comparability. In fact, we ran our own local full-dataset reproductions of SSG on both COCO-2014 30k (30,000 images) and COCO-2017 5k (5,000 images), as well as local evaluations of Unguided SDXL, CFG, PAG, and FreeU, using the exact same evaluation script (
calculate_metrics.py), reference statistics, resolution (256×256 for FID/IS), and reward evaluators (ImageReward-v1.0,PickScore_v1,CLIP ViT-B/32,LAION Aesthetic v2) as Zhang et al. (2026):Table R3.1: Published (
P) vs. Author-Evaluated Local (L) Metrics Under the Identical Evaluation PipelineP)L)L)P)L)P)L)L)L)P)L)P)L)L)Conclusion:
0.02FID on COCO-2014 30k (21.71vs.21.73) and0.17FID on COCO-2017 5k (31.75vs.31.92),0.0005on CLIP,0.002on ImageReward, and0.003on PickScore, confirming that our evaluation pipeline is 100% aligned with Zhang et al. (2026).Section 4.2, Tables 1–2andAppendix D.2, Table 8), we explicitly report both the Published (P) rows and our Locally Re-evaluated (L) rows side-by-side so that every comparison is 100% apples-to-apples under the exact same evaluator code.