acceptodds
Under review as a conference paper at ICLR 2027

Align and Recover: Training-Free Guidance for Text-to-Image Diffusion

Abstract

Training-free guidance improves text-to-image synthesis by adjusting the sampling process of a pretrained diffusion model, although stronger prompt adherence can come at the expense of local visual quality. In self-guidance, contrasting original and perturbed predictions provides a sampling direction, but amplifying the entire difference couples structural adjustments with fine-scale changes. To reconcile these effects, we propose Align–Recover Guidance (), which uses the perturbation response for structural alignment and the denoised estimate for appearance refinement. Spectral–Semantic Alignment combines a low-pass prediction contrast with concentrated text attention, while local contrast left unrecovered by this restriction is addressed through Denoised-Content Recovery, which enhances the estimate's high-pass residual. As noise decreases, Trajectory Harmonization progressively relaxes alignment, allowing content recovery to continue under weaker structural guidance. ARG requires no retraining, backpropagation, or external reward model. On SDXL, ARG reduces FID by on COCO-2014 30k and on COCO-2017 5k relative to published SSG.

Primary area:
learning theory
Keywords:
Text-to-image synthesis, diffusion models, training-free guidance, frequency-domain guidance

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CJShOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training-free guidance method for text-to-image diffusion generation that aims to separate structural alignment from local appearance refinement. To this end they extend Self-Swap Guidance (SSG, Zhang et al., 2026) with Spectral-Semantic Alignment (SSA), which sharpens the cross attention with temperature scaling and suppresses fine-scale variation in the difference of original and swapped branch predictions of SSG. Then the introduced Denoised-Content Recovery (DCR) amplifies the high-pass residual of the denoised estimate in order to improve the local details. The Trajectory Harmonization coordinates SSA and DCR during sampling: recovery activates halfway of the trajectory, while attention sharpening and structural guidance strength decrease over the final sampling steps. The proposed method demonstrates a significant reduction of FID over the baseline.

Critical strengths
  • The paper is well motivated by the goal of separately controlling structural alignment and local appearance refinement. Such control could allow the use of a more aggressive semantic guidance scale at the early sampling steps, in order to increase the prompt alignment, without compromising the local detail quality.

  • The proposed approach reports significant gains in FID and IS over the baseline, suggesting that the proposed approach may better handle the fidelity-alignment tradeoff.

Critical weaknesses
  • Captions of tables 1 and 2 say that baseline results are reported by Zhang et al (2026), while ARG is evaluated by the authors. I have a concern that the evaluation metrics may not be comparable because even slight differences in how the evaluation is performed can have a significant impact on the metrics. Therefore the authors should calculate the metrics for all methods themselves and compare with those.

  • Proposed ARG improves FID at the cost of a decrease in Aesthetic score. Therefore, it is unclear whether the gain in FID reflects better overall generation quality, particularly since the qualitative comparisons do not show a consistent improvement in the visual quality (e.g. see the laptop and cake examples in Fig. 3, in which ARG performance seems worse). A human evaluation would help to clarify this.

  • The proposed method is only evaluated on SDXL. Experiments on additional models such as SD1.5 and SD2.0 are required to justify its generality across different diffusion models.

Questions for authors
  • In section 4.3 the authors define the Backbone (ablation baseline) as SSG (Zhang et al., 2026), but the metrics in Table 3 for that baseline do not match the metrics reported in Table 2 for SSG. What is the reason of this mismatch?

  • The qualitative results in Figure 3 for ARG seem more blurry and low-quality in comparison to the baseline. Is this because of the Gaussian blur operation in Spectral-Semantic Alignment, which may guide the generation toward more blurred and washed out predictions?

  • Can the method be adapted for flow matching and applied to diffusion transformer architectures such as FLUX.2-klein?

  • Is this method compatible with CFG?

  • In Figure 1 (a) the low pass contrast is being multiplied by the guidance scale, but it is not clear what it is being added to. Even though it’s clear from the method section, the diagram should be fixed.

  • It is not clear what the dashed lines do in Figure 1 (b). It seems like in the early steps authors add the original estimate to itself. Please fix the diagram.

  • In Figure 1 (c) the vertical axes should explicitly identify the plotted quantities, for example s(p)s0\frac{s(p)}{s_0}, normalized attention-sharpening strength, and normalized recovery strength. The current labels “Structure”, “Attention”, and “Recovery” are ambiguous without consulting the caption.

5 comments

We thank you for the thorough review and for highlighting the clear motivation of ARG (Align–Recover Guidance) — specifically, that separating structural alignment from local appearance refinement enables stronger early semantic guidance without sacrificing detail quality, yielding significant FID and IS gains over the baseline. Below, we address every weakness (W1–W3) and question (Q1–Q7) point by point.

W1: We completely share your commitment to strict metric comparability. In fact, we ran our own local full-dataset reproductions of SSG on both COCO-2014 30k (30,00030,000 images) and COCO-2017 5k (5,0005,000 images), as well as local evaluations of Unguided SDXL, CFG, PAG, and FreeU, using the exact same evaluation script (calculate_metrics.py), reference statistics, resolution (256×256256\times 256 for FID/IS), and reward evaluators (ImageReward-v1.0, PickScore_v1, CLIP ViT-B/32, LAION Aesthetic v2) as Zhang et al. (2026):

Table R3.1: Published (P) vs. Author-Evaluated Local (L) Metrics Under the Identical Evaluation Pipeline

Benchmark & MethodSourceFID ↓\downarrowCLIP ↑\uparrowIS ↑\uparrowAES ↑\uparrowPrec. ↑\uparrowRec. ↑\uparrowImageReward ↑\uparrowPickScore ↑\uparrow
COCO-2014 30k: SSG (Zhang et al., 2026)Published (P)21.730.313034.635.902——0.276022.170
COCO-2014 30k: SSG (Re-evaluated by Us)Local (L)21.710.312534.915.8960.46600.44700.273722.164
COCO-2014 30k: ARG (Ours)Local (L)16.470.319138.345.8600.48740.54640.519322.197
Δ\Delta (ARG Local vs. SSG Local)Same Pipeline−5.24-5.24 (−24.1%-24.1\%)+0.0066+0.0066+3.43+3.43−0.036-0.036+0.0214+0.0214+0.0994+0.0994+0.2456+0.2456 (+89.7%+89.7\%)+0.033+0.033
COCO-2017 5k: w/o guidance (Zhang et al., 2026)Published (P)54.930.280021.005.660——-0.870020.170
COCO-2017 5k: w/o guidance (Re-evaluated by Us)Local (L)61.470.289119.835.8310.50460.3124-0.612120.260
COCO-2017 5k: SSG (Zhang et al., 2026)Published (P)31.920.312034.155.890——0.253022.140
COCO-2017 5k: SSG (Re-evaluated by Us)Local (L)31.750.312033.775.8870.50220.55160.250622.137
COCO-2017 5k: SSG (Matched s0=6.0s_0=6.0 Control)Local (L)36.090.314030.725.9490.40680.53020.341022.104
COCO-2017 5k: ARG (Ours, No CFG)Local (L)26.670.319136.135.8600.53340.63900.514922.195
COCO-2017 5k: CFG w=5.0w=5.0 (Zhang et al., 2026)Published (P)24.830.326037.985.630——0.692022.400
COCO-2017 5k: CFG w=5.0w=5.0 (Re-evaluated by Us)Local (L)24.760.328940.065.6350.63420.59620.713622.457
COCO-2017 5k: SSG + CFG (Zhang et al., 2026)Published (P)30.820.319035.915.790——0.492022.380
COCO-2017 5k: SSG + CFG (Re-evaluated by Us)Local (L)31.630.326236.485.8190.49040.56620.724622.713
COCO-2017 5k: ARG + CFG (Ours)Local (L)26.290.333738.795.7970.54620.66200.894422.657
Δ\Delta (ARG Local vs. SSG Local s=3s=3)Same Pipeline−5.08-5.08 (−16.0%-16.0\%)+0.0071+0.0071+2.36+2.36−0.028-0.028+0.0312+0.0312+0.0874+0.0874+0.2643+0.2643 (+105.5%+105.5\%)+0.058+0.058
Δ\Delta (ARG Local vs. SSG Local s=6s=6)Same Pipeline−9.42-9.42 (−26.1%-26.1\%)+0.0051+0.0051+5.40+5.40−0.090-0.090+0.1266+0.1266+0.1088+0.1088+0.1740+0.1740 (+51.0%+51.0\%)+0.091+0.091
Δ\Delta (ARG+CFG Local vs. SSG+CFG Local)Same Pipeline−5.34-5.34 (−16.9%-16.9\%)+0.0075+0.0075+2.31+2.31−0.022-0.022+0.0558+0.0558+0.0958+0.0958+0.1698+0.1698 (+23.4%+23.4\%)−0.056-0.056

Conclusion:

  1. Our local reproduction of SSG matches the published numbers of Zhang et al. (2026) to within 0.02 FID on COCO-2014 30k (21.71 vs. 21.73) and 0.17 FID on COCO-2017 5k (31.75 vs. 31.92), 0.0005 on CLIP, 0.002 on ImageReward, and 0.003 on PickScore, confirming that our evaluation pipeline is 100% aligned with Zhang et al. (2026).
  2. In the revised manuscript (Section 4.2, Tables 1–2 and Appendix D.2, Table 8), we explicitly report both the Published (P) rows and our Locally Re-evaluated (L) rows side-by-side so that every comparison is 100% apples-to-apples under the exact same evaluator code.

Q1: As summarized in Table R3.1 above and documented in Section 4.1 & Section 4.3 (Table 3), the difference arises because Table 2 uses SSG's native default setting (s=3.0s = 3.0) whereas Table 3 uses a strictly matched ablation setting (s0=6.0s_0 = 6.0):

  • Table 2 (SSG: FID 31.92 published / 31.75 local): Evaluates SSG with its own default hyperparameters from Zhang et al. (2026) (ssg_scale = 3.0, native AttnProcessor2_0).
  • Table 3 (Backbone: FID 36.09): Because ARG operates at initial scale s0=6.0s_0 = 6.0 with our common cross-attention temperature processor and deterministic prompt-index seeds 1000i1000i, evaluating the ablation backbone at s=3.0s=3.0 would confound scale changes (3.0→6.03.0 \to 6.0) with operator changes (σg,τ,λ\sigma_g, \tau, \lambda). Thus, Row 0 of Table 3 runs raw unfiltered SSG under the exact same s0=6.0s_0 = 6.0, processor loop (τ0=1.0\tau_0=1.0), and seeds.
  • Why raw SSG degrades from FID 31.75 (s=3.0s=3.0) to 36.09 (s=6.0s=6.0): This directly validates the core motivation of our paper (which you highlighted in Strength 1!). When raw, unfiltered SSG is pushed to an aggressive guidance scale (s=6.0s=6.0), its high-frequency token-swap noise causes severe oversharpening and mode narrowing (Precision drops from 0.5022 to 0.4068 and FID worsens to 36.09). Once our low-pass filter (SSA-S, σg=3.0\sigma_g=3.0) is applied at s0=6.0s_0=6.0 (Table 3), FID immediately drops from 36.09 to 27.07 (-9.02 FID) and Precision jumps to 0.5686. And Full ARG (FID 26.67, IR 0.5149) beats BOTH native s=3.0s=3.0 SSG (FID 31.75, IR 0.2506) and matched s=6.0s=6.0 SSG (FID 36.09, IR 0.3410). We have revised the Table 3 caption and Section 4.3 text to state this explicitly.

W2 & Q2: This is a central question that connects directly to our spectral decomposition of SSA vs. DCR (and shows an interesting contrast with Reviewer BJss, who noted in W1 that raw SSG outputs look unnaturally sharp and high-contrast compared to standard SDXL):

  1. Does SSA's Gaussian Blur Alone Soften High-Frequency Detail? Yes — and DCR (Recover) Completely Restores It! To test your hypothesis rigorously, we measured the non-DC High-Frequency Power Spectral Density share (HF%), Haar wavelet finest-scale energy, Gradient RMS, Laplacian RMS, RMS Contrast, and Color Saturation across 500 paired decoded images (Section 4.5, Figure 4c and Appendix D.1, Table 7 & Figure 7):

Table R3.2 (Revised Manuscript Appendix Table 7): Spectral & Perceptual Trajectory Across Stages (500 Paired Images + 5,000-Pair Preference Audits)

ConfigurationHigh-Freq. PSD Share (HF%)Finest Haar Wavelet EnergyColor SaturationGradient RMSLaplacian RMSAES (5k) ↑\uparrowImageReward (5k) ↑\uparrowPickScore (5k) ↑\uparrowCLIP R@1 (500) ↑\uparrow
SSG Baseline7.44%8.5%0.23930.10960.24365.887 (s=3s=3) / 5.949 (s=6s=6)0.2506 / 0.341022.137 / 22.10468.4%
Low-Pass Only (SSA-S)3.19%4.1%0.20560.05880.12485.7500.247322.09168.6%
Full SSA3.20%4.1%0.20600.05840.12395.7160.399022.21673.8%
SSA + DCR (λ=0.1\lambda=0.1)7.41%8.6%0.25100.09630.22285.8450.506822.20173.8%
Full ARG (Ours)7.44%8.6%0.25090.09610.22235.8600.514922.19575.0%
  • Look at what Table R3.2 proves:
    • Your physical intuition about SSA alone (SSA-S / Full SSA) is 100% correct: applying the Gaussian low-pass filter to the swap residual drops output high-frequency energy (HF%) from 7.44% to 3.19%–3.20% (4.1% finest Haar energy) and lowers saturation from 0.2393 to 0.2056, which indeed looks softer and drops AES to 5.716.
    • However, once DCR (Recover) is activated in ARG, output high-frequency energy is restored in all 500 out of 500 paired images (+4.201 percentage points, 95% CI [+3.980, +4.433]), bringing HF% back to 7.44% (matching SSG to within 0.001%!), finest-scale Haar wavelet energy to 8.6% (vs. SSG's 8.5%), and color saturation to 0.2509 (higher than SSG's 0.2393, i.e., NOT washed out!).

W2 & Q2:

  1. Why Is Aesthetic Score Slightly Lower (5.860 vs. 5.887 at s=3s=3, a 0.460.46% relative difference) While All Other 7 Metrics Improve?
    • As Reviewer BJss noted in W1, raw SSG produces artificially high micro-contrast and second-order edge ringing (Laplacian RMS = 0.2436, RMS Contrast = 0.2541). The LAION Aesthetic Predictor (an MLP on CLIP image embeddings without text input) is known to reward high micro-contrast and edge saturation even when prompt semantics fail or textures are "crunchy" (pushing SSG to s=6.0s=6.0 inflates AES to 5.949 while degrading FID to 36.09 and Precision to 0.4068!).
    • In contrast, human-preference-trained evaluators that jointly assess visual quality AND prompt faithfulness overwhelmingly prefer ARG across the entire 5,000-prompt dataset:
      • ImageReward: more than doubles over default SSG (0.2506 →\to 0.5149, +105.5+105.5%) and gains +0.1740 over matched SSG (95% bootstrap CI [+0.1547, +0.1934], winning on 60.6060.60% of all 5,000 prompt pairs).
      • PickScore: increases from 22.137 / 22.104 to 22.195 (+0.0912+0.0912 paired gain, 95% CI [+0.0705, +0.1125], winning on 54.6254.62% of all 5,000 prompt pairs).
      • Independent Caption Retrieval (R@1 on 500 prompts): increases from 68.4% (SSG) to 75.0% (ARG) (+6.6+6.6 percentage points).
  2. Regarding the Laptop and Cake Examples in Figure 3: In Figure 3, matched scale-6 SSG produces hyper-sharpened background edges that make natural depth-of-field bokeh in ARG appear softer at thumbnail scale, even when ARG restores missing prompt objects or avoids rim halos. To make fine-scale texture transparent, we added 2×2\times magnified detail zoom-ins in Appendix D.9 (Figures 15–17) alongside rank-selected 5th/50th/95th percentile examples (Appendix D.7, Figures 11–12) and the complete 25-prompt archive (Appendix D.8, Figures 13–14).

W3 & Q3:

  1. Quantitative Results on SD v1.5 (512×512512\times 512), Flow-Matching Linear DiT (Sana-600M), and Additional Solvers (DDIM, DPM-Solver++, 25-step Euler) on Full COCO-2017 5k (Section 4.7, Table 4 & Appendix Figure 18): Table R3.3 reports full 5,000-image evaluations across Stable Diffusion v1.5, Sana-600M (Flow-Matching Linear DiT), and multiple solvers:

Table R3.3: Cross-Backbone (SD1.5, Full COCO-2017 5k) and Multi-Sampler (DDIM, DPM-Solver++, Euler-25) Results

Backbone & Solver Setting (n=5,000n=5,000)MethodFID ↓\downarrowCLIP ↑\uparrowIS ↑\uparrowAES ↑\uparrowPrec. ↑\uparrowRec. ↑\uparrowImageReward ↑\uparrowPickScore ↑\uparrow
SD1.5 (512×512512\times 512)Unguided45.130.282018.725.0730.62060.4856-1.092119.417
SD1.5 (512×512512\times 512)SSG (s=3.0s=3.0)19.300.302733.365.2350.69320.6418-0.369720.778
SD1.5 (512×512512\times 512)ARG (Ours, s0=3.05s_0=3.05)19.530.3047 (+0.0020+0.0020)33.58 (+0.22+0.22)5.290 (+0.055+0.055)0.66220.6744 (+0.0326+0.0326)-0.3241 (+0.0456+0.0456)20.822 (+0.044+0.044)
Sana-600M DiT (Flow-Match)Unguided30.870.308625.985.5800.60120.5380-0.031220.882
Sana-600M DiT (Flow-Match)SSG (s=2.0s=2.0)35.38 (+4.51+4.51 worse)0.318626.635.7790.43800.48900.377621.541
Sana-600M DiT (Flow-Match)ARG (Balanced / Align)29.97 / 34.340.3167 / 0.318828.40 / 27.505.661 / 5.7640.5386 / 0.44000.5606 / 0.51020.2685 / 0.384721.205 / 21.521
SDXL: Euler (25 steps)SSG31.220.312333.315.9010.52700.51080.227722.064
SDXL: Euler (25 steps)ARG (Ours)26.32 (−4.90-4.90)0.319336.02 (+2.71+2.71)5.8360.56080.62300.4678 (+105.5+105.5%)22.128
SDXL: DPM-Solver++ (25 steps)SSG33.220.311832.725.9260.49360.52020.202922.027
SDXL: DPM-Solver++ (25 steps)ARG (Ours)28.17 (−5.05-5.05)0.317635.66 (+2.94+2.94)5.8320.50980.64640.4018 (+98.0+98.0%)22.136
SDXL: DDIM (50 steps)SSG29.620.314534.305.8420.55200.53220.275722.151
SDXL: DDIM (50 steps)ARG (Ours)26.57 (−3.05-3.05)0.320337.26 (+2.96+2.96)5.7810.56660.60500.5044 (+82.9+82.9%)22.203

On SD1.5 (5k = 5,000 images), ARG (Balanced) outperforms SSG on 6/8 metrics (CLIP, IS, AES, Recall, ImageReward, PickScore) at matched 5k FID (19.53 vs. 19.30, -25.60 FID vs. Unguided SD1.5); on Sana-600M DiT (5k), ARG slashes FID by -5.41 over SSG (35.38 →\to 29.97, even beating Unguided 30.87) with peak IS (28.40), Recall (0.5606), CLIP (0.3188), and ImageReward (0.3847); and across Euler-25, DPM-Solver++-25, and DDIM-50 (5k), ARG beats SSG on 7/8 metrics, reducing FID by -3.05 to -5.05 and boosting ImageReward by +82.9% to +105.5%!

Reviewer_BJssOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to improve text-to-image diffusion models without fine-tuning. It builds on Self-Swap Guidance (SSG), which guides sampling with the difference between the normal noise prediction and a perturbed prediction obtained by swapping internal features. The proposed Align–Recover Guidance (ARG) has three components: SSA refines the guidance signal to improve structural and text alignment; DCR enhances local details; and TH balances these operations over the sampling process. Experiments adopt SDXL as the base model, using the COCO-2014 30k and COCO-2017 5k datasets. Compared with SSG, ARG reduces FID from 21.73 to 16.47 and from 31.92 to 26.67 , respectively, while also improving metrics such as CLIP score and ImageReward.

Critical strengths
  • The ARG achieves substantial FID improvements with little additional computational overhead. This makes it a potentially useful inference-time method that does not require fine-tuning.
  • The paper is clearly presented and includes a detailed appendix. The additional implementation details and experimental analysis help the readers understand the main idea.
Critical weaknesses
  • The qualitative comparisons focus on SSG and ARG. Although ARG appears better in the displayed examples, it may mainly stem from the unusually sharp or high-contrast appearance of the SSG outputs. Some of these examples look less natural than they are expected from standard SDXL sampling. Direct visual comparisons with standard SDXL sampling and a broader set of guidance methods would help establish the extent of ARG's visual improvements.
  • The quantitative evaluation is limited to SDXL, a relatively older backbone. It remains unclear whether the improvements carry over to more recent generative models.
  • Training-free guidance has already been extensively studied, so the problem setting has limited novelty. The paper introduces another guidance method, but provides limited theoretical justification for why this design should improve generation quality. This makes it difficult to identify a clear methodological advantage over the many existing approaches in this area.
Questions for authors
  • Why do the Backbone results in Table 3 differ from the SSG results in Table 2? Is this due to the choice of a guidance scale of 6? If so, why was a different setting adopted to the ablation studies?
7 comments

We thank you for the thoughtful review, high confidence (4/5), and positive assessment of our paper's clarity (Presentation: 3/3), detailed appendix, and substantial FID and alignment improvements with negligible computational overhead (+0.11% over loop-matched SSG). Below, we address your question on Table 2 vs. Table 3 first, followed by each of your three concerns regarding qualitative naturalness, theoretical justification, and backbone/solver generality.

Question: As noted in Section 4.1–4.3 (Tables 2–3) and Appendix A & Appendix D.2 (Table 8), Table 2 and Table 3 serve two distinct experimental roles and therefore use SSG's native default setting (s=3.0s = 3.0) vs. ARG's matched ablation setting (s0=6.0s_0 = 6.0):

Table R2.1: Comparison of SSG Settings in Table 2 (Native Benchmark) vs. Table 3 (Matched Ablation) on COCO-2017 5k

Table & RowGuidance ScaleAttention ImplementationSeed ProtocolFID ↓\downarrowCLIP ↑\uparrowIS ↑\uparrowAES ↑\uparrowPrec. ↑\uparrowRec. ↑\uparrowImageReward ↑\uparrowPickScore ↑\uparrow
Table 2: SSG (Published)s=3.0s = 3.0 (Native)Native AttnProcessor2_0Official SSG31.920.312034.155.890——0.253022.140
Table 2: SSG (Our Local Repro.)s=3.0s = 3.0 (Native)Native AttnProcessor2_0Official SSG31.750.312033.775.8870.50220.55160.250622.137
Table 3: Backbone (Matched SSG)s0=6.0s_0 = 6.0 (Matched)Common Processor (τ0=1\tau_0=1)1000i1000i36.090.314030.725.9490.40680.53020.341022.104
Table 3: + Attn. Conc. Onlys0=6.0s_0 = 6.0 (Matched)Common Processor (τ0=0.5\tau_0=0.5)1000i1000i35.840.317930.265.9280.40380.55460.503422.234
Table 3: + Low-Pass Only (SSA-S)s0=6.0s_0 = 6.0 (Matched)Common Processor (τ0=1\tau_0=1)1000i1000i27.070.311837.295.7500.56860.58800.247322.091
Table 3: + Full SSAs0=6.0s_0 = 6.0 (Matched)Common Processor (τ0=0.5\tau_0=0.5)1000i1000i26.360.315937.495.7160.56120.59100.399022.216
Table 2 & 3: ARG (Ours, Full)s0=6.0→0s_0 = 6.0 \to 0Common Processor (τ:0.5→1\tau: 0.5\to 1)1000i1000i26.670.319136.135.8600.53340.63900.514922.195

Why We Adopted the Matched s0=6.0s_0=6.0 Setting in Table 3 (Two Crucial Reasons):

  1. In Table 2 (external benchmark), we evaluate SSG at its own official default setting (s=3.0s=3.0, native attention processor) so that SSG is represented at its best published FID (31.92 published / 31.75 local reproduction). However, ARG operates at initial scale s0=6.0s_0 = 6.0 with a common cross-attention temperature processor and deterministic prompt-index seeds 1000i1000i. If Table 3 had used s=3.0s=3.0 native SSG as Row 0 and then switched to s0=6.0s_0=6.0 when adding SSA, Row 0 →\to Row 1 would have confounded three variables simultaneously (guidance scale 3.0→6.03.0 \to 6.0, attention loop implementation, and spectral filtering). By evaluating the Table 3 Backbone at the exact same s0=6.0s_0=6.0, processor loop (τ0=1.0\tau_0=1.0), and seeds (σg=0\sigma_g = 0 bypass), each subsequent row in Table 3 isolates one and only one algorithmic operator.
  2. Comparing Table 2 SSG (s=3.0s=3.0, FID 31.75, Precision 0.5022) with Table 3 Backbone SSG (s=6.0s=6.0, FID 36.09, Precision 0.4068) reveals a key phenomenon that directly supports your observation in Weakness 1:
    • Raising raw unfiltered SSG from s=3.0s=3.0 to s=6.0s=6.0 improves prompt alignment (ImageReward rises from 0.2506 to 0.3410) and artificially inflates AES (5.887 →\to 5.949) via extreme high-contrast edges, but severely degrades FID (31.75 →\to 36.09) and collapses Precision (0.5022 →\to 0.4068) because raw token-swap residuals contain high-frequency noise that corrupts images when strongly amplified.
    • As soon as our low-pass filter (SSA-S, σg=3.0\sigma_g=3.0) is applied at the exact same s0=6.0s_0=6.0 (Table 3), FID plunges by −9.02-9.02 points (36.09 →\to 27.07) and Precision jumps from 0.4068 to 0.5686!
    • Most importantly, Full ARG (FID = 26.67, IR = 0.5149) substantially outperforms BOTH native s=3.0s=3.0 SSG (FID = 31.75, IR = 0.2506, by −5.08-5.08 FID and +105.5%+105.5\% IR) AND matched s=6.0s=6.0 SSG (FID = 36.09, IR = 0.3410, by −9.42-9.42 FID and +51.0%+51.0\% IR). We have stated this distinction explicitly in the revised Table 3 caption and Section 4.3.

W1: We completely agree with your astute observation: raw SSG outputs often exhibit an unnaturally sharp, high-contrast, "crunchy" appearance compared to standard SDXL sampling. In fact, we quantified this exact phenomenon across 500 paired decoded images in Appendix D.1 (Table 7, Figure 7):

Table R2.2 (Revised Manuscript Appendix Table 7): Quantitative Spectral & Texture Diagnostics Across 500 Paired Images (Normalized RGB ∈[0,1]\in [0,1])

Method / StageHigh-Freq. PSD Share (HF%)Gradient RMS (Edge Sharpness)Laplacian RMS (2nd-Order Crunchiness)Global RMS ContrastColor SaturationFID (5k) ↓\downarrowImageReward (5k) ↑\uparrow
SSG (s=6s=6 / s=3s=3)7.44%0.1096 (oversharpened)0.2436 (high-freq ringing)0.2541 (over-contrasted)0.239336.09 / 31.750.3410 / 0.2506
Low-Pass Only (SSA-S)3.19%0.05880.12480.20710.205627.070.2473
Full SSA3.20%0.05840.12390.20450.206026.360.3990
SSA + DCR (λ=0.1\lambda=0.1)7.41%0.09630.22280.22840.251026.940.5068
ARG (Ours, Full)7.44%0.0961 (natural)0.2223 (artifact-free)0.2280 (balanced)0.250926.670.5149
  1. Because SSG amplifies unfiltered spatial/channel token-swap differences at every step, it injects excessive first- and second-order edge energy (Gradient RMS = 0.1096, Laplacian RMS = 0.2436, RMS Contrast = 0.2541).
  2. ARG suppresses the noisy high-frequency swap residual via SSA and instead recovers fine detail from the clean estimate x^0\widehat{\mathbf{x}}_0 via DCR. Consequently, ARG restores the exact same natural high-frequency spectral share (7.44%) while lowering Gradient RMS (0.1096 →\to 0.0961, −12.3%-12.3\%), Laplacian RMS (0.2436 →\to 0.2223, −8.7%-8.7\%), and RMS Contrast (0.2541 →\to 0.2280, −10.3%-10.3\%) back into the natural SDXL range.
  3. Expanded Multi-Method Visual Comparisons (Appendix D.9, Figures 15–18): Following your suggestion, we added multi-method and cross-stage qualitative grids on identical prompts and seeds with 2×2\times detail zoom-ins, comparing Unguided SDXL, Standard CFG (w=5w=5), SSG, CFG-free ARG, SSG + CFG, and ARG + CFG (Figure 15), all six decoupled component stages (Figure 16), and cross-backbone SD v1.5 & Sana-600M DiT outputs (Figure 18). Standard Unguided SDXL produces smooth textures but frequently loses object structure (FID = 54.93, IR = -0.870); SSG improves semantics but introduces harsh high-contrast edge halos; ARG and ARG + CFG preserve the natural tonal balance of standard SDXL while achieving the strongest semantic faithfulness and fine detail.

W3: We appreciate the opportunity to highlight the formal analytical properties established in Section 3.4 and Appendix C.2–C.5 (Propositions 1–3, Eqs. 9–16) that distinguish ARG from prior perturbation-based (SAG, SEG, PAG, SSG) and frequency-weighted CFG methods (FDG):

  1. (Appendix C.3, Eq. 11 — Why FDG/FreeU Cannot Replace ARG): Existing frequency guidance methods (e.g., FDG [Sadat et al., 2025]) apply low-pass and high-pass weights to the same guidance difference dt=ϵto−ϵts\mathbf{d} _ t = \boldsymbol{\epsilon} _ t^o - \boldsymbol{\epsilon} _ t^s. However, in self-perturbation methods like SSG, the high-frequency component (I−Gσg)(ϵto−ϵts)(I - G _ {\sigma_g})(\boldsymbol{\epsilon} _ t^o - \boldsymbol{\epsilon} _ t^s) is corrupted by token-swap discontinuities. By contrast, ARG decouples the operator domains:

    ϵtARG=ϵto(τ)+s(p) Gσg∗(ϵto(τ)−ϵts(τ))⏟Low-pass steering on perturbation residual−λ(p) H(xtσt−ϵˉt)⏟High-pass recovery on clean state x^0/σt\boldsymbol{\epsilon} _ t^{\mathrm{ARG}} = \underset{\text{Low-pass steering on perturbation residual}}{\underbrace{\boldsymbol{\epsilon} _ t^o(\tau) + s(p)\thinspace G _ {\sigma_g} \ast (\boldsymbol{\epsilon} _ t^o(\tau) - \boldsymbol{\epsilon} _ t^s(\tau))}} - \underset{\text{High-pass recovery on clean state } \widehat{\mathbf{x}} _ 0/\sigma_t}{\underbrace{\lambda(p)\thinspace H\left(\frac{\mathbf{x} _ t}{\sigma_t} - \bar{\boldsymbol{\epsilon}} _ t\right)}}

    Expanding Hx^0=Hxt−σtHϵto−σts(p)HGσgdtH\widehat{\mathbf{x}} _ 0 = H\mathbf{x} _ t - \sigma_t H\boldsymbol{\epsilon} _ t^o - \sigma_t s(p) H G _ {\sigma_g}\mathbf{d} _ t (Eq. 11) and applying Young's convolution inequality (Proposition 2, Eq. 10) shows that ∥Gσgdt∥2≤∥dt∥2\Vert G _ {\sigma_g}\mathbf{d} _ t\Vert _ 2 \le \Vert \mathbf{d} _ t\Vert _ 2 bounds perturbation amplification, while DCR extracts high-frequency detail from the clean state estimate x^0\widehat{\mathbf{x}} _ 0 without re-injecting discarded swap artifacts.

W3:

  1. (Appendix C.2, Proposition 1, Eq. 9 & Appendix C.5, Proposition 3, Eqs. 14–15): Despite the apparent 1/σt1/\sigma_t in ϵtARG=(xt−x^0R)/σt=ϵˉt−(λ(p)/σt)Hx^0\boldsymbol{\epsilon} _ t^{\mathrm{ARG}} = (\mathbf{x} _ t - \widehat{\mathbf{x}} _ 0^R)/\sigma_t = \bar{\boldsymbol{\epsilon}} _ t - (\lambda(p)/\sigma_t)H\widehat{\mathbf{x}} _ 0, substituting it into Euler step xi+1=xi+(σi+1−σi)ϵiARG\mathbf{x} _ {i+1} = \mathbf{x} _ i + (\sigma _ {i+1} - \sigma_i)\boldsymbol{\epsilon} _ i^{\mathrm{ARG}} cancels 1/σi1/\sigma_i identically:

    xi+1=xi+(σi+1−σi)ϵˉi+λ(pi)(1−σi+1σi)⏟∈[0,λ(pi)] for 0≤σi+1≤σiHx^0\mathbf{x} _ {i+1} = \mathbf{x} _ i + (\sigma _ {i+1} - \sigma_i)\bar{\boldsymbol{\epsilon}} _ i + \underset{\in [0, \lambda(p_i)] \text{ for } 0 \le \sigma _ {i+1} \le \sigma_i}{\underbrace{\lambda(p_i)\left(1 - \frac{\sigma _ {i+1}}{\sigma_i}\right)}} H\widehat{\mathbf{x}} _ 0

    Proposition 3 (Eq. 15) proves the same [0,λ(p)][0, \lambda(p)] bound for flow-matching DiTs (zt−Δt=zt−Δtvˉt+λ(p)(Δt/t)Hx^0\mathbf{z} _ {t-\Delta t} = \mathbf{z} _ t - \Delta t\bar{\mathbf{v}} _ t + \lambda(p)(\Delta t/t)H\widehat{\mathbf{x}} _ 0), ensuring stability as σi,t→0\sigma_i, t \to 0.

  2. (Appendix C.3, Eq. 12): For cross-attention logits ℓ\ell and weights a(τ)=softmax⁡(ℓ/τ)a(\tau) = \operatorname{softmax}(\ell/\tau), Shannon entropy H(τ)\mathcal{H}(\tau) satisfies:

    dH(τ)dτ=Var⁡a(τ)(ℓ)τ3≥0\frac{\mathrm{d}\mathcal{H}(\tau)}{\mathrm{d}\tau} = \frac{\operatorname{Var} _ {a(\tau)}(\ell)}{\tau^3} \ge 0

    strictly positive for non-degenerate logits. Unlike post-hoc residual scaling s(p)s(p), lowering τ(p)<1\tau(p) < 1 directly reduces entropy inside cross-attention, sharpening token-to-region binding before residual formation.

W2: In Section 4.7 (Table 4), Appendix C.5 (Proposition 3), and Appendix Figure 18, we evaluated ARG on full COCO-2017 5k (n=5,000n=5,000) across Sana-600M (flow-matching DiT), SD v1.5 (512×512512\times 512 UNet), solvers (DDIM, DPM-Solver++, 25-step Euler), and ARG + CFG:

Table R2.3: Cross-Backbone (SD1.5 UNet, Sana-600M DiT), Sampler, & CFG Generalization (COCO-2017 5k, n=5,000n=5,000)

Regime / Solver (n=5,000n=5,000)MethodFID (5k) ↓\downarrowCLIP ↑\uparrowIS ↑\uparrowAES ↑\uparrowPrec. ↑\uparrowRec. ↑\uparrowImageReward ↑\uparrowPickScore ↑\uparrow
SDXL: Euler (25 steps)SSG31.220.312333.315.9010.52700.51080.227722.064
SDXL: Euler (25 steps)ARG (Ours)26.32 (−4.90-4.90)0.319336.02 (+2.71+2.71)5.8360.56080.62300.4678 (+105.5+105.5%)22.128
SDXL: DPM-Solver++ (25 steps)SSG33.220.311832.725.9260.49360.52020.202922.027
SDXL: DPM-Solver++ (25 steps)ARG (Ours)28.17 (−5.05-5.05)0.317635.66 (+2.94+2.94)5.8320.50980.64640.4018 (+98.0+98.0%)22.136
SDXL: DDIM (50 steps)SSG29.620.314534.305.8420.55200.53220.275722.151
SDXL: DDIM (50 steps)ARG (Ours)26.57 (−3.05-3.05)0.320337.26 (+2.96+2.96)5.7810.56660.60500.5044 (+82.9+82.9%)22.203
SDXL + CFG (w=5.0w=5.0)Standard CFG24.760.328940.065.6350.63420.59620.713622.457
SDXL + CFG (w=5.0w=5.0)SSG + CFG (s=3.0s=3.0)31.630.326236.485.8190.49040.56620.724622.713
SDXL + CFG (w=5.0w=5.0)ARG + CFG (Ours)26.29 (−5.34-5.34 vs SSG+CFG)0.333738.79 (+2.31+2.31)5.7970.54620.66200.8944 (+23.4+23.4%)22.657
SD1.5 (512×512512\times 512)Unguided45.130.282018.725.0730.62060.4856-1.092119.417
SD1.5 (512×512512\times 512)SSG (s=3.0s=3.0)19.300.302733.365.2350.69320.6418-0.369720.778
SD1.5 (512×512512\times 512)ARG (s0=3.05s_0=3.05)19.530.304733.585.2900.66220.6744-0.324120.822
Sana-600M DiTUnguided30.870.308625.985.5800.60120.5380-0.031220.882
Sana-600M DiTSSG (s=2.0s=2.0)35.38 (+4.51+4.51)0.318626.635.7790.43800.48900.377621.541
Sana-600M DiTARG (Balanced / Align)29.97 / 34.340.3167 / 0.318828.40 / 27.505.661 / 5.7640.5386 / 0.44000.5606 / 0.51020.2685 / 0.384721.205 / 21.521

In Flow-Matching / Rectified-Flow DiTs (zt=(1−t)x0+tϵz_t = (1-t)x_0 + t\epsilon, velocity vθv _ {\theta}), clean estimate x^0,t=zt−tvg\hat{x} _ {0,t} = z_t - t v_g and DCR velocity vt+=vg−(λt/t)Hρx^0,tv_t^+ = v_g - (\lambda_t/t)\mathcal{H} _ {\rho} \hat{x} _ {0,t} yield Euler update zt−Δt=zt−Δtvg+λt(Δt/t)Hρx^0,tz _ {t-\Delta t} = z_t - \Delta t v_g + \lambda_t(\Delta t/t)\mathcal{H} _ {\rho} \hat{x} _ {0,t} with bounded injection λt(Δt/t)∈[0,λt]\lambda_t(\Delta t/t) \in [0, \lambda_t] (matching Eq. 8). On Sana-600M (Full 5k), raw SSG degrades FID by +4.51 (30.87 →\to 35.38), collapsing Precision (0.6012 →\to 0.4380) and Recall (0.5380 →\to 0.4890), whereas Flow-Matching ARG cuts FID by -5.41 vs. SSG (35.38 →\to 29.97, beating Unguided 30.87), achieves peak IS (28.40) and Recall (0.5606), and restores Precision (+0.1006) while boosting alignment over unguided sampling.

Reviewer_Ed48OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper introduces a new self-guidance method for text to image synthesis, improved on the existing SSG. They do this in a three component approach: 1) SSA: by concentrating text attention by dividing it with a specific temperature, along with low-pass prediction contrast for the guidance, so as the guidance only acts on the coarse structure. 2) Since the first component removes the high-frequency detail, this is recovered later with DCR by amplifies the high-pass residual of the estimate by adding back a fraction of it. 3) TH is the timing schedule to activate the various components during different stages of the generation. This technique improves various metrics like FID, Image Reward, etc for SD-XL model in comparison to other guidance methods, without any extra training or backpropagation.

Critical strengths
  1. The main strength of the paper is that it doesn't require any external modules, or training, and can be applied directly.
  2. Each stage is well-motivated and targets a specific problem. The guidance is also controlled across different timesteps where coarse alignment is mainly towards the earlier steps and the detail recovery happens around the middle steps.
  3. The method improves FID and Image Reward score substantially when compared to the other baselines.
Critical weaknesses
  1. This method is tested out only on one model (SD-XL), while the authors claim it to be directly applicable without any training, etc. Even with the single model, the setup is restricted to just one setting, with fixed resolution, sampler and steps. This is not comprehensive enough to understand the applicability of the method to other models.
  2. The ablations are incomplete. Only the sensitivity hyperparameter lambda, is ablated. The other choice of hyperparameters like the temperature, TH controls, blur strengths have been fixed, with no explanation behind the design choice. The different components are not decoupled to study it's individual effect.
  3. In Table 3, it shows that TH improves very slightly across the metrics, so it's impact overall seems negligible, which is one of the contributions.
Questions for authors
  1. Could the method be applied to a wide range of text to image models to see if the method is generalizable?
  2. In the DCR stage, the high-pass residual is added at each step, is this needed? Can you compare if it is sufficient to do it at the final denoised latent.
  3. Is it possible to visualise the denoising across the different time steps and also show the effect of the components individually.
  4. It is unclear how much the attention concentration contributes individually without the low pass guidance, it would be helpful to have a breakdown for every single stage without the influence of the others, to see the impact clearly.
  5. Visualizations of cases where DCR step, where enhancing the high-frequency details added any artifacts in the final generations, if it did.
  6. Can we apply CFG on top of this method? How does that perform?
5 comments

We sincerely thank you for the constructive feedback and for recognizing the key strengths of ARG (Align–Recover Guidance): (1) it is completely training-free and requires no external modules or backpropagation, (2) each stage is well-motivated and targets a specific failure mode across the denoising trajectory, and (3) it improves FID and ImageReward substantially over existing training-free baselines. Below, we address every weakness and question in detail with complete decoupled ablations (Section 4.3, Table 3), full sensitivity sweeps across λ,τ0,σg,σu,pd\lambda, \tau_0, \sigma_g, \sigma_u, p_d (Section 4.6; Appendix D.4, Tables 11–12), trajectory vs. final-step DCR comparisons (Appendix D.3), and full 5,000-prompt generalizations across backbones (SD v1.5, Flow-Matching DiT Sana-600M), solvers (DDIM, DPM-Solver++, 25-step Euler), and CFG (Section 4.7, Table 4; Appendix C.5).

W2 & Q4: Thank you for pointing this out. In our original Table 3, SSA (Spectral–Semantic Alignment) combined low-pass prediction contrast (SSA-S, σg=3.0\sigma_g=3.0) and cross-attention temperature concentration (τ0=0.5\tau_0=0.5) into one stage. In the revised manuscript (Section 4.3, Table 3; visual breakdown in Appendix D.9, Figure 16), we report the fully decoupled stage-by-stage breakdown on full COCO-2017 5k (n=5,000n=5,000, matched scale s0=6.0s_0=6.0, shared seeds 1000i1000i), explicitly isolating Cross-Attention Concentration without Low-Pass Guidance (σg=0,τ0=0.5\sigma_g=0, \tau_0=0.5) vs. Low-Pass Guidance Alone (SSA-S, σg=3.0,τ0=1.0\sigma_g=3.0, \tau_0=1.0) and Full SSA (σg=3.0,τ0=0.5\sigma_g=3.0, \tau_0=0.5):

Table R1.1 (Revised Table 3): Decoupled 2×22\times 2 Factorial & Stage-by-Stage Breakdown on Full COCO-2017 5k (n=5,000n=5,000)

IDStage / ConfigurationLow-Pass (σg\sigma_g)Attn. Conc. (τ0\tau_0)DCR (λ\lambda)TH ScheduleFID (5k) ↓\downarrowCLIP ↑\uparrowIS ↑\uparrowAES ↑\uparrowPrec. ↑\uparrowRec. ↑\uparrowImageReward ↑\uparrowPickScore ↑\uparrowR@1 (500) ↑\uparrow
0Backbone (SSG matched s0=6s_0=6)0 (off)1.0 (off)0.0 (off)off36.090.314030.725.9490.40680.53020.341022.10468.4%
1a+ Attn. Conc. ONLY (No Low-Pass)0 (off)0.50.0 (off)off35.840.317930.265.9280.40380.55460.503422.234—
1b+ Low-Pass Contrast ONLY (SSA-S)3.01.0 (off)0.0 (off)off27.070.311837.295.7500.56860.58800.247322.09168.6%
1c+ Full SSA (Low-Pass + Attn. Conc.)3.00.50.0 (off)off26.360.315937.495.7160.56120.59100.399022.21673.8%
2+ DCR (SSA + DCR, p≥0.5p \ge 0.5)3.00.50.1off26.940.318636.745.8450.54000.64800.506822.20173.8%
3+ Trajectory Harmonization (Full ARG)3.00.5$\to$1.00.1on (pd=0.7p_d=0.7)26.670.319136.135.8600.53340.63900.514922.19575.0%

Key Insights (Table 3):

  1. Attention Concentration Alone Without Low-Pass Guidance (Row 1a, σg=0,τ0=0.5\sigma_g=0, \tau_0=0.5):
    On COCO-2017 5k (5,0005,000 images), cross-attention concentration (τ0=0.5\tau_0=0.5) without low-pass filtering (σg=0\sigma_g=0) boosts semantic alignment (CLIP: 0.3140 →\to 0.3179, ImageReward: 0.3410 →\to 0.5034, PickScore: 22.104 →\to 22.234), but leaves high-frequency swap artifacts unfiltered, so distributional fidelity stays poor (5k FID = 35.84, +9.48 worse than Full SSA's 26.36; IS = 30.26 vs. 37.49; Precision = 0.4038 vs. 0.5612).

  2. Low-Pass Contrast + Attention Concentration (Rows 1b–1c):

    • Low-pass filtering alone (SSA-S, Row 1b) removes swap noise and cuts FID by −9.02-9.02 (36.09→27.0736.09 \to 27.07) while raising Precision (0.4068→0.56860.4068 \to 0.5686), but softens prompt alignment (CLIP: 0.3140→0.31180.3140 \to 0.3118, IR: 0.3410→0.24730.3410 \to 0.2473).
    • Combining both (Full SSA, Row 1c) reverses this semantic loss while keeping the distributional gains, cutting FID by −9.48-9.48 vs. Row 1a (35.84 →\to 26.36), raising IS by +7.23+7.23 (30.26 →\to 37.49) and Precision by +0.1574+0.1574 (0.4038 →\to 0.5612), and lifting ImageReward by +0.1517+0.1517 over SSA-S (0.2473→0.39900.2473 \to 0.3990, 95% CI [+0.1343,+0.1691][+0.1343, +0.1691]) and CLIP retrieval R@1 from 68.6%68.6\% to 73.8%73.8\%.
  3. Effect of DCR Alone (Row 1c →\to Row 2): Adding clean-latent high-pass recovery (λ=0.1,σu=2.0\lambda=0.1, \sigma_u=2.0 for p≥0.5p \ge 0.5) restores 5k Aesthetic Score by +0.1295+0.1295 (5.716→5.8455.716 \to 5.845), boosts Recall by +0.0570+0.0570 (0.5910→0.64800.5910 \to 0.6480), and raises ImageReward by +0.1078+0.1078 (0.3990→0.50680.3990 \to 0.5068, improving 68.44%68.44\% of all 5,000 prompt pairs).

W3: While TH (Row 2 →\to Row 3) improves cohort-mean metrics on COCO-2017 5k moderately (FID:26.94→26.67\text{FID}: 26.94 \to 26.67, CLIP:0.3186→0.3191\text{CLIP}: 0.3186 \to 0.3191, AES:5.845→5.860\text{AES}: 5.845 \to 5.860, ImageReward:0.5068→0.5149\text{ImageReward}: 0.5068 \to 0.5149), averaging over 5,000 prompts masks TH's primary role: preventing late-stage over-commitment and protecting worst-case failure tails. Paired distributional and process diagnostics across all 5,000 prompts demonstrate three statistically significant effects of TH:

  1. Statistically Significant Reduction in Worst-Case Failure Tail (p<0.01p < 0.01): Holding the baseline SSG 10th-percentile ImageReward failure threshold (q10=−1.1706q_{10} = -1.1706) fixed across all 5,000 prompts:
    • Backbone SSG has 500 severe failure images (10.00%10.00\%).
    • Low-pass SSA-S has 584 (11.68%11.68\%).
    • SSA + DCR (without TH) reduces this to 355 (7.10%7.10\%).
    • Full ARG (with TH) further reduces severe failures to 317 (6.34%6.34\%) — a paired tail-risk reduction of −0.76-0.76 percentage points over SSA+DCR whose 4,000-replicate bootstrap 95% CI is [−1.32%,−0.20%][-1.32\%, -0.20\%] (strictly excluding zero) and a 36.6%36.6\% relative reduction over SSG!
  2. Significant Gain in Own-Caption Margin & Retrieval R@1: On the 500-prompt independent CLIP ViT-B/32 retrieval benchmark, adding TH increases own-caption retrieval R@1 from 73.8%73.8\% (SSA+DCR) to 75.0%75.0\% (Full ARG) and significantly improves the own-caption cosine margin over the hardest distractor caption by +0.00121+0.00121 (95% CI [+0.00017,+0.00229][+0.00017, +0.00229], excluding zero).
  3. Late-Window Attention Entropy Restoration: Process instrumentation tracking cross-attention entropy on identical QK⊤QK^\top logits shows that without TH, late-stage cross-attention entropy remains artificially depressed by −0.2104-0.2104 (normalized by log⁡77\log 77) right up to step 50. TH smoothly restores this entropy gap to 0.0000 at the final step while cutting cumulative perturbation exposure by 15.71%15.71\%.

Q2: Both theoretical analysis of the Euler update (Appendix C.2, Proposition 1) and controlled 5,000-prompt evaluation (Appendix D.3) confirm that applying DCR across the mid-to-late denoising trajectory (p≥0.5p \ge 0.5, steps 25–49) is strictly superior to applying it only once at the final denoised latent (i=49i=49):

Table R1.2: Trajectory DCR (p≥0.5p \ge 0.5, Steps 25–49) vs. Final-Step-Only DCR (i=49i=49) on Full COCO-2017 5k (n=5,000n=5,000)

DCR Schedule Modeλ\lambdaActive StepsCLIP (5k) ↑\uparrowAES (5k) ↑\uparrowRecall (5k) ↑\uparrowImageReward (5k) ↑\uparrowAES Loss RecoveredIR Gain Achieved
No DCR (SSA + TH only)0.00.0None0.31625.7220.59380.40650.0%0.0\%0.0%0.0\%
Final-Step-Only DCR (i=49i=49)0.10.1Step 49 only0.31695.7470.62700.435818.1%18.1\%27.0%27.0\%
Trajectory DCR (Full ARG, Ours)0.10.1Steps 25–490.31915.8600.63900.5149100%100\% (ref.)100%100\% (ref.)

Why Trajectory DCR (p≥0.5p \ge 0.5) Outperforms Final-Step-Only DCR:

  1. Substituting DCR into the Euler update gives xi+1=xi+(σi+1−σi)ϵˉi+λ(pi)(1−σi+1/σi)Hx^0\mathbf{x} _ {i+1} = \mathbf{x} _ i + (\sigma _ {i+1} - \sigma_i)\bar{\boldsymbol{\epsilon}} _ i + \lambda(p_i)(1 - \sigma _ {i+1}/\sigma_i)H\widehat{\mathbf{x}} _ 0 (Eq. 9). When applied at steps i∈[25,48]i \in [25, 48], the detail-compensated latent xi+1\mathbf{x} _ {i+1} is fed back into the frozen UNet score network at step i+1i+1, which projects the boosted high-frequency features onto the natural image manifold and refines object boundaries (increasing independent CLIP similarity by +0.00226+0.00226, 95% CI [+0.00073,+0.00380][+0.00073, +0.00380], and 5k Recall to 0.63900.6390).
  2. With λ=0.1\lambda=0.1 applied only once at the final step (i=49i=49), a single injection recovers less than 1919% of the Aesthetic Score restoration (5.747 vs. 5.860) and only 2727% of the ImageReward gain (0.4358 vs. 0.5149) achieved by multi-step Trajectory DCR.

W2 & Q5: We swept all five hyperparameters (λ,τ0,σg,σu,pd\lambda,\tau_0,\sigma_g,\sigma_u,p_d) on 500 COCO-2017 prompts (Sec. 4.6, Fig. 6; App. D.4, Tables 11–12):

Table R1.3a (App. Table 11): λ\lambda Sensitivity & Over-Sharpening Regime

λ\lambdaFID500↓FID1k↓CLIP↑IS↑AES↑Prec↑Rec↑IR↑PS↑Visual Regime
0.0084.3354.400.315933.175.7100.6080.7280.324722.148Smooth (no recovery)
0.0584.2253.960.317932.895.7880.5940.7520.391022.181Mild recovery
0.10 (Ours)86.7154.810.319331.835.8550.5660.7260.415022.144Balanced (peak IR, natural)
0.1588.4755.490.320831.135.8830.5320.7200.409922.042Peak AES, over-contrast onset
0.2091.5658.480.323729.885.8180.5060.6920.387921.669Over-amplified (ringing/grain)

W2 & Q5:

Table R1.3b (Revised Manuscript Appendix Table 12): Sensitivity Sweeps for Temperature τ0\tau_0, Low-Pass Scale σg\sigma_g, DCR Smoothing Scale σu\sigma_u, and TH Onset pdp_d (500-Prompt Matched Benchmark)

Hyperparameter FamilySetting ValueFID (500) ↓\downarrowCLIP ↑\uparrowIS ↑\uparrowAES ↑\uparrowPrec. ↑\uparrowRec. ↑\uparrowImageReward ↑\uparrowPickScore ↑\uparrow
Cross-Attn Temp. τ0\tau_0τ0=0.3\tau_0 = 0.3 (Aggressive)84.650.318832.565.8590.5660.7740.460722.098
(Default: τ0=0.5\tau_0 = 0.5)τ0=0.5\tau_0 = 0.5 (Ours)84.630.317433.765.8640.5660.7820.461422.100
τ0=0.7\tau_0 = 0.7 (Mild)85.570.316231.485.8840.5700.7660.427522.105
τ0=1.0\tau_0 = 1.0 (Off / SSA-S)86.080.310331.775.7480.6020.7440.185922.034
Low-Pass Blur σg\sigma_gσg=0.0\sigma_g = 0.0 (No Low-Pass)92.530.316428.145.9390.5120.7140.455822.166
(Default: σg=3.0\sigma_g = 3.0)σg=1.5\sigma_g = 1.5 (Narrow Kernel)84.380.318133.145.9000.5440.8040.479922.158
σg=3.0\sigma_g = 3.0 (Ours)84.630.317433.765.8640.5660.7820.461422.100
σg=5.0\sigma_g = 5.0 (Over-Blurred)84.830.316930.755.8530.5680.7440.397622.010
DCR Scale σu\sigma_uσu=1.0\sigma_u = 1.0 (Fine Edges)84.370.316431.795.8690.6040.7320.445822.157
(Default: σu=2.0\sigma_u = 2.0)σu=2.0\sigma_u = 2.0 (Ours)84.630.317433.765.8640.5660.7820.461422.100
σu=3.0\sigma_u = 3.0 (Broad Detail)85.060.317933.105.8640.5740.7920.454722.103
TH Decay Onset pdp_dpd=0.50p_d = 0.50 (Early Decay)85.500.317931.155.8640.5800.7740.449722.103
(Default: pd=0.70p_d = 0.70)pd=0.70p_d = 0.70 (Ours)84.630.317433.765.8640.5660.7820.461422.100
pd=0.85p_d = 0.85 (Late Decay)84.220.317631.615.8570.5920.7820.455822.110
  • Across the entire interior neighborhood (τ0∈[0.3,0.7]\tau_0 \in [0.3, 0.7], σg∈[1.5,3.0]\sigma_g \in [1.5, 3.0], σu∈[1.0,3.0]\sigma_u \in [1.0, 3.0], pd∈[0.5,0.85]p_d \in [0.5, 0.85]), ARG remains remarkably stable (FID ~ 84.22–85.57, ImageReward ~ 0.428–0.480, PickScore ~ 22.10–22.16), with our default configuration (τ0=0.5,σg=3.0,σu=2.0,pd=0.70\tau_0=0.5, \sigma_g=3.0, \sigma_u=2.0, p_d=0.70) achieving the highest Inception Score (33.76) across all four sweeps and the highest ImageReward (0.4614) across τ0\tau_0, σu\sigma_u, and pdp_d.
  • As shown in Table R1.3a and visualized with 2×2\times detail zoom-ins in Appendix D.9, Figure 17, pushing λ\lambda to 0.15–0.200.15\text{–}0.20 introduces high-frequency edge ringing around high-contrast boundaries, causing 1k FID to rise (54.81 →\to 58.48), AES to drop after peaking at λ=0.15\lambda=0.15 (5.883 →\to 5.818), and PickScore to drop (22.144 →\to 21.669). At our default λ=0.10\lambda=0.10, high-frequency PSD share is restored from 3.20%3.20\% (SSA) to 7.44%7.44\% (matching SSG's 7.44%7.44\%) while keeping Gradient RMS (0.09610.0961 vs. 0.10960.1096) and Laplacian RMS (0.22230.2223 vs. 0.24360.2436) strictly lower than SSG (Appendix D.1, Table 7).

Q3 & Presentation: In the revised manuscript and Appendix:

  1. Appendix D.1, Figure 8 plots the discrete 50-step temporal activity map across the entire trajectory (25 alignment-only steps, 10 full-overlap steps, and 15 annealing steps), while Appendix D.7, Figure 12 and Appendix D.9, Figure 16 show same-seed visual outputs across all six decoupled stages (Backbone SSG, Low-Pass Only [SSA-S], Attn. Concentration Only, Full SSA, SSA + DCR [no TH], and Full ARG), alongside multi-method 2×2\times zoom-ins (Figure 15), λ\lambda-sweeps (Figure 17), and cross-backbone comparisons (Figure 18).
  2. We have completely redrawn Figure 1 and unified all mathematical notation (σg,τ(p),s(p),λ(p),σu\sigma_g, \tau(p), s(p), \lambda(p), \sigma_u) across Sections 3.1–3.4 and Appendix C so every signal path and schedule curve is immediately clear at a glance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.