acceptodds
Withdrawn from ICLR 2027

Selective Low-Frequency Recovery for Diffusion Guidance

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Classifier-free guidance (CFG) improves conditional alignment in diffusion models, but high guidance scales can cause oversaturation, artifacts, and reduced sample diversity. Frequency-decoupled guidance (FDG) mitigates these effects by weakening low-frequency guidance while retaining stronger high-frequency guidance. However, uniformly suppressing the low-frequency residual can also attenuate information important for global structure and conditional consistency. Building on frequency-decoupled and projected guidance, we propose selective recovery guidance (SRG), a training-free method that restores low-frequency guidance according to its direction and denoising stage. SRG decomposes the low-frequency guidance residual into components parallel and orthogonal to the conditional low-frequency prediction. The method restores only the orthogonal component to a moderate guidance level while keeping the parallel component unchanged. This design preserves the projection of the conditional low-frequency prediction. A normalized log-SNR window confines the restoration to intermediate denoising stages, where structural predictions are reliable and global layout remains adjustable. The proposed method maintains original high-frequency guidance and requires no additional denoiser evaluations. Across five pretrained models, SRG improves conditional fidelity, prompt alignment and learned preference scores for text-to-image generation. On ImageNet and COCO 2017, it maintains FID, Precision and Recall comparable to FDG, preserving distributional quality and sample diversity. Ablations support the use of directional and temporal constraints with a moderate recovery strength. Selective low-frequency recovery improves the trade-off between conditional alignment and generation quality. Code is available at https://anonymous.4open.science/r/SRG-6E6D.

Primary area:
generative models
Keywords:
diffusion models, classifier-free guidance, frequency-domain guidance, low-frequency restoration, text-to-image generation

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_13AfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper discusses the trade-off in classifier-free guidance (CFG) between conditional alignment and sample quality/diversity, focusing on frequency-decoupled guidance (FDG). FDG suppresses low-frequency guidance to reduce oversaturation, but may discard useful structural information. The authors propose selective recovery guidance (SRG), a training-free extension of FDG that decomposes the low-frequency guidance residual into components parallel and orthogonal to the conditional low-frequency prediction, and restores only the orthogonal component (Eq. (7), Eq. (9)), gated by a normalized log-SNR window (Eq. (8)). High-frequency guidance is unchanged, and no additional denoiser forward passes are required.

Critical strengths
  1. Training-free and low computational overhead. SRG reuses the conditional and unconditional predictions already computed by CFG and requires no additional denoiser forward passes. This matters because the method is easy to adopt in practice and carries low deployment risk.
  2. Broad empirical evaluation across backbones and tasks. The paper evaluates five pretrained models, spanning class-conditional generation (EDM2-S, DiT-XL/2) and text-to-image generation (SD2.1, SDXL, SD3). This breadth supports the generality of the claimed improvements.
Critical weaknesses
  1. On COCO 2017, SRG ties FDG exactly or is slightly worse in FID (SD2.1/SDXL 21.96/25.68; SD3 28.31 vs. 28.10), and several Precision/Recall differences are small with no variance or significance reported; meanwhile, the SD3 DrawBench ImageReward gain from 0.695 to 0.931 is highlighted. Table 2 also contains near-zero improvements such as Parti Prompts SDXL ImageReward 0.696 to 0.697. Because an accept recommendation depends heavily on whether gains are stable rather than driven by isolated highlights, the authors need to support this with confidence intervals, multiple seeds, and per-prompt analyses across benchmarks. Otherwise, I would worry that the conclusion is setting-specific.
  2. The window 0.15<u_t<0.70 and η=2/3 are used uniformly across all five models, but the temporal and strength ablations are conducted mainly on SD2.1; the appendix also acknowledges that the same log-SNR interval corresponds to different numbers of effective steps across models/samplers. At the same time, the closest competing alternative FDG+APG is not evaluated, and the SD3 APG configuration is borrowed from SDXL and is not SD3-specific, which may disadvantage the baseline. For a method claimed to be training-free and general across models, this directly reduces my confidence in transferability and in the fairness of the relative comparison.
  3. Eq. (5) writes the recovery coefficient as A_t = A g(u_t), but the inner-product expressions mix A_t and A without clarifying that the identity holds for the product; the notation is slightly inconsistent between Eq. (5) and Eq. (9) (Sec. 4.1 vs. Sec. 4.2).
  4. Eq. (7) defines α_t with a denominator guard (‖C_L,t‖² > 0), but the orthogonal component ΔD^{low,⊥}_t = ΔD^{low}_t − ΔD^{low,∥}_t is stated in prose rather than as a numbered equation (Sec. 4.2), making the construction harder to verify quickly.
  5. Figures 1 and 2 only qualitatively illustrate oversaturation, but the paper provides no quantitative saturation or artifact metric to support the claim that orthogonal recovery alleviates oversaturation.
  6. The paper does not report negative results or cases where SRG underperforms FDG, which limits understanding of when the method should be preferred.
  7. The frequency decomposition (5×5 Gaussian filter, σ=1, factor-two downsampling, bilinear reconstruction) is specified only in Appendix B.3; there is no ablation on filter size, σ, or single-level vs. multi-level decomposition, so sensitivity to this design choice is unknown.
Questions for authors
  1. Please provide multi-seed variance or confidence intervals for key cells in Table 1 and Table 2, and extend the paired bootstrap analysis in Figure 5 to additional model-benchmark pairs such as SDXL DrawBench and SD3 Parti Prompts. Is the SD3 DrawBench improvement from 0.695 to 0.931 stable across all seeds and prompt slices?
  2. How were the window 0.15<u_t<0.70 and η=2/3 chosen, and were they validated even coarsely on SDXL, DiT, or SD3? Please report the number of effective denoiser steps inside the window for each model and sampler, so that the cross-model operational meaning of the interval is clear.
  3. Please unify the notation A_t and A·g(u_t) in Eq. (5) and Eq. (9), and present ΔD^{low,⊥} as a numbered equation. Also clarify the relationship between the stated bound 0≤A≤w_high−w_low and the η=4/3 ablation that exceeds it, and whether w_low+A·g(u_t)≤w_high holds under the default constraint.
  4. Was the SD3 APG baseline using the SDXL configuration tuned at all for SD3? If not, why not evaluate FDG+APG as a more direct selective-recovery baseline, so that APG is not underestimated due to a non-optimal configuration?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 13Af

Dear Reviewer 13Af,

Thank you for the detailed feedback. We have expanded the robustness analyses and direct comparisons and unified the mathematical notation.

W1 & Q1: Repeated sampling and uncertainty

We added SD3 Medium evaluations with three seeds per prompt on DrawBench, Parti Prompts, and HPS Prompts, including the original run (Appendix C.1/Tables 8–9). SRG achieves higher mean scores than CFG, APG, and FDG on all four metrics across all three benchmarks. Relative to FDG, mean ImageReward increases from 0.736 to 0.957, 0.982 to 1.157, and 0.990 to 1.115, respectively. These repeated evaluations support consistent improvements in alignment and learned preferences under different sampling noise.

Table 8 reports mean ±\pm standard deviation across the three runs, showing how much the scores vary with sampling randomness. Table 9 reports 95% confidence intervals for each method's average score, estimated by bootstrap resampling prompts. The calculation details are provided in Appendix C.1.

W2 & Q2: Window and strength transfer

The default window and η=2/3\eta=2/3 follow the timing/strength analysis on SD2.1 (Table 3). We added corresponding SD3 ablations in Appendix C.2/Table 10. Full-trajectory recovery has higher ImageReward (0.964 versus 0.931), while SRG's window yields lower FID (28.31 versus 29.06) and higher Recall (0.398 versus 0.381). Increasing η\eta to 11 or 4/34/3 worsens FID/Recall and lowers ImageReward/PickScore relative to the default.

The recovery window is defined by normalized log-SNR, which describes the noise level during denoising. For each sampling schedule, we rescale its sampled log-SNR values to [0,1][0,1] and activate recovery when 0.15<ut<0.700.15<u_t<0.70 (Appendix A.2). The window therefore selects evaluations according to noise level; different sampling schedules can yield different active step indices and counts. The added SD3 ablations support the effectiveness of this shared window rule and moderate recovery strength beyond the original SD2.1 setting.

W2 & Q4: FDG+APG and APG configuration

We added FDG+APG to the revised main Tables 1–2, covering all five distributional model settings and all nine text-to-image model/benchmark pairs. SRG improves prompt alignment and learned preference scores over FDG+APG while retaining competitive distributional quality, supporting the benefit of selective recovery beyond combining frequency decoupling with directional filtering. Selected core results are also provided in our response to Reviewer nnEh, W1 & Q1. For SD3, APG uses the transferred SDXL parameters documented in Appendix B.4. We also evaluated APG using the SD2.1 configuration reported in the APG paper (w=10w=10) under shared DDIM20 sampling conditions. SRG achieves higher scores on all four metrics across three benchmarks; please see Appendix D.1/Table 12 for the complete results.

W3–W4 & Q3: Unified notation and coefficient bound

We have unified the notation and component definitions in the revised equations. AA denotes the recovery amplitude, while At=Ag(ut)A_t=A g(u_t) is its gated value. For 0≤η≤10\le\eta\le1, the orthogonal low-frequency coefficient satisfies wL≤wL+At≤wHw_L\le w_L+A_t\le w_H. This constrains the coefficient, not the update norm. The η=4/3\eta=4/3 ablation deliberately tests stronger recovery beyond this range.

W5–W6: Interpretation of quality and non-improving cases

Figures 1–2 provide qualitative comparisons of saturation and artifacts, while Tables 1–2 evaluate distributional quality, alignment, and learned preferences. The fixed-state geometric property describes the added update and does not by itself quantify saturation or artifact frequency.

Our conclusion concerns improved alignment/preferences with competitive distributional quality. Table 1 also reports trade-offs: EDM2-S FID changes from 7.51 to 7.61 and Recall from 0.662 to 0.651; SD3/COCO2017 FID changes from 28.10 to 28.31. The SDXL/Parti Prompts ImageReward gain is small (0.696→0.6970.696\to0.697, Table 2). Figure 6 in the revised main paper further examines the quality–alignment trade-off across guidance settings, showing generally higher SRG scores at comparable FID in the tested range (Appendix D.2).

W7: Frequency-decomposition sensitivity

Appendix C.3/Table 11 adds SD2.1/DrawBench comparisons using 3×33\times3, 5×55\times5, and 7×77\times7 kernels, Gaussian σ∈{1,1.5}\sigma\in\{1,1.5\}, and one/two decomposition levels. SRG improves all four metrics over the corresponding FDG baseline across the tested operators. These results support recovery gains beyond the default frequency decomposition.

AuthorsOpenReview ↗

Response to Reviewer 13Af

Dear Reviewer 13Af,

Thank you again for your careful review and helpful comments. We have addressed your concerns in our responses and supplemented the manuscript with additional comparisons and robustness analyses.

When you have a chance, we would be grateful to hear your thoughts on these updates. We hope there will be an opportunity to discuss any remaining questions and would be happy to provide further clarification.

Best regards,

The Authors

Reviewer_ATyqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a low-frequency correction to classifier free guidance. The velocity field is split into high and low frequency bands. Then, for the low band, only the commponent that is orthogonal to the conditional velocity is considered. This means that structural components can be boosted by increasing the low frequency weight, without adding artifacts known we too much guidance, like duplicate objects. Experiments cover class-conditional and text-to-image generation.

Critical strengths

The main idea of the paper, that the low frequencies drive the structure of the generation process and that guidance in that band can overshoot the condition creating artifacts is really interesting. The proposed solution is very simple to implement which is a strength.

Critical weaknesses

I would like to get more insights on what is happening. For example, visualizations of the denoising process (ie, generated images at several timesteps) that compare the effect ot the parallel, orthogonal and full component would help understand what each component is encoding in the image generation process. I was also expecting more through ablation on the frequency decomposition, to see how it can isolate structure from the rest.

The experiments on imagenet are outdated. Recent models can do around 2 of FID without guidance. I don't think the scores in the paper are meaningful. For T2I, I think the scores are also very far from the state of the art. on PartiPrompt, SD3-medium should be around 1.15 for ImageReward and 0.3 for HPSv2, much better than the reported scores. I think the good benchmark for the method is to provide a trade-off between image quality (like FID or ImageReward) and complex prompt following (like geneval or other similar benchmarks). I assume you could get a higher ImageReward score by using a higher CFG without destroying geneval scores with duplicate objects or other artifacts. If the method is on the pareto line, that's a win.

Questions for authors

I was surprised that it's the orthogonal component of the LF that has to be boosted by guidance. I would have guessed the opposite: the LF contains the structure, so the concept (eg, a cat) and thus that's what you should boost if you want more condition following. I can see (thanks to the paper!) how this could "overshoot" the condition mode and create artifacts (such as duplicating objects). But conversely, what does that mean for the HF part? What does it means to follow the parallel direction in the HF? I think the paper needs a better story and it needs to be accompanied with convincing ablations.

Similarly, the experimental setup is not on par with the current state of the art. Why are the scores so bad compared to what is common in the literature? Since the method is not compute intensive, I would expect better comparisons. I have a positive opinion of the paper, but I may downgrade my score depending on where the paper is headed to.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ATyq

Dear Reviewer ATyq,

Thank you for the suggestions on interpretation, process visualizations, and evaluation. We have added evidence addressing both the recovery mechanism and compositional prompt following.

Q: Low-frequency recovery and high-frequency interpretation

Frequency content and geometric direction describe different properties: a low-frequency residual can contain both parallel and orthogonal components relative to the conditional prediction. Orthogonality therefore does not imply an absence of structural information.

SRG retains FDG's parallel coefficient and high-frequency update and restores only the attenuated orthogonal low-frequency component. At a fixed state, the added correction has zero inner product with the conditional low-frequency prediction. This preserves that projection, not the total norm or subsequent trajectory (Sec. 4.1; Appendix A.1). The aim is to restore useful coarse corrections without uniformly strengthening the suppressed band. In the high-frequency band, a parallel component is geometrically aligned with the conditional high-frequency prediction; this alone does not determine its perceptual role. SRG leaves FDG's high-frequency update unchanged, focusing recovery on the band deliberately attenuated by FDG.

The added SD3 direction ablation fixes recovery timing and strength within each dataset. SRG matches Full LF's DrawBench CLIPScore (28.60), improves ImageReward (0.856→0.931), and gives lower COCO2017 FID (28.84→28.31) and higher Recall (0.388→0.398). Parallel-only recovery gives limited alignment gains. These results support the directional choice; please see Appendix C.2/Table 10.

W1: Denoising trajectories and frequency decomposition

Figure 8/Appendix A.4 compares FDG, Full LF, Parallel-only, and SRG at ten intermediate steps and the final output on SD3 Medium. Methods share initial noise, base scales, and sampling conditions; recovery variants share amplitude and window. Panels show guided clean-sample predictions before each scheduler update.

For the displayed donkey/octopus/rope prompt, Full LF and SRG place the cat above the rope, whereas FDG and Parallel-only keep it closer to the ground. SRG achieves this layout correction using only orthogonal recovery, illustrating its effects on global composition, spatial relationships, and object interactions in this example.

Appendix C.3/Table 11 adds SD2.1/DrawBench comparisons using 3×3/5×5/7×7 kernels, Gaussian sigma=1/1.5, and one/two decomposition levels. Recovery retains positive gains over the corresponding FDG baseline on all four metrics across the tested operators, supporting improvements beyond the default decomposition.

W2 & W3: Evaluation scope and reported scores

Our experiments assess guidance changes within five fixed pretrained backbones spanning mainstream U-Net and Transformer architectures. On ImageNet, CFG→SRG FID changes are 11.67→7.61 for EDM2-S and 12.58→9.62 for DiT-XL/2 (Table 1). These controlled comparisons assess the guidance rule under shared sampling protocols.

For SD3/Parti Prompts, Table 2 reports SRG ImageReward 1.154 and HPSv2 0.298, consistent with the approximate 1.15/0.3 values you cite. ImageReward 0.931 belongs to DrawBench. Thus, the reported Parti Prompts results already reach the scores highlighted in your comment.

W4: Quality–alignment trade-offs and compositional evaluation

We added an SD2.1 Base guidance sweep on 5,000 fixed COCO2017 captions. CFG spans w=3–15, while FDG and SRG share base band configurations. For each method and setting, FID and all four alignment/preference metrics are evaluated on the same generated images. Figure 6 in the revised main paper shows generally higher SRG scores at comparable FID over the sampled range. Increasing CFG does not consistently improve every preference metric; Appendix D.2 provides the protocol.

We additionally evaluated GenEval on SD2.1 using the Table 2 guidance settings and report the results here. All entries are percentages; Overall is the unweighted mean of the six task scores.

MethodOverallSingle obj.Two obj.CountingColorPositionColor attr.
CFG-340.94793.43839.14133.43869.4154.2506.000
FDG42.22395.00039.14135.93871.0114.7507.500
SRG44.69398.43840.65737.50076.0645.50010.000

SRG outperforms CFG and FDG in all six GenEval categories, improving Overall by 2.470 percentage points over FDG. Counting, spatial position, and color attribution increase from 35.938%, 4.750%, and 7.500% to 37.500%, 5.500%, and 10.000%, respectively, providing quantitative evidence of improved compositional prompt following. GenEval uses the Table 2 guidance settings, complementing Figure 6's quality–alignment analysis across guidance settings.

AuthorsOpenReview ↗

Response to Reviewer ATyq

Dear Reviewer ATyq,

Thank you again for your thoughtful suggestions. We have provided detailed responses, together with additional experiments and visualizations addressing the issues you raised.

We would appreciate hearing your thoughts on these updates as soon as your schedule allows. Please let us know if any points would benefit from further clarification; we would be happy to discuss them.

Best regards,

The Authors

Thanks for the answers. I'll discuss this with the other reviewers and the AC and then modify my score accordingly.

Reviewer_5uZ2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the issue that frequency decoupling guidance (FDG) may reduce useful low-frequency information, including problems such as the balance between the intensity of prompts and color artifacts in image generation. The authors propose a method that, during the denoising process, appropriately restores information that has been suppressed, such as parts of layout, while striving to maintain the quality of the image. The authors evaluated on five pre-trained models and reported an improvement in prompt alignment metrics and a distribution quality that is generally close to that of FDG.

Critical strengths
  1. The method is simple and does not need training. It can be easily integrated into existing models.
  2. The experimental results are consistent. The three models outperformed the baseline in all four indicators on the three prompt word datasets.
Critical weaknesses
  1. Tables 1 and 2 contain only individual values, with no standard deviations or confidence intervals. The impact of different seeds on image generation is not shown. No uncertainty analysis was performed on the main results.
  2. The authors claim that “SRG improves text-to-image alignment and learned preference scores while maintaining FID, Precision, and Recall generally comparable to FDG.” . However, Table 6 in the appendix shows that the authors used different guidance strengths across different evaluations. For example, SD2.1, SDXL, and SD3 also differ, and the authors did not report numerical results under identical parameters. 3.Table 3 of the paper presents ablation experiments on SD2.1 for time windows, restoration intensity, and restoration direction; however, there are no corresponding ablation experiments for SDXL or SD3, nor is the generalizability of the ablation experiments demonstrated. 4.The paper’s text-to-image experiments primarily cover SD2.1, SDXL, and SD3; the other two are category-conditional models. The lack of testing on the latest models makes it difficult to demonstrate that the method can be reliably generalized to a wider range of state-of-the-art text-to-image models. 5.Figure 6 does not specify which model was used, which dataset was employed, or whether the results were averaged. Please provide a detailed explanation of how the curves were aggregated.
Questions for authors
  1. Regarding the innovation aspect, could the authors explain which concepts from existing methods—such as the FDG APG Guidance Interval (GI)—they have adopted, and what unique and insightful innovations they have introduced? If the authors can provide a reasonable explanation, a higher score may be considered.
  2. The paper argues that restoring low-frequency information can reduce excessive color saturation and image artifacts. Can the authors provide quantitative evidence demonstrating these issues of excessive color saturation and artifacts?
  3. Could the authors provide standard deviations or confidence intervals for the main results?
  4. Could the authors report experimental results under the same guidance parameters and compare them with existing methods?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 5uZ2

Dear Reviewer 5uZ2,

Thank you for the constructive comments. We have expanded the direct comparisons and robustness analyses.

Q1 & Q2: Contribution and motivation

SRG addresses a limitation of uniform low-frequency suppression: it can weaken useful coarse structural corrections along with undesirable amplification. Our contribution is a selective recovery policy that specifies which attenuated corrections to restore and when to restore them.

Compared with FDG. FDG uniformly attenuates low-frequency guidance. SRG selectively recovers its orthogonal component while retaining FDG's parallel coefficient and high-frequency update, allowing additional correction without strengthening the entire band.

Compared with APG and FDG+APG. APG downweights the guidance component parallel to the conditional prediction to reduce oversaturation. FDG+APG adds this geometric suppression to FDG's frequency attenuation, without restoring the weakened orthogonal low-frequency component. SRG instead increases this orthogonal low-frequency coefficient during selected stages while retaining FDG's parallel coefficient.

Compared with GI. GI restricts full guidance according to noise level. SRG gates only the additional orthogonal low-frequency correction, while base FDG guidance remains active throughout sampling. Its window is motivated by the different temporal roles of coarse structure and fine detail, targeting an early-to-intermediate interval for coarse structural correction.

Direct comparisons with CFG, FDG, APG, and the added FDG+APG baseline support improved prompt alignment and learned preference scores. Please see our response to Reviewer nnEh for selected comparisons, and Sec. 5/Tables 1–2 and Appendix D.1/Table 12 for the complete results. At a fixed state, the added correction preserves the projection onto the conditional low-frequency prediction (Appendix A.1). Our saturation/artifact evidence remains qualitative (Figures 1–2).

W1 & Q3: Multiple random seeds and uncertainty

We added SD3 Medium evaluations with three seeds per prompt on all three prompt benchmarks, including the original run. SRG retains higher means than CFG, FDG, and APG on all four metrics, supporting consistent alignment and preference gains under repeated sampling.

Appendix C.1/Table 8 reports all four metrics as mean±sample SD (ddof=1); Table 9 provides individual-method 95% CIs from 10,000 prompt-bootstrap resamples after seed averaging. These complement Figure 5's paired difference intervals. The new uncertainty estimates concern SD3 prompt-conditioned results.

W2 & Q4: Guidance settings and joint evaluation

The main tables follow FDG's task-specific settings, documented in Appendix B.4/Table 6. Within each comparison, FDG and SRG share base band coefficients and sampling conditions; the added recovery term is the intervention. A scalar CFG coefficient and frequency-dependent coefficients do not define identical updates.

We therefore added a guidance sweep on SD2.1 Base using 5,000 fixed COCO2017 captions, varying CFG strength and matched FDG/SRG base band coefficients. For each method and setting, FID and all four alignment/preference metrics are evaluated on the same generated images. Figure 6 in the revised main paper shows generally higher SRG scores at comparable FID, supporting improved quality–alignment trade-offs across settings. Appendix D.2 provides the protocol.

W3 & W4: Ablation transfer and architecture coverage

Our evaluation spans five backbones with mainstream U-Net and Transformer architectures, covering text- and class-conditional generation. We added SD3 timing, strength, and direction ablations on DrawBench and COCO2017 to examine the design beyond SD2.1 (Appendix C.2/Table 10).

Orthogonal recovery improves over parallel-only recovery. Full-trajectory recovery has higher ImageReward (0.964 versus 0.931), while SRG's window gives lower FID (28.31 versus 29.06) and higher Recall (0.398 versus 0.381). Stronger recovery worsens this balance, supporting moderate, stage-selective recovery on the tested architectures. Figure 8/Appendix A.4 illustrates its effects on global composition and object interactions. Appendix C.3/Table 11 further reports positive gains over FDG across the tested frequency-decomposition configurations.

W5: Original Figure 6, now Figure 7

The figure uses SD2.1 Base on all 200 DrawBench prompts with four seeds per prompt: 800 standard CFG trajectories at w=3. At each node, we compute each trajectory's low/high L2 norm shares from the unscaled conditional residual, average the ratios over seeds within each prompt, then equally over prompts. These are averages of ratios, not ratios of averaged norms. The caption and Appendix A.3 document the setup and aggregation.

AuthorsOpenReview ↗

Response to Reviewer 5uZ2

Dear Reviewer 5uZ2,

Thank you again for your time and constructive feedback. We have responded to your comments and added experiments and analyses addressing the main concerns.

We would be grateful if you could review our responses at your earliest convenience and let us know whether any concerns remain. We would welcome the opportunity to clarify any outstanding points and continue the discussion.

Best regards, The Authors

Reviewer_nnEhOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper works on the alignment-versus-quality trade-off in classifier-free guidance. FDG mitigates oversaturation at high guidance scales by applying a weaker scale to the low-frequency part of the guidance residual, but it suppresses that band uniformly and thereby discards low-frequency information that carries global structure and layout. SRG, the proposed method, decomposes the attenuated low-frequency residual into components parallel and orthogonal to the conditional low-frequency prediction and adds back only the orthogonal component, within a fixed normalized log-SNR window (0.15 < u < 0.70) and at strength bounded by the high-low scale gap. The method is training-free and needs no additional denoiser evaluations. Across five pretrained models, SRG improves ImageReward / HPSv2 / PickScore / CLIP over FDG in every model-benchmark combination (e.g. SD3 on DrawBench, ImageReward 0.695 → 0.931), while FID, precision and recall remain close to FDG's.

Critical strengths
  1. The geometric idea is clean, and the paper tests it honestly. Eq. (5) correctly shows that an orthogonal correction leaves the projection onto the conditional low-frequency prediction unchanged, and the direction ablation (Table 3c) is exactly the right experiment: full and parallel-only recovery degrade FID and recall, while orthogonal recovery keeps FID at FDG's level with the alignment gains intact. This is the part of the paper I find solid.

  2. The method is cheap to adopt — no retraining, no extra forward passes, one fixed setting (η = 2/3, same window) across five rather different backbones — and the evaluation includes a per-prompt paired analysis with bootstrap CIs (Fig. 5), which is more than most guidance papers offer. I also checked Table 6 against A = η(w_H − w_L) for every setting; it is consistent.

Critical weaknesses
  1. My main concern: the contribution is a combination of existing components, and the experiment that would justify this particular combination is missing. The parallel/orthogonal decomposition is from APG; FDG's own appendix already combines FDG with APG's projection; restricting guidance to a noise-level interval is the Guidance Interval idea. What is specific to this paper is the claim that applying the orthogonal correction inside the low-frequency band recovers information that FDG+APG still suppresses (argued in Sec. 2, p. 3). That claim is directly testable, and the natural test — a comparison against FDG+APG — is absent. As it stands, the increment over the existing combination is argued, not demonstrated.

  2. The empirical case that is presented has a weak link: the APG baseline looks misconfigured. On several text-to-image setups APG scores far below even vanilla CFG (DrawBench/SD2.1 ImageReward −0.832 vs CFG's −0.174; Parti/SD2.1 −0.810 vs 0.029), while APG reports gains over CFG in its own paper; Appendix B.4 also states the SD3 settings are borrowed from SDXL. So both the "SRG beats APG" rows and the framing that APG-style projection is not enough rest on a baseline that appears off.

  3. The headline trade-off claim is assembled from two different operating points: for SD2.1, the COCO runs use (w_L, w_H) = (3, 7) and the preference benchmarks (3, 12). "FID comparable, alignment better" would require comparing methods at matched FID — a single alignment-vs-FID curve — which the paper does not provide.

Questions for authors
  1. Please compare against FDG+APG as described in the FDG appendix, under your evaluation protocol.
  2. Did you use APG's recommended per-model settings? How does the ordering change once APG is configured properly?
  3. Can you provide an alignment-vs-FID comparison at matched FID across CFG / APG / FDG / SRG?
3 comments
AuthorsOpenReview ↗

Response to Reviewer nnEh

Dear Reviewer nnEh,

Thank you for recognizing the geometric argument and identifying the missing direct comparisons. We address each concern in detail below.

W1 & Q1: Contribution and direct comparison with FDG+APG

Recovering useful corrections within a suppressed band. FDG lowers low-frequency guidance to limit oversaturation and diversity loss, but applying a single scale to the entire band also weakens potentially useful coarse structural corrections. When configured to suppress parallel updates, FDG+APG further reduces the parallel component while keeping the orthogonal component at FDG’s conservative base scale. The orthogonal corrections attenuated by FDG therefore remain weak. SRG recovers these corrections during selected denoising stages, allowing stronger low-frequency updates without increasing guidance in every direction. Our contribution is this constrained recovery policy, rather than the projection operator itself. Motivated by the different temporal roles of low- and high-frequency guidance, recovery is confined to an early-to-intermediate interval for coarse structural correction. Unlike Guidance Interval (GI), which gates full guidance, our window controls only the additional low-frequency correction. The base FDG update remains active throughout sampling.

Direct comparison with FDG+APG. We have added FDG+APG as an additional baseline. Selected results are shown below; Tables 1 and 2 in the revised main paper provide the full comparison.

BenchmarkModelMethodImageReward↑HPSv2↑PickScore↑CLIPScore↑
DrawBenchSD2.1FDG+APG-0.1550.23920.8225.83
SRG0.0350.25421.0526.37
SDXLFDG+APG0.4730.24922.1127.83
SRG0.6710.26722.3728.34
SD3FDG+APG0.7140.27622.2328.22
SRG0.9310.29122.4128.60
DatasetModelMethodFID↓Precision↑Recall↑
COCO2017SD2.1FDG+APG22.200.6480.521
SRG21.960.6670.511

These results show that SRG improves prompt alignment and learned preference scores relative to FDG+APG. The ImageReward improvement is most pronounced on SD3/DrawBench, where the score increases from 0.714 to 0.931. On SD2.1/COCO2017, SRG maintains distributional quality comparable to FDG+APG. We kindly invite you to consult Tables 1 and 2 in the revised main paper for the complete results.

W2 & Q2: APG baseline verification and supplementary evaluation

We would first like to clarify the scope of APG's reported improvements: better FID and Recall with comparable Precision. Its Table 8 reports the same CLIP-Score (0.31) for CFG and APG, and it does not compare ImageReward, PickScore, or HPS. These findings therefore do not establish that APG outperforms CFG on the preference metrics used here. Our original SD2.1 comparison follows FDG's sampling protocol, using shared checkpoints and initial noise with CFG/APG at w=3. APG's implementation and projection, norm threshold, and momentum settings follow its original appendix.

To complement this protocol, we additionally evaluated APG using the SD2.1 configuration reported in the APG paper (w=10) with DDIM20, with SRG evaluated under the same sampling conditions. SRG obtains higher alignment and preference scores across all three benchmarks. The largest ImageReward improvement is on Parti Prompts, from 0.200 to 0.375.

BenchmarkModelMethodImageReward↑HPSv2↑PickScore↑CLIPScore↑
DrawBenchSD2.1APG-100.0360.24721.2625.90
SRG0.1340.26021.3026.69
PartiPromptsSD2.1APG-100.2000.25521.3626.70
SRG0.3750.26521.3827.06
HPSPromptsSD2.1APG-100.1770.24921.0228.51
SRG0.3190.25921.1028.96

W3 & Q3: Distribution–alignment trade-offs

To compare distributional quality and conditional alignment jointly, we added an SD2.1 Base guidance sweep on 5,000 fixed COCO2017 captions. CFG spans w=3–15, while FDG and SRG share six base frequency-band configurations. Initial noise is paired across methods. For each method and setting, FID and all four alignment/preference metrics are evaluated on the same generated images. Figure 6 in the revised main paper shows that SRG generally achieves higher prompt-alignment and learned preference scores than CFG and FDG at comparable FID. This supports a favorable quality–alignment trade-off across guidance settings, extending the evidence beyond the individual operating points in Tables 1–2. Detailed settings are provided in Appendix D.2.

Thanks for the authors rebuttal, adding new experiements and explainations, here's my comments for considerations.

  1. APG's paper reports no preference-metric results, you are right on this point, and I withdraw that part of my concerns.
  2. what's specific to this paper is a recovery policy: a fixed window and a fixed strength, neither derived quantitatively. SRG leads FDG in all 36 cells of table2, but narrowly. Whether a tuning-level policy on top of FDG with this gain profile constitutes a sufficient contribution is the part of W1 that new experiments cannot fully address
AuthorsOpenReview ↗

Response to Reviewer nnEh

Dear Reviewer nnEh,

Thank you for revisiting the additional results and clarifying the APG point. We understand that your remaining concern is whether the recovery policy offers a substantive contribution beyond parameter tuning, and whether the gains justify it.

1. Significance of frequency- and stage-selective recovery

Frequency-resolved temporal analysis of CFG residuals and its use for selective recovery in image generation remain relatively underexplored. Yet this temporal behavior matters: a structural correction must be applied when it can influence composition. Uniform low-frequency attenuation can weaken useful global-structure corrections, while coarse structural correction and high-frequency refinement have different temporal requirements.

SRG connects these observations through a specific intervention: it restores the orthogonal low-frequency component during selected stages while retaining FDG's parallel coefficient and high-frequency update. Simply increasing FDG's low-frequency scale strengthens both parallel and orthogonal components. FDG+APG regulates directions without explicitly introducing this recovery term; GI gates full guidance, whereas SRG gates only the additional low-frequency correction.

The research value is a guidance design principle that jointly considers frequency, direction, and denoising stage, providing finer control than changing overall guidance strength. The temporal analysis and controlled ablations examine which suppressed corrections to recover and when they are useful (Figure 7; Tables 3 and 10). As you recognized, the direction ablation tests the mechanism with timing and strength fixed. These analyses provide a mechanism-level contribution alongside the benchmark gains.

2. Parameter rationale and cross-model validation

An optimal recovery window depends on the model's denoising dynamics, sampling schedule, and quality–alignment objective. The fixed-state geometric identity constrains the correction direction but does not describe its accumulated effects along the sampling trajectory. It therefore cannot, by itself, determine a universally optimal window across models.

We combine the frequency-temporal analysis with ablations to select the default and evaluate the same normalized window, 0.15<ut<0.700.15<u_t<0.70, and η=2/3\eta=2/3 across five backbones. The amplitude follows A=η(wH−wL)A=\eta(w_H-w_L), adapting to each model's existing FDG scales. For 0≤η≤10\le\eta\le1, the effective orthogonal coefficient stays between wLw_L and wHw_H.

The added SD3 ablations support this shared configuration: full-trajectory recovery obtains higher ImageReward (0.964 versus 0.931), but worse FID (29.06 versus 28.31) and Recall (0.381 versus 0.398). Stronger recovery also worsens this balance (Appendix C.2/Table 10). Thus, the default is supported by an explicit coefficient constraint, controlled ablations, and consistent performance across the tested backbones.

3. Interpreting the gain profile

The gains are not uniformly narrow. On SD3/DrawBench, SRG achieves ImageReward 0.931, compared with 0.695 for FDG and 0.714 for FDG+APG. On SDXL/HPS Prompts, ImageReward increases from 0.736 to 0.931 over FDG. The gains persist under repeated noise sampling (Appendix C.1/Tables 8–9). Figure 6 in the revised main paper also shows generally higher alignment/preference scores at comparable FID across the sampled guidance settings. These results support the practical value of the recovery mechanism without additional denoiser evaluations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.