- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Pattern-based Classifier-Free Guidance (PCFG), a training-free guidance method that assigns spatially varying CFG scales during diffusion sampling. It converts the conditional and unconditional prediction fields into quantized channel-norm maps, encodes ordered local patches, estimates neighborhood pattern histograms, and measures their Jensen-Shannon divergence. Regions with larger conditional–unconditional pattern discrepancies receive less additional extrapolation, whereas regions with similar patterns receive stronger guidance. Favorable empirical results are reported on SD3.5 Medium and SDXL across multiple datasets.
Critical strengths
-
The proposed method is well motivated by the observation and analysis that identical pointwise residual energy can conceal different spatial arrangements. Using local pattern-distribution discrepancy as a feedback signal for spatial guidance is interesting and potentially useful.
-
The proposed method is sufficiently concrete in its design and clearly presented. Extensive experiments demonstrate generally consistent gains across diverse settings and metrics.
Critical weaknesses
-
The evaluation relies entirely on automated metrics. A human preference study would strengthen the performance claim but is absent. This is particularly important because the metrics exhibit non-negligible disagreement: PCFG consistently improves ImageReward and HPSv2.1, but does not consistently lead PickScore or Aesthetic Score.
-
The stated implementation forms a tensor of shape [B, , H, W] = [B, 19683, H, W]. At a 128×128 latent in FP32, this is approximately 1.2 GiB per branch before other tensors. No latency, peak-memory, or FLOP results are reported. Consequently, “no additional denoiser evaluations” does not establish practical efficiency.
-
The generalization evidence remains limited to the Stable Diffusion family, although the models cover velocity and noise predicition and involve different architectures. It is unclear whether the proposed method generalizes to other modern model families, such as FLUX.
Questions for authors
-
Could the authors report wall-clock latency, throughput, FLOP, or peak-memory measurements and make a direct efficiency comparison against CFG and SAMG?
-
Could the authors clarify whether PCFG is applicable only to models exposing explicit conditional and unconditional prediction branches? Does it work with FLUX models?
-
Why is the S-CFG result absent from the experimental comparison despite being recognized a closely related spatially adaptive CFG method?
-
Do all mean values reported are obtained with a single seed of 4416? If yes, authors are encouraged to report the mean over different seeds.
4 comments
Dear Reviewer vqcs,
We sincerely thank you for your constructive feedback, which has helped us strengthen the paper and clarify its contributions.
We hope that our detailed responses and additional results have adequately addressed your concerns. Should any questions or concerns remain, we would be more than happy to provide further clarification.
If you find that our responses have satisfactorily resolved your concerns, we would greatly appreciate your reconsideration of the score and stronger support for the paper.
Thank you again for your time and thoughtful evaluation.
Best regards,
The Authors
Thanks for the authors' response. It addresses most of my concerns, and I decided to raise my rating.
Thank you for your thoughtful feedback and for reconsidering our response. We are glad that our clarifications addressed your concerns, and sincerely appreciate your decision to raise the rating.
Efficiency, Reliability, and More Evaluation
We thank the reviewer for the careful and constructive review, and for finding the method "well motivated", "clearly presented", and supported by "extensive experiments". Every question raised is concrete and testable. We answer each with new measurements, and in each case we include more baselines than were requested.
W1: Metric disagreement
We conducted a multi-seed study. Against CFG, PCFG improves all five metrics, including PickScore and Aesthetic, in every seed and with intervals excluding zero. In this SD3.5 study, PCFG significantly improves Aesthetic relative to SAMG and CFG-Rescale. PickScore intervals span zero, and no metric shows a statistically significant decrease relative to these baselines. Image-level paired win rates against CFG are 69% (HPS), 61% (IR), 61% (Aesthetic), 57% (Pick), and 55% (CLIP). In addition, PCFG improves distribution-level fidelity on COCO-5k: precision 0.726 → 0.740, coverage 0.729 → 0.753, FID 31.8 → 30.7 (Table G4).
W2 / Q1: Memory, latency, throughput, and comparison with CFG and SAMG
The estimate concerns the dense histogram used to define the statistic in the paper; our implementation never builds it. The reason is simple: a k×k neighborhood contains only (k−p+1)2=121 patches per branch, so at any location at most 242 of the 39=19,683 possible patterns actually occur. All other patterns have zero probability in both branches and contribute exactly zero to the JS divergence. We therefore count only the patterns that occur, which gives exactly the same S with no approximation. This reduces the per-step cost from 8.7 GiB and 93 ms (dense) to 126 MiB and 2.7 ms; App. G of the revised paper gives the derivation.
SD3.5-Medium, A100, 10242, 20 steps.
Compared with standard CFG, PCFG's inference-time overhead at 20 steps is only +1.43%. General Response Table G1 covers all nine methods.
W3 / Q2: Beyond SD; FLUX; the branch requirement
PCFG requires exactly what CFG already computes: a conditional and an unconditional prediction on a spatial grid. It needs no attention maps, so it applies to any model sampled with CFG. We ran FLUX.2 Klein Base 4B (50 steps; 600 prompts) and deliberately kept the SD3.5 configuration.
PCFG runs on this model family without retuning. HPS improves significantly and PickScore decreases significantly; CLIP, IR, and Aesthetic have small mean differences whose 95% intervals include zero. In summary, PCFG is now evaluated on SD3.5 (MMDiT, velocity), SDXL and SD2.1 (UNet, noise), and FLUX.2, as well as four solvers (Table G3a, 39/40 significant gains).
Q3: S-CFG
S-CFG derives its regions from the cross-attention maps of UNet text-to-image models; the official code supports SD1.5. To provide a fair comparison, we ported the official implementation to SD2.1-base on a shared pipeline, preserving its attention processor and mask computation and keeping the setup as close as possible to the official pipeline. We also included CFG-Rescale and APG. The experiments use 600 prompts, DPM-Solver++, and two sampling budgets; full results are given in Table G3b of the General Response.
PCFG gives the best CLIP, IR, and HPS at 20 steps and the best scores on all five quality metrics at 50 steps. It is also faster than S-CFG at both budgets: 0.80 vs. 0.87 s/image at 20 steps and 1.85 vs. 1.99 s/image at 50 steps.
Q4: Multiple seeds
Yes, the original results used seed 4416. We now report four seeds, and the conclusions are unchanged. SD3.5-M, 20 steps, 600 prompts × seeds {4416,5525,6636,7749}, paired.
The "x/4" entries give the number of seeds in which PCFG is better. Further details are given in Tables G2a–b.
We hope these experiments fully address the reviewer's questions, and we would be happy to provide further details.