Where Guidance Matters Most: Spatially Adaptive CFG for Diffusion Models

Classifier-free guidance (CFG) improves prompt adherence by extrapolating beyond conditional predictions, but applies a uniform guidance scale to spatial regions that may respond differently to conditioning. We introduce pattern-based classifier-free guidance (PCFG), a training-free method that adaptively regulates further extrapolation by measuring how conditioning alters local spatial patterns. PCFG represents small patches by ordered prediction amplitudes and constructs neighborhood-level pattern distributions from their occurrence frequencies, capturing local spatial relationships beyond pointwise residual magnitudes. It then compares the corresponding conditional and unconditional distributions using JS divergence to quantify local pattern discrepancy. Finally, a bounded power mapping converts this discrepancy into spatially varying guidance scales, suppressing further extrapolation in regions with pronounced pattern changes while allowing stronger amplification where the two distributions remain similar. At each sampling step, PCFG updates the guidance map from the current conditional and unconditional predictions without additional denoiser evaluations. Experiments on SD3.5 Medium across COCO, DrawBench, and GenEval2 show that PCFG achieves the highest reported ImageReward and HPSv2.1 means in all nine benchmark–budget settings against standard CFG and seven competing guidance methods. It surpasses the metric-specific best-scanned scalar CFG in 41 of 45 comparisons; moreover, its 20-step results outperform the best-scanned 30-step scalar CFG on both ImageReward and HPSv2.1 across all three benchmarks. These results demonstrate that local pattern discrepancy provides an effective signal for spatially adaptive guidance.