PatchAFR: Noise-Corrected Patch-Adaptive Frequency Routing for Pixel-Space Diffusion Models

Pixel space diffusion models generate images directly from raw pixels without relying on pretrained tokenizers, but their high dimensional input space makes training from scratch expensive. Existing frequency based approaches mainly characterize frequency recoverability at the global image level. We observe that this global treatment overlooks an important property of images: the same frequency band can have different local signal strength relative to the diffusion noise across spatial regions, resulting in spatially varying recoverability. Based on this observation, we introduce PatchAFR, a lightweight **Patch**-**A**daptive **F**requency **R**outing method that can be integrated into pixel space diffusion models. We formulate frequency recoverability through local spectral evidence and derive a timestep aware correction to estimate this evidence from noisy inputs. PatchAFR then jointly uses the corrected local evidence and diffusion timestep to adaptively route frequency bands for each spatial patch. Experiments on the pixel space diffusion model JiT demonstrate improvements throughout training. On JiT B/32, PatchAFR reduces FID by 8.4%, 7.7%, and 9.1% at epochs 60, 120, and 200, respectively, and requires 29% fewer training epochs to reach the same FID over 600 epochs.