- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper extends spectral forcing by changing the high-frequency filtering to depend on the patch, rather than having one global cut off. For this, PatchAFR splits the image into 4 DCT freq bands, and for evey patch a tiny MLP decides how much of each band should pass through. To showcase the approach, experiments are conducted on ImageNet, using primarily a JiT-B/32 model.
Critical strengths
- The method is simple and parameters efficient, acting a plug-in, making it potentially easy to adopt.
- Good improvements for the B/32 model.
- Reasonable number of ablations where conducted for some of the main components.
Critical weaknesses
-
Novelty is limited and the related work is incomplete. The method appear to be essentially SF+a learned per patch soft gate over the DCT bands, with the gate budget taken direct from the SF's mask. Most of this components were used in a similar manner before and the paper doesn't currently explain how they differ. E.g: [1] Frequency-Aware Flow Matching for High-Quality Image Generation, CVPR 2026 (they use a similar timestep-cindioned gating of freq bands), [2] Denoising, Fast and Slow: Difficulty-Aware Adaptive Sampling for Image Generation, CVPR 2026 and [3] Diffusion Models With Learned Adaptive Noise, NeurIPS 2025 (both establishing the same premise that denoising difficulty varies spatially).
-
The theory its rather standard and is classical spectral substraction. The unbiasedness and variance results from appendix R are also textbook moments of a Gaussian quadtratic form (see Mathay&Provost, "Quadratic forms in Random Variables: Theory and Application"). The derivations look correct anyway, but the paper should present this parts (including in the main) as background and as a theoretical grounding/contribution.
-
The claims of general improvements are overstated, especially that the scaling appears to be poor beyound B/32 models. In the standard JiT setting, PatchAFR is worse than the JiT baseline at 120 and 200 epochs. So it would appear that both under a longer scheduler and using larger models the approach doesn't offer any notable gains. This is particularly important because the B/32 setting at 256px is rather constricted configuration.
-
Some baselines and evaluations protocols make the results harder to interpret and may be unfair/overclaim: For example, herein all methods use a cfg of 4.0 while JiT paper tunes the CFG per model. As a results the JiT paper reports 4.37 for the B/16 model at 200 epochs vs only 4.77 in this work.
-
Continuing on the same line as above, other methods that speed up JiT training are not compared at matched epochs. WiT: Waypoint Diffusion Transformers via Trajectory Conflict Navigation is cited but not compared, and appears to be better at 200 epochs.
-
PixelREPA is not reported standalone. The work reports 4.14 for B/16 + PixelREPA which is bellow PixelREPA standalone number of 4.0 in the asme setting.
-
All experiments are done in small scale and constrained setting, unclear if they can scale and current evidence points in the opposite direction.
Questions for authors
-
Because N_pb is a global band projector averaged over a patch, it is nearly constant across patches, minus the boundary effects. Is in this case the "noise correction" close to a deterministic function which the MLP could in principle learn on its own since it already receives t?
-
Can the paper explain how the out of domain baselines are adapted?
-
What would be the accuracy under random, shuffled or fixed-per-band gates under the same budget? Without these, the gains could come from a different global attenuation shape (i.e. soft per-band gate vs SF's sigmoid cutoff).
2 comments
Part II: W4-Q3
W4: evaluation protocols.
Thank you for pointing it out. We use CFG=4.0 consistently in the manuscript to directly compare different methods under the same sampling conditions and avoid the influence of different CFG settings on the comparison. Evaluation results across different CFG scales and random seeds are also provided in the appendix.
Following your suggestion and the CFG tuning protocol in the original JiT paper, we further searched for the best CFG separately for each model. For B/32 and L/32, the tuned results are consistent with the performance reported in the manuscript. For B/16, the best CFG and corresponding FID-50K are shown below:
| Model | Best CFG | FID ↓ |
|---|---|---|
| JiT-B/16 (reproduced) | 3.5 | 4.49 |
| PatchAFR-B/16 | 3.2 | 4.72 |
| PatchAFR-B/16 + PixelREPA | 3.5 | 4.00 |
Compared with the results reported in the manuscript using a fixed CFG=4.0, CFG tuning improves the FID of JiT-B/16, PatchAFR-B/16, and PatchAFR-B/16 + PixelREPA from 4.77, 5.12, and 4.14 to 4.49, 4.72, and 4.00, respectively. We will include these evaluation results in the revised manuscript and clearly distinguish between fixed-CFG comparisons and best-CFG evaluations.
W5: Other methods compared at matched epochs.
WiT reports its results on JiT-B/32 using ImageNet 512×512, while our experiments use ImageNet 256×256.Therefore, FID results at the same epoch are not directly comparable due to the different experimental settings.
W6: PixelREPA standalone.
We thank the reviewer for highlighting this important baseline. We have added experiments with JiT-B/16 + PixelREPA up to epoch 100 due to time and resource limitations.
On B/16, our method combined with PixelREPA achieves a large improvement over PixelREPA at epoch 100, meeting our goal of accelerating training.
The early-stage convergence improvement observed on vanilla JiT (Table 2) also extends to the combination with PixelREPA. At epoch 100, PatchAFR + PixelREPA reduces FID from 5.92 to 5.09, while matching PixelREPA's reported FID of 4.00 at epoch 200. These results suggest that PatchAFR complements PixelREPA by further improving training efficiency without sacrificing its later-stage performance.
| Epoch | JiT + PixelREPA (paper) | JiT + PixelREPA (our reproduction) | Ours + PixelREPA |
|---|---|---|---|
| 100 | 5.9 | 5.91 | 5.09 |
| 200 | 4.0 | N/A | 4.00 |
Q1: effectiveness of N_pb
That’s an insightful question! Mathematically, () is indeed deterministic. Ideally, the router may be able to learn this transformation. In practice, there is a trade-off between model capacity and overhead. A lightweight MLP has limited learning capacity. Noise correction provides an inductive bias rather than additional information. By explicitly subtracting the expected noise floor, we use the known relationship from the forward diffusion process. This means the MLP does not need to learn this relationship implicitly through optimization alone, helping it improve its capability more quickly.
As shown in Table 5, the MLP using raw energy does not reach the performance of explicit noise correction in the middle and later training stages. This shows that explicit guidance is still beneficial.
Q2: details about the out of domain baselines
Thank you for your question. We have updated the implementation details of out-of-domain in the Appendix C “Adaptation of Frequency-Based Baselines.”
Q3: accuracy under random, shuffled or fixed-per-band gates under the same budget?
Relation and differences from SF. Spectral Forcing (SF) applies a timestep-dependent DCT low-pass mask, while PatchAFR uses its mean value only as a budget prior. Our learned router allocates this budget across patches and frequency bands.
Following your suggestion, we conducted budget-matched ablations with random, shuffled, and fixed-per-band gates (Appendix A, The Router Effectively Learns Frequency Allocation). Their FID-50K scores are 229, 249, and 243, respectively (CFG=4.0), with no high-quality samples observed. These results show that the SF-derived budget alone is insufficient, highlighting the importance of learned frequency allocation.
We sincerely appreciate your insightful and constructive review! Following your suggested experiments and evaluation protocols, we have significantly improved our work, obtained more comprehensive results, and built stronger connections with related studies. More excitingly, we have also uncovered previously underexplored benefits of our method, showing the great potential of B/16 when combined with PixelREPA and achieving even better results. We are truly grateful for your thoughtful feedback and suggestions, which have helped us make meaningful improvements to our work.
Part I: W1-W3
Dear Reviewer 5DF9,
We sincerely thank you for your positive assessment of our work and for your questions.
PatchAFR learns different routing weights for frequency bands in each patch. SF uses a timestep-dependent DCT low-pass mask for frequency filtering. PatchAFR uses only the mean value of SF mask as a scalar budget prior, while the learned router determines the frequency allocation for each patch.
Raw frequency energy measured from noisy pixel inputs contains diffusion noise. PatchAFR uses the known diffusion noise contribution to construct routing features. Most previous work does not model the noisy inputs of diffusion models.
FreqFlow[1] and our method both introduce frequency-aware modules, but their architectures and roles are different. [1] employs a frequency branch to fuse features with spatial branch. In contrast, PatchAFR is a lightweight input routing plug-in. Its router predicts per-patch gates for four DCT frequency bands of the noisy input, directly adjusts the contribution of each band, and passes the recombined input to the original backbone. Therefore, the frequency weights in [1] are used for feature fusion, while PatchAFR's gates are used for input frequency routing without introducing an additional frequency modeling branch.
PatchForcing[2] also considers differences in denoising difficulty across patches, but mainly controls the denoising process by adjusting patch-wise denoising timesteps. PatchAFR instead focuses on how information is allocated across different frequency bands for each patch.
MuLAN[3] learns a data-dependent noise process to adjust how noise is injected, while PatchAFR learns how to allocate and use information from different frequency bands given a noisy state.
General related statistical theory about noise and energy has been well explored in prior work. Indeed, the purpose of the theory in our current manuscript is to motivate the noise-corrected routing features and as a theoretical background. We will polish wording for proper statement.
We appreciate your insightful observation, which closely aligns with our experimental settings and findings presented in Table 2 and Figure 4 of our manuscript.
Model scaling. We observe consistent performance improvements not only on JiT-B/32 but also on the larger JiT-L/32. As shown in Table 2, PatchAFR outperforms the corresponding JiT baselines on both B/32 and L/32 at all reported training stages (60, 120, and 200 epochs).
Training scaling. Figure 3 presents the training results on B/32 up to 600 epochs. PatchAFR maintains its performance advantage throughout longer training and requires 29% fewer training epochs to reach the same FID.
Patch 32 settings are important. Large-patch modeling is a widely studied direction for efficient pixel-space generation. Recent works such as MiniT2I, PixelU, etc. use patch size 32 in experiments. Notably, PixelDiT also experiments with B/32 at 256×256 resolution. Larger patch sizes reduce the number of tokens and the computational cost of Transformers. Therefore, improving generation quality and training convergence for large-patch pixel diffusion has practical value.