- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces Wiener-informed Decoding (WiD), a method for improving the training efficiency and generation quality of pixel-space flow models by incorporating second-order image statistics into the pixel decoder. Motivated by the observation that frozen Wiener estimators substantially improve velocity prediction, the authors propose a trainable statistical prediction pathway with PCA-initialized coordinate transforms, noise-dependent spectral gains, and coefficient interfaces that interact with the native decoder. The method retains the original decoder blocks and flow-matching objective while introducing a relatively small number of additional parameters. Experiments on ImageNet-1K 256×256 demonstrate that PixelDiT-XL with WiD achieves an FID of 1.61 at 260 epochs, matching the reported baseline performance at 320 epochs with approximately 0.94% additional parameters. The approach also improves DeCo-XXL on text-to-image generation, increasing GenEval from 0.43 to 0.49 and DPG-Bench from 73.1 to 78.8. Additional ablations and diagnostic analyses suggest that statistical initialization and coefficient-space interactions provide complementary benefits.
Critical strengths
-
The paper identifies a meaningful difficulty in pixel-space velocity learning and connects it to classical Wiener estimation. The progression from frozen analytic predictors to trainable decoder components is intuitive and supported by preliminary experiments. This provides an interesting connection between traditional image estimation and modern flow-based generative modeling.
-
WiD introduces statistical prediction and coefficient interfaces without modifying the internal blocks of the original decoder. The initialization strategy is particularly well designed: it preserves the native decoder's initial input representation while initializing the overall velocity prediction with a Wiener estimate. This makes the method relatively straightforward to integrate into existing architectures.
-
The improvements on PixelDiT and DeCo suggest that WiD is not restricted to a particular decoder implementation. The ImageNet results demonstrate faster progress toward competitive FID values, while the text-to-image experiments provide additional evidence of practical relevance. The gains on GenEval and DPG-Bench are particularly encouraging.
-
The ablations indicate that statistical initialization, direct Wiener prediction, and coefficient interfaces each contribute to the overall performance. The parameter and FLOP increases are also relatively small, making the approach attractive for integration into existing pixel-space generation systems.
Critical weaknesses
-
The paper argues that Wiener priors facilitate learning beyond their explicit linear predictions. However, the error decomposition measures agreement with a pretrained PixelDiT-XL reference rather than directly evaluating the learned velocity against its training target. Furthermore, the remaining component can still contain linear dependencies outside the selected affine fitting family, as acknowledged in Appendix B.1. Consequently, the reported improvements in the remaining error do not necessarily establish improved nonlinear representation learning. The diagnostic analysis also focuses primarily on frozen Wiener bases rather than the final WiD architecture.
-
The experimental design does not sufficiently isolate Wiener-specific benefits from more general preconditioning and additional model capacity. WiD combines statistical initialization, trainable analysis and synthesis transforms, time-dependent coefficient scaling, feature injection, and an additional prediction head. Although individual ablations are provided, it remains unclear how much of the improvement specifically requires Wiener estimation. Comparisons with parameter-matched decoder extensions, generic trainable residual pathways, and simpler preconditioning schemes would help establish whether the proposed statistical structure is essential.
-
The highlighted result of FID 1.61 at 260 epochs uses different CFG weights and guidance intervals from the reported PixelDiT-XL baseline at 320 epochs. Appendix A.3 shows that the 260-epoch WiD checkpoint obtains FID 1.65 under CFG 2.75, while the reported FID 1.61 uses CFG 3.25 with a different guidance interval. Although selecting favorable inference configurations is reasonable, a matched evaluation protocol would better isolate the contribution of the training method. Moreover, the paper does not report whether the improvements persist when WiD is trained to the same 320-epoch budget or beyond.
-
The paper primarily measures efficiency through training epochs rather than end-to-end training time or total computational expenditure. Appendix A.3 reports approximately 1.33% additional forward FLOPs and reduced inference throughput, with observed slowdowns of roughly 5–6%. These overheads are relatively small, but they highlight the importance of reporting training time and quality-versus-compute curves. The current results support improved convergence in epochs more directly than actual wall-clock training efficiency.
Questions for authors
-
Can the authors provide direct comparisons between PixelDiT-XL and PixelDiT-XL+WiD under identical CFG weights, guidance intervals, and sampling settings? It would also be useful to compare both methods at the same 320-epoch budget and report training time versus generation quality.
-
How does WiD compare with simpler alternatives, such as a frozen Wiener residual connection, a trainable Wiener branch without coefficient interfaces, a parameter-matched additional decoder head, or generic preconditioning? These comparisons would help identify the essential source of the improvement.
-
Does WiD remain effective at higher resolutions, such as 512×512, or under different patch sizes and image distributions? Additionally, the text mentions an ablation removing Wiener gain, but the corresponding numerical result is not included in Table 3. Could the authors provide this result?
5 comments
Response to Reviewer nLnu (2/3)
W2 & Q2: Comparisons with simpler alternatives.
We added two controls on native PixelDiT-B/16. All use 40 epochs, global batch 256, REPA 0.5, and EMA/no-CFG FID-50K with FlowDPMSolver-100:
| Method | FID |
|---|---|
| Native PixelDiT-B/16 | 60.01 |
| + time-conditioned linear skip | 48.46 |
| + scalar flow preconditioning | 56.18 |
| + WiD | 41.71 |
Time-conditioned linear skip. For each noisy patch , we add
to the corresponding native velocity output. The trainable matrices and are shared across patches. The time modulation is produced by a MLP with SiLU. We randomly initialize and zero-initialize and the MLP's last layer. This branch is linear in the image input at fixed , and adds 7.41M parameters versus WiD's approximately 7.48M.
Scalar flow preconditioning. We adapt EDM's scalar input, skip, and output preconditioning [1] to our linear-flow velocity parameterization. From the same patch second-moment matrix used by WiD, we compute and set . The prediction is
where is native PixelDiT and is the class label. This combines an isotropic Wiener skip with noise-dependent input and output scaling, without adding trainable parameters.
The frozen additive-base comparison is already shown separately without REPA in Fig.1(c): its early gains narrow with training. In Table 5, we have tested the trainable statistical path without coefficient interfaces, which also underperforms full WiD. Random initialization of and gives FID 43.99, while replacing only the empirical spectrum with ones gives FID 48.92. Both are worse than full WiD's 41.71. Together, these controls support the value of statistical initialization and coefficient interaction.
[1] Karras et al. Elucidating the Design Space of Diffusion-Based Generative Models.
Response to Reviewer nLnu (3/3)
W3 & Q1: Extended training and comparable settings.
Our original comparison reported tuned generation quality. CFG scale and guidance interval are inference hyperparameters. As discussed in JiT [1], the best CFG scale can vary across model configurations. We agree that shared-setting evaluation complements reporting tuned performance, and we are pleased to provide additional results in the following.
Shared-setting evaluation at 256 x 256. We now report both models under CFG 4.0 and interval [0.14, 0.55], with EMA, FlowDPMSolver-100, and 50K samples:
| Model | Epochs | FID |
|---|---|---|
| PixelDiT-XL/16 | 320 | 1.58 |
| PixelDiT-XL/16 + WiD | 260 | 1.59 |
| PixelDiT-XL/16 + WiD | 320 | 1.58 |
Under identical guidance settings, WiD reaches comparable quality with 18.75% fewer training epochs and matches the baseline at 320 epochs. These results correspond with our claim of accelerated training with comparable quality retained. These comparisons are included in Tables 1 and 7.
Longer training. At 500 epochs, WiD reaches FID 1.53, comparable to PixelDiT's officially reported 1.54 at 800 epochs, with 37.5% fewer epochs. We use the published baseline result because its 800-epoch checkpoint is not available. This longer-training comparison further supports the benefit in training efficiency.
Higher-resolution support (also Q3). At 512 x 512, WiD reaches FID 1.68 using the baseline's reported CFG 3.5 and interval [0.10, 1.00], compared with its published 1.81. Keeping CFG at 3.5 and adjusting only the interval to [0.10, 0.90] further lowers WiD's FID to 1.64. Our method uses 260 epochs of 256-resolution pretraining plus 50 epochs of 512-resolution fine-tuning, while the baseline reports 320 + 530 epochs. This provides additional support for the training-efficiency benefit under the baseline's own reported guidance settings.
W4 & Q1: Training time estimations.
With 8 H200 GPUs, we obtain median update times of 0.174s for native PixelDiT-XL/16 and 0.183s with WiD on ImageNet 256×256. Extrapolating the rates, we estimate 66.0 hours for PixelDiT-XL/16+WiD at 260 epochs versus 77.1 hours for the native model at 320 epochs, a 14.4% saving. For the longer-training comparison, the estimates are 127.0 hours for PixelDiT-XL/16+WiD at 500 epochs versus 192.9 hours for the native model at 800 epochs, accounting for a 34.2% saving.
Q3: Patch size and image distributions.
Patch size. These experiments use B-size models: native PixelDiT-B and PixelDiT-B + WiD, each trained for 40 epochs with global batch 256 and REPA 0.5. WiD lowers FID at all three patch sizes:
| Patch size | Native PixelDiT-B | + WiD |
|---|---|---|
| 8 | 46.53 | 32.83 |
| 16 | 60.01 | 41.71 |
| 32 | 82.92 | 75.27 |
Text-to-image training. We initialize DeCo-XXL/16 + WiD using statistics from 51,200 images in the 256-resolution pretraining set. Following DeCo's three-stage recipe, we train for 200K steps at 256 resolution, continue for 80K at 512, and fine-tune for 40K on BLIP3-o-60K at 512. WiD remains effective after both resolution and training-data changes, with its parameters adapting jointly with the network.
At 512×512, our method achieves GenEval 0.86 and DPG-Bench 82.6, versus 0.82 and 82.3 from our evaluation of the official DeCo-XXL/16 checkpoint, strictly following its official sampling settings. We report detailed results in Table 4.
DeCo officially reports GenEval 0.86 and DPG-Bench 81.4. Our reevaluated GenEval 0.82 is also consistent with independent studies: RaPD [2] and DMA² [3]. These studies likewise obtain GenEval around 0.82–0.83. Our results extend WiD's validation to another decoder architecture and image-text distribution, including subsequent higher-resolution training and fine-tuning.
Q3: Numerical result for removing the Wiener gain.
Thank you for pointing out the ambiguous wording. This referred to removing the direct Wiener prediction, not setting the spectrum to one. The row labeled "w/o Wiener prediction" gives FID 49.54, sFID 10.88, and IS 29.9, versus full WiD's 41.71, 9.18, and 35.1. We have aligned the text with this row name in Sec.5.3, Table 5.
[1] Tianhong Li and Kaiming He. Back to Basics: Let Denoising Generative Models Denoise.
[2] Ge et al. RaPD: Resolution-Agnostic Pixel Diffusion via Semantics-Enriched Implicit Representations.
[3] Lin et al. DMA²: Pixel-space Distribution Matching with Adversarial and Anchor Losses.
Response to Reviewer nLnu (1/3)
Thank you for the comments. We have added reference-independent diagnostics, controls, and extended experiments to address these concerns. We have also revised the manuscript accordingly.
The original analysis. We agree that the remaining prediction is not necessarily purely nonlinear: it excludes the chosen affine family but may retain other linear dependencies. Our claim is that gains can extend beyond the explicit additive prediction, not that the decomposition proves improved nonlinear representations. At each fixed t, our affine fits can represent the corresponding frozen base exactly. Therefore, subtracting the fitted component removes the base’s direct additive contribution. The smaller remaining error reflects closer agreement of the learned remainder with the reference.
Direct validation against flow targets. We now compare complete predictions against v=x−ϵ. Although the sampled target differs from the marginal velocity, the expected MSE against this target equals the marginal-velocity MSE plus a predictor-independent term. This term cancels when comparing models (Appendix B.3, Eq.20).
We use 3,072 held-out images, two fixed noises per image, and five fixed time steps, with EMA and no CFG. We report mean MSE reductions in units of 10−3 in the table below.
All five comparisons favor the added base, decoder, or WiD at both checkpoints (Table 11). This validates prediction improvement independently of the chosen reference.
Analysis of trained WiD. We examine the direct statistical prediction, coefficient correction, and native pixel output separately. At 40 epochs, the direct path's MSE is 0.24440. Adding the coefficient correction reduces it to 0.11797, and the full output reaches 0.09864. Removing the direct statistical, coefficient-correction, or native pixel term from the trained output, with all other computation held fixed, increases MSE by 1.14861, 0.10750, and 0.01933, respectively. These measurements show that all three output terms contribute to the combined prediction (Appendix B.4, Table 12).