- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the persistent VAE encoding‑decoding overhead in one‑step super‑resolution and proposes LiPDiT‑SR, a VAE‑free model. Built upon PixelDiT, the approach first adapts the full model for one‑step SR. It then leverages MDBS to evaluate the impact of each patch block on downstream AdaLN modulation, progressively pruning from 26 patch blocks down to 18. Meanwhile, SMA constrains modulation signals before and after pruning, while keeping the pixel‑level pathway intact. Evaluations are conducted on ImageNet‑Test, RealSR and RealSet65. On unified hardware, the final model achieves 16.54 ms per image, delivering approximately 1.6× speedup compared with the fastest existing one‑step baseline.
Critical strengths
- Practically‑motivated design supported by runtime profiling. For one‑step latent diffusion, VAE overhead is no longer negligible. The authors further profile the full PixelDiT‑SR and find that the patch‑level pathway accounts for 77.2 % of total runtime whereas the pixel‑level pathway occupies only 19.5 %. Therefore prioritizing compression of the patch‑level pathway while preserving the pixel‑level renderer is not an ad‑hoc heuristic.
- More reasonable block‑selection criterion via MDBS. Instead of directly comparing block‑level features, MDBS measures changes in scale, shift and gate signals fed into the pixel pathway upon bypassing each block. For PixelDiT’s dual‑path architecture, this criterion has a more direct connection to the final rendering module.
Critical weaknesses
- Moderate methodological novelty; heavy dependence on PixelDiT’s specific architecture. The VAE‑free pixel‑diffusion backbone and patch‑/pixel‑dual‑path structure are inherited from PixelDiT. The genuinely new components introduced in this work are mainly block‑selection and modulation‑alignment for the patch‑level pathway. While the idea is sensible, it remains unclear whether it can generalize to other DiT variants without such explicit dual‑path design and AdaLN interfaces.
- The phrasing “comparable visual quality” appears somewhat over‑optimistic. The main strength of this method lies in speed rather than absolute restoration quality. For instance, on RealSR, LiPDiT‑SR reaches a CLIPIQA score of 0.6881 versus TSD‑SR’s 0.7290. On RealSet65, it obtains MUSIQ 65.735 compared to TSD‑SR’s 71.642. Similar gaps persist across multiple perceptual metrics. A more accurate conclusion would highlight its favourable quality‑efficiency trade‑off rather than claiming overall parity against top‑performing methods.
Questions for authors
- Under the identical PixelDiT‑SR backbone, fixed 18‑block budget and exactly matched training schedule, could you add direct comparisons against one or two established standard layer‑importance / pruning criteria? The current consecutive / uniform baselines are too simplistic to fully demonstrate the inherent advantages brought by MDBS.
- MDBS produces a fixed block ranking by individually bypassing each single block within the full model, even though 8 blocks are ultimately removed simultaneously. Strong interactions may exist among different blocks. Have you investigated recomputing the importance ranking after each pruning iteration? Have you also evaluated ranking stability across different calibration sets?
2 comments
Response to Reviewer MVRg (2/2)
Q2: Recomputing the ranking after each pruning stage, and ranking stability
R: (1) Re-scoring. Yes. Our implementation recomputes the scores at every pruning stage. We thank the reviewer for this question, which led us to identify an inaccuracy in the submitted description. It presented MDBS as a single ranking on the full model. At the start of each stage we load the EMA weights of the trained model of the preceding stage, bypass each of its remaining blocks in turn on the calibration set (128 ImageNet-training images), compute the modulation deviation against that model's own modulation signals, and keep the highest-scoring blocks. All reported results use this procedure. Section 3.2 (Eqs. 5–8), Section 3.4, Section 4.1 and the caption of Fig. 2 have been corrected.
Table R2. Blocks removed at each stage (index in the original 26-block pathway, starting from 1).
| Stage | 26→22 | 22→18 | 18→14 | 14→10 |
|---|---|---|---|---|
| Blocks removed | 5, 19, 22, 23 | 6, 12, 18, 21 | 3, 7, 16, 20 | 2, 4, 14, 17 |
(2) Effect of re-scoring. A single ranking on the full model would remove blocks 3, 4, 5, 6, 19, 21, 22 and 23. The stage-wise procedure removes blocks 5, 6, 12, 18, 19, 21, 22 and 23. The two agree on six of eight blocks: after the 22-block model is trained, re-scoring removes blocks 12 and 18 where the single ranking would remove blocks 3 and 4. Before fine-tuning, the re-scored selection also gives higher PSNR (26.71 vs. 26.44 dB) and lower LPIPS (0.4036 vs. 0.4121) than the single ranking (Table R1).
(3) Stability across calibration sets. We re-scored the full model and the 22-block model on seven fresh 128-image draws from the ImageNet training set, each with a new degradation seed and disjoint from the others and from the original calibration set.
Table R3. Stability of the MDBS selection (Appendix A.2).
| Stage | Mean Spearman correlation with the original ranking (minimum) | Draws removing the same four blocks | Draws sharing at least three of the four |
|---|---|---|---|
| 26→22 | 0.991 (0.982) | 0 of 7 | 7 of 7 |
| 22→18 | 0.990 (0.982) | 2 of 7 | 7 of 7 |
The ranking is stable: its Spearman correlation with the original ranking is at least 0.98 in all 14 runs, and the removed set shares at least three of its four blocks with the original in every draw. The fourth block lies at the cut-off and changes with the draw. At 26→22, block 23 is replaced by block 4 in five draws and by block 21 in two; these three blocks are the fourth-, fifth- and sixth-lowest in the original ranking. At 22→18, block 18 is replaced by block 3 in five draws. This variation is confined to near-tied blocks. At 26→22, the scores of blocks 23, 21 and 4 lie within 1.9% of each other on the original calibration set and within 4.0% on the new draws; at 22→18, the scores of blocks 18 and 3 differ by 4.3% and 2.6%. The full analysis is in Appendix A.2.
(4) Interactions within a stage. Because scores are recomputed after every stage, later selections are made on a model that has adapted to the earlier removals. Within a stage, four blocks are removed together on the basis of single-block scores. On the 22-block model, over 200 random four-block sets, the Spearman correlation between the summed single-block scores and the jointly measured deviation is 0.87, and the set selected by MDBS has a lower joint deviation than 99% of the random sets (198 of 200). The joint deviation is smaller than the sum of the single-block scores (0.140 against 0.341 for the MDBS set), so the effects are not additive in magnitude, but their ordering is preserved (Appendix A.3).
In summary, we attribute the architecture to PixelDiT and state the scope of the method, state what the quality claim covers, compare MDBS with sensitivity-based criteria under a fixed budget, and clarify that the ranking is recomputed at every stage.
References
[1] Y. Yu et al. PixelDiT: Pixel Diffusion Transformers for Image Generation. CVPR 2026.
[4] Z. Ma et al. DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation. CVPR 2026.
Response to Reviewer MVRg (1/2)
We thank the reviewer for the careful assessment and constructive suggestions. In response, we have clarified which components are inherited from PixelDiT and stated the architectural scope of the method (Sections 3.1 and 5); stated the scope of the quality claim (Section 4.2); compared MDBS with sensitivity-based selection criteria (Appendix A.1); and corrected the description of MDBS and added a ranking-stability analysis (Section 3.2; Appendix A.2). All changes to the main text are highlighted in blue, and the appendices are new. Below, we provide point-by-point responses to the weaknesses and questions raised in your review.
W1: Novelty and dependence on PixelDiT's architecture
R: (1) Attribution. PixelDiT [1] provides the backbone, as the paper states. Our contribution is how to compress it for one-step SR: which pathway to prune, which blocks to remove (MDBS), and how to keep the pruned pathway compatible with the pixel-level renderer (SMA). None of these is part of PixelDiT, and Section 3.1 now states this division explicitly.
(2) Scope. MDBS and SMA need a trunk whose output reaches a retained module through an identifiable conditioning interface. This structure is not specific to PixelDiT. DeCo [4], for example, conditions its pixel decoder on the DiT output through AdaLN. For single-path DiTs without such an interface, MDBS reduces to feature sensitivity at a chosen layer, and we make no claim for that setting. Experiments on other dual-path architectures are outside the scope of this paper, and Section 5 now states this as a limitation.
W2: The phrase "comparable visual quality"
R: The paper claims quality comparable to one-step diffusion SR methods as a group, not parity with the best method on each metric, and Table 1 supports this claim: on 17 of its 18 metric–dataset pairs, LiPDiT-SR falls inside the range spanned by its seven baselines. The exception is LPIPS on ImageNet-Test.
On RealSR, LiPDiT-SR ranks second in SSIM and fourth in LPIPS. It trails most baselines in MUSIQ, MANIQA and NIQE, which includes the gap to TSD-SR that the reviewer points out. It reaches this quality at 16.54 ms/image, 1.6× faster than AdcSR and 3.5× faster than TSD-SR; among the compared methods, only TinySR is faster at 512×512 (Section 4.2 and Appendix B.1).
We have therefore kept the wording. So that it is not read as parity with the best method, Section 4.2 now states this definition explicitly, together with the metrics on which LiPDiT-SR trails.
Q1: Comparison with established layer-importance criteria
R: With the PixelDiT-SR backbone and the 18-block budget fixed, we compared the blocks selected by MDBS with those selected by output sensitivity, feature sensitivity and a single ranking without re-scoring. Output sensitivity scores a block by the change in the SR output when it is bypassed, and feature sensitivity by the change in the final semantic tokens; the formulas are given in Appendix A.1.
Table R1. Blocks removed at 22→18 by each criterion (index in the original 26-block pathway, starting from 1), and RealSR quality of the resulting 18-block model before any fine-tuning (Appendix A.1).
The criteria are not equivalent: output sensitivity agrees with MDBS on two of the four removed blocks and feature sensitivity on three. Before any fine-tuning, the MDBS selection preserves fidelity best, with the highest PSNR (26.71 dB, 0.27 dB above the next criterion) and the lowest LPIPS. This matches its design, since MDBS keeps the blocks that most affect the conditioning consumed by the pixel-level renderer. No-reference scores are low for every selection before fine-tuning, and for MDBS the fine-tuning stage raises CLIPIQA from 0.5016 to 0.6881 in the final model.
After training, MDBS outperforms position-based selection under the same budget and schedule (Table 4 of the paper), improving LPIPS from 0.4112 (consecutive) and 0.3838 (uniform) to 0.3798, and CLIPIQA from 0.6407 and 0.6144 to 0.6598.