acceptodds
Under review as a conference paper at ICLR 2027

Divide and Lighten: Lightweight Pixel Diffusion Transformers for One-Step Image Super-Resolution

Abstract

One-step diffusion substantially reduces the denoising cost of super-resolution, yet most existing methods still rely on latent diffusion architectures and thus remain burdened by expensive VAE encoding and decoding. We therefore propose LiPDiTSR, a Lightweight Pixel Diffusion Transformer that eliminates the VAE and performs one-step super-resolution directly in pixel space. LiPDiT-SR first builds a full VAE-free one-step PixelDiT model for super-resolution, where the patch-level and pixel-level pathways primarily handle semantic reasoning and texture reconstruction, respectively. We then derive a compact model by pruning the computationally dominant patch-level pathway while preserving the pixel-level pathway. Modulation Deviation Block Selection (MDBS) selects the patch blocks whose removal least disrupts the modulation signals delivered to the pixel-level pathway, while a Semantic Modulation Alignment (SMA) loss aligns these signals with those of the preceding reference model at each pruning stage. Experiments on multiple synthetic and real-world benchmarks demonstrate that LiPDiT-SR achieves visual quality comparable to state-of-the-art one-step diffusion SR methods while delivering a 1.6× inference speedup for ×4 super-resolution.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Super Resolution, Diffusion, Pixel Space, One-step

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MVRgOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the persistent VAE encoding‑decoding overhead in one‑step super‑resolution and proposes LiPDiT‑SR, a VAE‑free model. Built upon PixelDiT, the approach first adapts the full model for one‑step SR. It then leverages MDBS to evaluate the impact of each patch block on downstream AdaLN modulation, progressively pruning from 26 patch blocks down to 18. Meanwhile, SMA constrains modulation signals before and after pruning, while keeping the pixel‑level pathway intact. Evaluations are conducted on ImageNet‑Test, RealSR and RealSet65. On unified hardware, the final model achieves 16.54 ms per image, delivering approximately 1.6× speedup compared with the fastest existing one‑step baseline.

Critical strengths
  1. Practically‑motivated design supported by runtime profiling. For one‑step latent diffusion, VAE overhead is no longer negligible. The authors further profile the full PixelDiT‑SR and find that the patch‑level pathway accounts for 77.2 % of total runtime whereas the pixel‑level pathway occupies only 19.5 %. Therefore prioritizing compression of the patch‑level pathway while preserving the pixel‑level renderer is not an ad‑hoc heuristic.
  2. More reasonable block‑selection criterion via MDBS. Instead of directly comparing block‑level features, MDBS measures changes in scale, shift and gate signals fed into the pixel pathway upon bypassing each block. For PixelDiT’s dual‑path architecture, this criterion has a more direct connection to the final rendering module.
Critical weaknesses
  1. Moderate methodological novelty; heavy dependence on PixelDiT’s specific architecture. The VAE‑free pixel‑diffusion backbone and patch‑/pixel‑dual‑path structure are inherited from PixelDiT. The genuinely new components introduced in this work are mainly block‑selection and modulation‑alignment for the patch‑level pathway. While the idea is sensible, it remains unclear whether it can generalize to other DiT variants without such explicit dual‑path design and AdaLN interfaces.
  2. The phrasing “comparable visual quality” appears somewhat over‑optimistic. The main strength of this method lies in speed rather than absolute restoration quality. For instance, on RealSR, LiPDiT‑SR reaches a CLIPIQA score of 0.6881 versus TSD‑SR’s 0.7290. On RealSet65, it obtains MUSIQ 65.735 compared to TSD‑SR’s 71.642. Similar gaps persist across multiple perceptual metrics. A more accurate conclusion would highlight its favourable quality‑efficiency trade‑off rather than claiming overall parity against top‑performing methods.
Questions for authors
  1. Under the identical PixelDiT‑SR backbone, fixed 18‑block budget and exactly matched training schedule, could you add direct comparisons against one or two established standard layer‑importance / pruning criteria? The current consecutive / uniform baselines are too simplistic to fully demonstrate the inherent advantages brought by MDBS.
  2. MDBS produces a fixed block ranking by individually bypassing each single block within the full model, even though 8 blocks are ultimately removed simultaneously. Strong interactions may exist among different blocks. Have you investigated recomputing the importance ranking after each pruning iteration? Have you also evaluated ranking stability across different calibration sets?
2 comments
AuthorsOpenReview ↗

Response to Reviewer MVRg (1/2)

We thank the reviewer for the careful assessment and constructive suggestions. In response, we have clarified which components are inherited from PixelDiT and stated the architectural scope of the method (Sections 3.1 and 5); stated the scope of the quality claim (Section 4.2); compared MDBS with sensitivity-based selection criteria (Appendix A.1); and corrected the description of MDBS and added a ranking-stability analysis (Section 3.2; Appendix A.2). All changes to the main text are highlighted in blue, and the appendices are new. Below, we provide point-by-point responses to the weaknesses and questions raised in your review.

W1: Novelty and dependence on PixelDiT's architecture

R: (1) Attribution. PixelDiT [1] provides the backbone, as the paper states. Our contribution is how to compress it for one-step SR: which pathway to prune, which blocks to remove (MDBS), and how to keep the pruned pathway compatible with the pixel-level renderer (SMA). None of these is part of PixelDiT, and Section 3.1 now states this division explicitly.

(2) Scope. MDBS and SMA need a trunk whose output reaches a retained module through an identifiable conditioning interface. This structure is not specific to PixelDiT. DeCo [4], for example, conditions its pixel decoder on the DiT output through AdaLN. For single-path DiTs without such an interface, MDBS reduces to feature sensitivity at a chosen layer, and we make no claim for that setting. Experiments on other dual-path architectures are outside the scope of this paper, and Section 5 now states this as a limitation.

W2: The phrase "comparable visual quality"

R: The paper claims quality comparable to one-step diffusion SR methods as a group, not parity with the best method on each metric, and Table 1 supports this claim: on 17 of its 18 metric–dataset pairs, LiPDiT-SR falls inside the range spanned by its seven baselines. The exception is LPIPS on ImageNet-Test.

On RealSR, LiPDiT-SR ranks second in SSIM and fourth in LPIPS. It trails most baselines in MUSIQ, MANIQA and NIQE, which includes the gap to TSD-SR that the reviewer points out. It reaches this quality at 16.54 ms/image, 1.6× faster than AdcSR and 3.5× faster than TSD-SR; among the compared methods, only TinySR is faster at 512×512 (Section 4.2 and Appendix B.1).

We have therefore kept the wording. So that it is not read as parity with the best method, Section 4.2 now states this definition explicitly, together with the metrics on which LiPDiT-SR trails.

Q1: Comparison with established layer-importance criteria

R: With the PixelDiT-SR backbone and the 18-block budget fixed, we compared the blocks selected by MDBS with those selected by output sensitivity, feature sensitivity and a single ranking without re-scoring. Output sensitivity scores a block by the change in the SR output when it is bypassed, and feature sensitivity by the change in the final semantic tokens; the formulas are given in Appendix A.1.

Table R1. Blocks removed at 22→18 by each criterion (index in the original 26-block pathway, starting from 1), and RealSR quality of the resulting 18-block model before any fine-tuning (Appendix A.1).

CriterionBlocks removed at 22→18Shared with MDBSPSNR / LPIPS before fine-tuningCLIPIQA / MUSIQ before fine-tuning
MDBS (ours)6, 12, 18, 214 of 426.707 / 0.40360.5016 / 44.46
Output sensitivity6, 7, 18, 202 of 426.028 / 0.40440.5757 / 50.29
Feature sensitivity3, 6, 12, 183 of 425.684 / 0.41990.6362 / 59.39
Single ranking3, 4, 6, 212 of 426.442 / 0.41210.5819 / 49.60

The criteria are not equivalent: output sensitivity agrees with MDBS on two of the four removed blocks and feature sensitivity on three. Before any fine-tuning, the MDBS selection preserves fidelity best, with the highest PSNR (26.71 dB, 0.27 dB above the next criterion) and the lowest LPIPS. This matches its design, since MDBS keeps the blocks that most affect the conditioning consumed by the pixel-level renderer. No-reference scores are low for every selection before fine-tuning, and for MDBS the fine-tuning stage raises CLIPIQA from 0.5016 to 0.6881 in the final model.

After training, MDBS outperforms position-based selection under the same budget and schedule (Table 4 of the paper), improving LPIPS from 0.4112 (consecutive) and 0.3838 (uniform) to 0.3798, and CLIPIQA from 0.6407 and 0.6144 to 0.6598.

AuthorsOpenReview ↗

Response to Reviewer MVRg (2/2)

Q2: Recomputing the ranking after each pruning stage, and ranking stability

R: (1) Re-scoring. Yes. Our implementation recomputes the scores at every pruning stage. We thank the reviewer for this question, which led us to identify an inaccuracy in the submitted description. It presented MDBS as a single ranking on the full model. At the start of each stage we load the EMA weights of the trained model of the preceding stage, bypass each of its remaining blocks in turn on the calibration set (128 ImageNet-training images), compute the modulation deviation against that model's own modulation signals, and keep the highest-scoring blocks. All reported results use this procedure. Section 3.2 (Eqs. 5–8), Section 3.4, Section 4.1 and the caption of Fig. 2 have been corrected.

Table R2. Blocks removed at each stage (index in the original 26-block pathway, starting from 1).

Stage26→2222→1818→1414→10
Blocks removed5, 19, 22, 236, 12, 18, 213, 7, 16, 202, 4, 14, 17

(2) Effect of re-scoring. A single ranking on the full model would remove blocks 3, 4, 5, 6, 19, 21, 22 and 23. The stage-wise procedure removes blocks 5, 6, 12, 18, 19, 21, 22 and 23. The two agree on six of eight blocks: after the 22-block model is trained, re-scoring removes blocks 12 and 18 where the single ranking would remove blocks 3 and 4. Before fine-tuning, the re-scored selection also gives higher PSNR (26.71 vs. 26.44 dB) and lower LPIPS (0.4036 vs. 0.4121) than the single ranking (Table R1).

(3) Stability across calibration sets. We re-scored the full model and the 22-block model on seven fresh 128-image draws from the ImageNet training set, each with a new degradation seed and disjoint from the others and from the original calibration set.

Table R3. Stability of the MDBS selection (Appendix A.2).

StageMean Spearman correlation with the original ranking (minimum)Draws removing the same four blocksDraws sharing at least three of the four
26→220.991 (0.982)0 of 77 of 7
22→180.990 (0.982)2 of 77 of 7

The ranking is stable: its Spearman correlation with the original ranking is at least 0.98 in all 14 runs, and the removed set shares at least three of its four blocks with the original in every draw. The fourth block lies at the cut-off and changes with the draw. At 26→22, block 23 is replaced by block 4 in five draws and by block 21 in two; these three blocks are the fourth-, fifth- and sixth-lowest in the original ranking. At 22→18, block 18 is replaced by block 3 in five draws. This variation is confined to near-tied blocks. At 26→22, the scores of blocks 23, 21 and 4 lie within 1.9% of each other on the original calibration set and within 4.0% on the new draws; at 22→18, the scores of blocks 18 and 3 differ by 4.3% and 2.6%. The full analysis is in Appendix A.2.

(4) Interactions within a stage. Because scores are recomputed after every stage, later selections are made on a model that has adapted to the earlier removals. Within a stage, four blocks are removed together on the basis of single-block scores. On the 22-block model, over 200 random four-block sets, the Spearman correlation between the summed single-block scores and the jointly measured deviation is 0.87, and the set selected by MDBS has a lower joint deviation than 99% of the random sets (198 of 200). The joint deviation is smaller than the sum of the single-block scores (0.140 against 0.341 for the MDBS set), so the effects are not additive in magnitude, but their ordering is preserved (Appendix A.3).

In summary, we attribute the architecture to PixelDiT and state the scope of the method, state what the quality claim covers, compare MDBS with sensitivity-based criteria under a fixed budget, and clarify that the ranking is recomputed at every stage.

References

[1] Y. Yu et al. PixelDiT: Pixel Diffusion Transformers for Image Generation. CVPR 2026.

[4] Z. Ma et al. DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation. CVPR 2026.

Reviewer_UEgaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LiPDiT-SR, a compact PixelDiT-based model for one-step super-resolution without a VAE. It prunes the patch pathway using Modulation Deviation Block Selection (MDBS) and preserves conditioning of the pixel pathway through Semantic Modulation Alignment (SMA). Experiments on synthetic and real-world benchmarks compare restoration quality and runtime, with component ablations examining the proposed compression strategy.

Critical strengths

The patch pathway accounts for 77.2% of the full model's latency, providing an empirical basis for choosing what to compress. Under the authors' reported common evaluation setting, the model runs in 16.54 ms, approximately 1.59 times faster than AdcSR at 26.29 ms, while improving RealSR LPIPS from 0.3867 to 0.3792. This is a meaningful deployment benefit; a larger parameter count alone does not invalidate it.

MDBS selects retained layers according to changes in modulation signals, while SMA preserves the conditioning signals passed to the pixel pathway. Table 4 shows that MDBS improves LPIPS, CLIPIQA, and MUSIQ over contiguous and uniform layer selection. Neither mechanism adds computation at deployment, making the approach worth further investigation.

Critical weaknesses

(1) About the new mechanisms

The dual-path architecture and pixel-wise AdaLN originate from PixelDiT, so the main new contributions should be attributed to MDBS and SMA. MDBS is currently compared only with contiguous and uniform selection; the experiments do not establish that modulation deviation is a better selection criterion than output or feature sensitivity. SMA improves CLIPIQA from 0.6598 to 0.6788 but does not improve LPIPS, and output distillation achieves better LPIPS in Table 5. These results indicate a trade-off rather than an unambiguous advantage. The distillation baselines' loss weights and the stability of the results need clarification to establish the benefit of aligning this particular interface.

(2) About the comparisons and correspondence between key results

TinySR addresses compression of one-step DiT super-resolution, while GenDR-Pix addresses removing the VAE. Both are directly relevant to the paper's efficiency claims but are absent from the experimental comparisons. The reported speed also entails quality costs: for example, ImageNet-Test LPIPS is worse than that of every external baseline listed in the table. Furthermore, the final model's RealSR CLIPIQA is 0.6881 in the main table but 0.6788 in the ablation table. This discrepancy is substantial relative to the reported benefit of SMA and requires explanation. The concern is whether the evidence establishes a competitive overall trade-off and reliable component attribution, rather than whether the method ranks first on every metric.

Questions for authors

With the number of retained layers and the training budget fixed, does MDBS outperform layer selection based on output sensitivity?

With comparable tuning of loss weights, how does SMA compare with output distillation in reconstruction quality?

What explains the difference between the final model's RealSR CLIPIQA in the main table (0.6881) and the ablation table (0.6788)?

Under a common evaluation protocol, how does the proposed model compare with TinySR in its quality–latency trade-off?

6 comments
AuthorsOpenReview ↗

Response to Reviewer UEga (1/4)

We thank the reviewer for the careful assessment and constructive suggestions. In response, we have compared MDBS with output- and feature-sensitivity selection (Appendix A.1); reported the loss weights of all supervision targets and revised the discussion of SMA (Sections 4.1 and 4.3); added TinySR under a common protocol and a discussion of GenDR-Pix (Sections 2.3 and 4.2; Appendix B); and stated the training schedule of every reported model (Section 4.1; Tables 4 and 6). All changes to the main text are highlighted in blue, and the appendices are new. Below, we provide point-by-point responses to the weaknesses and questions raised in your review.

W1: Attribution of the new mechanisms

R: The paper builds on PixelDiT [1] and cites it for the dual-path architecture and pixel-wise AdaLN; we do not claim them. Our contributions are the adaptation of PixelDiT to one-step SR together with the finding that its patch-level pathway accounts for most of the latency, MDBS, and SMA. Section 3.1 now states this division explicitly.

W1 & Q1: MDBS versus output- and feature-sensitivity selection

R: We compared MDBS with output- and feature-sensitivity selection on the same trained 22-block model and the same calibration set, with the number of retained blocks fixed at 18. Each criterion scores the 22 blocks by bypassing them one at a time and removes the four lowest-scoring blocks:

  • Output sensitivity: mean absolute change in the SR output.
  • Feature sensitivity: normalized change in the final semantic tokens, i.e., the MDBS score measured before the AdaLN projections.
  • Single ranking: the MDBS ranking computed once on the full model, without re-scoring.

In the notation of Section 3.2, the superscript ∖i\setminus i marks the preceding Ks−1K_{s-1}-block model with block ii bypassed (Eq. 5), and dd is the normalized ℓ1\ell_1 distance of Eq. 6. The four scores are:

qiMDBS=Ey[1Lp∣U∣∑j=1Lp∑u∈Ud(Mj,u∖i,Mj,uKs−1)]q^{\mathrm{MDBS}}_i = \mathbb{E}_y\Big[\frac{1}{L_p|\mathcal{U}|}\sum_{j=1}^{L_p}\sum_{u\in\mathcal{U}} d\big(\mathbf{M}^{\setminus i}_{j,u},\mathbf{M}^{K_{s-1}}_{j,u}\big)\Big]

qiout=Ey[13HW∥x^∖i−x^Ks−1∥1],x^∖i=Rpix(y↑,{Mj∖i}j=1Lp)q^{\mathrm{out}}_i = \mathbb{E}_y\Big[\frac{1}{3HW}\big\Vert \hat{x}^{\setminus i}-\hat{x}^{K_{s-1}}\big\Vert _1\Big], \qquad \hat{x}^{\setminus i}=\mathcal{R}_{\mathrm{pix}}\big(y_\uparrow,\lbrace \mathbf{M}^{\setminus i}_j\rbrace _{j=1}^{L_p}\big)

qifeat=Ey[d(S∖i,SKs−1)]q^{\mathrm{feat}}_i = \mathbb{E}_y\Big[d\big(\mathbf{S}^{\setminus i},\mathbf{S}^{K_{s-1}}\big)\Big]

qisingle=Ey[1Lp∣U∣∑j=1Lp∑u∈Ud(Mj,u∖i,Mj,uF)]q^{\mathrm{single}}_i = \mathbb{E}_y\Big[\frac{1}{L_p|\mathcal{U}|}\sum_{j=1}^{L_p}\sum_{u\in\mathcal{U}} d\big(\mathbf{M}^{\setminus i}_{j,u},\mathbf{M}^{F}_{j,u}\big)\Big]

The first three are computed on the trained 22-block model. The single-ranking score is computed once on the full model, with the bypass applied to the full pathway, and its ranking is reused at every stage. Each criterion then retains the highest-scoring blocks as in Eq. 8.

Table R1. Blocks removed at 22→18 by each criterion (index in the original 26-block pathway, starting from 1), and RealSR quality of the resulting 18-block model before any fine-tuning (Appendix A.1).

CriterionBlocks removed at 22→18Shared with MDBSPSNR / LPIPS before fine-tuningCLIPIQA / MUSIQ before fine-tuning
MDBS (ours)6, 12, 18, 214 of 426.707 / 0.40360.5016 / 44.46
Output sensitivity6, 7, 18, 202 of 426.028 / 0.40440.5757 / 50.29
Feature sensitivity3, 6, 12, 183 of 425.684 / 0.41990.6362 / 59.39
Single ranking3, 4, 6, 212 of 426.442 / 0.41210.5819 / 49.60

The criteria are not equivalent: output sensitivity agrees with MDBS on two of the four removed blocks and feature sensitivity on three. Before any fine-tuning, the MDBS selection preserves fidelity best, with the highest PSNR (26.71 dB, 0.27 dB above the next criterion) and the lowest LPIPS. This matches its design, since MDBS keeps the blocks that most affect the conditioning consumed by the pixel-level renderer. No-reference scores are low for every selection before fine-tuning, and for MDBS the fine-tuning stage raises CLIPIQA from 0.5016 to 0.6881 in the final model.

After training, MDBS outperforms position-based selection under the same budget and schedule (Table 4 of the paper), improving LPIPS from 0.4112 (consecutive) and 0.3838 (uniform) to 0.3798, and CLIPIQA from 0.6407 and 0.6144 to 0.6598.

AuthorsOpenReview ↗

Response to Reviewer UEga (2/4)

W1 & Q2: SMA versus output distillation, loss weights and stability

R: (1) Loss weights. In Stage II the task loss is L1 (weight 1.0) plus LPIPS (weight 0.5), with no adversarial term. Each supervision target in Table 5 of the paper adds one term to it.

Table R2. Supervision targets and their weights (Section 4.3).

TargetAdded termWeight
SMA (ours)Normalized L1 between the modulation signals of the current and preceding models (Eq. 9)0.2, increased linearly from 0 over the first 1,000 iterations of each stage
Output KDL1 and LPIPS between student and teacher outputs0.2 (L1) and 1.0 (LPIPS)
Semantic feature KDMean cosine distance between the final semantic tokens1.0

(2) Reconstruction quality. SMA and output KD sit at different ends of the fidelity–perception trade-off. Output KD gives better reconstruction quality (PSNR 25.751 vs. 25.373; LPIPS 0.3719 vs. 0.3800), but it lowers perceptual quality below the model trained with the task loss only (CLIPIQA 0.6377 vs. 0.6598; MUSIQ 65.04 vs. 66.08). SMA is the only target in Table 5 that raises both perceptual metrics over that model (CLIPIQA 0.6788; MUSIQ 66.18), and it does so with PSNR and LPIPS within 0.02 dB and 0.0002 of it. Since each target was trained at a single weight, Section 4.3 now presents Table 5 as a comparison of operating points on this trade-off, among which SMA is the only one that improves perception at essentially unchanged fidelity.

(3) Stability. Variance across seeds and checkpoints was not measured, and Section 4.3 now states this.

AuthorsOpenReview ↗

Response to Reviewer UEga (3/4)

W2 & Q4: Comparison with TinySR and GenDR-Pix

R: (1) TinySR. We ran the released TinySR model [2] under the protocol of Tables 1–2, with the same GPU, timing script and metric code.

Table R3. TinySR and LiPDiT-SR under a common protocol (Appendix B.1).

Methodms/imageMemory (GiB)FLOPs (T)Params (M)RealSR PSNR / LPIPSRealSR CLIPIQA / MUSIQ
TinySR11.710.8470.974362.8324.840 / 0.39940.7125 / 68.264
LiPDiT-SR16.542.511.053605.0425.351 / 0.37920.6881 / 66.167

LiPDiT-SR gives higher fidelity than TinySR on every paired benchmark: higher PSNR and SSIM on all four (on RealSR, 25.351 vs. 24.840 dB and 0.7681 vs. 0.7224), and better LPIPS on RealSR and DRealSR. TinySR gives better no-reference scores on all five benchmarks. The two methods therefore occupy different points of the fidelity–perception trade-off.

At 512×512, TinySR is more efficient: it is faster (11.71 vs. 16.54 ms/image) and uses less memory (0.847 vs. 2.51 GiB), fewer FLOPs (0.974T vs. 1.053T) and fewer parameters (362.83M vs. 605.04M).

Table R4. Latency and peak memory across output resolutions (Appendix B.1, LiPDiT-SR at 512×512 as in Table 2). Following its official settings, TinySR runs tiled inference at the larger sizes (9 tiles at 1024×1024 and 25 tiles at 2048×2048); LiPDiT-SR processes the full image in one pass.

Output resolutionTinySR time (ms)LiPDiT-SR time (ms)FasterTinySR memory (GiB)LiPDiT-SR memory (GiB)
512×51211.7116.54TinySR, 1.41×0.8472.510
1024×102483.3340.74LiPDiT-SR, 2.05×1.3223.138
2048×2048250.77234.02LiPDiT-SR, 1.07×3.2245.648

TinySR is faster at 512×512 and uses less memory at every resolution. LiPDiT-SR is faster at 1024×1024 and 2048×2048, by 2.05× and 1.07×. At the larger sizes we report latency and memory only. Restoration quality above 512×512 is not evaluated, as the limitations of the submission note. Section 4.2 now summarizes the comparison with TinySR, including the effect of resolution, and Appendix B.1 gives the full results.

(2) GenDR-Pix. No code or weights are publicly available, so we compare with the results reported in its paper [3] on ImageNet-Test. Runtime is omitted because the hardware differs. Compute is given relative to AdcSR, which both papers measure, because [3] reports MACs and we report FLOPs.

Table R5. GenDR-Pix as reported in [3] and LiPDiT-SR on ImageNet-Test.

MethodParams (M)Compute (× AdcSR)PSNRSSIMNIQECLIPIQAMUSIQ
GenDR-Pix [3]8660.69 without PadCFG, 1.50 with25.490.72864.1890.716872.85
LiPDiT-SR605.040.9624.8920.74814.5540.742071.818

LiPDiT-SR has about 30% fewer parameters and is better in SSIM and CLIPIQA; GenDR-Pix is better in PSNR, NIQE and MUSIQ. Compute is of the same order. LPIPS is omitted because the two papers' values are not on the same scale: for the baselines both evaluate, our VGG-based LPIPS is about 1.3× the value reported in [3] (AdcSR: 0.3302 vs. 0.2432). Protocol differences between the two papers remain, so this comparison is indicative only.

(3) Related work. Section 2.3 now discusses GenDR-Pix as prior VAE-free one-step SR. It replaces the VAE encoder of a latent one-step model with pixel unshuffle and its decoder with pixel shuffle, whereas LiPDiT-SR starts from a native pixel-space backbone and prunes its semantic pathway.

W2: ImageNet-Test LPIPS relative to the external baselines

R: The LPIPS gap on ImageNet-Test comes from the pixel-space backbone and not from our compression: the full PixelDiT-SR has an LPIPS of 0.3421 and LiPDiT-SR has 0.3392, so removing 8 of the 26 patch blocks does not worsen it. On the same benchmark, the SSIM of LiPDiT-SR (0.7481) is exceeded only by CTMSR and SinSR, and its CLIPIQA (0.7420) only by TSD-SR and TinySR among the external baselines. Section 4.2 now reports the gap to the baselines of Table 1 (0.2854–0.3311) and states that LPIPS is computed with the VGG backbone.

Reviewer_3D7rOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LiPDiT-SR, a one-step SR model that works directly in pixel space and avoids VAE costs. It prunes the computation-heavy patch pathway while preserving the pixel pathway for detail reconstruction. Experiments show comparable visual quality and a 1.6× speedup for ×4 super-resolution.

Critical strengths
  1. The optimization target is clear and supported by end-to-end latency measurements.

  2. MDBS ranks blocks by their effect on downstream modulation signals, rather than pruning only by depth or position. SMA then constrains these signals during compression. Ablations show that MDBS performs better than consecutive and uniform selection on several perceptual metrics, while SMA gives small gains on some no-reference perceptual metrics. These results provide preliminary support for the design.

Critical weaknesses
  1. For the 18-block model, Table 2 reports 16.54 ms, while Table 6 reports 15.06 ms. The RealSR results also differ: Table 1 reports PSNR 25.351, LPIPS 0.3792, and CLIPIQA 0.6881 for Ours; Table 4 reports 25.373, 0.3800, and 0.6788 for LiPDiT-SR. The paper does not explain whether these differences come from input resolution, evaluation protocol, checkpoint, or random seed. Since efficiency and quality are central claims, these discrepancies affect the assessment of the main results.

  2. LiPDiT-SR is faster and uses fewer FLOPs than AdcSR, but it is not smaller or more memory-efficient. It has 605.04M parameters and uses 2.51 GiB of memory, compared with 456.07M parameters and 2.19 GiB for AdcSR. Thus, the results demonstrate latency and FLOPs improvements, but not consistent reductions in model size or memory across baselines.

  3. The qualitative comparison in Figure 3 does not clearly demonstrate the advantage of the proposed method. Generalization to real-world degradations remains a concern. On RealSR, the proposed method shows no clear advantage over strong baselines in either full-reference or no-reference metrics. The authors should evaluate and illusrate more challenging real-world degradations.

Questions for authors

How robust is the MDBS ranking to calibration data and joint pruning? What is the size and source of the calibration set? Is the block ranking stable across different calibration subsets or random seeds? Do the individual bypass scores reliably predict the effect of removing multiple blocks together? This is important for determining whether MDBS is more effective than generic pruning method.

5 comments
AuthorsOpenReview ↗

Response to the Review (1/3)

We thank the reviewer for the careful assessment and constructive suggestions. In response, we have unified the timing protocol and stated the training schedule of every reported model (Section 4.1; Tables 2 and 6); added the full PixelDiT-SR to the efficiency comparison (Table 2); added evaluations on DRealSR and ScreenSR (Appendix C); and corrected the description of MDBS and added stability and joint-removal analyses (Section 3.2; Appendix A). All changes to the main text are highlighted in blue, and the appendices are new. Below, we provide point-by-point responses to the weaknesses and questions raised in your review.

W1: Inconsistent latency and RealSR results across tables

R: The differences have two separate causes. Neither involves input resolution, evaluation data or random seed.

(1) Latency (16.54 ms in Table 2 vs. 15.06 ms in Table 6). Table 2 measured end-to-end time including CPU–GPU data transfer, whereas Table 6 measured GPU time only; the caption of Table 6 was inaccurate. All entries of Table 6 now use the protocol of Table 2.

Table R1. Inference time under the unified end-to-end protocol (revised Table 6).

Patch blocks26 (full)22181410
ms/image21.1618.5616.5414.3011.74

The 18-block entry matches Table 2 (16.54 ms). The time saved per four removed blocks is similar at every step (2.0–2.6 ms), and removing 8 of the 26 patch blocks lowers latency from 21.16 to 16.54 ms (21.8%). The percentages in Section 4.3 have been updated to match.

(2) RealSR quality (PSNR / LPIPS / CLIPIQA of 25.351 / 0.3792 / 0.6881 in Table 1 vs. 25.373 / 0.3800 / 0.6788 in Table 4). Both rows are the same architecture with identical retained blocks, at different training lengths. The model in Table 1 was trained for 20K iterations per pruning stage. Because of limited computational resources, every variant in Tables 4–6, including the LiPDiT-SR rows, was trained for 10K iterations per stage with all other settings unchanged, so that all ablation comparisons are made under identical conditions.

Table R2. LiPDiT-SR (18 patch blocks) on RealSR at the two training lengths.

ScheduleReported inPSNRLPIPSCLIPIQAMUSIQ
10K per stageTables 4–625.3730.38000.678866.18
20K per stageTable 1; Figs. 3–425.3510.37920.688166.17

The ablation conclusions therefore do not depend on this difference. Section 4.1 now states both schedules, and the 20K model is listed in Tables 4 and 6 for reference.

W2: Model size and memory relative to AdcSR

R: Our efficiency claim concerns latency and FLOPs, and on both LiPDiT-SR leads AdcSR: 16.54 vs. 26.29 ms/image and 1.053T vs. 1.099T FLOPs. LiPDiT-SR does have more parameters (605.04M vs. 456.07M) and uses more memory (2.51 vs. 2.19 GiB), but parameter count does not determine latency in this comparison: CTMSR has 171.52M parameters and takes 39.05 ms. The term "lightweight" refers to the reduction relative to the full PixelDiT-SR from which the model is derived, which the submitted Table 2 did not show. We have added that row.

Table R3. Full and lightweight models (revised Table 2).

Modelms/imageMemory (GiB)FLOPs (T)Params (M)
Full PixelDiT-SR (26 blocks)21.163.2361.352796.23
LiPDiT-SR (18 blocks)16.542.511.053605.04

Relative to the full model, LiPDiT-SR reduces latency by 21.8%, memory by 22.4%, FLOPs by 22.1% and parameters by 24.0%. Section 4.2 now states explicitly that the gain over the baselines is in latency and FLOPs.

AuthorsOpenReview ↗

Response to the Review (2/3)

W3: Qualitative comparison and generalization to real-world degradations

R: (1) Scope of the claim. LiPDiT-SR is designed to reach the quality range of one-step diffusion SR at a much lower latency, not to exceed the strongest baseline on each metric, and the results show that it does. On RealSR it ranks second in SSIM and fourth in LPIPS among the eight methods of Table 1, at 16.54 ms/image, faster than every compared method except TinySR. Section 4.2 now states this scope explicitly.

(2) Additional real-world benchmarks. We evaluated LiPDiT-SR, the full model and three baselines (AdcSR, TSD-SR and TinySR) on DRealSR [6] and ScreenSR [5] with the protocol used for RealSR.

Table R4. Results on DRealSR and ScreenSR (Appendix C).

DatasetMetricsAdcSRTSD-SRTinySRFull PixelDiT-SRLiPDiT-SR
DRealSRPSNR / LPIPS28.221 / 0.382327.823 / 0.390927.509 / 0.407428.008 / 0.374628.102 / 0.3819
DRealSRCLIPIQA / MUSIQ0.7058 / 66.2990.7251 / 66.6360.7089 / 65.3560.8177 / 62.3950.6993 / 62.221
ScreenSRPSNR / LPIPS22.230 / 0.336721.589 / 0.329821.369 / 0.354722.394 / 0.365322.286 / 0.3769
ScreenSRCLIPIQA / MUSIQ0.6773 / 72.0190.7179 / 73.9210.7086 / 72.9870.8473 / 72.0940.6715 / 70.428

On both benchmarks LiPDiT-SR has the highest SSIM of the five methods (0.8117 on DRealSR, 0.6542 on ScreenSR). Its PSNR is above all three external baselines on ScreenSR and within 0.12 dB of the best one on DRealSR. Its LPIPS is better than all three external baselines on DRealSR and behind them on ScreenSR. On the no-reference metrics it is below the external baselines on both sets, as on RealSR. Compared with the full model, pruning preserves PSNR and SSIM (within 0.11 dB and 0.004).

(3) Training data. LiPDiT-SR reaches this quality with ImageNet-only training and no text-to-image prior, whereas several of the compared baselines build on large pretrained text-to-image models. Section 5 notes that this bounds its no-reference scores on real-world images.

AuthorsOpenReview ↗

Response to the Review (3/3)

Q1: Robustness of the MDBS ranking to calibration data and joint pruning

R: (1) Calibration set. The calibration set contains 128 images sampled from the ImageNet training set. The same set is used at every stage.

(2) MDBS re-scores at every stage. The scores are recomputed at the start of each pruning stage, in our implementation and in every reported result. We thank the reviewer for this question. It led us to find that the submitted text described the procedure inaccurately as a single ranking computed once on the full model, and we have corrected the description. The procedure is:

  1. Load the EMA weights of the trained model of the preceding stage (the full 26-block model for the first stage).
  2. On the calibration set, bypass each of its remaining patch blocks in turn and compute the modulation deviation (Eq. 7) against that model's own modulation signals.
  3. Keep the highest-scoring blocks in their original order, initialize the next model from the preceding one, and train it.

Table R5. Blocks removed at each stage (index in the original 26-block pathway, starting from 1).

Stage26→2222→1818→1414→10
Blocks removed5, 19, 22, 236, 12, 18, 213, 7, 16, 202, 4, 14, 17

No reported number changes. Section 3.2 (Eqs. 5–8), Section 3.4, Section 4.1 and the caption of Fig. 2 have been corrected.

(3) Stability across calibration sets. We re-scored the full model and the 22-block model on seven fresh 128-image draws from the ImageNet training set, each with a new degradation seed and disjoint from the others and from the original calibration set.

Table R6. Stability of the MDBS selection (Appendix A.2).

StageMean Spearman correlation with the original ranking (minimum)Draws removing the same four blocksDraws sharing at least three of the four
26→220.991 (0.982)0 of 77 of 7
22→180.990 (0.982)2 of 77 of 7

The ranking is stable: its Spearman correlation with the original ranking is at least 0.98 in all 14 runs, and the removed set shares at least three of its four blocks with the original in every draw. The fourth block lies at the cut-off and changes with the draw. At 26→22, block 23 is replaced by block 4 in five draws and by block 21 in two; these three blocks are the fourth-, fifth- and sixth-lowest in the original ranking. At 22→18, block 18 is replaced by block 3 in five draws. This variation is confined to near-tied blocks. At 26→22, the scores of blocks 23, 21 and 4 lie within 1.9% of each other on the original calibration set and within 4.0% on the new draws; at 22→18, the scores of blocks 18 and 3 differ by 4.3% and 2.6%. The full analysis is in Appendix A.2.

(4) Joint removal. Because scores are recomputed after every stage, later selections are made on a model that has adapted to the earlier removals. Within a stage, four blocks are removed together on the basis of single-block scores. On the 22-block model, over 200 random four-block sets, the Spearman correlation between the summed single-block scores and the jointly measured deviation is 0.87, and the set selected by MDBS has a lower joint deviation than 99% of the random sets (198 of 200). The joint deviation is smaller than the sum of the single-block scores (0.140 against 0.341 for the MDBS set), so the effects are not additive in magnitude, but their ordering is preserved (Appendix A.3).

(5) Comparison with generic pruning criteria. On the same 22-block model, output sensitivity agrees with MDBS on two of the four removed blocks and feature sensitivity on three, so the criteria select different blocks (Appendix A.1). Before fine-tuning, the MDBS selection gives the highest PSNR and the lowest LPIPS of the four criteria. After training, MDBS outperforms position-based selection under the same budget and schedule (Table 4 of the paper).

In summary, the two discrepancies come from the timing protocol and the training length and are now resolved in the tables; the efficiency claim is limited to latency and FLOPs; and MDBS re-scores at every stage, with a stable ranking across calibration sets (Spearman correlation of at least 0.98) and three of the four removed blocks unchanged in every draw.

References

[5] R. Wu et al. VOSR: A Vision-Only Generative Model for Image Super-Resolution. CVPR 2026.

[6] P. Wei et al. Component Divide-and-Conquer for Real-World Image Super-Resolution. ECCV 2020.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Major Manuscript Revisions

We are grateful to all reviewers for their constructive feedback and valuable suggestions. The main revisions are summarized below. Changes to the main text are highlighted in blue, and Appendices A to C are new.

  • Method description. Corrected the description of MDBS. Block scores are recomputed on the trained model of the preceding stage at the start of every pruning stage; the submission described a single ranking on the full model. All reported results already use the stage-wise procedure, so no number changes. (Section 3.2, Eqs. 5–8; Section 3.4; caption of Fig. 2)
  • Training and evaluation details. Added the Stage II settings (20K iterations per stage, learning rate 5×10⁻⁶, no adversarial term, SMA weight schedule), the 10K schedule used for all ablations, the calibration set, and the LPIPS variant. (Section 4.1)
  • Consistency of reported results. Reported all latencies under one end-to-end protocol and stated the training schedule of the ablations, which resolves the differences between the main and ablation tables. (Section 4.1; Tables 2, 4 and 6; Section 4.3)
  • Block-selection baselines. Compared the blocks selected by MDBS with those selected by output sensitivity, feature sensitivity and a single ranking on the same model and calibration set. (Appendix A.1)
  • Analysis of MDBS. Added the blocks removed at each stage, the stability of the selection across calibration sets, and a joint-removal analysis. (Appendix A.2–A.3)
  • Additional comparisons. Added TinySR under the unified protocol, including the effect of output resolution, a discussion of GenDR-Pix with its reported results, and the full PixelDiT-SR in the efficiency table. (Sections 2.3 and 4.2; Table 2; Appendix B)
  • Real-world evaluation. Added results on DRealSR and ScreenSR. (Appendix C)
  • Claims and scope. Stated what "comparable quality" covers and where LiPDiT-SR trails the strongest methods, clarified which components are inherited from PixelDiT, and stated the architectural scope of the method. (Sections 3.1, 4.2 and 5)
AuthorsOpenReview ↗

Summary of the Discussion

We sincerely thank all reviewers and the AC for their time and constructive feedback. The discussion is summarized below.

  • Recognized strengths. The reviewers noted that the design is grounded in runtime profiling, since the patch-level pathway accounts for 77.2% of the latency (UEga, MVRg); that LiPDiT-SR delivers practical latency gains, running at 16.54 ms/image, about 1.6× faster than the fastest baseline in Table 2, while improving RealSR LPIPS over AdcSR (3D7r, UEga, MVRg); and that MDBS ties block selection to the modulation consumed by the pixel-level pathway, outperforms consecutive and uniform selection, and, like SMA, adds no inference cost (3D7r, UEga, MVRg).
  • Resolved concerns. We explained the differences between the main and ablation tables by the timing protocol and the training length, scoped the efficiency claim to latency and FLOPs, added DRealSR and ScreenSR results, clarified the attribution to PixelDiT, corrected the description of MDBS as stage-wise re-scoring, and showed that the MDBS ranking is stable across calibration sets and consistent under joint removal. Reviewer 3D7r found that most concerns were addressed and raised the recommendation to weak accept, and Reviewer MVRg noted that several concerns were well addressed. (Sections 3.1, 3.2, 4.1 and 4.2; Appendices A and C)
  • Additional comparisons. We added TinySR under a unified protocol, including the effect of output resolution, and GenDR-Pix with its reported results; Reviewer UEga found the additional analyses and the fine-tuned comparisons with consecutive and uniform selection helpful. (Sections 2.3 and 4.2; Appendix B)
  • Remaining points. How strongly the results support "comparable visual quality", and matched fine-tuned comparisons of MDBS and SMA with sensitivity-based selection and output distillation. The revised paper states these claims within the scope of the reported evidence, defining the quality claim and the metrics on which LiPDiT-SR trails, and presenting the supervision targets as operating points on the fidelity–perception trade-off. (Sections 4.2 and 4.3; Appendix A.1)

The revisions are summarized below, and the point-by-point responses are in each reviewer's thread.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.