- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Sketch2Pixel, a VAE-free image restoration diffusion model that recovers a low-frequency structural representation earlier than the full image, allowing progressively restored structure to guide detail generation. In detail, it proposes Dual-Time Joint Diffusion, which jointly models the image and a wavelet-derived low-frequency structure with separate diffusion times, a shared DiT backbone, and two prediction branches. Besides, it introduces a structure-forcing schedule to control relative denoising progress. Experiments show advancement of the proposed model in performance and efficiency trade-offs.
Critical strengths
- The key concept of "Sketch Before Pixel" is intuitive and telling, and the overall performance surpasses existing methods to some extent.
Critical weaknesses
- The innovation is incremental and most concepts came from latent forcing. The previous work has already validated a similar schema that models pixels and auxiliary representations (including downsampled pixels) in the generation task. More precisely, it seems an extension of latent forcing in the restoration task, providing little insight.
- The motivation is intuitive but vague, lacking broader pre-validation. The structure-pre schema makes sense, but it just selects the wavelet operation without explaining "why wavelet should be selected", nor providing proof that it outperforms other structural or semantical representations.
- Despite mentioning previous VAE-free work in the image restoration task, e.g., PixelIR and PixRestore, it provides none detailed comparison or discussion.
Questions for authors
See Weaknesses.
9 comments
Official Response by Authors – Part 2
Weakness 3: Comparison with VAE-Free Restoration Methods
Answer: Thanks for your valuable comment. As suggested, we have added quantitative and qualitative comparisons with PixRestore [2] under a consistent evaluation protocol. Since PixRestore was trained on part of ScreenSR and its training/test split differs from ours, we exclude direct comparisons on this dataset to avoid potential overlap between its training data and our test data.
(1) Quantitative Comparison. As shown in Table R2, PixRestore achieves higher PSNR and SSIM, but this fidelity advantage comes with poorer perceptual quality, as reflected by all four no-reference metrics. Specifically, on RealSR and LSDIR, our method improves MUSIQ by 12.06 and 7.51 points and MANIQA by 0.1991 and 0.2170, respectively. These results support the perceptual quality advantages of our method, while highlighting the trade-off between visual realism and structural fidelity.
(2) Qualitative Comparison. Compared with PixRestore, our method more faithfully preserves fine textures and text structures while recovering more realistic details, as illustrated by the examples in Figure 5 of the revised manuscript.
We did not include a comparison with PixelIR because its source code was not publicly available. The complete comparison results and accompanying discussion have been added to the revised manuscript.
Table R2. Quantitative comparison with PixRestore on RealSR and LSDIR. Bold indicates the better result.
| Dataset | Method | PSNR↑ | SSIM↑ | LPIPS↓ | NIQE↓ | MUSIQ↑ | MANIQA↑ | DeQA↑ | FID↓ |
|---|---|---|---|---|---|---|---|---|---|
| RealSR | PixRestore | 27.41 | 0.7695 | 0.1657 | 6.14 | 56.28 | 0.3166 | 3.00 | 66.74 |
| RealSR | Sketch2Pix-OS (Ours) | 25.63 | 0.7171 | 0.2843 | 5.11 | 68.34 | 0.5157 | 3.58 | 107.01 |
| LSDIR | PixRestore | 21.42 | 0.5558 | 0.2741 | 4.72 | 63.38 | 0.3228 | 3.66 | 39.81 |
| LSDIR | Sketch2Pix-OS (Ours) | 21.15 | 0.5399 | 0.2139 | 4.25 | 70.89 | 0.5398 | 3.91 | 42.07 |
References:
[2] Lingchen Sun et al. PixRestore: Unified Image Restoration via Pixel Diffusion Transformer. arXiv preprint arXiv:2608.16793, 2026.
We hope these explanations clear up your confusion. We have provided clearer explanations and experimental analysis in the revised version. If you have any further concerns, we would be pleased to provide more detailed clarification.
Official Comment by Authors
Dear Reviewer Seub,
Thank you for the careful evaluation of our paper. We want to know whether our previous response has relieved or successfully addressed any of your initial concerns. If not and you have any further questions, please feel free to share them with us so that we can use this last chance to address them. We want to thank you again for your valuable time and efforts. We will be more than happy to receive your response and resolve any further problems.
Best regards,
The Authors
Official Response by Authors – Part 1
We would like to sincerely thank the reviewer for the detailed and constructive comments on our work. We take every comment seriously and hope our response can address the reviewer’s concerns. If there are any remaining questions, we are more than happy to address them.
Weakness 1: Distinctions from Latent Forcing and Insights into Real-World Super-Resolution
Answer: Thank you for your insightful comment. We would like to clarify our contributions in terms of Motivation, Method, and New Insight for Real-SR.
(1) Motivation. Latent Forcing [1] studies the coordination of auxiliary representations and pixel generation. In contrast, we investigate how to restore and exploit the imperfect structural cues already present in degraded observations to constrain detail synthesis. We therefore treat structure as an explicit recovery objective that continuously updates during joint denoising, providing progressive guidance for detail reconstruction.
(2) Method. Compared with the representation configurations in Latent Forcing that rely on pretrained encoders, we adopt a computationally efficient, learning-free wavelet transform to construct spatially aligned low-frequency structural targets. Conditioned on the LR input, we jointly restore structure and the full image while independently controlling their denoising progress, forming a self-contained pixel-space restoration framework without an external encoder.
(3) New Insight for Real-SR. We find that jointly restoring structure and coordinating its progress with detail synthesis improves Real-SR reconstruction compared with conditioning on fixed structural cues from degraded LR images. Specifically, Figure 6 demonstrates the benefits of joint structural recovery over fixed LR structural conditioning. Figure 4 and Table 4 further show that moderate structural precedence improves reconstruction, whereas excessive precedence degrades quality. Motivated by these findings, we propose the Sketch Before Pixel paradigm, which explicitly coordinates structural recovery and detail synthesis within joint denoising to balance structural fidelity and visual realism.
Weakness 2: Motivation for Wavelet-Based Structural Representation
Answer: Thank you for your constructive comment. We would like to clarify that we choose wavelets to construct a spatially aligned structural target that captures coarse-scale layout and can be explicitly restored. Wavelet decomposition explicitly separates low-frequency structure from high-frequency details, allowing the structural branch to guide the coarse spatial layout and the full-image branch to recover fine details, consistent with our design of coordinating structural recovery and detail synthesis through their relative denoising progress. Moreover, this representation requires neither learning nor an external pretrained encoder and incurs little computational overhead.
Following your suggestion, we have added an ablation study replacing the wavelet-based structural target with Gaussian low-pass filtering. As shown in Table R1, the LL-based wavelet representation achieves better PSNR, SSIM, LPIPS, MUSIQ, MANIQA, and DeQA on ScreenSR, whereas Gaussian low-pass filtering performs better in NIQE and FID. These results provide empirical support for our choice of a wavelet-based structural representation. The corresponding results and discussion have been added to the revised manuscript.
Table R1. Ablation of structural representations on ScreenSR. Bold indicates the better result.
References:
[1] Alan Baade et al. Latent Forcing: Reordering the Diffusion Trajectory for Pixel-Space Image Generation. International Conference on Machine Learning (ICML), 2026.