- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper repeatedly applies an autoencoder to its own output (, ) and names the resulting trajectory Self-Reconstruction Dynamics (SRD). The paper also proposes to feed the reverse steps , together with , to a small CNN to predict the latent correction term (SRD-RR). The proposed method also illustrates a "x-free" variant that trains on the pseudo-label . On convolutional AEs and MNIST, FMNIST, CIFAR-10, SVHN, ImageNet100, and CelebA datasets, SRD-RR shows +2.0 dB results, and the x-free variant gains +1.7 dB.
Critical strengths
- Different variants are well studied and established. Ablation studies are well conducted.
- The proposed method shows low inference cost compared to per-sample latent optimization.
Critical weaknesses
- The core idea is classical, and the paper does not position the proposed method well against related work.
- M4, , is the first step of van Cittert's iterative deconvolution [1], with , using the AE in place of the blur operator. The paper only notes a resemblance to unsharp masking but omits this.
- M5, , is exactly one step of the gradient-free decoder inversion of [2] (Eq. 4), , with , and . The paper cites [2] but does not note this equivalence or compare with it.
- Closely related prior work is not well discussed:
- re-applying a codec to its own decoded output to enhance it (Nosratinia, 2001);
- re-applying the known degradation to the observation as self-supervision (ZSSR, Shocher et al., 2018);
- The setting is a toy one, and no practical benefit is demonstrated.
- Toy AEs. The main AEs have 3 conv stages, 16–128 channels, and at most 50 training epochs (App. C.2). This implies a 1024-dim latent for 784-pixel MNIST/FMNIST and only about 1.5× compression on ImageNet/CelebA, which is a very simple setting. No standard VAE, tokenizer, or learned codec is tested.
- On the pretrained RAE, M6 gains +0.12 dB PSNR, but LPIPS rises from 0.156 to 0.445 and rFID from 3.91 to 21.76.
Reference:
- P. H. van Cittert. Zum Einfluß der Spaltbreite auf die Intensitätsverteilung in Spektrallinien. II. Zeitschrift für Physik, 69:298–308, 1931. doi:10.1007/BF01391351
- Seongmin Hong, Suh Yoon Jeon, Kyeonghyun Lee, Ernest K. Ryu, Se Young Chun. Gradient-free Decoder Inversion in Latent Diffusion Models. NeurIPS, 2024.
- Aria Nosratinia. Enhancement of JPEG-Compressed Images by Re-application of JPEG. Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology, 27(1–2):69–79, 2001. doi:10.1023/A:1008167430544
- Assaf Shocher, Nadav Cohen, Michal Irani. "Zero-Shot" Super-Resolution using Deep Internal Learning. CVPR, 2018
Questions for authors
Please see the weakness.
Revised Manuscript Uploaded: Systematic Qualitative Evaluation, Additional Experiments, and Clarified Limitations
We sincerely thank the reviewer for the thoughtful and constructive feedback, which has been particularly valuable in helping us strengthen the experimental evaluation and clarify the scope of our contribution. We have now uploaded a substantially revised manuscript. In particular, we have made the following improvements to address the reviewer's concerns.
1. Systematic and less selectively biased qualitative evaluation (Appendices B, D, and E)
To address the concern about potentially cherry-picked reconstruction examples, we substantially expanded the qualitative evaluation. Rather than selecting visually favorable images, we now use a deterministic, rank-based selection strategy based on M6's PSNR improvement over M1. For the pretrained RAE and KL-VAE, we present examples at the top 10%, 25%, 75%, and 90% rank positions among 5,000 test images. We similarly broadened the ImageNet100 and CelebA examples to cover widely separated performance levels. Importantly, these examples include cases with limited improvements and perceptual degradation. We also provide full-test-set pixel–perceptual analyses (Figs. A12 and A22) to complement the selected examples with distribution-wide evidence.
2. Additional evaluation on pretrained autoencoder families and higher compression ratios (Sections 5.5–5.6; Appendices A.4, D, and E) We extended our experiments to pretrained RAE and KL-VAE models, which have substantially different architectures and latent representations, and evaluated convolutional AEs at compression ratios up to 24×. These experiments consistently show that M6 benefits from correctly paired SRD features relative to trajectory-free and shuffled controls. Appendix F further consolidates the evidence across 14 AE settings: M6 outperforms M6-LO in mean PSNR in all 14 settings and M6-Shuf in all eight settings with reported numerical comparisons.
3. Explicit analysis of perceptual quality and failure cases (Sections 5.6 and 6.3–6.4; Appendices D and E)
We now provide a more detailed analysis of the discrepancy between pixel fidelity and perceptual similarity, including PSNR, LPIPS, rFID, and qualitative examples. We explicitly acknowledge that MSE-trained M6 can improve PSNR while substantially degrading perceptual quality on pretrained AEs. The x-free variant better preserves perceptual quality but does not outperform M1 in PSNR on these models. These results clarify an important distinction: the presence of useful SRD features does not guarantee that the current refinement objective exploits them in a perceptually desirable manner.
4. Clarification of the contribution relative to M4 and the limitations of latent refinement (Sections 3.3 and 6.2–6.4)
We explicitly acknowledge M4 as a strong image-space alternative and discuss the limitations revealed by AE reapplication. We also clarify that improved reconstruction does not establish downstream benefits of refined latent representations, such as editing or interpolation. Our central contribution is therefore not a claim of universal superiority in final-image quality, but the demonstration that naturally generated self-reconstruction trajectories provide useful features for learned, decoder-compatible latent correction without per-sample test-time optimization. We greatly appreciate the reviewer's suggestions, which motivated these substantial revisions. We believe the revised manuscript now provides a more comprehensive, transparent, and balanced evaluation of both the strengths and limitations of SRD-RR. Thank you again for your careful consideration. We would be grateful for any further feedback or questions.