- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes RefPLC for reference-based super-resolution. It combines latent reference recovery with state-conditioned pixel modulation and reports improvements over the adapted PiD baseline on several benchmarks.
Critical strengths
1.The method addresses both reference alignment and state-dependent reference usage.
2.The paper includes external benchmarks, module ablations, and a reference-condition study.
Critical weaknesses
-
The methodological novelty is limited. ROLT and SCVM mainly combine existing correspondence, residual conditioning, and spatial modulation techniques.
-
The ablation does not demonstrate clear synergy. SCVM alone achieves better SSIM and LPIPS than the complete model.
-
The qualitative evidence is weak. In Figure J.9, outputs under different references and Null appear nearly identical.
-
The computational overhead is unclear. Runtime, memory usage, and FLOPs are not reported despite repeated decoder evaluations.
-
The internal training set and Single717 benchmark lack sufficient details about data sources, reference selection, and public availability.
Questions for authors
-
Why does SCVM alone achieve better SSIM and LPIPS than the complete model?
-
How different are the outputs under Top, Medium, Distant, and Null references?
-
What are the inference latency, memory usage, and FLOPs of RefPLC?
-
How are the Top, Medium, and Distant references selected?
-
What are the data sources of the internal training set and Single717 benchmark?
2 comments
Reminder for Rebuttal
Dear Reviewer,
The authors have provided their rebuttal. Please leave your comments asap.
Your AC
Dear Reviewer rJ4S,
Thank you for your questions. We address each point using the revised manuscript.
[W1] Methodological contribution
RefPLC connects two restoration residuals (Sec. 4; Table I.1). Before PiD's condition projection, ROLT transports sharp-minus-blurred VAE detail, applies a target-conditioned shared-map difference, and writes a gated, RMS-limited latent increment. At the RGB-velocity output, SCVM differences shared-decoder responses to reference and zero evidence at the same state, then applies a learned spatial gain. ROLT's recovered condition determines the base prediction seen by SCVM, connecting evidence preparation to its state-dependent use. Figs. I.1–I.4 trace component effects on signals and GT error; Appendix K measures off/on corrections and learned/unit gains at fixed states.
[Q1] SCVM-only versus Full
Table 4 uses the same training protocol. ROLT changes zL to zL within the same VAE space; SCVM's base prediction and correction consequently depend on this condition. Differences in PSNR (dB)/SSIM/LPIPS-VGG are:
SCVM improves all three metrics under both conditions. SCVM-only has higher SSIM (0.7151 vs. 0.7079) and lower LPIPS (0.4382 vs. 0.4463), while Full gains 0.13 dB PSNR and 2.49 MUSIQ over SCVM-only and leads all eight no-reference metrics among the four configurations. Sec. 5.3 distinguishes these final rankings from the gain of adding SCVM under each condition.
Fig. I.4 fixes weights, target, reference, and noise. On the same 64×64 arm–grille crop, Full obtains 22.086 dB, exceeding H bypass, caption scale 1, global-only, and local-only controls by 0.093/0.098/0.080/0.126 dB. Figs. I.1–I.3 trace the individual operations.
[Q2] Reference-dependent output differences
Across 90 targets, Top/Medium/Distant improve mean PSNR over the same model's Null output by 0.7614/0.6380/0.0758 dB; all reduce LPIPS (Tables 3, G.7). Fig. J.9 now shows common enlarged crops, crop PSNR, and GT-error maps.
In J.10, Top improves map-crop PSNR over Null by 5.731 dB, lowers MSE from 119.76 to 32.00 (73.28%), and changes the output by a mean absolute 6.22 8-bit RGB levels. In J.11, Top improves gong crop/full-image PSNR by 2.302/1.042 dB and lowers crop MSE from 134.98 to 79.45 (41.14%). MSE-to-GT measures accuracy; MAE-to-Null locates output changes. Each heatmap row shares a scale across conditions, revealing measurable local changes within similar overall compositions.
[Q3] Computational cost
Table F.2 uses one H20, inference batch 1, 25 steps, guidance 5, BF16 mixed precision, and an FP32 sampler, covering multiple aspect ratios.
At 2K, added latency is 3.747 s (8.89%) and peak memory increases by 1.16 GiB. Pair-level evidence is cached; both SCVM decoder passes share one base prediction at each velocity evaluation. CUDA-synchronized timing includes transfer, preprocessing, text embedding, and restoration; it excludes model loading, offline caption generation, and image I/O. Peak allocated memory includes weights. Counted FLOPs cover convolutions, matrix multiplication, and attention (two operations per multiply–add), excluding normalization and most elementwise operations.
[Q4] Reference tiers
Video candidates are quality-screened frames sampled at 1 fps from the GT's 4K source video; non-video references use action/viewpoint/lighting edits with subject/text checks. Overall-similarity and local-correspondence scores fuse DINOv2/VGG16 with weights 0.7/0.3. Top prioritizes overall similarity, Medium local correspondence, and Distant is the least globally similar member of the selected triplet. GT, LQ, LQ caption, and seed are fixed. Each run uses one reference with its caption; Null omits it. Single717 video targets use Top only (Sec. 3; G.1, G.6).
[Q5] Sources and availability
The datasets derive from purchased image/video resources authorized for AI training, covering people, products, scene text, and video frames. Training contains 97,589 GT–Ref pairs from 64,427 targets; Single717 has 717 targets with one reference each. Neither is currently public. Upon acceptance, we will release the training/test data permitted for redistribution, with pairings, tier annotations, and construction scripts.