- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper brings classical sub-pixel accumulation into learned latent features for continuous space-time video super-resolution, using flow-based residual trajectories and direct LR-to-HR splatting.
Critical strengths
-
The motivation is clear. By predicting residual motion and directly splatting low-resolution features at their estimated sub-pixel positions onto a high-resolution grid, the method preserves sub-pixel positional information and follows the classical principle of precise alignment and accumulation in multi-frame super-resolution.
-
The method achieves leading results on GoPro and Adobe240, with convincing quantitative and qualitative improvements over strong baselines. The discussion of failure cases also helps clarify its limitations.
Critical weaknesses
-
Two key ablations are missing: upsample-then-splat versus direct LR-to-HR splatting, and full-trajectory prediction versus flow-residual prediction. Without these comparisons under matched training settings, it is difficult to isolate the contribution of the proposed design choices.
-
Sub-pixel accumulation and B-spline motion modeling have both been explored in prior work. The novelty is relatively modest, making stronger ablations important for establishing the value of the combined design.
-
The architectural presentation is difficult to follow. A detailed pipeline diagram and a side-by-side schematic comparison with prior methods would improve readability.
-
I could not find a code attachment. Do the authors plan to release the code and pretrained models?
-
Metric reporting is inconsistent across experiments. Please provide PSNR, SSIM, and LPIPS consistently across the main comparison tables to support a more complete assessment. In table 1, only PSNR and LPIPS are reported. However, in table 2, PSNR and SSIM are reported.
Questions for authors
None.
5 comments
Follow-up to Reviewer KND3: residual vs full-displacement comparison
The residual-on-flow vs full-displacement comparison you asked for is now in the general thread: the residual is better on average by a small margin and markedly better on large motion (+0.18 / +0.37 dB PSNR-Y on the 10% of frames with the largest motion). The same post explains that the Table 6 reproduced in our reply was trained with a wrong version of the trajectory implementation; we will share the updated table on Wednesday 18th.
The updated version of the Table 6 reproduced in our reply to you is now in the general thread, re-trained with the released implementation. The conclusions are unchanged: our splat is better than fusing at low resolution then PixelShuffle (+0.24 / +0.36 dB PSNR-Y on interpolated frames, GoPro / Adobe240) and than rounding to the high-resolution pixel (+0.15 / +0.24 dB), and linearly lifting first matches it at higher cost.
Response to Reviewer KND3
We thank the reviewer for the positive assessment and the precise requests.
Two missing ablations, with the encoder, decoder, data and training schedule fixed (new Appendix D.1, Table 6):
Novelty. We agree that sub-pixel splatting and B-spline motion exist individually. What is new is how the trajectories are obtained and where the samples land. First, and most important, our B-spline trajectories are estimated from features propagated over the whole sequence (Sec. 3.1). MoTIF and BF-STVSR also splat along learned trajectories at intermediate times, but estimate them without multi-frame context (BF-STVSR from two frames, where acceleration is not observable); to our knowledge, ours is the first C-STVSR method to estimate explicit trajectories from multi-frame context. Appendix A shows that the trained estimator relies on this context, and that given the same four frames it outperforms QVI's closed-form quadratic model on every video. Figures 4, 5 and 9 illustrate its effect under complex motion. Second, we splat each low-resolution feature directly at its precise sub-pixel position, whereas MoTIF and BF-STVSR lift each frame with a nonlinear implicit representation first. The controls requested by 2eK7 test this placement: our splat outperforms fusing at LR then PixelShuffle, and rounding to the HR grid, in PSNR-Y on every dataset and frame type (Table 6 of the revised paper, reproduced below).
PSNR-Y ↑ / LPIPS ↓, ×4 space / ×8 time on GoPro and Adobe240, key frames (τ = 0) and interpolated frames, ordered by how much of each sample's sub-pixel position survives until fusion; * = mean of two seeds; best of the controlled variants in bold, as in Table 6 (the last row is the efficiency control).
Architecture presentation. New Figure 2 is a detailed pipeline diagram: for one interval, it shows the model's intermediate outputs (optical flow, spline residual and visibility, splatted features, output) and which steps run once per interval and once per output time. New Appendix F (Table 13) gives the side-by-side comparison, in table form, with prior methods (SPMC, MoTIF, BF-STVSR, GS-STVSR, V³): from which frames each estimates motion, which trajectory model it predicts and on which grid, and where it fuses the frames at an output time.
Code. Anonymized inference code and pretrained weights are in the supplementary material.
Metric consistency. Tables 1 and 2 now report PSNR-Y, SSIM-Y and LPIPS for every method with public weights (published values otherwise). Ours is best in PSNR-Y and SSIM-Y in every setting of Tables 1 and 2, and in LPIPS on GoPro and Adobe240 in Table 1. At ×12 space, BF-STVSR has the best LPIPS; in the crops of these outputs (new Figure 8), ours is higher in PSNR-Y and shows more detail (e.g. the window frames of the facade in the first crop), so we read the LPIPS ranking at this scale with caution (Appendix E.4).
We thank the reviewer for the constructive feedback, which improved the paper.