- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes MatchVR, which reframes reference-guided generative video restoration as a spatiotemporal reference alignment problem. The method first establishes reliable correspondences between low-quality target frames and high-quality reference frames through degradation-robust feature matching. It then resamples the matched reference content into each target frame’s layout in the latent space, and injects it as per-frame local guidance into a DiT backbone via temporal relocation and attention masking. In addition, it distills a full-reference teacher into a single-reference student to improve reference utilization under the single-reference setting.
Critical strengths
- The motivation is clear, and the problem is well defined. The authors reveal a “reference utilization gap” through diagnostic experiments: the single-reference curve degrades as the target frame moves away from the reference frame, whereas the idealized full-reference setting remains high and flat. The target-to-reference attention also peaks near the reference frame. This analysis is intuitive and convincing.
- The method design is reasonable, and the logical pipeline is complete. Degradation-robust matching, selective spatial alignment, temporal relocation, attention masking, and reference distillation work together to transform a single reference frame into per-frame guidance that is spatially aligned and temporally local.
- The efficiency overhead is small. Table 4 shows that adding the aligned reference reduces FPS from 2.77 to 2.70 and increases peak GPU memory from 19.9 GB to 20.1 GB, indicating that the locally visible aligned tokens do not introduce substantial inference overhead.
Critical weaknesses
- The diagnosis of the reference utilization gap is not rigorous enough. Figure 1 uses a low-motion, occlusion-free SPMCS subset, but temporal distance also reduces true overlap due to motion, occlusion, and new content. Without reporting matching coverage, occlusion/new-content ratios, and their relation to PSNR, it remains unclear how much of the performance drop comes from underutilization rather than the reference no longer covering the target.
- The method relies heavily on high-quality references. Table 3 and Figure 5 show that output quality is strongly correlated with reference quality. The paper does not sufficiently examine the robustness of the method when the reference quality is limited or when the target regions do not appear in the reference.
- The paper does not demonstrate the accuracy of LQ matching using DINOv3 features. It would be helpful to compare and visualize LQ–Ref matching against GT–Ref matching to show the accuracy of the current matching stage.
Questions for authors
- Can you report matching coverage, the proportion of valid aligned tokens, and occlusion/new-content ratios at different temporal distances, and relate them to PSNR gains? When the aligned reference is empty, does MatchVR degenerate into blind restoration?
- What is the essential difference between MatchVR and existing reference-based VSR methods? RefVSR and RefVSR++ already perform reference alignment and propagation. Is MatchVR’s degradation-robust matching plus latent warping merely transferring these mechanisms to a diffusion/DiT backbone? Can you provide a flow-warp or deformable-attention baseline under the same backbone?
5 comments
Responses to Reviewer 1NtJ(Part II)
[W2]Reference-quality Study
We add a 7-level reference-degradation study on SPMCS: across all levels, the PSNR(23.05–25.92 dB) is always higher than SeedVR2-3B (22.19 dB), and using the first frame restored by SeedVR2-3B itself as the reference still gives 22.40 dB. The discussion has been expanded in the Influence of Reference Quality paragraph of Appendix D (Table 15 and Table 16).
| Reference source | Reference vs. GT first frame (PSNR / SSIM) | Output PSNR ↑ | Output SSIM ↑ | Output LPIPS ↓ |
|---|---|---|---|---|
| L0 clean | - / - | 25.51 | 0.7919 | 0.1053 |
| L1 mild | 32.31 / 0.9277 | 25.92 | 0.7823 | 0.1406 |
| L2 light | 29.10 / 0.8636 | 25.83 | 0.7706 | 0.2136 |
| L3 moderate | 24.89 / 0.6734 | 24.23 | 0.6635 | 0.3641 |
| L4 heavy | 22.21 / 0.4824 | 23.13 | 0.5753 | 0.4564 |
| L5 severe | 20.27 / 0.3867 | 23.07 | 0.5858 | 0.4364 |
| L6 extreme | 18.93 / 0.3552 | 23.05 | 0.5936 | 0.4203 |
| First frame restored by SeedVR2 | 21.77 / 0.6042 | 22.4 | 0.6117 | 0.3007 |
| SeedVR2-3B no reference (Table 2) | – | 22.19 | 0.6367 | 0.3194 |
We also add a partial-overlap evaluation on 10 high-motion clips from REDS, with the GT first frame as the reference:
| Method | Reference | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|---|
| MatchVR | GT first frame | 23.69 | 0.6527 | 0.2675 |
| MatchVR | None (blind restoration) | 22.92 | 0.6307 | 0.4273 |
| SeedVR2-3B | None | 23.04 | 0.6491 | 0.2764 |
| FlashVSR-Full | None | 21.43 | 0.5653 | 0.273 |
The matching coverage decreases monotonically with temporal distance (96.6% at frame 0 down to 19.1% at frame 90), yet under strong motion and partial overlap the reference still brings consistent gains over blind restoration, and remains better than SeedVR2-3B and FlashVSR-Full. The experiment has been added to the Large Motion and Partial Overlap paragraph of Section 5.5 "Reference Quality and Robustness Analysis" in the revised manuscript (Table 8 and Fig. 6(b)).
Responses to Reviewer 1NtJ(Part III)
[W3]LQ–GT Matching Robustness
We add Fig. 7 in the updated manuscript to visualize the matching results using GT and degraded target frames. We further quantify matching robustness under degradation using the protocol in Appendix B, where coordinate agreement measures the fraction of matches within one feature cell of the ground-truth correspondences:
| Dataset | Target frame type | Matching coverage | Valid token ratio | Coordinate agreement rate | confidence |
|---|---|---|---|---|---|
| SPMCS | GT (ideal upper bound) | 0.9204 | 0.9204 | 1 | 0.8428 |
| SPMCS | Mild | 0.9251 | 0.9251 | 0.8539 | 0.8442 |
| SPMCS | Moderate | 0.9084 | 0.9084 | 0.7686 | 0.836 |
| SPMCS | Severe | 0.8116 | 0.8116 | 0.6678 | 0.8084 |
| YouHQ40 | GT (ideal upper bound) | 0.9075 | 0.9075 | 1 | 0.8391 |
| YouHQ40 | Mild | 0.9083 | 0.9083 | 0.8437 | 0.8398 |
| YouHQ40 | Moderate | 0.8975 | 0.8975 | 0.7272 | 0.8356 |
| YouHQ40 | Severe | 0.8468 | 0.8468 | 0.6154 | 0.8173 |
Under degradation the matching still maintains very high spatial coverage (81.2–92.5 %), and even under severe degradation the coordinate agreement rate remains above 60 %. A lower agreement rate does not necessarily mean wrong matches: in videos the same region often has multiple plausible matching candidates (especially background regions), and the final restored video benefits from the matching results, as corroborated by the full-reference results in Table 2 and Table 6. The comparison has been added to Appendix B.4 "LQ-Ref vs. GT-Ref Matching" of the revised manuscript (Table 11 and Fig. 7).
[Q2]Optical Flow Comparison
This question helps clarify the positioning of our method. RefVSR and RefVSR++ align and propagate reference features within regression-based video super-resolution frameworks. In contrast, MatchVR differs in that: (i) our matcher is explicitly trained to be robust to severe degradations, with confidence-based filtering of unreliable matches; (ii) our alignment aims to improve reference utilization by spatially aligning reference latents, relocating them to each target time step, and restricting their visibility to the corresponding frame; (iii) restoration is performed by a one-step generative DiT. Our key distinction lies in making reliable reference information more accessible to the generative model.
We further add a comparison between optical flow and our matcher under the same evaluation protocol and train a RAFT-based restoration variant by replacing our matcher with RAFT:
| Dataset | Method | Mild | Moderate | Severe | PSNR | SSIM | LPIPS |
|---|---|---|---|---|---|---|---|
| SPMCS | MatchVR | 0.9023 | 0.8929 | 0.8702 | 25.51 | 0.7919 | 0.1053 |
| SPMCS | RAFT | 0.9725 | 0.7429 | 0.639 | 24.54 | 0.7694 | 0.1127 |
| YouHQ40 | MatchVR | 0.9064 | 0.8984 | 0.8825 | 24.94 | 0.7458 | 0.1360 |
| YouHQ40 | RAFT | 0.9756 | 0.8991 | 0.7877 | 24.30 | 0.7278 | 0.14 |
Our matcher drops much less in accuracy than optical flow as the degradation becomes more severe, which is what enables the rejection behavior above. Moreover, MatchVR outperforms the RAFT-based variant on all three restoration metrics. The comparison has been added to the Matching Accuracy paragraph of Appendix B.3 (Table 12), and the complete accuracy–coverage results under multiple thresholds are given in Table 9 and Table 10 of Appendix B.
Responses to Reviewer 1NtJ(Part I)
To reviewer 1NtJ: We sincerely thank the reviewer for the careful reading and constructive comments. Responses to specific comments are given below, which are also added to the revised paper.
[W1, Q1]Matching Reliability Analysis
We add the matching statistics at different temporal distances on the low-motion SPMCS subset (GT target frames, η=0.70):
The valid aligned token ratio is identical to the coverage under this setting, so the two rows coincide. Even at the largest temporal distance, 81.4% of the target tokens still receive valid matches from the reference under the specified confidence threshold, so the observed quality degradation cannot be attributed to "the reference no longer covering the target". Moreover, the decay rates of the two quantities are clearly mismatched: from frame 1 to frame 30 the coverage decreases by only 0.18, whereas PSNR drops by 2.12 dB, so most of the quality loss cannot be explained by the shrinking matchable area. The statistics and discussion have been added to Section 3 (Motivation) of the revised manuscript (Table 1).
We also add an unrelated-reference experiment on SPMCS:We feed the GT first frame of a different video as the reference. When the reference frame is completely unrelated, the valid matching coverage drops to 6.7 % and the restoration quality smoothly falls back to the level of its own blind restoration, confirming that erroneous reference information is effectively rejected. The results have been added to the Unrelated References paragraph of Section 5.5 "Reference Quality and Robustness Analysis" (Table 7 and Fig. 6(a)).