- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether a pretrained video generation model can perform video restoration without task-specific training. Its method gives the frozen generator two complementary references: the degraded video for motion, layout, and content, and a frame restored by an image expert for visual appearance. A frequency-gated data-consistency step then constrains the generated result to the observation primarily at low frequencies, allowing high-frequency detail to be synthesized. The paper evaluates the approach across eight benchmarks and four low-level video tasks. Its experiments emphasize perceptual quality, temporal behavior, and the respective contributions of appearance conditioning, motion conditioning, and frequency-gated consistency.
Critical strengths
-
The paper tests pretrained video generators directly on restoration and identifies two concrete failure patterns: retaining the input degradation and changing content that should be preserved.
-
The paper proposes a method that is built around those failures. The degraded video supplies motion and structure, a restored frame supplies appearance, and frequency-gated consistency anchors the output to the observation. Each part has a distinct purpose.
Critical weaknesses
-
A close baseline is absent. DiTVR is directly relevant to both the task and one of the method’s central mechanisms. Without it, the comparison set is broad across restoration systems but incomplete for judging the paper’s methodological advance.
-
The method performs strongly on no-reference visual-quality measures, while its UDM10 PSNR is 21.68 dB, below several restoration baselines. This supports a claim about perceptually appealing restoration more directly than one about uniformly more faithful reconstruction.
Questions for authors
-
How much of the gain comes from the image expert? And how are results affected by the choice of the first frame? What happens if it is occluded, unusually degraded, unrepresentative of later frames, or contains objects that subsequently change?
-
What evidence isolates a latent restoration capability in the frozen video generator? The appearance reference introduces restored information from an external model. Which controlled experiment separates the generator’s contribution from propagation of that reference?
-
Which parts of the pipeline require benchmark-specific information?
4 comments
Response to Reviewer 48sC (2/2)
Q1: Contribution of the image expert, and the effect of the first frame.
A3: Swapping the expert (UDM10; Table 8 of the revision): only frame 0 changes; the no-expert row is the generator alone, without the consistency step.
| First frame | MUSIQ | DOVER | PSNR | LPIPS | DISTS |
|---|---|---|---|---|---|
| none (degraded video only) | 27.9 | 0.176 | 19.86 | 0.494 | 0.235 |
| Nano Banana Pro (Table 1) | 65.3 | 0.661 | 22.86 | 0.348 | 0.154 |
| LucidFlux (open-source) | 67.4 | 0.622 | 21.53 | 0.346 | 0.158 |
| PiSA-SR ×4 (open-source, Table 1) | 62.1 | 0.538 | 23.76 | 0.289 | 0.148 |
| ODTSR (open-source) | 58.8 | 0.510 | 23.98 | 0.303 | 0.149 |
| TADSR (open-source) | 56.1 | 0.473 | 24.01 | 0.335 | 0.177 |
| HarnessIR + NBP | 59.5 | 0.581 | 23.78 | 0.325 | 0.157 |
| ground-truth frame (oracle) | 60.6 | 0.578 | 26.16 | 0.122 | 0.064 |
It shows that without an expert the generator does not restore, and with any of the six experts, four of them open-source, it does; the output inherits the character of the expert, and a more accurate frame 0 translates directly into a more faithful output (26.16 dB with the ground-truth frame). With every expert, Gen4LV carries the expert's texture through the whole clip along the input's motion (Fig. 10), at an E_warp of 1.44–1.89, against 3.50 when PiSA-SR restores every frame (Table 6).
Content that frame 0 does not show (Fig. 11, Appendix B). Frame 0 sets the appearance but is not warped forward, so content that it does not show is generated from the degraded video. In a YouHQ40 clip a pedestrian walks into the view after frame 0 (out-of-view content), and in a UDM10 clip a plant hidden behind a person in frame 0 appears later (disocclusion). Gen4LV draws both as in the ground truth (the pedestrian seen from behind, with cap and backpack; the plant on the shelf), whereas AnyV2V, which carries frame 0 forward, smears the pedestrian and omits the plant, and SeedVR2-7B turns the pedestrian towards the camera.
Q2: A controlled experiment that separates the generator's contribution from propagation of the reference.
A4: Three controls give exactly the same expert frame to alternative downstream solvers, so the differences measure what the generator adds (Tables 6–7, Figs. 5 and 9):
- the per-frame expert with temporal smoothing keeps much of the flicker or removes the detail with it: PiSA-SR on every frame has E_warp 3.50, Deep Video Prior lowers it to 0.97 but MUSIQ falls to 32.8, while Gen4LV with the same expert reaches 1.55 with the best LPIPS and DISTS (0.289 and 0.148);
- other video models through their native channels (Wan2.1-VACE, LTX-2) copy the degraded video or regenerate the clip and depart from the observation;
- first-frame-guided propagation (AnyV2V) is the closest control:
The same frame 0 given to AnyV2V or to Gen4LV (UDM10, 32 frames; Table 6 of the revision). Bold: the better of the two.
| Frame 0 | Method | PSNR | LPIPS | DISTS | DOVER | E_warp×10³ |
|---|---|---|---|---|---|---|
| NBP | AnyV2V | 20.25 | 0.410 | 0.183 | 0.654 | 2.12 |
| NBP | Gen4LV | 22.86 | 0.348 | 0.154 | 0.661 | 1.64 |
| PiSA-SR ×4 | AnyV2V | 20.66 | 0.367 | 0.178 | 0.598 | 2.15 |
| PiSA-SR ×4 | Gen4LV | 23.76 | 0.289 | 0.148 | 0.538 | 1.55 |
It shows that, given the same frame 0, Gen4LV is better on PSNR, LPIPS, DISTS and E_warp with both experts; on all three benchmarks it gains 1.4–3.1 dB PSNR, while AnyV2V reaches a higher DOVER in four of the six benchmark–expert pairs. Beyond these metrics, AnyV2V follows the motion but alters the content of the scene (Fig. 5), and it smears or omits content that frame 0 does not contain (Q1). Together, these controls isolate what the generator adds beyond the reference frame: temporal consistency, lower LPIPS and DISTS, and content that the frame does not contain.
Q3: Which parts of the pipeline require benchmark-specific information.
A5: None of the tuned quantities uses a benchmark. The operator family is a task-level assumption: nominal ×4 bicubic downsampling for super-resolution and the exact grayscale operator for colorization (σ_y = 1/255, one 8-bit quantization step); archival film and generated video use no operator and no consistency step. For super-resolution, σ_y, g, c and N are calibrated once on 30 clips disjoint from all benchmarks (Appendix C, Tables 12–13) and used unchanged on every VSR benchmark; the calibration selects the same g, c and N as the submission, and Table 1 is re-run with the calibrated σ_y. The steps of the N corrections follow from the sampler's noise schedule (Appendix H). The image expert, the captioner and the 50 sampling steps are fixed per task.
Revision: Sec. 4.2, Fig. 5, Appendices A–C and H.
We hope these responses address your concerns, and we would be glad to discuss any remaining questions.
Official Comment by Reviewer 48sC
Thank you to the authors for their detailed responses and additional experiments. I believe that most of my concerns have been addressed, and after reading the responses to the other reviewers, I remain willing to maintain my current score.
Dear Reviewer,
Thank you for your positive feedback and for taking the time to carefully consider our responses and additional experiments. We appreciate your acknowledgment that most of your concerns have been addressed.
Thank you again for your time and constructive feedback!
Response to Reviewer 48sC (1/2)
We sincerely thank Reviewer 48sC for the careful review and the pointer to DiTVR, and for recognizing that the paper tests pretrained video generators directly on restoration, identifies two concrete failure patterns, and builds a method around them in which each part has a distinct purpose. Below, W and Q follow the numbering of the review, and A1, A2, … are our answers in order; all revisions are marked in red in the revised PDF.
W1: DiTVR as a close baseline.
A1: We now cite and discuss DiTVR (Sec. 2) and add a DiTVR-style row to Table 5. DiTVR builds on an image diffusion transformer, links frames with trajectory-aware attention, and enforces consistency by a DDNM-style range-space correction of the wavelet low band at every sampling step. Its code is not public, so we emulated this consistency term inside our pipeline: a hard projection onto the observation at every step, with gain 1, under a ×4 average-pooling operator, for which it is exactly the projection of the wavelet LL band (the flow-guided averaging is not emulated).
DiTVR-style consistency against our frequency-gated correction (UDM10, NBP expert, the configuration of Table 1; Table 5 of the revision). LR-PSNR compares the downsampled output with the degraded input; the ground truth itself scores 25.8 dB.
It shows that the hard projection raises PSNR by only 0.37 dB (7/10 clips), and by 0.03 dB at our ten steps, but its LR-PSNR far exceeds the ground truth's: the output reproduces the degraded input, noise included, and LPIPS, DISTS, MUSIQ and DOVER are worse than with our correction on 10/10 clips. The difference is where the observation enters: in DiTVR the consistency term is the only path for the observation, so it must be strong; in Gen4LV the observation is already a condition, so the correction only fixes a residual low-band error, softly and at a few steps.
Revision: Sec. 2, Table 5.
W2: Perceptually appealing restoration rather than uniformly more faithful reconstruction.
A2: We agree, and the revision states the claim as perceptual restoration with a controllable fidelity trade-off (Sec. 1). Part of the gap was geometric (a shrink of about 1% of every frame), now removed by registration without ground truth (Appendix D): UDM10 PSNR rises from 21.68 to 22.86 dB, and to 23.76 dB with the open-source PiSA-SR ×4 expert, which matches or exceeds FlashVSR-Full in PSNR, SSIM and DISTS on all three benchmarks, while DOVE and SeedVR2-7B keep higher PSNR and SSIM (Table 1). In Fig. 3 (DOVER against DISTS), Gen4LV with NBP sets the perceptual end on all three benchmarks and SeedVR2-7B the fidelity end. A blinded user study (Sec. 4.3, Appendix E) separates the two: 24 raters from diverse backgrounds, none of them an author, compared Gen4LV (NBP or PiSA-SR ×4) with DOVE, SeedVR2-7B and FlashVSR-Full on 30 VSR clips, judging visual appeal from the two outputs alone and fidelity to the reference after the ground truth is revealed.
Preference for Gen4LV, (wins + ½ × cannot-tell) / n, with 95% bootstrap intervals; 50% means no preference (Table 14 of the revision).
It shows that the advantage of Gen4LV is largest in appearance, as the reviewer suggests, while in fidelity to the ground truth it is preferred or at parity against every baseline, including those that lead in PSNR: with NBP, 78% against DOVE and 67% against SeedVR2-7B; with PiSA-SR ×4, 69% and 45% (parity).
Revision: Sec. 1, Sec. 4.3, Fig. 3, Appendices D–E.