acceptodds
Under review as a conference paper at ICLR 2027

Video Generation Models are Zero-Shot Low-Level Vision Solvers

Abstract

Video generation models learn rich priors over appearance and motion, enabling instruction-guided synthesis and editing. Here we investigate a central question: can these priors also support low-level vision without task-specific fine-tuning? We show that a single frozen video generator can serve as a zero-shot solver for a spectrum of low-level video tasks, including blind video super-resolution, colorization, archival film restoration, and restoration of AI-generated video. Our framework, Gen4LV, casts these tasks as reference-conditioned generation with complementary visual cues. That is, the degraded video guides motion, layout, and content, while a single frame restored by an image editing expert guides appearance. However, this conditioning scheme enables the generator to synthesize detailed video but does not, by itself, ensure fidelity to the input. For tasks requiring high fidelity, we further introduce frequency-gated data consistency to constrain the output using reliable observation information.In blind super-resolution, for instance, the gate restricts observation-based corrections to low frequencies, improving structural alignment while preserving generated texture and enabling inference-time control over the perception–fidelity trade-off. Across eight benchmarks spanning four tasks, Gen4LV delivers strong no-reference video quality, matching or surpassing specialized models in multiple restoration settings without task-specific fine-tuning. These results demonstrate that pretrained video generators can serve as generalists for low-level vision, pointing toward a future in which diverse visual tasks are addressed by specifying instructions, references, and constraints at inference time.

Primary area:
generative models
Keywords:
video generation, low-level vision, zero-shot learning, video restoration, generative priors

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_48sCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether a pretrained video generation model can perform video restoration without task-specific training. Its method gives the frozen generator two complementary references: the degraded video for motion, layout, and content, and a frame restored by an image expert for visual appearance. A frequency-gated data-consistency step then constrains the generated result to the observation primarily at low frequencies, allowing high-frequency detail to be synthesized. The paper evaluates the approach across eight benchmarks and four low-level video tasks. Its experiments emphasize perceptual quality, temporal behavior, and the respective contributions of appearance conditioning, motion conditioning, and frequency-gated consistency.

Critical strengths
  1. The paper tests pretrained video generators directly on restoration and identifies two concrete failure patterns: retaining the input degradation and changing content that should be preserved.

  2. The paper proposes a method that is built around those failures. The degraded video supplies motion and structure, a restored frame supplies appearance, and frequency-gated consistency anchors the output to the observation. Each part has a distinct purpose.

Critical weaknesses
  1. A close baseline is absent. DiTVR is directly relevant to both the task and one of the method’s central mechanisms. Without it, the comparison set is broad across restoration systems but incomplete for judging the paper’s methodological advance.

  2. The method performs strongly on no-reference visual-quality measures, while its UDM10 PSNR is 21.68 dB, below several restoration baselines. This supports a claim about perceptually appealing restoration more directly than one about uniformly more faithful reconstruction.

Questions for authors
  1. How much of the gain comes from the image expert? And how are results affected by the choice of the first frame? What happens if it is occluded, unusually degraded, unrepresentative of later frames, or contains objects that subsequently change?

  2. What evidence isolates a latent restoration capability in the frozen video generator? The appearance reference introduces restored information from an external model. Which controlled experiment separates the generator’s contribution from propagation of that reference?

  3. Which parts of the pipeline require benchmark-specific information?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 48sC (1/2)

We sincerely thank Reviewer 48sC for the careful review and the pointer to DiTVR, and for recognizing that the paper tests pretrained video generators directly on restoration, identifies two concrete failure patterns, and builds a method around them in which each part has a distinct purpose. Below, W and Q follow the numbering of the review, and A1, A2, … are our answers in order; all revisions are marked in red in the revised PDF.

W1: DiTVR as a close baseline.

A1: We now cite and discuss DiTVR (Sec. 2) and add a DiTVR-style row to Table 5. DiTVR builds on an image diffusion transformer, links frames with trajectory-aware attention, and enforces consistency by a DDNM-style range-space correction of the wavelet low band at every sampling step. Its code is not public, so we emulated this consistency term inside our pipeline: a hard projection onto the observation at every step, with gain 1, under a ×4 average-pooling operator, for which it is exactly the projection of the wavelet LL band (the flow-guided averaging is not emulated).

DiTVR-style consistency against our frequency-gated correction (UDM10, NBP expert, the configuration of Table 1; Table 5 of the revision). LR-PSNR compares the downsampled output with the degraded input; the ground truth itself scores 25.8 dB.

ConsistencyPSNRLR-PSNRLPIPSDISTSMUSIQDOVERE_warp×10³
none21.5823.160.3570.15565.20.6721.73
frequency-gated, g = 0.5, 10 steps (ours)22.8625.880.3480.15465.30.6611.64
DiTVR-style, every step23.2332.700.6640.38224.30.0593.00
DiTVR-style, at our 10 steps22.8931.970.5950.31739.10.0773.36

It shows that the hard projection raises PSNR by only 0.37 dB (7/10 clips), and by 0.03 dB at our ten steps, but its LR-PSNR far exceeds the ground truth's: the output reproduces the degraded input, noise included, and LPIPS, DISTS, MUSIQ and DOVER are worse than with our correction on 10/10 clips. The difference is where the observation enters: in DiTVR the consistency term is the only path for the observation, so it must be strong; in Gen4LV the observation is already a condition, so the correction only fixes a residual low-band error, softly and at a few steps.

Revision: Sec. 2, Table 5.

W2: Perceptually appealing restoration rather than uniformly more faithful reconstruction.

A2: We agree, and the revision states the claim as perceptual restoration with a controllable fidelity trade-off (Sec. 1). Part of the gap was geometric (a shrink of about 1% of every frame), now removed by registration without ground truth (Appendix D): UDM10 PSNR rises from 21.68 to 22.86 dB, and to 23.76 dB with the open-source PiSA-SR ×4 expert, which matches or exceeds FlashVSR-Full in PSNR, SSIM and DISTS on all three benchmarks, while DOVE and SeedVR2-7B keep higher PSNR and SSIM (Table 1). In Fig. 3 (DOVER against DISTS), Gen4LV with NBP sets the perceptual end on all three benchmarks and SeedVR2-7B the fidelity end. A blinded user study (Sec. 4.3, Appendix E) separates the two: 24 raters from diverse backgrounds, none of them an author, compared Gen4LV (NBP or PiSA-SR ×4) with DOVE, SeedVR2-7B and FlashVSR-Full on 30 VSR clips, judging visual appeal from the two outputs alone and fidelity to the reference after the ground truth is revealed.

Preference for Gen4LV, (wins + ½ × cannot-tell) / n, with 95% bootstrap intervals; 50% means no preference (Table 14 of the revision).

Expertvs.Appeal (no reference)Fidelity to the reference
NBPDOVE96% [88, 100]78% [65, 90]
NBPSeedVR2-7B90% [79, 98]67% [51, 81]
NBPFlashVSR-Full94% [83, 99]81% [65, 93]
NBPall three93% [87, 98]75% [64, 85]
PiSA-SR ×4DOVE77% [62, 89]69% [55, 83]
PiSA-SR ×4SeedVR2-7B53% [38, 70]45% [29, 61]
PiSA-SR ×4FlashVSR-Full68% [53, 82]68% [53, 80]
PiSA-SR ×4all three66% [55, 76]61% [50, 71]

It shows that the advantage of Gen4LV is largest in appearance, as the reviewer suggests, while in fidelity to the ground truth it is preferred or at parity against every baseline, including those that lead in PSNR: with NBP, 78% against DOVE and 67% against SeedVR2-7B; with PiSA-SR ×4, 69% and 45% (parity).

Revision: Sec. 1, Sec. 4.3, Fig. 3, Appendices D–E.

AuthorsOpenReview ↗

Response to Reviewer 48sC (2/2)

Q1: Contribution of the image expert, and the effect of the first frame.

A3: Swapping the expert (UDM10; Table 8 of the revision): only frame 0 changes; the no-expert row is the generator alone, without the consistency step.

First frameMUSIQDOVERPSNRLPIPSDISTS
none (degraded video only)27.90.17619.860.4940.235
Nano Banana Pro (Table 1)65.30.66122.860.3480.154
LucidFlux (open-source)67.40.62221.530.3460.158
PiSA-SR ×4 (open-source, Table 1)62.10.53823.760.2890.148
ODTSR (open-source)58.80.51023.980.3030.149
TADSR (open-source)56.10.47324.010.3350.177
HarnessIR + NBP59.50.58123.780.3250.157
ground-truth frame (oracle)60.60.57826.160.1220.064

It shows that without an expert the generator does not restore, and with any of the six experts, four of them open-source, it does; the output inherits the character of the expert, and a more accurate frame 0 translates directly into a more faithful output (26.16 dB with the ground-truth frame). With every expert, Gen4LV carries the expert's texture through the whole clip along the input's motion (Fig. 10), at an E_warp of 1.44–1.89, against 3.50 when PiSA-SR restores every frame (Table 6).

Content that frame 0 does not show (Fig. 11, Appendix B). Frame 0 sets the appearance but is not warped forward, so content that it does not show is generated from the degraded video. In a YouHQ40 clip a pedestrian walks into the view after frame 0 (out-of-view content), and in a UDM10 clip a plant hidden behind a person in frame 0 appears later (disocclusion). Gen4LV draws both as in the ground truth (the pedestrian seen from behind, with cap and backpack; the plant on the shelf), whereas AnyV2V, which carries frame 0 forward, smears the pedestrian and omits the plant, and SeedVR2-7B turns the pedestrian towards the camera.

Q2: A controlled experiment that separates the generator's contribution from propagation of the reference.

A4: Three controls give exactly the same expert frame to alternative downstream solvers, so the differences measure what the generator adds (Tables 6–7, Figs. 5 and 9):

  • the per-frame expert with temporal smoothing keeps much of the flicker or removes the detail with it: PiSA-SR on every frame has E_warp 3.50, Deep Video Prior lowers it to 0.97 but MUSIQ falls to 32.8, while Gen4LV with the same expert reaches 1.55 with the best LPIPS and DISTS (0.289 and 0.148);
  • other video models through their native channels (Wan2.1-VACE, LTX-2) copy the degraded video or regenerate the clip and depart from the observation;
  • first-frame-guided propagation (AnyV2V) is the closest control:

The same frame 0 given to AnyV2V or to Gen4LV (UDM10, 32 frames; Table 6 of the revision). Bold: the better of the two.

Frame 0MethodPSNRLPIPSDISTSDOVERE_warp×10³
NBPAnyV2V20.250.4100.1830.6542.12
NBPGen4LV22.860.3480.1540.6611.64
PiSA-SR ×4AnyV2V20.660.3670.1780.5982.15
PiSA-SR ×4Gen4LV23.760.2890.1480.5381.55

It shows that, given the same frame 0, Gen4LV is better on PSNR, LPIPS, DISTS and E_warp with both experts; on all three benchmarks it gains 1.4–3.1 dB PSNR, while AnyV2V reaches a higher DOVER in four of the six benchmark–expert pairs. Beyond these metrics, AnyV2V follows the motion but alters the content of the scene (Fig. 5), and it smears or omits content that frame 0 does not contain (Q1). Together, these controls isolate what the generator adds beyond the reference frame: temporal consistency, lower LPIPS and DISTS, and content that the frame does not contain.

Q3: Which parts of the pipeline require benchmark-specific information.

A5: None of the tuned quantities uses a benchmark. The operator family is a task-level assumption: nominal ×4 bicubic downsampling for super-resolution and the exact grayscale operator for colorization (σ_y = 1/255, one 8-bit quantization step); archival film and generated video use no operator and no consistency step. For super-resolution, σ_y, g, c and N are calibrated once on 30 clips disjoint from all benchmarks (Appendix C, Tables 12–13) and used unchanged on every VSR benchmark; the calibration selects the same g, c and N as the submission, and Table 1 is re-run with the calibrated σ_y. The steps of the N corrections follow from the sampler's noise schedule (Appendix H). The image expert, the captioner and the 50 sampling steps are fixed per task.

Revision: Sec. 4.2, Fig. 5, Appendices A–C and H.

We hope these responses address your concerns, and we would be glad to discuss any remaining questions.

Reviewer_48sCOpenReview ↗

Official Comment by Reviewer 48sC

Thank you to the authors for their detailed responses and additional experiments. I believe that most of my concerns have been addressed, and after reading the responses to the other reviewers, I remain willing to maintain my current score.

Dear Reviewer,

Thank you for your positive feedback and for taking the time to carefully consider our responses and additional experiments. We appreciate your acknowledgment that most of your concerns have been addressed.

Thank you again for your time and constructive feedback!

Reviewer_HgYaOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates whether a frozen video generator can solve low-level video tasks in a zero shot manner. Their findings suggests that direct prompting fails due to degradation or drifting from the input. They provide results on four tasks across several benchmarks.

Critical strengths

The fact that one frozen generator handles four important tasks with no fine-tuning is practical and useful.

Implementation details are listed in detail, and ablations are provided.

Critical weaknesses

Zero-shot claim seems to be a bit stretch since σ_y is estimated from paired data and c and g are tuned on UDM10 dataset.

Evaluations seems to be quite small on 10 clips.

No limitations section is listed, and reliance on Nano banana raises concerns about reproducibility.

Questions for authors

How much the performance of restoration comes from nano banana vs video prior is not clear. Can authors share some insights?

4 comments
AuthorsOpenReview ↗

Response to Reviewer HgYa (1/2)

We sincerely thank Reviewer HgYa for the constructive review, and for recognizing that one frozen generator handling four tasks without fine-tuning is practical and useful, and that the implementation details and ablations are thorough. W1–W3 and Q1 follow the order of the review, and A1–A4 are our answers; revisions are in red in the revised PDF.

W1: The zero-shot claim, given that σ_y is estimated from paired data and c and g are tuned on UDM10.

A1: "Zero-shot" refers to the generator, whose weights are never trained or fine-tuned. We agree that the hyperparameters must not depend on test data: the revision calibrates σ_y, g, c and N once on 30 clips from DOVE's training set, disjoint from every benchmark (Appendix C), and the calibration selects the same g, c and N as the submission, so the submitted results do not rest on tuning on UDM10. σ_y (12.61/255) is measured on these clips, not on a test set; Table 1 is re-run with it.

Gain g and cutoff c on the calibration set at N = 10: change against no correction on the same clip, both experts pooled (30 clips × 2 experts); all nine settings in Table 12 of the revision.

(g, c)ΔPSNRΔLPIPSΔDISTSΔDOVERΔMUSIQ
(0.5, 0.03)+1.13−0.010−0.001−0.005+0.13
(0.5, 0.06), ours+1.66−0.015+0.001−0.005+0.47
(0.75, 0.06)+2.11−0.015+0.004−0.013+0.33
(0.5, 0.12)+2.35−0.007+0.009−0.021+0.22
(0.75, 0.12)+2.97+0.006+0.019−0.058−1.04

It shows that the knee lies at (0.5, 0.06): PSNR +1.66 dB, on every clip with both experts, at a DOVER change of −0.005; one step further, the LPIPS gain stops or halves and DOVER drops 2.6–4× more. N = 10 is calibrated on the same clips (Table 13). The same values raise PSNR by 1.3 dB on UDM10 (Table 5) and, on SPMCS and YouHQ40, which never entered any choice, on every clip (by 0.98 and 1.62 dB on average). Colorization uses σ_y = 1/255 for its exact operator.

Revision: Appendix C (Tables 12–13, Fig. 7); Table 1 re-run with the calibrated σ_y.

W2: Small evaluation (10 clips).

A2: The VSR comparison already covers 80 clips (UDM10 10, SPMCS 30, YouHQ40 40). The revision extends the black-and-white film and generated-video sets from 10 to 30 and 40 clips with the same method, settings and baselines (Tables 3–4): Gen4LV remains first on DOVER, FastVQA, Q-Align and CDC on the 30 film clips, and has the highest Q-Align and the highest preference rates of both VLM judges on the 40 generated clips. It also adds a blinded user study on 30 VSR clips with 24 raters (Sec. 4.3, Appendix E), the disjoint calibration set, and the AnyV2V and video-generator controls on SPMCS and YouHQ40 (Table 7).

Revision: Tables 3, 4 and 7; Sec. 4.3; Appendices C and E.

AuthorsOpenReview ↗

Response to Reviewer HgYa (2/2)

W3: No limitations section; reproducibility with Nano Banana.

A3: The revision adds a Limitations paragraph (Sec. 5): the perceptual gains come at some cost in fidelity, and inference is computationally demanding, since we run the original generator without multi-GPU parallelism, quantization or few-step distillation (cost per clip in Table 18). The pipeline also works with an open-source expert: with PiSA-SR ×4 instead of Nano Banana Pro (NBP), same pipeline and settings, PSNR rises on all three benchmarks (UDM10 22.86 → 23.76, SPMCS 20.18 → 21.45, YouHQ40 21.68 → 22.41 dB) and DOVER falls, and no method in Table 1 has a lower E_warp; Table 1 reports both configurations.

Revision: Sec. 5; Table 1; Table 18.

Q1: How much of the performance comes from Nano Banana and how much from the video prior.

A4: Both are necessary, for different things. We vary each with the other fixed.

Same generator, different first frames (UDM10; Table 8 of the revision): only frame 0 changes; the no-expert row is the generator alone, without the consistency step.

First frameMUSIQDOVERPSNRLPIPSDISTS
none (degraded video only)27.90.17619.860.4940.235
Nano Banana Pro (Table 1)65.30.66122.860.3480.154
LucidFlux (open-source)67.40.62221.530.3460.158
PiSA-SR ×4 (open-source, Table 1)62.10.53823.760.2890.148
ODTSR (open-source)58.80.51023.980.3030.149
TADSR (open-source)56.10.47324.010.3350.177
HarnessIR + NBP59.50.58123.780.3250.157
ground-truth frame (oracle)60.60.57826.160.1220.064

It shows that the video prior alone does not restore, that the expert sets the perceptual tier, and that a more accurate frame 0 gives a more faithful output.

Same NBP frame, other video generators through their native channels, without our consistency step (UDM10; Table 6 and Fig. 5 of the revision). "PSNR to input" compares the output with the degraded input; high values mean that the input is copied.

Backbone · channelMUSIQPSNRPSNR to inputBehaviour
Wan2.1-VACE-14B · expert as frame 0 of the control video20.822.9929.2copies the degraded video
LTX-2 · expert as frame 0 + retake67.115.2016.1regenerates; departs from the observation (dispersion 23.5 px)
MiniMax-H3 · Gen4LV (Table 1, NBP)65.322.8623.9restores

It shows that the expert frame alone does not make a video model restore: Wan2.1-VACE and LTX-2 copy the degraded video (their other channels alike, MUSIQ 19.3–21.5) or regenerate the clip and depart from the observation, whereas the same frame through the interface of Gen4LV restores. The expert alone, in turn, restores frames but not the video: PiSA-SR on every frame flickers (E_warp 3.50), while the same expert used once, for frame 0 of Gen4LV, reaches 1.55, with LPIPS 0.289 instead of 0.362 and 0.6 dB higher PSNR (Table 6). The generator also supplies content that frame 0 does not contain, such as a pedestrian who enters the view or a plant hidden in frame 0 (Fig. 11, Appendix B).

Revision: Appendix A (Tables 6 and 8, Figs. 9–10), Appendix B (Fig. 11), Fig. 5.

We hope these responses address your concerns, and we would be glad to discuss any remaining questions.

Thanks a lot for the detailed rebuttal. I increased my score to Weak Accept.

Dear Reviewer HgYa,

Thank you for your time and for raising your score. We sincerely appreciate your constructive feedback.

Best regards,

The Authors

Reviewer_aUgoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Gen4LV, a training-free framework that aims to apply a frozen pretrained video generation model to four low-level vision tasks: blind video super-resolution, video colorization, archival film restoration, and AI-generated video restoration. Its main contributions include two techniques. First, a motion–appearance decoupled conditioning strategy uses the degraded video to preserve motion and structural information, while leveraging the first frame restored by an image restoration model to guide appearance generation. Second, a frequency-gated data consistency mechanism constrains structural deviations while preserving synthesized high-frequency details. The authors evaluate Gen4LV on eight benchmarks and compare it with general-purpose video generation models, zero-shot restoration methods, and task-specific models using metrics for perceptual quality, fidelity, and temporal consistency. The experimental results demonstrate that Gen4LV achieves competitive performance on multiple no-reference perceptual quality metrics and delivers competitive results on some full-reference metrics.

Critical strengths
  1. Novel problem setting: The paper explores using a frozen pretrained video generation model to solve multiple low-level vision tasks without task-specific fine-tuning, offering a potentially generalizable restoration framework.

  2. Effective technical design: The combination of motion–appearance decoupled conditioning and frequency-gated data consistency addresses two key challenges in generative restoration: degradation retention and deviation from the observed content.

Critical weaknesses
  1. Limited methodological novelty: The proposed components build on existing techniques, including reference-conditioned generation and plug-and-play data consistency.

  2. Dependence on external models: The framework relies on an image restoration expert (Nano Banana Pro) and a VLM (Qwen3-VL-4B), making it difficult to attribute the performance gains solely to the pretrained video generator and introducing additional computational costs.

  3. Limited validation on real-world data: Several evaluations use small datasets, including only ten clips for archival film and AI-generated video restoration.

  4. Trade-off between perceptual quality and fidelity: Although Gen4LV achieves strong results on several no-reference perceptual metrics, it underperforms specialized methods on some full-reference metrics, particularly PSNR and SSIM.

Questions for authors

1. How does Gen4LV distinguish reliable structural information from corrupted content when applying frequency-gated data consistency under unknown real-world degradations? Can the authors provide further evidence of its robustness to different degradation types and mismatches in the assumed forward operator?

2. Since Gen4LV relies on an external image restoration model and a VLM for appearance guidance and semantic conditioning, how much does each component contribute to the final performance? Could the authors provide ablation studies to clarify whether the improvements primarily come from the proposed conditioning strategy or the capabilities of these auxiliary models?

6 comments
AuthorsOpenReview ↗

Response to Reviewer aUgo (1/3)

We sincerely thank Reviewer aUgo for the constructive review, and for recognizing the novel setting of solving several low-level video tasks with a frozen pretrained video generator, and the effectiveness of combining decoupled conditioning with frequency-gated data consistency against degradation retention and deviation from the observation. Below, W and Q follow the numbering of the review, and A1, A2, … are our answers in order; all revisions are marked in red in the revised PDF.

W1: Methodological novelty.

A1: We agree that both ingredients exist. Beyond them, the paper contributes a systematic study of a question that prior work leaves open: whether a frozen, instruction-following video generator can perform low-level video tasks, where the output must agree with the observation pixel by pixel and frame by frame. Used directly, it cannot: four state-of-the-art generators retain the degradation or depart from the observation (Sec. 4.2). Conditioned through its reference interface, one frozen generator handles four tasks on eight benchmarks, and the controls below show that where the observation enters, not the two ingredients alone, is what makes it restore.

Where the observation enters: prior zero-shot methods against Gen4LV.

Prior zero-shot methodsGen4LV
Generative modelimage diffusion prior with temporal propagation (DiffIR2VR-Zero, VISION-XL); video consistency model (LVTINO); image DiT (DiTVR)frozen instruction-following video generator
How the observation entersonly through the consistency (likelihood) term, which must anchor the content on its own and presumes a known operatorthrough the generator's reference interface: the degraded video for motion and layout, the expert frame for appearance
Role of consistencyprimary anchor, applied at (almost) every step; DiTVR projects the wavelet low band onto the observation at every stepsoft, noise-weighted correction of the low band, at 10 of 50 steps

Each alternative we tested fails, which shows that this specific design matters:

  • either reference alone retains the degradation or departs from the observation (Table 5);
  • other video models given the same restored frame copy the degraded video or regenerate the clip (Table 6, Fig. 5);
  • first-frame propagation (AnyV2V) flickers more and alters the content (Table 6, Fig. 5);
  • the expert on every frame flickers or blurs (Table 6, Fig. 9);
  • a DiTVR-style hard projection used as the anchor copies the degradation into the output (Table 5).

Revision: Sec. 2 (related work rewritten in these terms), Sec. 4.2.

W2 & Q2: Dependence on the image expert and the VLM, and the contribution of each.

A2: Both auxiliary models are needed, for different reasons, and neither replaces the generator: the expert sets the perceptual tier, the scene description must come from the restored frame, and the generator contributes temporal consistency and the content that frame 0 does not contain. Each is called once per clip.

Swapping the expert: same pipeline and settings, only frame 0 changes; the no-expert row is the generator alone, without the consistency step (UDM10, the configuration of Table 1; Table 8 of the revision).

First frameMUSIQDOVERPSNRLPIPSDISTS
none (degraded video only)27.90.17619.860.4940.235
Nano Banana Pro (Table 1)65.30.66122.860.3480.154
LucidFlux (open-source)67.40.62221.530.3460.158
PiSA-SR ×4 (open-source, Table 1)62.10.53823.760.2890.148
ODTSR (open-source)58.80.51023.980.3030.149
TADSR (open-source)56.10.47324.010.3350.177
HarnessIR + NBP59.50.58123.780.3250.157
ground-truth frame (oracle)60.60.57826.160.1220.064

It shows that without an expert the generator does not restore, and with any of the six experts, four of them open-source, it does; no-reference quality follows the expert's perceptual tier and full-reference quality its fidelity, and with the ground-truth frame the same pipeline reaches 26.16 dB and DISTS 0.064. Given the same expert, the generator adds what the frame cannot: PiSA-SR on every frame flickers (E_warp 3.50, against 1.55 for Gen4LV with the same expert; Table 6), and content that frame 0 does not show is generated from the degraded video (Fig. 11).

AuthorsOpenReview ↗

Response to Reviewer aUgo (2/3)

(A2, continued) The scene description is written once by the captioner from the expert's frame 0 and describes content, not degradation. Replacing it, with everything else fixed (UDM10; Table 9 of the revision):

Scene description (NBP expert)MUSIQ↑DOVER↑PSNR↑LPIPS↓
from the expert frame (Table 1)65.30.66122.860.348
from the degraded frame46.70.42923.790.408
generic sentence52.50.50723.090.409
none53.00.51423.070.406

The effect depends on the clip: on 5–7 of the 10 clips the output hardly changes, while on the others the generator falls back, fully or in part, to the degraded video (MUSIQ lower by 9–51; PSNR rises, as for any output closer to the blurred input). A description of the degraded frame is no better than none: it describes the degradation as scene content (e.g. "scattered colorful specks that look like digital noise or static"), and the output keeps it. Cost: the expert and the captioner take 7–25 s per clip together, out of 594–617 s in total (Table 18); nothing is optimized for speed.

Revision: Appendix A (Tables 6, 8, 9; Figs. 9–10), Appendix B (Fig. 11), Appendix H.

W3: Limited validation on real-world data.

A3: We extended both main-text sets with the method, settings and baselines of the submission: MovieLQ-BW from 10 to 30 clips (10 public-domain films, 3 each) and AIGC-LQ from 10 to 40 clips (5 generators, 8 each). For VSR, Table 1 already covers 80 clips, and the revision adds a blinded user study on 30 clips (Appendix E) and a disjoint 30-clip calibration set (Appendix C).

Gen4LV against the best baseline on each metric (Tables 3 and 4 of the revision).

SetMetricGen4LVBest baseline
MovieLQ-BW (30 clips)DOVER↑0.6260.468 (ColorMNet)
FastVQA↑0.8220.592 (ColorMNet)
Q-Align↑4.503.37 (ColorSurge-Tiny)
CDC×10³↓0.2440.425 (ColorSurge-Tiny)
AIGC-LQ (40 clips)Q-Align↑4.574.53 (FlashVSR-Full)
HPSv3↑8.518.55 (Vivid-VR)
realness / structure, GPT-6.1-Sol↑0.82 / 0.870.80 / 0.76 (Vivid-VR)
realness / structure, Gemini-3.8-Flash↑0.92 / 0.930.67 / 0.68 (Vivid-VR)

It shows that the black-and-white film results hold on 30 clips (Gen4LV's Q-Align is above all three baselines on every clip), and on generated video Gen4LV remains in the leading group with Vivid-VR and FlashVSR-Full, with the highest Q-Align and, under both judges, the highest preference rates. Its CLIP agreement with the source is second only to SeedVR2-7B; its motion deviation (EPE) is the largest, which we read as a reference rather than a target, since part of the source motion in generated video is itself an artifact.

Revision: Sec. 4.4, Tables 3–4, Appendices F–G.

W4: Trade-off between perceptual quality and fidelity.

A4: We agree that the task-specific methods remain ahead in PSNR and SSIM; the revision states the claim as perceptual restoration with a controllable fidelity trade-off (Sec. 1).

  • After the correction. Part of the gap was geometric (a shrink of about 1% of every frame), now removed by registration without ground truth (+0.76–1.18 dB PSNR with NBP, higher SSIM; Appendix D). With the open-source PiSA-SR ×4 expert, Gen4LV matches or exceeds FlashVSR-Full in PSNR, SSIM and DISTS on all three benchmarks, while DOVE and SeedVR2-7B keep higher PSNR and SSIM; all three are trained in-distribution, whereas Gen4LV is zero-shot (Table 1).
  • Position. Lower PSNR and SSIM are expected at the perceptual end of the perception–distortion trade-off (Blau & Michaeli, 2018); in Fig. 3 (DOVER against DISTS), Gen4LV with NBP sets that end on all three benchmarks.
  • Control. Without retraining, the gain g moves the operating point: on UDM10, PSNR rises from 21.58 to 23.14 dB as g goes from 0 to 1, with MUSIQ within 64.6–65.5 (Appendix I, Table 19); a more accurate frame 0 gives a more faithful output (26.16 dB with the ground-truth frame).
  • Human judgment. In the blinded study (30 VSR clips, 24 raters; Appendix E), Gen4LV was preferred in fidelity to the ground truth over DOVE, SeedVR2-7B and FlashVSR-Full overall (75% with NBP, 61% with PiSA-SR ×4); with NBP it was judged more faithful even than DOVE (78%) and SeedVR2-7B (67%), and with PiSA-SR ×4 it is at parity with SeedVR2-7B (45%).

Revision: Sec. 1, Sec. 4.3, Fig. 3, Appendices D, E, I.

AuthorsOpenReview ↗

Response to Reviewer aUgo (3/3)

Q1: Reliable structure under unknown degradations, and robustness to a mismatched operator.

A5: By frequency. The correction trusts only the band of the observation below the cutoff c, where blind degradations leave the input closest to the ground truth; above c, the content comes from the generator. Measured directly (Table 10), the upsampled input agrees with the ground truth only at the lowest frequencies on all three benchmarks: 19–22 dB below 0.03 of the output Nyquist frequency, 8–10 dB up to the deployed cutoff c = 0.06, and 1 dB or less above 0.125; with the exact operator of REDS4 it is reliable up to 0.125. The cutoff, calibrated on disjoint clips (Appendix C), coincides with this band, and the result depends little on the mismatch level σ_y. The deployed setting is itself mismatched: the benchmarks use blind degradations, while our operator is a nominal ×4 bicubic downsampling.

Mismatched operators: change from the consistency step (with − without), UDM10 content, PiSA-SR ×4, 10 clips per row; in brackets the clips on which the step helps. Noise and JPEG act on the low-resolution frames (Table 11 of the revision).

True degradationΔPSNRΔLPIPSΔDISTSΔMUSIQΔDOVERΔQ-AlignΔE_warp×10³
bicubic ×4 (exact)+2.09 (10/10)−0.010 (10/10)−0.003 (9/10)+0.68 (8/10)+0.003 (6/10)−0.02 (5/10)+0.065 (3/10)
blur σ = 2, then bicubic ×4+2.60 (10/10)−0.025 (10/10)−0.003 (9/10)+0.25 (9/10)−0.001 (8/10)−0.03 (6/10)+0.110 (4/10)
bicubic ×4, then noise 10/255+2.89 (10/10)−0.018 (10/10)−0.006 (8/10)−0.38 (4/10)−0.038 (3/10)−0.27 (4/10)+0.355 (0/10)
bicubic ×4, then JPEG q = 30+1.64 (10/10)−0.009 (10/10)−0.000 (4/10)+0.48 (8/10)−0.027 (1/10)−0.09 (2/10)+0.028 (4/10)
blur σ = 2, DiTVR-style hard projection instead+4.31 (9/10)+0.105 (2/10)+0.140 (0/10)−20.3 (0/10)−0.249 (0/10)−1.02 (1/10)+0.085 (2/10)

It shows that under every degradation the gated step raises PSNR by 1.6–2.9 dB and lowers LPIPS on every clip, and changes MUSIQ by less than 0.7. Noise is the one case with a clear cost (E_warp +0.35 on every clip, DOVER −0.04, Q-Align −0.27), yet the step does not write the noise back: what it changes below c is uncorrelated with the noise there (correlation −0.03). A DiTVR-style hard projection raises PSNR more under blur but copies the blur into the output (MUSIQ −20, DOVER −0.25). In the band the step trusts, all four synthetic degradations are milder than the blind ones of the benchmarks (23.6–46.5 dB against 16.7–19.4 dB), so Table 1, where the same step raises PSNR on every clip of SPMCS and YouHQ40, is the strong-mismatch end of this sweep.

Revision: Appendix C (Tables 10–11).

We hope these responses address your concerns, and we would be glad to discuss any remaining questions.

Reviewer_3mzjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Gen4LV uses a frozen video generator for restoration, conditioned on degraded video for motion and content and an image-expert-restored frame for appearance. A frequency-gated data-consistency correction improves alignment while restricting changes to generated texture. The paper evaluates super-resolution, colorization, archival restoration, and generated-video repair, emphasizing no-reference quality and a controllable perception–fidelity trade-off.

Critical strengths
  1. A simple conditioning strategy with a convincing within-backbone ablation. In Table 5 (p. 9), appearance-only guidance yields high MUSIQ but large geometric dispersion; adding the degraded-video reference reduces dispersion from 28.28 to 1.17 pixels while retaining MUSIQ of 65.32. This establishes why both references are needed.
  2. A useful empirical diagnosis of ungated consistency. Full-band correction reduces MUSIQ from 65.32 to 46.92, while the gated correction keeps it at 65.46 and improves PSNR from 20.04 to 21.98 dB (Table 5). This is direct evidence for the proposed correction strategy, with additional dose-matched analyses in the appendix.
Critical weaknesses
  1. The restoration claim exceeds the fidelity evidence. On UDM10 (Table 1, p. 6), Gen4LV has PSNR 21.68 and LPIPS 0.3636 versus DOVE's 26.44 and 0.2674; SPMCS also shows a substantial fidelity gap. In generated-video repair, source-motion EPE is 0.102, compared with 0.033–0.056 for the listed baselines (Table 4). These costs are meaningful, not merely an omission of PSNR. Independent human assessment of fidelity and visual quality is needed to establish when the generated output is a successful restoration rather than a plausible replacement. The VLM judge shares an encoder family with the generator, as the paper itself notes.
  2. The source of the gain and the zero-shot boundary need stronger controls. All full-system experiments rely on MiniMax-H3 plus a Nano Banana Pro frame expert and a captioner. The same expert frame should be supplied to a strong propagation/reference-based control to isolate what the proposed video interface adds. Appendix B estimates operator mismatch on benchmark-specific training probes, and Section 4.5 selects correction settings on named benchmarks. This is compatible with a frozen generator but must be distinguished from the absence of task-level calibration data. End-to-end runtime and expert-query cost are also needed to assess the practical generalist claim.
Questions for authors
  1. Can you add blinded human comparisons that separately measure visual appeal, identity/text fidelity, and motion preservation? How often is the preferred-looking result materially inconsistent with the source?
  2. Can you compare with a control that propagates exactly the same expert-restored frame, and report sensitivity to expert choice or expert hallucination? Does the interface work on another compatible backbone?
  3. Please specify probe-set sizes and disjointness, hyperparameter selection, and performance with a fixed transferable calibration. Please include expert, captioner, and decoded consistency-step costs in runtime.
6 comments
AuthorsOpenReview ↗

Response to Reviewer 3mzj (1/3)

We sincerely thank Reviewer 3mzj for the detailed and constructive review, and for recognizing that the decoupled conditioning is supported by a convincing within-backbone ablation (Table 5) and that the gated correction rests on a direct diagnosis of ungated consistency. Below, W and Q follow the numbering of the review, and A1, A2, … are our answers in order; all revisions are marked in red in the revised PDF.

W1 & Q1: Fidelity evidence, and a blinded human comparison of appeal and fidelity.

A1: We agree that the fidelity costs are real, and the revision states the claim as perceptual restoration with a controllable fidelity trade-off (Sec. 1). Part of the gap was geometric: the generator shrinks every frame by about 1%, which full-reference metrics count as error. Two affine registrations that use no ground truth remove it (Appendix D) and raise PSNR by 1.18 / 0.76 / 0.95 dB on UDM10 / SPMCS / YouHQ40 with Nano Banana Pro (NBP); the image content is unchanged, and the no-reference and temporal metrics stay within noise.

Fidelity and perceptual quality (Table 1 of the revision). Each cell: PSNR↑ / DISTS↓ / DOVER↑ / CLIPIQA+↑; bold: best of these five.

MethodUDM10SPMCSYouHQ40
DOVE26.44 / 0.150 / 0.586 / 0.50922.98 / 0.170 / 0.492 / 0.62923.94 / 0.155 / 0.669 / 0.561
SeedVR2-7B26.01 / 0.121 / 0.438 / 0.42223.02 / 0.150 / 0.437 / 0.55723.13 / 0.113 / 0.686 / 0.553
FlashVSR-Full23.59 / 0.149 / 0.583 / 0.56421.42 / 0.182 / 0.468 / 0.62121.73 / 0.141 / 0.689 / 0.633
Gen4LV (NBP)22.86 / 0.154 / 0.661 / 0.59020.18 / 0.171 / 0.525 / 0.71021.68 / 0.128 / 0.701 / 0.644
Gen4LV (PiSA-SR ×4)23.76 / 0.148 / 0.538 / 0.54521.45 / 0.180 / 0.442 / 0.65222.41 / 0.139 / 0.634 / 0.593

It shows that with the open-source PiSA-SR ×4 expert, Gen4LV matches or exceeds FlashVSR-Full in PSNR and DISTS on all three benchmarks, and with NBP its DISTS is on par with DOVE or better, while DOVE and SeedVR2-7B remain ahead in PSNR. With NBP, Gen4LV has the highest DOVER and CLIPIQA+ of all methods in Table 1. A more accurate frame 0 translates directly into a more faithful output: with the ground-truth frame 0, the same pipeline reaches 26.16 dB on UDM10 (Table 8).

Blinded human comparison (Q1). 24 raters from diverse backgrounds, none of them an author, compared Gen4LV, with the NBP or the PiSA-SR ×4 expert, against DOVE, SeedVR2-7B and FlashVSR-Full on 30 VSR clips (180 pairs; sides and order randomized, with a "cannot tell" option; all raters passed the attention checks). Each pair received two separate questions: visual appeal, judged from the two outputs alone, and fidelity to the reference in content, details and motion, judged after the ground truth is revealed; identity/text fidelity and motion are judged together in the second question. Text legibility is shown in Fig. 4, where Gen4LV keeps text legible and stable across frames while task-specific restorers smear or fragment the strokes, and motion preservation is measured by E_warp and tLPIPS in Table 1.

Preference for Gen4LV, (wins + ½ × cannot-tell) / n, with 95% bootstrap intervals; 50% means no preference (Table 14 of the revision).

Expertvs.Appeal (no reference)Fidelity to the reference
NBPDOVE96% [88, 100]78% [65, 90]
NBPSeedVR2-7B90% [79, 98]67% [51, 81]
NBPFlashVSR-Full94% [83, 99]81% [65, 93]
NBPall three93% [87, 98]75% [64, 85]
PiSA-SR ×4DOVE77% [62, 89]69% [55, 83]
PiSA-SR ×4SeedVR2-7B53% [38, 70]45% [29, 61]
PiSA-SR ×4FlashVSR-Full68% [53, 82]68% [53, 80]
PiSA-SR ×4all three66% [55, 76]61% [50, 71]

It shows that raters preferred Gen4LV in appeal over every baseline with either expert, except PiSA-SR ×4 against SeedVR2-7B (parity), and in fidelity overall, by smaller margins. On your second question: among the comparisons in which Gen4LV's appearance was preferred, it was judged less faithful than the baseline in 20% (82/401) with NBP and 14% (36/262) with PiSA-SR ×4, a broader criterion than "materially inconsistent". The appeal advantage depends on the expert (93% with NBP, 66% with PiSA-SR ×4, lower for 23 of the 24 raters), so raters did not simply favour the look of our pipeline.

AuthorsOpenReview ↗

Response to Reviewer 3mzj (2/3)

(A1, continued) The VLM judge. We agree: the submitted judge shares its encoder with our generator, as the submission noted, and our captioner is also a Qwen model, so it was not appropriate. Table 4 now uses two judges outside the Qwen family, GPT-6.1-Sol and Gemini-3.8-Flash, which compare all six methods blind, each pair in both orders.

Blind pairwise comparison on AIGC-LQ (40 clips; Tables 4 and 17 of the revision). Each cell: the method's preference rate (share of its 200 comparisons won, a tie counting half) · wins / ties / losses of Gen4LV against it.

MethodGPT-6.1-Sol realnessGPT-6.1-Sol structureGemini-3.8 realnessGemini-3.8 structure
VEnhancer0.30 · 35/4/10.34 · 37/2/10.33 · 37/2/10.34 · 38/1/1
SeedVR2-7B0.27 · 38/2/00.28 · 38/2/00.30 · 38/0/20.31 · 38/0/2
Vivid-VR0.80 · 13/13/140.76 · 16/14/100.67 · 31/3/60.68 · 31/3/6
FlashVSR-Full0.67 · 24/11/50.61 · 29/9/20.48 · 34/5/10.46 · 34/5/1
SparkVSR0.14 · 39/1/00.15 · 39/1/00.29 · 39/1/00.29 · 39/1/0
Gen4LV0.820.870.920.93

It shows that both judges rank the six methods identically (Kendall τ = 1) and put Gen4LV first on realness and structure. Under GPT-6.1-Sol, Gen4LV leads Vivid-VR, which is trained for generated video, narrowly on realness and more clearly on structure, and the two are on par head to head; Gemini-3.8-Flash prefers Gen4LV on both questions.

Motion EPE on generated video. On the 40 clips, EPE is 0.094 against 0.038–0.074 for the baselines (Table 4). In generated video part of the source motion is itself an artifact that restoration may need to correct, so we read EPE as a reference rather than as a target; both VLM judges rate the structural plausibility of Gen4LV highest. This task has no ground truth, so the human study covers VSR only.

Revision: Sec. 1, Sec. 4.3, Sec. 4.4, Table 4, Appendices D, E and G.

W2 & Q2: The same expert frame given to propagation and reference-based controls; sensitivity to the expert; another backbone.

A2: Given exactly the same expert frame, Gen4LV is better than per-frame restoration with temporal smoothing and than first-frame-guided propagation (AnyV2V) in PSNR, LPIPS and DISTS, with low flicker; through the native channels of two other open video generators, the same frame does not restore.

The same PiSA-SR ×4 expert, on every frame with temporal smoothing or once as frame 0 of Gen4LV (UDM10, 32 frames; Table 6 of the revision).

DownstreamExpert callsPSNRLPIPSDISTSMUSIQDOVERE_warp×10³
per-frame, no smoothing3223.150.3620.19266.60.5553.50
+ Deep Video Prior3223.110.4560.27132.80.2070.97
+ All-in-One Deflicker3223.280.3530.19964.70.5312.27
Gen4LV (Table 1)123.760.2890.14862.10.5381.55

It shows that smoothing either keeps much of the flicker (All-in-One Deflicker) or removes the detail with it (Deep Video Prior), whereas Gen4LV, with one expert call per clip, has the best LPIPS and DISTS (better on 9–10 of 10 clips against each alternative) and low flicker.

The same frame 0 given to AnyV2V or to Gen4LV (UDM10, 32 frames; Table 6 of the revision). Bold: the better of the two.

Frame 0MethodPSNRLPIPSDISTSDOVERE_warp×10³
NBPAnyV2V20.250.4100.1830.6542.12
NBPGen4LV22.860.3480.1540.6611.64
PiSA-SR ×4AnyV2V20.660.3670.1780.5982.15
PiSA-SR ×4Gen4LV23.760.2890.1480.5381.55

On all three benchmarks (Tables 6–7), Gen4LV is better on PSNR (by 1.4–3.1 dB), LPIPS, DISTS, MUSIQ and both temporal metrics, while AnyV2V reaches a higher DOVER in four of the six benchmark–expert pairs. Beyond these metrics, AnyV2V follows the motion but alters the content of the scene (Fig. 5), and it smears or omits content that frame 0 does not show, which Gen4LV draws from the degraded video in the appearance of frame 0 (Fig. 11, Appendix B).

Sensitivity to the expert and expert hallucination. With six experts and the ground-truth frame (Table 8), no-reference quality follows the expert's perceptual tier and full-reference quality its fidelity. Content that the expert invents in frame 0 is carried into every frame; fidelity-oriented open-source experts (PiSA-SR ×4, ODTSR), whose frame 0 is closer to the ground truth, lower this risk at some cost in no-reference scores.

AuthorsOpenReview ↗

Response to Reviewer 3mzj (3/3)

(A2, continued) Another backbone. We gave the same NBP frame to Wan2.1-VACE-14B and LTX-2 through their own conditioning channels (UDM10; Table 6, Fig. 5). "PSNR to input" compares the output with the degraded input; high values mean that the input is copied.

Backbone · channelMUSIQPSNRPSNR to inputBehaviour
Wan2.1-VACE-14B · expert as frame 0 of the control video20.822.9929.2copies the degraded video
LTX-2 · expert as frame 0 + retake67.115.2016.1regenerates; departs from the observation (dispersion 23.5 px)
MiniMax-H3 · Gen4LV (Table 1, NBP)65.322.8623.9restores

The other channels behave alike (VACE with the expert as reference image and LTX-2 in-context also copy the input, MUSIQ 19.3–21.5). Gen4LV needs a soft video reference that guides motion and layout without being copied, and a separate image reference for appearance, which MiniMax-H3's reference mode provides; VACE and LTX-2 fail in exactly the two ways the paper diagnoses. We therefore claim that a frozen generator with such an interface restores, not that every generator does; Gen4LV trains nothing, so it applies unchanged to any generator that offers these two channels.

Revision: Sec. 4.2, Fig. 5, Appendix A (Tables 6–8, Figs. 9–10), Appendix B (Fig. 11).

W2 & Q3: Probe sets and disjointness, hyperparameter selection, a fixed transferable calibration, and runtime.

A3: σ_y, g, c and N are now calibrated once on clips disjoint from every benchmark and applied unchanged to all three VSR benchmarks; Table 1 of the revision is re-run with the calibrated σ_y, and the calibrated g, c and N coincide with the values of the submission.

  • Calibration set: 30 clips sampled with a fixed seed from the 2,055 videos of DOVE's training set, which is public, provides clean ground truth and shares no frame with any benchmark; each clip is cropped to 1248×704×32 and degraded with the standard Real-ESRGAN high-order model in its RealBasicVSR video extension, which also synthesizes DOVE's training pairs.
  • σ_y: 12.61/255, the root-mean-square residual of the nominal operator on these clips; λ in Eq. 1 is fixed to 1 and never tuned, since λ and σ_y enter only through λσ_y².
  • g and c: swept over g ∈ {0.25, 0.5, 0.75} × c ∈ {0.03, 0.06, 0.12} with both experts of Table 1 (600 generations).

Gain g and cutoff c on the calibration set at N = 10: change against no correction on the same clip, both experts pooled (30 clips × 2 experts); all nine settings in Table 12 of the revision.

(g, c)ΔPSNRΔLPIPSΔDISTSΔDOVERΔMUSIQ
(0.5, 0.03)+1.13−0.010−0.001−0.005+0.13
(0.5, 0.06), ours+1.66−0.015+0.001−0.005+0.47
(0.75, 0.06)+2.11−0.015+0.004−0.013+0.33
(0.5, 0.12)+2.35−0.007+0.009−0.021+0.22
(0.75, 0.12)+2.97+0.006+0.019−0.058−1.04

It shows that the knee lies at (0.5, 0.06): PSNR +1.66 dB, on every clip with both experts, at a DOVER change of −0.005; one step further, the LPIPS gain stops (g = 0.75) or halves (c = 0.12), and DOVER drops 2.6–4× more.

  • N: N = 10 retains 87% of the geometric gain of all-step correction at one-fifth of its added time (Table 13); the steps follow from the sampler's noise schedule (Appendix H).
  • Transfer: on UDM10 the calibrated values raise PSNR by 1.3 dB over the decoupled conditioning alone (Table 5); on SPMCS and YouHQ40, which never entered any choice, they raise PSNR on every clip, by 0.98 and 1.62 dB on average. Colorization uses the exact grayscale operator with σ_y = 1/255; archival film and generated video use no consistency step.

Runtime per clip in seconds on one H200 (32 frames at 1248×704; 50 sampling steps, ten with the consistency step; median of three UDM10 clips; Table 18 of the revision). Model loading reads the weights and moves them between CPU and GPU; other I/O covers input preparation, final decoding and writing; peak GPU memory 71.4 GB.

ExpertImage expertCaptionerSampling (50 steps)Consistency (10 steps)Model loadingOther I/OTotal
NBP22.7 (API)2.5226.5106.4226.931.8617
PiSA-SR ×44.52.3230.4107.7219.330.0594

The expert and the captioner run once per clip and take 7–25 s together, under 5% of the total; each consistency step adds about 10.6 s, mostly for two VAE encodings; model loading takes 220–230 s with the framework's default memory management. Nothing is optimized for speed.

Revision: Appendix C (Tables 12–13, Figs. 7 and 12), Appendix H (Table 18).

We hope these responses address your concerns, and we would be glad to discuss any remaining questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We sincerely thank all reviewers for their careful and constructive reviews. To facilitate the discussion among the reviewers and the area chair, the table below summarizes the feedback, and the list that follows gives our main responses. In each reply, W and Q follow the numbering of the review, and A1, A2, … are our answers in order; all revisions are marked in red in the revised PDF.

R-3mzjR-aUgoR-HgYaR-48sC
Strengths
A frozen video generator for several low-level tasks, without fine-tuning✓✓✓
Diagnosis of two failure modes of generators used directly✓✓
A method built around these failures (decoupled conditioning, gated consistency)✓✓
Convincing ablations and a direct diagnosis of each component✓✓
Concerns
Fidelity gap; scope of the claim✓✓✓
Contribution of the image expert vs. the generator✓✓✓✓
Benchmark-specific calibration; zero-shot claim✓✓✓
Human evaluation; VLM judge✓
Evaluation scale✓✓
Novelty; DiTVR✓✓
Robustness to operator mismatch✓
Runtime and cost✓✓
Another backbone✓
Limitations; reproducibility✓

Main responses and revisions:

  • Calibration disjoint from every benchmark (R-3mzj, R-HgYa, R-48sC). σ_y, g, c and N are calibrated once on 30 clips from DOVE's training set, disjoint from all benchmarks, and used unchanged on all three VSR benchmarks; the calibration selects the same g, c and N as the submission, and Table 1 is re-run with the calibrated σ_y (Fig. 7, Appendix C).
  • Fidelity and the claim (R-3mzj, R-aUgo, R-48sC). The claim is stated as perceptual restoration with a controllable fidelity trade-off (Sec. 1). The generator shrinks every frame by about 1%, which registration without ground truth removes (Appendix D); with the open-source PiSA-SR ×4 expert, Gen4LV matches or exceeds FlashVSR-Full in PSNR, SSIM and DISTS. A perception–distortion figure (Fig. 3) and the gain g (Appendix I) show the trade-off and its control.
  • Blinded user study (R-3mzj, R-aUgo, R-48sC). 24 raters on 30 VSR clips preferred Gen4LV over DOVE, SeedVR2-7B and FlashVSR-Full in appearance (93% with NBP, 66% with PiSA-SR ×4) and in fidelity to the ground truth (75% and 61%) (Sec. 4.3, Appendix E).
  • What the generator adds (all reviewers). The same expert frame given to per-frame restoration with smoothing, to AnyV2V and to two other video models (Sec. 4.2, Fig. 5, Tables 6–7); six experts and the ground-truth frame (Table 8); the scene description (Table 9); content that frame 0 does not contain (Appendix B, Fig. 11).
  • Archival film and generated video (R-aUgo, R-HgYa, R-3mzj). Tables 3 and 4 now cover 30 and 40 clips; Table 4 replaces the Qwen judge with GPT-6.1-Sol and Gemini-3.8-Flash, which compare all six methods blind (Sec. 4.4, Appendix G).
  • DiTVR and positioning (R-48sC, R-aUgo). Related work is rewritten against zero-shot methods built on generative priors, including DiTVR (Sec. 2); Table 5 gains a DiTVR-style row.
  • Operator mismatch (R-aUgo). Reliability of the observation by frequency band, and mismatched operators (Appendix C, Tables 10–11).
  • Runtime (R-3mzj, R-aUgo). Cost by stage, including the expert, the captioner and the decoded consistency steps (Appendix H, Table 18).
  • Limitations (R-HgYa). A Limitations paragraph in Sec. 5.
  • Correction. The submitted SPMCS evaluation cropped the ground truth with a 2-pixel vertical offset relative to the input, for all methods; all SPMCS numbers are re-computed with a consistent crop. Table 5 and the knob table of Appendix I now follow the protocol of Table 1; their conclusions are unchanged.

We sincerely thank all reviewers and the area chair for their time and constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.