acceptodds
Under review as a conference paper at ICLR 2027

LiveVSR: Reviving Live Photos via Cover-Guided Video Super-Resolution

Abstract

Live Photo pairs a high-quality cover image with a lower-quality video of the same moment. Existing cover-guided methods mainly enhance individual frames, while complete video restoration has received less attention. Two challenges hinder this task: uncertain cover-video correspondence and residual degradation in the cover. To address these challenges, we propose LiveVSR, a cover-guided one-step diffusion framework for complete Live Photo super-resolution. We combine continuous reference features with a discrete visual prior to better use the cover. First, we build an in-context framework to jointly model cover and video tokens through bidirectional attention. Flow-guided local retrieval combines motion cues with content-based reference matching. Meanwhile, we introduce correspondence-guided codebook injection. A codebook pretrained on high-quality images provides discrete cover features. These features are transferred through contextualized correspondences to complement continuous reference guidance. Training with clean and mildly degraded covers further accounts for reference imperfections. We compare the proposed LiveVSR with conventional VSR, reference-guided VSR, and Live Photo image enhancement methods. Experiments on synthetic benchmarks and real Live Photos demonstrate favorable visual quality, reconstruction fidelity, and temporal consistency. Code will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Live Photos, Diffusion, Video Super-Resolution

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_epriOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses cover-guided Live Photo super-resolution: restoring the entire low-quality Live Photo video using its high-quality native cover, whereas prior Live Photo methods (LiveMoments, Live2K) output a single image.

LiveVSR adapts Wan2.1-1.3B into a one-step diffusion restorer with two components:

  1. In-context framework: jointly processes cover and video tokens with bidirectional self-attention, plus flow-guided local retrieval (RAFT-based, confidence-weighted mixing of local and global cover attention).
  2. Correspondence-guided codebook injection: uses hard top-1 matching (trained with STE) to inject discrete features from a codebook pretrained on HQ images, intended to mitigate residual cover degradation.

Training mixes clean and mildly degraded references. Experiments on UDM10, SPMCS, YouHQ40 (reference = mildly degraded GT middle frame) and a new 40-clip RealLive set report large fidelity/perceptual gains on synthetic data.

Critical strengths
  1. Practical and well-motivated task. Extending cover guidance from a single frame to the full Live Photo video is a natural, useful step, and the planned RealLive release could benefit the community.
  2. Strong fidelity gains when a good reference is available. On synthetic benchmarks, LiveVSR substantially outperforms all baselines in PSNR/SSIM/LPIPS/DISTS (e.g., UDM10 PSNR 29.18 vs. 26.62; LPIPS 0.096 vs. 0.258), and the ablations (Tab. 2b) show that joint self-attention uses the reference much better than cross-attention or ReferenceNet, with a modest one-step 1.3B model.
Critical weaknesses

1. Key claims are not supported by the results.

  • Temporal consistency. The contribution list (Sec. 1) claims that LiveVSR outperforms existing methods in temporal consistency, but in Table 1 its Ewarp∗E^*_{warp} ranks only 3rd on all synthetic sets (e.g., UDM10 1.105 vs. 0.781 for SparkVSR*) and 7th of 10 on RealLive (3.990 vs. 2.577 for ERVSR). The promised temporal analysis "in Appendices C and D" does not exist (C is efficiency, D is visual results). The Table 1 caption ("outperforms ... across various datasets and metrics") is also overstated (e.g., DOVER ranks 2nd–4th; several relative NR metrics on YouHQ40/RealLive favor LiveMoments).
  • Codebook injection. Presented as a core contribution for handling degraded covers, but its gain is marginal (+0.235 dB PSNR over +Flow, with slightly worse DISTS and DOVER in Tab. 2a; all codebook variants worsen CLIPre in Tab. 3b). There is no experiment varying cover degradation and no visualization (e.g., retrieved codes/correspondences) showing that it actually suppresses cover artifacts.

2. Evaluation fairness and numerical inconsistencies.

  • SparkVSR is not given the same cover. SparkVSR, itself a keyframe-reference method, uses a PiSA-SR-generated reference instead. Since LiveVSR's NoRef variant (25.59 dB) is below DOVE (26.53 dB), its lead comes almost entirely from the reference. Moreover, on synthetic sets that reference is the GT middle frame degraded with exactly the training degradation (strength 0.10), i.e., a near-oracle, train-test-matched reference. Without giving SparkVSR the same cover, the comparison does not isolate the contribution of the proposed method.
  • Table inconsistencies. Attention and Ours have identical CLIPre (0.1997) in Tab. 2b despite very different other metrics; the full model's DOVER is 0.7986 in Tab. 1 but 0.8014 in Tab. 2a.

Minor (presentation/reproducibility)

  • Broken references ("Appendix ??", "detailed in .", "Figs. 5, ??, and 6").
  • Duplicated sentences in Sec. 4.3.
  • Undefined footnotes in Tab. 5, whose runtime is from an "earlier pipeline".
  • Missing hyperparameters (λf\lambda_f, λtemp\lambda_{temp}, τ\tau, neighborhood size, codebook size, stage-2 LR, iterations).
Questions for authors
  1. Please reconcile the temporal-consistency claim with Table 1 (especially Ewarp∗E^*_{warp} on RealLive), and provide the temporal analysis referenced in Appendices C/D.
  2. Please correct/explain the identical CLIPre values in Tab. 2b and the DOVER mismatch between Tab. 1 and Tab. 2a.
  3. Fairness: results for SparkVSR using the same cover as its keyframe reference, and PSNR as a function of frame distance from the reference.
  4. Codebook: results with and without codebook injection under varying cover degradation (clean, 0.10, stronger, real-cover-like), plus visualizations showing that it suppresses cover artifacts.
Reviewer_XVP9OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents LiveVSR for Live Photo VSR. LiveVSR leverages a pretrained diffusion model with flow-guided reference matching. It shows improvements in both reconstruction fidelity and temporal consistency over existing VSR models and other reference-based methods.

Critical strengths
  • The paper shows improvements over multiple previous baselines on multiple datasets (Table 1).
  • Using the high-quality cover image as a reference for VSR makes sense, especially in the context of using generative models, which might bring hallucinations.
Critical weaknesses
  • As a paper working on videos, there are not any video results shown in the supplementary material, making it hard to judge if the method actually works and if it is good with temporal consistency.
  • The scope of the problem is somehow limited and the problem formulation has little differences from reference-based VSR. I do not see why we need to formulate a new topic, i.e. cover-based SR.
  • It is also unclear why a codebook is used here. There is no sufficient proof/explanation that it works better than continuous latent codes.
Questions for authors

N/A

Reviewer_WUxxOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces LiveVSR, a novel framework for cover-guided video super-resolution specifically designed for Live Photo videos. This task addresses a practically valuable yet relatively underexplored problem in smartphone imaging. The proposed approach jointly models cover-image and video-frame latents via bidirectional attention within a diffusion transformer (DiT), while a codebook-based branch injects high-quality detail priors. Extensive evaluations demonstrate substantial performance gains across multiple reconstruction and perceptual metrics.

Critical strengths
  1. Well-Motivated Problem. Enhancing Live Photo videos using their corresponding high-quality cover images is a compelling and practical application for mobile photography.

  2. Effective Reference Integration. The proposed reference-conditioning mechanism is a key driver of performance. Ablation studies clearly demonstrate the benefits of incorporating the cover reference and validate the effectiveness of the bidirectional attention design.

  3. Strong Quantitative Performance. The method achieves state-of-the-art results across several synthetic benchmarks, outperforming existing approaches by significant margins in metrics such as PSNR, SSIM, LPIPS, and DISTS.

Critical weaknesses
  1. Reproducibility of Architecture and Training Details. While the paper outlines the overall training pipeline, several key implementation details needed for reproducibility remain missing. These include the architecture and pretraining objectives of the codebook mapper and reference selector, as well as specific training configurations for the codebook adapter. Providing a more comprehensive description of these components would greatly benefit readers trying to reproduce the work.

  2. Controlled Comparison Protocol. It is currently unclear whether the baseline methods were retrained or fine-tuned under identical conditions (e.g., using the same training data and degradation pipeline). Without a tightly controlled comparison against strong video super-resolution baselines adapted to use the same cover reference, it is difficult to isolate whether the performance gains stem purely from the proposed reference-injection design or from differences in training configurations.

  3. Trade-offs in Temporal Consistency. As shown in Table 1, the proposed method exhibits higher temporal warping error than the reference-free SparkVSR variant and falls short of achieving the best DOVER scores. Given the importance of temporal stability in video super-resolution, these results warrant a deeper discussion alongside the frame-level improvements.

  4. Mixed Impact of Codebook Injection. According to Table 2(a), adding the codebook branch improves pixel-level metrics (PSNR and SSIM) but slightly degrades perceptual metrics like DISTS and DOVER. This suggests that the module's contribution is less consistent than the reference-conditioning component. Further analysis is needed to justify its necessity and clarify what specific benefits it brings.

  5. Minor Presentation Issues. Several cross-references remain unresolved, notably the ?? placeholders around lines 430 and 737, which should be corrected to ensure a polished presentation.

Questions for authors
  1. Could the authors provide a more detailed description of the codebook mapper and reference selector, including their architecture, pretraining objectives, and training protocols? Additionally, please clarify the architecture of the codebook adapter and any component-specific training strategies used across the training stages.

  2. Were the baseline methods in Table 1 evaluated using their original pretrained checkpoints, or were they retrained/fine-tuned on the same dataset under the identical degradation pipeline? Please clarify.

  3. Would it be possible to include a strong video super-resolution baseline (e.g., FlashVSR) augmented with the same cover reference and trained under identical conditions? This comparison would help isolate the specific contributions of the proposed reference-injection mechanism.

  4. How do the authors account for the higher temporal warping error compared to the reference-free SparkVSR variant, as well as the lower DOVER scores relative to the strongest baselines in Table 1? Do these findings reflect an inherent trade-off between spatial detail enhancement, temporal consistency, and overall video quality?

  5. How should the mixed results in Table 2(a)—where codebook injection improves PSNR and SSIM but slightly worsens DISTS and DOVER—be interpreted? Could the authors provide further analysis or representative visual examples to clarify which aspects of reconstruction actually benefit from this module?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.