acceptodds
Under review as a conference paper at ICLR 2027

Efficient Video Super-Resolution via Hybrid Attention and Reward-Yield Distribution Matching

15 views

Abstract

Recent video super-resolution (VSR) models achieve efficient one-step generation through diffusion distillation. However, high-resolution inference remains costly, and distilled models are difficult to align with human preferences. These challenges call for improvements in both model architecture and training strategy. We present HyR-VSR to address both limitations. On the architecture side, we introduce hybrid attention based on the strong locality observed in pretrained VSR models, while retaining sparse global interaction for long-range consistency. It achieves up to DiT speedup and 45% lower peak memory, enabling spatially untiled 4K VSR on a single 24GB RTX 4090. On the training side, we propose Reward-Yield Distribution Matching Distillation (Ry-DMD) for preference-aligned one-step generation. Ry-DMD uses reward-weighted teacher rollouts to incorporate black-box rewards into distribution matching, without backpropagating through the reward model. We further show that Ry-DMD generalizes beyond VSR to T2I generation. Together, HyR-VSR delivers stronger restoration quality and substantially improves high-resolution efficiency, with consistent gains in user preference.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video super-resolution, One-step diffusion post-train, Hybrid attention

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_tqDqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HyR-VSR combines a hybrid-attention architecture for one-step video super-resolution with Ry-DMD, a reward-guided distribution-matching distillation method. Most global attention blocks are replaced by local spatiotemporal attention while periodic global blocks remain. Ry-DMD estimates a reward-weighted teacher endpoint shift from a small number of rollouts using scalar rewards, without differentiating through the reward model. Experiments cover video quality and efficiency, OCR-guided VSR, text-to-image transfer, and a blinded preference study.

Critical strengths
  1. The efficiency evaluation is relatively concrete, covering long videos, multiple resolutions, and two substantially different hardware platforms. Reporting the RTX 4090 implementation with KV-cache offloading rather than spatial tiling also makes the practical inference comparison more meaningful.
  2. The reward-distillation method is compared with relevant alternatives and tested beyond a single video super-resolution configuration. Its application to both image-generation backbones and different sampling budgets suggests that the proposed formulation may have utility beyond the primary setting.
Critical weaknesses
  1. The human-preference evidence is difficult to assess. Table 1 reports 20 videos per method pair, but does not make the number of raters, agreement, or uncertainty of preference margins clear.
  2. Reward-guided VSR centers on a synthetic text dataset and OCR reward. The SDXL and SD3.5-M results also emphasize OCR prompts. This shows task-directed behavior, but not broad preference alignment on natural video. Multi-stage training cost should also be distinguished from inference savings.
Questions for authors
  1. How many raters evaluated each pair, how were ties and disagreements handled, and can the authors report confidence intervals or per-video preference statistics?
  2. Does Ry-DMD improve preferences on natural, non-text VSR with a non-OCR reward, or is the benefit currently specific to text-oriented objectives?
Reviewer_7MxJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HyR-VSR, a framework for efficient and preference-aligned one-step video super-resolution. It introduces two main components: (1) a hybrid attention architecture that combines local spatiotemporal attention with sparse global attention to reduce computational and memory costs, and (2) Reward-Yield Distribution Matching Distillation (Ry-DMD), which incorporates black-box reward signals into one-step distillation through reward-weighted teacher rollouts.

Critical strengths
  1. The proposed hybrid attention achieves substantial latency and memory reductions while maintaining competitive restoration quality.
  2. The authors analyze attention locality in pretrained VSR models to motivate their hybrid attention design.
  3. Ry-DMD provides a simple way to incorporate non-differentiable rewards into one-step distillation. The improvements in OCR accuracy and additional results on text-to-image generation suggest potential applicability beyond VSR.
Critical weaknesses
  1. The proposed method estimates a reward-weighted endpoint shift from a limited number of teacher rollouts and uses it to modify the DMD teacher term. However, the relationship between this local correction and the globally reward-tilted distribution is not fully established.
  2. The main Ry-DMD evaluation focuses on OCR-guided restoration using synthetic text data. How does Ry-DMD compare with alternative reward-guided methods under matched training budgets, and does it generalize to rewards beyond OCR?
  3. The reported 4K inference results rely on additional system optimizations, including CPU KV-cache offloading, asynchronous prefetching, and hardware-specific attention implementations. Their individual contributions are not sufficiently quantified.
Questions for authors

See the Weaknesses.

1 comment

Thank you for the detailed response. The authors have addressed my concerns, and I will maintain my original score.

Reviewer_3xe5OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

HyR-VSR combines two proposals: hybrid local and global attention in a one-step VSR student, and reward-yield distribution matching distillation. The architecture retains one global block every five DiT blocks and supports spatially untiled 4K output. On an H200, the 4K 89-frame Ours-Turbo configuration takes 30.59 s of GPU time with 26.33 GB peak memory, versus 46.34 s and 45.00 GB for Ours-Base. A separate synthetic OCR-guided VSR experiment improves OCR score.

Critical strengths
  • Hybrid attention yields a cost-quality tradeoff. The adapted configuration gives large DiT latency and memory reductions with only a small DOVER change. Fig. 6 reports substantial full-system savings at 2K/4K.
  • Ry-DMD has a clear operational advantage for non-differentiable rewards. With the same one-step VSR starting point, OCR-guided training raises the reported score, while PSNR and LPIPS also improve on the synthetic test.
Critical weaknesses
  • The human-preference claim is not isolated for Ry-DMD. Table 1 compares Ours-Base to other restorers on only 20 videos per method pair, but does not show a matched Ours-Base with and without Ry-DMD comparison or annotator counts/uncertainty. The OCR VSR experiment uses procedurally rendered text and the PaddleOCR-based reward as its main reported OCR measure. An independent recognizer and natural-text test are needed to rule out reward-specific gains.
  • The consumer-GPU result depends on custom Triton attention kernels plus pinned-CPU KV-cache offloading and prefetching. Spatially untiled 4K is a useful feasibility result, but the RTX 4090 wall-clock chart does not establish real-time 4K.
Questions for authors
  • What is the human preference difference between DMD and Ry-DMD on natural VSR videos? What are the rater counts and uncertainty?
  • Does the improvement hold for real-world text when tested with a different OCR model? Please also separate kernel and offload effects from hybrid-attention savings in the RTX 4090 timings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.