- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SwiftVR, a one-step generative video restoration framework designed for high-resolution real-time streaming. The method introduces Mask-Free Shifted-Window Attention (MFSWA) to reduce spatial attention costs using standard dense attention kernels, and a lightweight Restoration-aware Autoencoder (ReAE) to accelerate video encoding and decoding. A three-stage training pipeline combines flow matching, attention distillation, and adversarial fine-tuning. Experiments demonstrate improved throughput, achieving 31 FPS at 2K on an H100 and 26 FPS at 1080p on an RTX 5090.
Critical strengths
- Practical motivation and strong efficiency. The paper addresses important computational bottlenecks in generative video restoration and demonstrates impressive high-resolution streaming throughput on both server-class and consumer-grade GPUs.
- Comprehensive experimental evaluation. The paper provides comparisons with recent video restoration methods, detailed runtime analysis, component ablations, and qualitative results across multiple benchmarks.
Critical weaknesses
- Limited methodological novelty (the primary reason for my rejection recommendation). MFSWA is primarily an implementation-level modification of existing shifted-window attention, while ReAE follows the lightweight autoencoder strategy already adopted by FlashVSR[1]. The training pipeline also combines established flow matching, distillation, and adversarial fine-tuning techniques. Although the engineering improvements are meaningful, the methodological novelty appears relatively limited.
- Insufficient attribution of performance improvements. SwiftVR adopts Wan2.2-TI2V instead of the smaller Wan2.1-1.3B backbone used by FlashVSR, together with a more compressed latent representation. The reported 3.3× throughput improvement over FlashVSR may therefore partly originate from the backbone and latent compression differences rather than the proposed modules. This makes the paper's actual technical contribution less substantial than its claimed breakthrough suggests.
[1] Zhuang, Junhao, et al. "Flashvsr: Towards real-time diffusion-based streaming video super resolution." In CVPR, 2026.
Questions for authors
- Can the authors provide full-resolution visual comparisons (rather than only small cropped patches), along with video demonstrations? Given the relatively weak full-reference metrics and the lack of temporal consistency metrics, the current evidence is insufficient to convincingly assess the restoration quality.
- SwiftVR does not use cross-chunk DiT attention or KV caching, relying solely on ReAE boundary states to maintain temporal continuity. It remains unclear whether such low-level state propagation is sufficient to preserve generated details across chunks, especially under fast motion. Moreover, a short KV cache may already provide effective temporal context in VSR with relatively limited computational overhead. Can the authors further justify this design choice and provide stronger evidence that ReAE boundary states alone are sufficient?
4 comments
(Part 2/2)
Q2. Are ReAE boundary states alone sufficient? Why not a KV cache?
We now provide direct evidence (Table R5, repeated from our response to rBep):
Table R5. Temporal consistency at chunk boundaries (20 videos at 2K, ; tLPIPS ).
| Variant | tLPIPS all | boundary | interior | PSNR | LPIPS | FPS |
|---|---|---|---|---|---|---|
| w/o ReAE cross-chunk states | 3.41 | 16.85 | 2.84 | 27.89 | 0.1936 | 31.35 |
| ReAE states only (default) | 2.37 | 3.61 | 2.32 | 28.45 | 0.1866 | 31.35 |
| + 1-latent DiT overlap | 2.34 | 2.90 | 2.31 | 28.45 | 0.1863 | 27.51 |
| + 2-latent DiT overlap | 2.33 | 2.87 | 2.31 | 28.45 | 0.1861 | 24.57 |
The ReAE states are necessary. Removing them raises boundary tLPIPS and costs 0.56 dB PSNR, at no change in speed.
The ReAE states are sufficient. Overlapping one or two latent frames gives the DiT the same kind of short cross-chunk context that a short KV cache would provide. It lowers boundary tLPIPS from 3.61 to 2.90, but overall tLPIPS moves only from 2.37 to 2.33, and PSNR and LPIPS are unchanged. The cost is 12–22% of throughput.
A KV cache conflicts with the throughput target. Each chunk’s attention would then span the current tokens plus the cached ones, adding compute and memory to every chunk. With full temporal visibility inside each window, a cache of even one latent frame enlarges every window’s sequence. With ReAE states alone, whole-video temporal quality is already within tLPIPS of the overlapped variants, so the cache-free design is the right default for real-time streaming, and overlap is available as an option.
Each point above is answered with new measurements, and we are happy to provide further details during the discussion.
Thank you for the detailed rebuttal and additional experiments. Several of my concerns have been addressed, but I still have some remaining questions.
W1. I acknowledge the novelty and effectiveness of MFSWA. However, I still have concerns regarding ReAE. The authors describe it as a module that "turns a tiny autoencoder into a restoration-grade streaming codec" and "carries cross-chunk states". To my understanding, propagating temporal states across chunks is already a common design in causal video autoencoders, including TAEHV.
W2. I agree that the proposed modules substantially accelerate the Wan2.2-5B backbone. However, I find the claim that the "gain does not come from the backbone's latent compression" still somewhat overstated. SwiftVR and FlashVSR follow similar acceleration strategies: efficient attention and lightweight decoding. The current experiments do not sufficiently disentangle the contribution of differences in latent compression to the approximately 3× speedup over FlashVSR. This also underlies my concern about the incremental nature of the contribution, as SwiftVR largely follows the same acceleration pipeline as FlashVSR, with refinements to existing components rather than fundamentally new designs.
Q1. My concern has been addressed.
Q2. A comparison with actual causal streaming is still missing. Overlap may smooth transitions across chunk boundaries (e.g., the central yellow pattern in the provided SPMCS video appears inconsistent across chunks, gradually changing near chunk boundaries). In contrast, causal streaming can condition subsequent chunks on previously generated content to maintain consistency. Moreover, KV caching reuses previous computations, potentially introducing less overhead than overlap. Could the authors clarify the actual efficiency and quality trade-offs of causal streaming with KV caching?
If I have misunderstood any aspects of the method or experiments, I welcome further clarification and discussion from the authors.
We thank the reviewer for the continued discussion and for acknowledging the novelty and effectiveness of MFSWA.
W1 (ReAE). We do not claim ReAE as a new architecture, and cross-chunk state propagation is inherited from TAEHV. ReAE adapts an existing tiny autoencoder to the two requirements of this task: latency within the real-time budget, and reconstruction quality sufficient for restoration. Neither available option meets both (Table R2). Wan2.2-VAE reaches 35.48 dB but decodes in 2714 ms, and the released TAEHV decodes in 40 ms but reaches only 30.89 dB. ReAE reaches 32.74 dB at 99 ms: +0.46 dB comes from our recipe, which fine-tunes ReAE jointly with the DiT along the one-step inference path, and a further +1.39 dB comes from the wider decoder.
W2 (attribution and relation to FlashVSR). We revise the sentence to "the gain does not come only from the backbone's latent compression"; the word "only" was missing in our previous response. Compression alone is far from real time. The unmodified Wan2.2-TI2V-5B, with the same latent, runs at about 2.9 FPS at 1080p on an H100 (Fig. 2), and MFSWA and ReAE bring the same backbone to 54 FPS. The speedup over FlashVSR compares complete systems. Directly adopting FlashVSR's acceleration would not reach this result, for two reasons:
- Portability. FlashVSR relies on a Block-Sparse-Attention kernel that has no supported acceleration path on Blackwell consumer GPUs. MFSWA uses one dense SDPA call per layer, and the same weights run unchanged on five attention backends (Table 8) and on the RTX 5090.
- Effectiveness under stronger compression. In a more compressed latent space, each token carries more image content and the sequence is shorter. This leaves less redundancy for restricted attention to exploit, so it is not given in advance that windowed attention stays accurate. MFSWA does: it stays within 0.28 dB of the full-attention teacher.
We do not claim fundamentally new designs. Our contribution is to extend the deployment boundary of generative VR. SwiftVR is the first generative VR method to run in real time at 1080p on a consumer GPU (26 FPS on an RTX 5090). It is also the only diffusion-based method in our evaluation that runs at 4K on a single H100.
Q2 (causal streaming with KV caching). We do not claim that our chunk-wise processing is better than actual causal streaming. The paper provides a simple and effective base pipeline. Overlap is not part of our method: the paper splits the stream into non-overlapping chunks (Sec. 3.3), and all reported SwiftVR results, including the comparisons in Table 1, are obtained without overlap (App. B.1). Our core contribution is real-time high-resolution generative VR on consumer hardware through MFSWA and ReAE, so we did not discuss cross-chunk DiT modeling in depth. Following the suggestion, we implemented causal KV caching. Each DiT layer caches the keys and values of the last 1 or 2 latent frames of the previous chunk, and current tokens attend within their MFSWA window to the cached and current frames. The variant is training-free and uses the same weights. Results at 2K (L=24, one H100; tLPIPS ×10⁻²):
| Cross-chunk DiT context | tLPIPS boundary | tLPIPS all | PSNR | LPIPS | FPS |
|---|---|---|---|---|---|
| None (default) | 3.61 | 2.37 | 28.45 | 0.1866 | 31.35 |
| 1-latent overlap | 2.90 | 2.34 | 28.45 | 0.1863 | 27.51 |
| 2-latent overlap | 2.87 | 2.33 | 28.45 | 0.1861 | 24.57 |
| 1-latent KV cache | 2.84 | 2.33 | 28.46 | 0.1861 | 30.35 |
| 2-latent KV cache | 2.77 | 2.31 | 28.47 | 0.1858 | 29.73 |
As the reviewer anticipated, KV caching is both cheaper and more effective than overlap. With the same context, it matches or improves on overlap in every quality metric at a much smaller throughput cost: 3.2% versus 12.2% for one latent, and 5.2% versus 21.6% for two. Overlap re-denoises the shared frames in every chunk, whereas the cache reuses their keys and values. The effect concentrates at the boundaries: two cached latents lower boundary tLPIPS by 23% (3.61→2.77). For the video as a whole the gain is small, at most 0.06×10⁻² tLPIPS, 0.02 dB PSNR, and 0.0008 LPIPS, because boundaries are only 12 of 296 frame pairs. With the ReAE states, the base pipeline already provides most of the temporal consistency at the highest throughput.
Both mechanisms are inference-time options that need no retraining, and a deployment can enable KV caching when boundary consistency matters more than the last few percent of throughput. We will report this comparison in the final version.
We hope these results resolve the remaining concerns. If so, we would be grateful if the reviewer would consider raising the score.
Response to Reviewer UJf1
We thank the reviewer for the clear summary of concerns. We address the novelty question first, since the reviewer names it as the primary reason for rejection. New results are given in Tables R1–R6 (R1–R5 are in our response to rBep and R6 in our response to Lst9). Paper table numbers refer to the revised PDF, whose wording changes are marked in blue.
(Part 1/2)
W1. Limited methodological novelty.
SwiftVR contributes a new attention design, MFSWA, and a restoration-adapted autoencoder, ReAE, and combines them into the first generative VR system that runs in real time on a consumer GPU. Each part has a measured, independent effect:
It solves an open deployment problem. SwiftVR is the first generative VR method shown to sustain real-time 1080p streaming on a consumer GPU (26 FPS on an RTX 5090). It is also the only diffusion-based method in our evaluation that runs at 4K on a single H100. The closest streaming method, FlashVSR, reports 17 FPS at 768×1408 on an A100 and depends on a dedicated sparse kernel.
MFSWA is a new window design that outperforms existing ones. Its defining property is that every window has the same shape at every resolution, obtained by boundary clamping and priority scatter. This differs from Swin (cyclic shift and mask), from SeedVR (variable-sized windows with varlen kernels), and from plain non-shifted windows. In a new controlled comparison on the same backbone and budget (Table R6), MFSWA gives the best quality of all windowed variants (LPIPS 0.2508 vs. 0.2618–0.2637) while running within 3% of the fastest variant on standard dense SDPA only:
Table R6. Same backbone and budget, (16,16) windows, 2K.
flash_attn)Because no custom kernel is involved, the same weights run unchanged on five attention backends (Table 8) and on Blackwell consumer GPUs, where sparse-kernel methods such as FlashVSR currently have no supported acceleration path.
W2. Attribution of the performance gains (Wan2.2-TI2V-5B and latent compression).
The 3.3× figure over FlashVSR-Tiny compares complete systems, and Sec. 4.3 of the final version will state this explicitly. The gain comes from MFSWA and ReAE, not from the backbone’s latent compression. Fig. 2 profiles the unmodified Wan2.2-TI2V-5B (original VAE and full-attention DiT) on an H100. It takes 8.37 s per 1080p chunk, VAE decoding alone takes 2.70 s at 1080p and 25.8 s at 4K, and its memory forces tiled inference at 2K and above. The compressed latent alone is therefore far from real-time.
The contribution of each module on this same backbone is isolated as follows:
ReAE versus Wan2.2-VAE (Table 5): decoding takes 99 instead of 2714 ms (27× faster) and encoding 34 instead of 818 ms. This also frees enough memory to run the DiT without tiling.
MFSWA versus full attention (Table 4): the DiT time drops from 1039 to 566 ms at 2K, giving 1.62× end-to-end throughput.
Starting from the unmodified 5B pipeline, these two modules bring it from 8.37 s per chunk to 54 FPS at 1080p on the H100.
Q1. Full-resolution visuals and video demos.
The Supplementary Material now includes
comparison_visual.zip, with three full-resolution side-by-side comparison videos, one each from VideoLQ, SPMCS, and YouHQ40. Temporal consistency is also quantified in Tables R1 and R5.