acceptodds
Under review as a conference paper at ICLR 2027

DeCache: Decoupled and Damped Memory Purification for Robust Streaming Video Enhancement

12 views

Abstract

Streaming Video Super-Resolution (VSR) relies on causal Key-Value (KV) memory for constant-memory, low-latency inference, but suffers critically from causal cache contamination, where corrupted frames pollute the historical context, causing irreversible error accumulation and hallucination drift. To address this, we propose DeCache, which introduces a Read-Write Decoupled Routing mechanism to separate the dual roles of each frame: raw degraded inputs anchor current-frame generation for physical fidelity, while model-restored “pseudo-clean” representations update the temporal cache to protect future memory. Through a causal Gauss-Seidel/Jacobi analysis, we prove both schedules converge exactly to the same causal fixed point, and show that one round decoupling is empirically Pareto-optimal, whereas converging to the fixed point triggers self-reinforcing hallucination loops and over-sharpening. We further design an Asymmetric Damped KV scheme, using a convex combination for Values to calibrate the perception-distortion trade-off. Finally, a 1-chunk lookahead asynchronous streaming architecture achieves mathematical equivalence to offline one-step decoupling in single-pass online inference. Experiments show DeCache sets new state-of-the-art results in perceptual quality while maintaining competitive full-reference fidelity and temporal consistency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Generative Model, Inverse Problem, Memory, Causal Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_kwv3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to improve streaming video restoration by reducing the effect of degraded inputs on historical memory. The authors focus on how errors in the cache affect the restoration of later video chunks. The proposed method uses the original low-quality input for rendering and a pseudo-clean input for memory writing. The pseudo-clean input comes from downsampling a restoration draft generated by the same frozen model. The method also selects keys from either pass and combines their values with a mixing weight to update the cache. Experiments with FlashVSR show gains in several quality metrics under different degradation settings. These gains come with additional computation, and some other metrics decrease.

Critical strengths

The paper focuses on the visual restoration problem in streaming video. I find the distinction between restoring the current chunk and writing memory for later chunks interesting. Here are two good points.

(1) The method gives a straightforward way to use a restored input for memory writing while keeping the original observation for rendering. It uses a frozen model and requires no additional training. Figure 6 also provides useful evidence that changes to historical memory affect the restored output.

(2) The evaluation covers several degradation settings. Table 1 reports higher MUSIQ, CLIP-IQA, FasterVQA, and DOVER scores than FlashVSR across all reported subsets, although some differences are small. I also appreciate the additional tests in Appendix E. These use a fixed configuration on four standard VSR benchmarks and provide evidence under milder degradations.

Critical weaknesses

Here are my major concerns.

(1) The contribution of the rendering input is not clear to me. Figure 6 examines the effect of historical memory, and Figure 3 studies refinement rounds and KV settings. These experiments provide useful evidence, but they do not directly isolate the benefit of keeping the original input for rendering. A matched variant could use the same fixed pseudo-clean draft for both rendering and memory writing. Comparing the two designs would help distinguish the effect of using different inputs from the benefit of refinement with the draft. This matters because the full pipeline in Appendix A takes about 2.2 to 2.3 times as long as FlashVSR. Some quality metrics in the main evaluation also decrease.

(2) I find the parameter selection procedure difficult to assess from the current description. Appendix B.1 gives the search grid and explains that selection considers fidelity, perceptual quality, and temporal consistency together. The description does not identify the tuning videos or explain the choice when these metrics favor different settings. This leaves some uncertainty about performance beyond the videos used for tuning. The fixed-configuration tests in Appendix E are a useful addition. Their focus is on milder degradations, however, so the same question remains for the main benchmarks with more severe degradations.

Questions for authors

(1) I wonder whether using the same fixed pseudo-clean draft for both rendering and memory writing gives similar results. Using memory keys and lambda = 1 in both variants, with the same noise and initial cache, would make this a useful comparison. This would help me understand the specific benefit of retaining the original rendering input.

(2) I am interested in how the tuning videos are chosen and whether they are separate from the final evaluation videos. This distinction would help me assess how well the selected settings generalize.

(3) I find the runtime analysis in Appendix A useful, and I wonder how the cost varies across the configurations used for the main quality results. Appendix A measures the setting with memory keys and lambda = 1. Runtime and peak memory under the same measurement protocol would give a clearer picture of the balance between quality and cost.

3 comments
AuthorsOpenReview ↗

Official Comment by Authors (Part 1)

We thank the reviewer for the constructive feedback. We are pleased that the reviewer finds the distinction between restoring the current chunk and writing memory for later chunks "interesting," considers our method "a straightforward way". Below, we provide detailed responses to the concerns raised.


1. W1 / Q1 (Contribution of the Rendering Input)

We ran the comparison as suggested. Both variants use KmemK^{mem} and λ=1\lambda = 1, the same noise seed, an empty initial cache, the same draft x~\tilde{x} and the default cache length. With these settings the two variants write identical cache contents at every step, so the rendering input is the only difference:

  • Draft rendering: x~\tilde{x} is used for both rendering and memory writing. This is equivalent to running the backbone a second time on x~\tilde{x}, i.e. a self-refinement pass.
  • DeCache: the original input is rendered and x~\tilde{x} is used only for memory writing.

We evaluated the four benchmarks with ground truth. BSCV uses a random sample of 100 of its 526 clips because of the rebuttal time limit.

DatasetRendering inputPSNR↑SSIM↑LPIPS↓NIQE↓MUSIQ↑CLIP-IQA↑DOVER↑FasterVQA↑Ewarp↓tOF↓
TAPEDraft x~\tilde{x}19.360.67170.29604.72271.590.57208.840.79943.5661.662
TAPEOriginal (DeCache)20.010.68060.28754.69971.050.56769.070.79943.4931.401
BSDDraft x~\tilde{x}19.570.59180.37284.93460.460.40782.090.622811.11312.204
BSDOriginal (DeCache)21.290.64010.34434.98257.670.39382.090.61737.8089.805
AbsDeg-synDraft x~\tilde{x}16.670.43530.39563.11561.630.4570-0.100.558223.8715.807
AbsDeg-synOriginal (DeCache)17.600.46840.37373.10161.380.4594-0.120.572121.3115.059
BSCV (100 clips)Draft x~\tilde{x}20.670.61670.37663.40168.460.576410.980.83796.4144.034
BSCV (100 clips)Original (DeCache)22.130.65560.34583.47967.580.572910.990.83865.1263.353

The two variants do not give similar results. Retaining the original rendering input is better on PSNR, SSIM and LPIPS on all four benchmarks. It also improves temporal consistency: on BSD, AbsDeg-syn and BSCV, it lowers Ewarp by 30%, 11% and 20%, and tOF by 20%, 13% and 17%. On the no-reference metrics NIQE, CLIP-IQA, DOVER and FasterVQA the two variants are close, with neither consistently ahead. Draft rendering therefore produces outputs that a per-frame quality predictor rates slightly higher but that are further from the ground truth, which is the usual signature of added generated detail.

Retaining the original input helps because it keeps first-pass errors from being compounded. The draft x~\tilde{x} is a re-degraded copy of a first restoration. Whatever the first pass removed or hallucinated is already fixed in x~\tilde{x}, and no information about the true scene can be added to it afterwards. When x~\tilde{x} is also the rendering input, the second pass treats these errors as observations and restores on top of them, so first-pass errors are carried into the output and compounded. This matches the consistent drop in PSNR, SSIM and LPIPS. DeCache gives the draft a narrower role. The draft only decides what is written into memory, where it keeps corrupted content out of the cache, while every output frame remains conditioned on the actual observation. The output therefore stays anchored to the measured content, and the draft's errors can reach it only indirectly, as temporal context, and not as the signal being restored.

Changes in the revision. We have added this comparison and its explanation as Appendix B.4 (Table 8), referenced from Sec. 3.2.

AuthorsOpenReview ↗

Official Comment by Authors (Part 2)

2. W2 / Q2 (Parameter Selection and Tuning Videos)

The two hyperparameters follow from the role each one plays in the method, and we explain this first. We then describe the tuning videos, and finally report applied to all benchmarks.

How λ\lambda is chosen. λ\lambda follows the severity of the corruption: the heavier the corruption, the larger λ\lambda. Writing ρ∈[0,1]\rho \in [0, 1] for the severity, i.e. how much of the content entering the cache is expected to be corrupted, we use λ=g(ρ)\lambda = g(\rho), with gg non-decreasing, g(0)=0g(0) = 0 and g(1)=1g(1) = 1, and the simplest reading is λ≈ρ\lambda \approx \rho: replace as much of the memory as is corrupted. When the corruption is mild or local, most of the cached content from the original input is reliable, and a small λ\lambda keeps it and preserves fidelity. When the corruption is heavy, the cached content is mostly unreliable, and a large λ\lambda is needed to keep it out of memory. ρ\rho is a coarse property of a benchmark and is not estimated per video, so in practice λ\lambda is taken at the level that matches the benchmark.

  • BSD, BSCV and AbsDeg-syn: the corruption affects part of the content, and λ\lambda = 0.15.
  • TAPE: dropout, mistracking and scan-line flicker damage whole frames in large numbers, and λ\lambda = 0.5.
  • AbsDeg-real and SRWOV/real: real old films with heavy, persistent degradation, and λ\lambda = 1.

λ\lambda thus gives the method flexibility across degradation levels with one mechanism.

How the key source is chosen. The key decides which memory entries a query attends to. Memory keys are the default, since keys from the draft are not misled by the corruption. Real keys are preferred when the draft itself is less reliable in structure, as on TAPE, where the corruption is sampled independently for each frame; keys from the original input then keep the attention correspondence consistent with what the rendering pass sees.

Tuning videos. TAPE, BSD and BSCV provide official splits, and their configurations were selected on videos separate from the test videos: the training split of TAPE, the validation split of BSD and the training split of BSCV. AbsDeg-syn, AbsDeg-real and SRWOV/real have no official split, and no separate tuning set was held out for them.

One configuration for all benchmarks. We agree with the reviewer that the hyperparameters should not be tuned too specifically for each benchmark. We therefore propose a single benchmark-independent configuration, KmemK^{mem} with λ\lambda = 0.15, and evaluated it on all benchmarks without any adjustment. For BSD, BSCV and AbsDeg-syn this is the configuration already used in the main paper, so their results are reported in Table 1 and are not repeated here. The remaining three benchmarks are reported below.

BenchmarkMethodPSNR↑SSIM↑LPIPS↓NIQE↓MUSIQ↑CLIP-IQA↑DOVER↑FasterVQA↑Ewarp↓tOF↓
TAPEFlashVSR20.880.71750.24294.80569.770.57318.940.80202.8011.529
TAPEDeCache20.820.71370.24724.73269.810.57948.980.80282.6461.725
SRWOV/realFlashVSR–––5.00063.150.65031.380.5544––
SRWOV/realDeCache–––4.92963.690.65891.470.5568––
AbsDeg-realFlashVSR–––3.27559.830.5391-2.090.6382––
AbsDeg-realDeCache–––3.20560.850.5669-1.830.6457––

The single configuration is a conservative, fidelity-preserving operating point. On TAPE it stays close to FlashVSR on every metric. On the two real old-film benchmarks it improves the video quality scores. These gains are smaller than those of the benchmark-specific configurations in Table 1, which is consistent with the analysis above: the heavier corruption of these three benchmarks calls for a larger λ\lambda than 0.15. The benchmark-specific settings remain available as an option when the degradation level is known. A larger λ\lambda, up to 1, is an option when perceptual quality is the priority and some risk of hallucinated detail is acceptable, as in old-film restoration.

Changes in the revision. We have added the selection principle and the tuning videos to Appendix B.1, and the single-configuration results as Table 6.

AuthorsOpenReview ↗

Official Comment by Authors (Part 3)

3. Q3 (Cost Across the Main Configurations)

We re-measured all four configurations used in Table 5 with the Appendix A.3 protocol in one environment. The table also includes a FlashVSR baseline with the same incremental decoding as DeCache, which is the fairer comparison.

ConfigurationTAPE 512×512: FPSWall (s)Peak alloc. (GiB)BSCV 1280×640: FPSWall (s)Peak alloc. (GiB)
FlashVSR, incremental decoding20.808.327.436.6925.8515.85
DeCache, KrealK^{real}, λ\lambda=0.59.0619.0910.052.9059.6724.03
DeCache, KmemK^{mem}, λ\lambda=0.159.1019.029.872.9159.4523.48
DeCache, KrealK^{real}, λ\lambda=19.1418.939.872.9458.8323.48
DeCache, KmemK^{mem}, λ\lambda=19.1019.029.792.9259.2523.21

Cost is nearly independent of the configuration. Throughput varies by less than 1.5%. All four configurations run three backbone forwards and two decoder invocations per step, and differ only in whether render-pass candidates are kept until the memory pass writes them. Choosing the key source and λ\lambda per dataset is therefore a quality decision that does not change the cost. The cost relative to the baseline is about 2.3× in runtime in every configuration. We present DeCache as a trade-off of additional compute and memory for the quality changes in Table 1.

Changes in the revision. We have added these measurements to Table 3 in Appendix A.3.

We thank the reviewer once again for the valuable comments. We hope our explanations have clarified all the questions. We are happy to address any further concerns or discussions the reviewer may have. We kindly hope the reviewer could consider raising the score if satisfied with our responses. Thank you!

Reviewer_PXG6OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training-free streaming video super-resolution method, DeCache, which alleviates the problem of historical cache contamination by degraded input by separating current frame generation from key-value cache writing: the original low-resolution input is retained during generation, while the cache uses the recovery results of independent baseline branches, and the feedback intensity is controlled by a mixture of key selection and value weighting. The method employs a single round of refinement, stably improving the no-reference quality metric on multiple benchmarks, but the full-reference metric shows both increases and decreases, with a total runtime approximately 2.2 times that of FlashVSR.

Critical strengths

Separating generation and cache writing balances the quality of original observation information and historical representations, eliminating the need for additional training; direct comparison with the original FlashVSR facilitates the assessment of method contributions.

It covers degradation issues such as tape artifacts, corrupted bitstreams, motion blur, and old images, improving four no-reference metrics across six main test subsets; testing on a standard dataset with a unified configuration supplements the evidence of applicability.

Experiments reveal that multi-round feedback may improve perceptual scoring but reduce reconstruction fidelity, providing a basis for using only a single round of refinement; streaming execution, state management, and time and memory costs are explained comprehensively.

Critical weaknesses

The convergence proof may not align with actual scheduling. The proof requires each step to depend only on previous outputs, but according to the frame intervals in the appendix, the buffer for step 0 requires frames 21–24, which can only be generated in step 1, potentially creating a circular dependency. The independent baseline branch addresses the data supply for the final single-round method but is insufficient to prove that the recursive algorithm satisfies the convergence assumption.

Each test subset selects parameters from ten configurations, but it is not specified whether an independent validation set is used or the specific selection criteria. The main gains are concentrated in the no-reference metrics; the full-reference metrics are unstable, for example, the PSNR for BSCV ​​drops from 23.11 to 22.59 dB, and the LPIPS for Tape increases from 0.2706 to 0.2799. The effectiveness of the uniform configuration under severe degradation remains to be verified.

Questions for authors

How does the buffer in step 0 obtain frames 21–24 while avoiding reliance on reading from step 1? Does the convergence theorem apply to practical implementations, or only to simplified theoretical models?

Which videos and standard parameters are used in Table 5? Are the parameters separated from the test data? If a uniform configuration is selected on an independent validation set, will the gains on severely degraded benchmarks be maintained?

4 comments
AuthorsOpenReview ↗

Official Comment by Authors (Part 1)

We thank the reviewer for the constructive feedback. We are pleased that the reviewer finds that our results "support the method's usefulness across different degradation settings." Below, we provide detailed responses to the concerns raised.


1. W1 / Q1 (Convergence Proof and the Recursive Schedule)

Corrected theory. The revised Appendix C analyzes a draft-anchored operator MDM_D. Seed frames that belong to the next chunk are held at a fixed draft DD, and all other seed frames come from the current trajectory. This directly answers the question: frames 21–24 are supplied to MEM(0) by the draft, so they do not depend on OUT(1). Step tt then reads only frames through 8t+128t+12 of the current trajectory, and all of these lie in J0∪⋯∪Jt−1J_0 \cup \dots \cup J_{t-1}, so MDM_D is strictly lower-triangular. Propositions 1 and 2 hold for MDM_D without further assumptions on the backbone. With D=Y(0)D = Y^{(0)}, the first iterate of MDM_D is exactly it1.

Scope of the theorem. The theorem applies to MDM_D, which is the recursive schedule as now defined. It does not apply to the unmodified operator. The fixed point of MDM_D depends on DD, and half of the seed frames remain anchored to the draft, so we have also rewritten the discussion in Appendix C.3 and restricted the corresponding statements in the abstract, Sec. 3.4, and the conclusion.

Changes in the revision. We have redefined the operator in Appendix C as the draft-anchored MDM_D, restated Lemma 1 and Propositions 1 and 2 for it, and restricted the corresponding statements in the abstract, Sec. 3.4, the conclusion and Appendix C.3.

AuthorsOpenReview ↗

Official Comment by Authors (Part 2)

2. W2 / Q2 (Parameter Selection and Tuning Videos)

The two hyperparameters follow from the role each one plays in the method, and we explain this first. We then describe the tuning videos, and finally report applied to all benchmarks.

How λ\lambda is chosen. λ\lambda follows the severity of the corruption: the heavier the corruption, the larger λ\lambda. Writing ρ∈[0,1]\rho \in [0, 1] for the severity, i.e. how much of the content entering the cache is expected to be corrupted, we use λ=g(ρ)\lambda = g(\rho), with gg non-decreasing, g(0)=0g(0) = 0 and g(1)=1g(1) = 1, and the simplest reading is λ≈ρ\lambda \approx \rho: replace as much of the memory as is corrupted. When the corruption is mild or local, most of the cached content from the original input is reliable, and a small λ\lambda keeps it and preserves fidelity. When the corruption is heavy, the cached content is mostly unreliable, and a large λ\lambda is needed to keep it out of memory. ρ\rho is a coarse property of a benchmark and is not estimated per video, so in practice λ\lambda is taken at the level that matches the benchmark.

  • BSD, BSCV and AbsDeg-syn: the corruption affects part of the content, and λ\lambda = 0.15.
  • TAPE: dropout, mistracking and scan-line flicker damage whole frames in large numbers, and λ\lambda = 0.5.
  • AbsDeg-real and SRWOV/real: real old films with heavy, persistent degradation, and λ\lambda = 1.

λ\lambda thus gives the method flexibility across degradation levels with one mechanism.

How the key source is chosen. The key decides which memory entries a query attends to. Memory keys are the default, since keys from the draft are not misled by the corruption. Real keys are preferred when the draft itself is less reliable in structure, as on TAPE, where the corruption is sampled independently for each frame; keys from the original input then keep the attention correspondence consistent with what the rendering pass sees.

Tuning videos. TAPE, BSD and BSCV provide official splits, and their configurations were selected on videos separate from the test videos: the training split of TAPE, the validation split of BSD and the training split of BSCV. AbsDeg-syn, AbsDeg-real and SRWOV/real have no official split, and no separate tuning set was held out for them.

One configuration for all benchmarks. We agree with the reviewer that the hyperparameters should not be tuned too specifically for each benchmark. We therefore propose a single benchmark-independent configuration, KmemK^{mem} with λ\lambda = 0.15, and evaluated it on all benchmarks without any adjustment. For BSD, BSCV and AbsDeg-syn this is the configuration already used in the main paper, so their results are reported in Table 1 and are not repeated here. The remaining three benchmarks are reported below.

BenchmarkMethodPSNR↑SSIM↑LPIPS↓NIQE↓MUSIQ↑CLIP-IQA↑DOVER↑FasterVQA↑Ewarp↓tOF↓
TAPEFlashVSR20.880.71750.24294.80569.770.57318.940.80202.8011.529
TAPEDeCache20.820.71370.24724.73269.810.57948.980.80282.6461.725
SRWOV/realFlashVSR–––5.00063.150.65031.380.5544––
SRWOV/realDeCache–––4.92963.690.65891.470.5568––
AbsDeg-realFlashVSR–––3.27559.830.5391-2.090.6382––
AbsDeg-realDeCache–––3.20560.850.5669-1.830.6457––

The single configuration is a conservative, fidelity-preserving operating point. On TAPE it stays close to FlashVSR on every metric. On the two real old-film benchmarks it improves the video quality scores. These gains are smaller than those of the benchmark-specific configurations in Table 1, which is consistent with the analysis above: the heavier corruption of these three benchmarks calls for a larger λ\lambda than 0.15. The benchmark-specific settings remain available as an option when the degradation level is known. A larger λ\lambda, up to 1, is an option when perceptual quality is the priority and some risk of hallucinated detail is acceptable, as in old-film restoration.

Changes in the revision. We have added the selection principle and the tuning videos to Appendix B.1, and the single-configuration results as Table 6.

We thank the reviewer once again for the valuable comments. We hope our explanations have clarified all the questions. We are happy to address any further concerns or discussions the reviewer may have. We kindly hope the reviewer could consider raising the score if satisfied with our responses. Thank you!

I thank the author for the clarification and supplementary experiments. The revised theoretical analysis and unified configuration evaluation alleviated my main concerns. Although there is still room for improvement in parameter selection standardization and performance gains, I am raising the rating to 3.

AuthorsOpenReview ↗

Official Comment by Authors

Thank you for your thoughtful re-evaluation and for raising the score. We really appreciate your engagement with our rebuttal.

Reviewer_aWFKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies contamination of historical KV caches by degraded inputs in streaming video restoration and proposes a training-free method that decouples output generation from memory writing. The current output remains conditioned on the actual low-resolution input, while pseudo-clean inputs from an independent initial FlashVSR restoration trajectory are used to construct memory. The method controls feedback strength through a discrete choice of key source and continuous mixing of values, and implements a single round of draft-seeded refinement through interleaved streaming execution. The paper also analyzes fixed points of recursive feedback and offline multi-round iteration, arguing that one round of decoupled refinement is preferable to iterating to a fixed point.

Critical strengths
  1. The problem is concrete, and the intervention is clearly defined. Distinguishing the observational evidence needed for current reconstruction from the contextual quality needed by future cache entries is well motivated. The method requires no additional training and can be applied to existing backbones with persistent conditioning pathways and manageable KV caches.

  2. There are reasonably consistent gains in no-reference quality under severe degradation. Relative to FlashVSR, MUSIQ improves from 59.83 to 62.15 and DOVER from −2.09 to −0.47 on AbsoluteDegradation-real. On SRWOV/real, MUSIQ improves from 63.15 to 64.19. The appendix additionally evaluates four standard VSR benchmarks with milder degradation, providing useful breadth.

  3. The streaming implementation driven by an independent initial trajectory is described in considerable detail. The appendix and code explain the interleaved REF, MEM, and OUT scheduling, separate causal states, FIFO buffers, incremental decoding, and rolling seeds. The independent draft provides a feasible data source for the single-round it1 procedure, and the authors disclose the additional computational cost.

Critical weaknesses

W1. The fixed-point analysis does not yet cover the temporal dependencies of the actual implementation The proof in Appendix C is valid for the abstract operator under its strictly past-dependent assumption. The issue is that the paper does not establish that this assumption holds for the actual FlashVSR timing described in Appendix A. Appendix A.1 specifies that:

  • Step 0 takes input frames 0–24 and decodes output frames 0–20.
  • Step 1 decodes output frames 21–28.
  • MEM(0) requires restored frames 0–24. If the pseudo-clean inputs come from the same refined output trajectory, frames 21–24 required by MEM(0) must be produced by OUT(1), while OUT(1) itself reads MEM(0). This creates a dependency on the current output that is inconsistent with the strictly lower-triangular dependency structure required by the proof. Lines 10–18 of the supplementary decache/streaming.py also explicitly show this frame-window relationship. The authors need to explain the actual scheduling of the recursive online baseline in Figure 3 and establish that it corresponds to the analyzed operator, or provide more fine-grained conditions on token dependencies. This concern does not invalidate the independent-draft-driven it1 streaming scheme, because its missing frames are supplied by an independent baseline stream. A related issue is that Appendix C.3 states that actual LR inputs are no longer introduced after the first chunk, whereas Equation 6 retains conditioning on the current xtx_t at every step. This mechanistic explanation is inconsistent with the equation and should be corrected. The authors explicitly state that this discussion is not a theorem, so it should not be characterized as an incorrect mathematical proof.

W2. The reported evaluation sample count directly contradicts the code, and the parameter-selection protocol is unclear Appendix B.2 states that BSCV uses all 661 test clips, whereas lines 98–100 of the supplementary README state that only 526 of the 661 clips are retained because their dimensions are divisible by four. The _keep function in prepare_lq.py indeed applies this filter. This is a directly verifiable inconsistency in the evaluation protocol. The authors should provide the actual clip list and valid frame counts used for Table 1 and confirm that all baselines use the same samples, cropping, and temporal alignment. In addition, Appendix B.1 selects the final configuration separately for each evaluation dataset from two key sources and five mixing coefficients. It only states that multiple types of metrics are considered, without specifying an independent validation set or a reproducible selection criterion. A training-free method still requires a distinction between parameter selection and final testing. The available material does not rule out optimistic bias from benchmark-specific parameter selection.

W3. The results support a perceptual trade-off, but some claims about fidelity and temporal quality are too strong The BSCV PSNR gap relative to DOVE is 4.46 dB, which should not be described as a “modest margin.” The improvements in no-reference metrics are valuable, but they do not establish that generated textures are more faithful or that overall temporal performance is leading. Appendix D already discusses the perception–distortion trade-off and the sensitivity of temporal metrics to smoothing. However, these general explanations cannot replace direct evidence for this method. I recommend providing recovery curves after controlled corruption ends, comparisons with simple strategies such as clearing or shortening the cache, and full videos with difficult structures that can be compared against ground truth. These evaluations would help distinguish cache-purification benefits from additional sharpening.

W4: The efficiency analysis requires comparisons using the final quality configurations and consistent implementations The authors disclose the cost: at 512×512, throughput decreases from 20.24 to 9.10 FPS; at 1280×640, it decreases from 6.56 to 2.88 FPS. Total runtime is approximately 2.22–2.28× that of the baseline. The remaining issue is that the efficiency evaluation uses Kmem, λ=1, while most datasets in the main table use smaller mixing coefficients or keys derived from the actual conditioning inputs. These settings require retaining additional rendering candidates, so memory usage should be reported again using the final quality configurations. The long-video memory comparison also involves implementation differences: the baseline decodes after processing, whereas the proposed method uses incremental decoding. The baseline should first receive the same incremental decoding and buffering optimizations before memory growth is compared. Moreover, an identical input-lookahead range does not imply identical processing latency. Time to first output and steady-state chunk latency should be reported.

Questions for authors
  1. How does the recursive online algorithm in Figure 3 obtain restored frames 21–24 required by MEM(0)? Are its dependencies fully consistent with Appendix C?
  2. Are the BSCV results in Table 1 based on 661 clips or 526 clips? Do all baselines use the same clip list, valid frame ranges, and cropping rules?
  3. How are the key source and mixing coefficient selected for each dataset? Can the authors provide an independent validation protocol or cross-dataset results using one fixed configuration?
  4. Compared with clearing the cache, shortening the cache, and an ordinary additional restoration pass, what advantages does the method offer in recovery speed and ground-truth fidelity after corruption ends?
  5. What are the time to first output, throughput, and memory usage under the final quality configurations? Does the memory advantage remain when the baseline uses the same incremental decoding strategy?
7 comments
AuthorsOpenReview ↗

Official Comment by Authors (Part 1)

We thank the reviewer for the constructive feedback. We are pleased that the reviewer finds the problem "concrete" and "the intervention clearly defined," considers our method "well motivated". Below, we provide detailed responses to the concerns raised.


1. W1 / Q1 (Convergence Proof and the Recursive Schedule)

Corrected theory. The revised Appendix C analyzes a draft-anchored operator MDM_D. Seed frames that belong to the next chunk are held at a fixed draft DD, and all other seed frames come from the current trajectory. This directly answers the question: frames 21–24 are supplied to MEM(0) by the draft, so they do not depend on OUT(1). Step tt then reads only frames through 8t+128t+12 of the current trajectory, and all of these lie in J0∪⋯∪Jt−1J_0 \cup \dots \cup J_{t-1}, so MDM_D is strictly lower-triangular. Propositions 1 and 2 hold for MDM_D without further assumptions on the backbone. With D=Y(0)D = Y^{(0)}, the first iterate of MDM_D is exactly it1.

Scope of the theorem. The theorem applies to MDM_D, which is the recursive schedule as now defined. It does not apply to the unmodified operator. The fixed point of MDM_D depends on DD, and half of the seed frames remain anchored to the draft, so we have also rewritten the discussion in Appendix C.3 and restricted the corresponding statements in the abstract, Sec. 3.4, and the conclusion.

Changes in the revision. We have redefined the operator in Appendix C as the draft-anchored MDM_D, restated Lemma 1 and Propositions 1 and 2 for it, and restricted the corresponding statements in the abstract, Sec. 3.4, the conclusion and Appendix C.3.

2. W2 / Q2 (BSCV Evaluation Protocol)

We confirm that the supplementary README and code are correct, and the description in Appendix B.2 was inaccurate. All BSCV results reported in the paper, for DeCache and for every baseline, were computed on the same 526 clips (73,426 frames). The reported metrics are therefore unchanged and the comparison between methods is unaffected.

BSCV provides no LR inputs, so we generate them by bicubic ÷4 downsampling. We retained only clips whose native width and height are both divisible by 4, so that the LR frames correspond to the source frames at an exact ×4 ratio without rounding. This filter excludes 135 of the 661 clips.

Changes in the revision. We have corrected Appendix B.2: BSCV uses 526 of the 661 clips (73,426 frames), with the reason for the filter.

AuthorsOpenReview ↗

Official Comment by Authors (Part 2)

3. W2 / Q3 (Parameter Selection and Tuning Videos)

The two hyperparameters follow from the role each one plays in the method, and we explain this first. We then describe the tuning videos, and finally report applied to all benchmarks.

How λ\lambda is chosen. λ\lambda follows the severity of the corruption: the heavier the corruption, the larger λ\lambda. Writing ρ∈[0,1]\rho \in [0, 1] for the severity, i.e. how much of the content entering the cache is expected to be corrupted, we use λ=g(ρ)\lambda = g(\rho), with gg non-decreasing, g(0)=0g(0) = 0 and g(1)=1g(1) = 1, and the simplest reading is λ≈ρ\lambda \approx \rho: replace as much of the memory as is corrupted. When the corruption is mild or local, most of the cached content from the original input is reliable, and a small λ\lambda keeps it and preserves fidelity. When the corruption is heavy, the cached content is mostly unreliable, and a large λ\lambda is needed to keep it out of memory. ρ\rho is a coarse property of a benchmark and is not estimated per video, so in practice λ\lambda is taken at the level that matches the benchmark.

  • BSD, BSCV and AbsDeg-syn: the corruption affects part of the content, and λ\lambda = 0.15.
  • TAPE: dropout, mistracking and scan-line flicker damage whole frames in large numbers, and λ\lambda = 0.5.
  • AbsDeg-real and SRWOV/real: real old films with heavy, persistent degradation, and λ\lambda = 1.

λ\lambda thus gives the method flexibility across degradation levels with one mechanism.

How the key source is chosen. The key decides which memory entries a query attends to. Memory keys are the default, since keys from the draft are not misled by the corruption. Real keys are preferred when the draft itself is less reliable in structure, as on TAPE, where the corruption is sampled independently for each frame; keys from the original input then keep the attention correspondence consistent with what the rendering pass sees.

Tuning videos. TAPE, BSD and BSCV provide official splits, and their configurations were selected on videos separate from the test videos: the training split of TAPE, the validation split of BSD and the training split of BSCV. AbsDeg-syn, AbsDeg-real and SRWOV/real have no official split, and no separate tuning set was held out for them.

One configuration for all benchmarks. We agree with the reviewer that the hyperparameters should not be tuned too specifically for each benchmark. We therefore propose a single benchmark-independent configuration, KmemK^{mem} with λ\lambda = 0.15, and evaluated it on all benchmarks without any adjustment. For BSD, BSCV and AbsDeg-syn this is the configuration already used in the main paper, so their results are reported in Table 1 and are not repeated here. The remaining three benchmarks are reported below.

BenchmarkMethodPSNR↑SSIM↑LPIPS↓NIQE↓MUSIQ↑CLIP-IQA↑DOVER↑FasterVQA↑Ewarp↓tOF↓
TAPEFlashVSR20.880.71750.24294.80569.770.57318.940.80202.8011.529
TAPEDeCache20.820.71370.24724.73269.810.57948.980.80282.6461.725
SRWOV/realFlashVSR–––5.00063.150.65031.380.5544––
SRWOV/realDeCache–––4.92963.690.65891.470.5568––
AbsDeg-realFlashVSR–––3.27559.830.5391-2.090.6382––
AbsDeg-realDeCache–––3.20560.850.5669-1.830.6457––

The single configuration is a conservative, fidelity-preserving operating point. On TAPE it stays close to FlashVSR on every metric. On the two real old-film benchmarks it improves the video quality scores. These gains are smaller than those of the benchmark-specific configurations in Table 1, which is consistent with the analysis above: the heavier corruption of these three benchmarks calls for a larger λ\lambda than 0.15. The benchmark-specific settings remain available as an option when the degradation level is known. A larger λ\lambda, up to 1, is an option when perceptual quality is the priority and some risk of hallucinated detail is acceptable, as in old-film restoration.

Changes in the revision. We have added the selection principle and the tuning videos to Appendix B.1, and the single-configuration results as Table 6.

AuthorsOpenReview ↗

Official Comment by Authors (Part 3)

4. W3 / Q4 (Cache Clearing, Cache Shortening and an Additional Restoration Pass)

We have conducted additional experiments as suggested by the reviewer.

  • Shortened cache. FlashVSR keeps 3 cache units by default, about 24 frames. We ran it with 2 units (about 16 frames) and 1 unit (about 8 frames).
  • Additional restoration pass. We interpret an ordinary additional restoration pass as re-running the backbone on the re-degraded first-pass output, with the default cache.
  • Cleared cache. We did not run this baseline. FlashVSR cannot run with an empty cache after its first step, and clearing the cache at the right moment requires knowing when the corruption ends, which is not available at inference. The 1-unit cache is the closest setting we could run.

All rows below are FlashVSR variants measured in one environment with the same noise seed. BSCV uses a random sample of 100 of its 526 clips because of the rebuttal time limit. DeCache results on these benchmarks are in Table 1.

DatasetBaselinePSNR↑SSIM↑LPIPS↓NIQE↓MUSIQ↑CLIP-IQA↑DOVER↑FasterVQA↑Ewarp↓tOF↓
TAPEFlashVSR, 3 units (default)20.880.71750.24294.80569.770.57318.940.80202.8011.529
TAPEFlashVSR, 2 units20.910.71890.24124.79469.720.57419.020.80402.5301.438
TAPEFlashVSR, 1 unit20.930.71850.24254.80369.590.57518.950.80612.4201.344
TAPEAdditional pass19.360.67170.29604.72271.590.57208.840.79943.5661.662
BSDFlashVSR, 3 units (default)21.630.65510.32854.93856.970.38111.740.62077.4609.035
BSDFlashVSR, 2 units21.640.65550.32944.96256.900.37991.770.62147.3559.137
BSDFlashVSR, 1 unit21.640.65470.33174.97856.800.37871.790.62317.2659.413
BSDAdditional pass19.570.59180.37284.93460.460.40782.090.622811.11312.204
AbsDeg-synFlashVSR, 3 units (default)17.720.47720.36223.10860.730.4495-1.040.556520.8734.846
AbsDeg-synFlashVSR, 2 units17.750.47690.36023.10261.020.4555-0.950.559520.6184.807
AbsDeg-synFlashVSR, 1 unit17.800.47580.35983.11861.030.4612-1.020.561820.2554.815
AbsDeg-synAdditional pass16.670.43530.39563.11561.630.4570-0.100.558223.8715.807
BSCV (100 clips)FlashVSR, 3 units (default)22.830.68800.31253.57766.970.560710.500.83994.7423.097
BSCVFlashVSR, 2 units22.850.68800.31383.58367.080.563110.500.84044.6883.048
BSCVFlashVSR, 1 unit22.860.68720.31603.56767.160.567210.430.84134.6523.029
BSCVAdditional pass20.670.61670.37663.40168.460.576410.980.83796.4144.034

Shortening the cache changes very little. Relative to the default, PSNR moves by at most 0.08 dB, SSIM by at most 0.0014, LPIPS by at most 0.0035 and MUSIQ by at most 0.30. Ewarp decreases slightly, by 2% to 14% with one unit. Cutting the temporal context to a third therefore does not remove the effect of corrupted history. Whatever is in the cache, long or short, was still written from corrupted inputs. DeCache addresses a different factor, the content written into memory, and leaves the cache length unchanged.

An additional pass trades fidelity for perceptual scores. It raises MUSIQ by 0.90 to 3.49, but lowers PSNR by 1.05 to 2.16 dB and raises Ewarp by 14% to 49%. The second pass restores from the first-pass output, so details the first pass lost or hallucinated are treated as observations and carried into the result. DeCache uses the first-pass output only to decide what is written into memory and still renders every frame from the original input.

Changes in the revision. We have added this comparison as Appendix B.3 (Table 7).

Comments on the paper

1 comment
Area_Chair_Ldj8OpenReview ↗

Please review the authors’ responses and provide feedback

Dear Reviewers,

The authors have submitted their responses. Please review them carefully, assess whether they address your concerns, and leave a follow-up comment for the authors.

Thank you for your time and thoughtful feedback.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.