DeCache: Decoupled and Damped Memory Purification for Robust Streaming Video Enhancement

Streaming Video Super-Resolution (VSR) relies on causal Key-Value (KV) memory for constant-memory, low-latency inference, but suffers critically from causal cache contamination, where corrupted frames pollute the historical context, causing irreversible error accumulation and hallucination drift. To address this, we propose DeCache, which introduces a Read-Write Decoupled Routing mechanism to separate the dual roles of each frame: raw degraded inputs anchor current-frame generation for physical fidelity, while model-restored “pseudo-clean” representations update the temporal cache to protect future memory. Through a causal Gauss-Seidel/Jacobi analysis, we prove both schedules converge exactly to the same causal fixed point, and show that one round decoupling is empirically Pareto-optimal, whereas converging to the fixed point triggers self-reinforcing hallucination loops and over-sharpening. We further design an Asymmetric Damped KV scheme, using a convex combination for Values to calibrate the perception-distortion trade-off. Finally, a 1-chunk lookahead asynchronous streaming architecture achieves mathematical equivalence to offline one-step decoupling in single-pass online inference. Experiments show DeCache sets new state-of-the-art results in perceptual quality while maintaining competitive full-reference fidelity and temporal consistency.