Reliable and Propagation-Aware Test-Time Latent Adaptation for Video Compression

Neural video codecs rely on pretrained amortized encoders to generate coding latents, which may not be optimal for individual test videos. Encoder-side test-time latent adaptation provides an attractive way to improve instance-specific rate-distortion performance while keeping the pretrained codec and decoder unchanged. Extending latent refinement to chunk-based video coding, however, introduces two challenges: optimization is driven by differentiable surrogates that may not reflect native coding outcomes, and each adapted reconstruction changes the coding state used for subsequent chunks. We present, to our knowledge, the first encoder-side latent adaptation framework specifically designed for chunk-based neural video compression. At the chunk level, a gradient-conditioned learned optimizer generates coupled updates for the primary and hyper latents, while a distortion-constrained safety gate controls their local application with a fallback to the original encoder output. At the sequence level, candidate update policies are evaluated over a short observed prefix through separate causal closed-loop coding trajectories, using native bitstreams and decoded reconstructions to capture both actual coding behavior and temporal propagation effects. The selected policy is then causally applied from the state produced by the transmitted prefix, while all candidate trajectories remain private to the encoder. Our method requires no modification to the pretrained codec, decoder, or bitstream syntax, and introduces no additional signaling overhead. Experiments on six standard benchmark groups demonstrate consistent improvements over DCVC-UF, achieving an average BD-Rate reduction of 3.2% and up to 4.9% on individual datasets.