- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
From my understanding, the paper tries to add a small recurrent memory, “Temporal Residual” (TR), to bounded-context autoregressive video DiTs. Each of the 30 Wan2.1-1.3B blocks gets one d-dimensional state token, G^ℓ, initialized to zero. he model reads it in two ways: 1)AdaLN path: an MLP on [timestep embedding; pooled state] adds a correction to the six host modulation vectors. 2)Attention path: the state is projected with the host Q/K/V (RoPE at the chunk’s first frame, scaled by a learned γ^ℓ). Visual queries attend to it alongside the host KV context. The state’s own query attends to itself and the current chunk. TR adds 118.4M parameters and is post-trained jointly with the host for 2,000 iterations (400 generator updates) under the host’s DMD objective. Evaluation is on 60s videos from 256 MovieGenBench prompts × 5 samples using six VBench-Long dimensions.
Critical strengths
-
A cheap, host-preserving interface that transfers across two cache policies. TR leaves the host’s KV window and cache untouched. It adds negligible attention cost (~0.01% more query–key pairs) and a state that is about 0.003% the size of the KV cache. The same module attaches to a sink-free host (Self Forcing) and a sink-based one (LongLive). The design choices are sensible and well argued: chunk-synchronous writes, so that noisy denoising passes don’t pollute the state, and a read-only clean recache. On Self Forcing the improvement is visible and fairly large: +1.8 subject consistency and +3.8 imaging quality. The user study also favors it strongly (~86% tie-adjusted on consistency and overall quality). If those gains hold up against a matched control, this would be a practical add-on for streaming generators. That is the main thing in the paper’s favor.
-
TR+LongLive: dynamic degree rises from 35.18 to 52.85, with consistency and quality roughly unchanged (+0.02 to +0.63) while TR+Self Forcing: subject consistency rises from 96.05 to 97.85 and imaging quality from 66.59 to 70.39, while dynamic degree falls from 57.77 to 49.91.
Critical weaknesses
- I think the gains can’t be attributed to TR, because there is no matched post-training control. Both host baselines are the released checkpoints. TR variants get an extra 2,000 iterations of joint host+TR training under DMD. For example For Self Forcing, that training includes 93–240-frame extended self-rollouts on half of the iterations (Table 3). The released Self Forcing was trained on short rollouts only, and long-rollout tuning by itself is exactly what LongLive and Self-Forcing++ showed reduces long-horizon drift. MOreover, The closest methods the paper cites, VideoSSM and ARL2 (and also LongLive-RAG and AdaState), are discussed but never compared. Additionally, MALT Diffusion (Feb. 2025) is not cited at all. It uses a fixed-size per-layer memory updated after each segment from the clean segment, with stop-gradient on the prior memory and read through added attention. That is very close to TR’s state, write schedule and stop-gradient. Recurrent Memory Transformer-style memory tokens, TTT layers for minute-long video (CVPR 2025), and block-SSM memory with local attention (Po et al., ICCV 2025) are also absent.
2)I don't think “Long-range consistency” is actually measured, and the dynamic-degree evidence is fragile and read in both directions. VBench-Long splits each 60s video into 30 short clips and scores subject and background consistency within each clip. Those numbers can’t detect whether the child at t=60s matches the child at t=0s, which is the paper’s central claim (Fig. 1).
- The ablations make me doubt the headline delta is stable: Four identical state tokens, which the authors acknowledge carry the same information and only change attention mass, drop dynamic degree from 52.85 to 4.96. Restricting TR to the last 10 or first 10 blocks gives 26.78 / 25.83, while the last 20 give 56.00. All of these are single training runs with no seeds or variance.
Questions for authors
-
Please report each host post-trained with the identical recipe without TR: same iterations, data, extended rollouts and prompt switches, LoRA/full fine-tuning as in Table 3. Without this, I can’t attribute the Table 1 deltas to the memory. A TR-zeroed version of the same run (state forced to 0) would probably also work.
-
The AI-use statement says LLMs were used “to process the data used for the experiments”. What exactly does that mean: prompt extension, prompt-pair creation for switches, or filtering?
3 Can you add a metric that measures long-range identity directly? For example, DINO/CLIP or face-ID similarity between the first clip and clips at 20/40/60s, or a retrieval test where an object reappears after leaving the cache window. Does TR improve on the host there?