- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DeltaVideo adds a fixed-size recurrent memory to causal video generation. A gated Delta rule updates the memory using information that the current state fails to predict, while two readout branches guide attention and supply historical features. Training on repeated histories encourages the memory and its readouts to remain stable under redundant observations. On 30-second videos, the method improves object-revisit and memory-continuity scores over the tested baselines while maintaining comparable overall video quality. Ablations examine the update rule, redundancy-aware training and readout branches, and controlled experiments probe how the memory responds to repeated content and influences generation.
Critical strengths
-
The interaction between Delta updates and redundancy-aware training is supported by the ablations. RA raises ORS from 24.60 to 32.74 with Delta updates, while it does not improve ORS with additive updates (Table 2).
-
The evaluation directly tests memory retention. CoM reaches 47.70 versus 41.81 for the strongest baseline (Table 1), and the controlled repetition tests show smaller updates during repeated inputs followed by recovery when natural observations resume.
Critical weaknesses
-
It is unclear whether the CF++ baseline receives the same additional OpenVid training and long-rollout adaptation. Table 2 controls the memory components but lacks a memory-free variant, so the main CoM gain is not cleanly separated from the additional training.
-
The practical cost of the memory is not quantified. Section 3.3 re-encodes completed frames through the DiT, but no end-to-end latency or peak GPU memory measurements are reported. Fixed-size state storage alone does not establish the cost of using this method for streaming generation.
Questions for authors
-
Was the CF++ baseline trained with the same data, additional training budget and long-rollout adaptation? Please report this matched control without the memory module.
-
What are the latency and peak GPU memory with and without Delta-Memory? Does clean-frame re-encoding reuse the baseline's KV-cache update pass, or require an additional pass?
4 comments
Official Reply for Reviewer AJQu(1/2)
(Weakness #1) It is unclear whether the CF++ baseline receives the same additional OpenVid training and long-rollout adaptation. Table 2 controls the memory components but lacks a memory-free variant, so the main CoM gain is not cleanly separated from the additional training.
(Question #1) Was the CF++ baseline trained with the same data, additional training budget and long-rollout adaptation? Please report this matched control without the memory module.
Re: Thank you for pointing out the issue of attribution. The current paper does not establish that the CF++ baseline undergoes exactly the same additional training and long-horizon generation adaptation. Although Table 2 isolates the memory components under a reduced training schedule, it does not include a memory-free control. Therefore, we agree that these comparisons alone cannot clearly disentangle the contributions of the complete memory design from those of additional training.
To address this concern, we have clarified the training procedure of the CF++ baseline and introduce an additional memory-free baseline under matched training conditions. Both models start from the same publicly released CF++ AR diffusion checkpoint and use identical datasets, optimizer update steps, and applicable causal distillation, DMD, and long-horizon generation settings. The RA stage is exclusively designed to train the newly introduced memory module, with the generation backbone frozen throughout this stage. Therefore, the memory-free CF++ baseline naturally skips RA, as it has no memory module or corresponding RA objectives to optimize.
Table A1. 30-second evaluation of memory-free and memory-enabled models under matched additional training conditions.
| Model | ORS ↑ | Camera Control ↑ | Memory Continuity ↑ | Quality ↑ | Subject Consistency ↑ | Dynamic Degree ↑ | Imaging Quality ↑ |
|---|---|---|---|---|---|---|---|
| Self Forcing | 35.86 | 16.75 | 25.37 | 78.43 | 97.67 | 30.13 | 68.46 |
| Causal Forcing++ (2-step per frame) | 35.18 | 17.34 | 41.81 | 78.36 | 98.21 | 23.26 | 70.24 |
| LongLive 1.0 | 36.93 | 15.36 | 32.55 | 79.39 | 98.61 | 19.51 | 70.89 |
| LongLive 2.0 | 31.69 | 17.61 | 38.18 | 79.71 | 98.39 | 33.43 | 69.87 |
| Rolling Forcing | 42.27 | 14.60 | 36.37 | 78.86 | 98.35 | 17.65 | 71.06 |
| Deep Forcing | 31.62 | 11.79 | 37.62 | 80.47 | 97.88 | 39.90 | 71.40 |
| CF++ w/o Memory (same pipeline) | 38.35 | 16.41 | 42.03 | 77.95 | 97.39 | 26.03 | 64.66 |
| Delta Video (Ours) | 45.02 | 16.49 | 47.70 | 80.99 | 96.74 | 58.71 | 71.12 |
Official Reply for Reviewer AJQu(2/2)
(Weakness #2) The practical cost of the memory is not quantified. Section 3.3 re-encodes completed frames through the DiT, but no end-to-end latency or peak GPU memory measurements are reported. Fixed-size state storage alone does not establish the cost of using this method for streaming generation.
(Question #2) What are the latency and peak GPU memory with and without Delta-Memory?
Re: We agree that a fixed storage size for recurrent memory does not necessarily imply low inference overhead. The actual cost includes memory retrieval during denoising, memory writing after clean-frame generation, and any additional computation required to obtain clean-frame features.
Table A2. Actual inference costs under identical hardware and generation settings.
| Model | Steady-state Latency per Latent Frame (ms) ↓ | Peak GPU Memory (GiB) ↓ |
|---|---|---|
| CF++ w/o Memory | 323.49 ± 0.69 ms | 18.562 GiB |
| Delta Video | 351.18 ± 1.77 ms (+8.56%) | 19.042 GiB (+2.59%) |
During steady-state generation, the memory-free model takes 323.49 ± 0.69 ms per latent frame, while Delta Video takes 351.18 ± 1.77 ms, corresponding to an additional overhead of 8.56%. These values represent the mean and sample standard deviation of the per-run averages across six generation runs. During the measurement interval, the peak allocated GPU memory reported by PyTorch increase from 18.562 GiB to 19.042 GiB, an increase of 0.480 GiB (2.59%).
Comment:
(Question #2) Does clean-frame re-encoding reuse the baseline's KV-cache update pass, or require an additional pass?
Re: Both configurations perform two denoising forward passes and one clean-context cache-update forward pass per latent frame during steady-state generation. Memory writing reuses the latter and therefore does not introduce an additional full DiT forward pass.
Regarding clean-frame re-encoding, we clarify that the memory-writing features are obtained from the layer-wise hidden representations produced by the clean-context DiT forward pass at (t=0). Importantly, this forward pass is shared with the baseline model's KV-cache update and does not require an additional DiT forward pass. Therefore, the memory module introduces zero additional full DiT forward passes per generated latent frame.
Thank you again for your support and for raising these important practical considerations. We hope that the matched-training comparison and explicit accounting of inference costs address your remaining concerns.
Hope to Get Your Reply
Dear Reviewer AJQu,
As the discussion deadline approaches, we wanted to briefly follow up on the two questions you raised. In the 30-second evaluation, Delta Video improved CoM from 42.03 to 47.70 over the memory-free baseline with all shared training stages matched. We also measured an 8.56% increase in steady-state latency per latent frame and a 2.59% increase in peak allocated GPU memory. We hope these results address your concerns.Your feedback is highly valuable to us, and we would appreciate any updates or further guidance you might have regarding our revisions and responses.
Thank you again for your time and constructive feedback.
Rebuttal by Authors
Dear Reviewer AJQu,
Thank you very much for taking the time to review Delta Video, for your constructive suggestions, and for your support of our work. We particularly appreciate your recognition of the interaction between Delta updates and redundancy-aware training, as well as our evaluation of long-term memory retention. In your review, you asked whether the performance gains persist under matched additional training and what practical inference overhead the memory introduces. During the rebuttal period, we added a memory-free baseline with matched shared training stages and long-rollout adaptation, measured steady-state generation latency and peak allocated GPU memory, and clarified how memory writing reuses the baseline's KV-cache update pass.
We hope that these additional results address your remaining concerns and further strengthen your confidence in Delta Video. Specifically, we have added the following experiments and clarifications:
Below, we provide detailed responses to your questions, including the matched comparison and inference-cost measurements.