acceptodds
Under review as a conference paper at ICLR 2027

DeltaVideo: Learning Residual Updates for Long-Term Video Memory

Abstract

World modeling requires maintaining persistent information about scenes and objects as the environment evolves. Causal streaming video generation offers a promising route toward this goal, yet limited attention windows restrict access to distant history, making long-term consistency difficult to maintain. Existing context-based memory methods extend this accessible history, but often retain redundant visual information across neighboring frames, leading to inefficient use of memory capacity. Efficiently retaining this history therefore requires exploiting the substantial visual redundancy across frames. We introduce Delta Video, a learnable online memory mechanism that learns residual memory updates, suppressing information already captured by the current state while preserving newly observed content. For memory updating, its frame-wise Delta rule predicts incoming visual representations from the current memory state and writes only the prediction residual, suppressing redundant updates for already well-represented information. Retrieved historical context is injected into attention layers to guide ongoing generation. To further teach the memory to handle redundant observations, we introduce redundancy-aware post-training that enforces consistency in both the recurrent memory state and its readout under constructed redundant history, encouraging stable and redundancy-aware memory preservation. On 30-second autoregressive generation, Delta Video improves Continuity of Memory by 14.1% over the strongest baseline. Controlled analyses further show that redundant observations induce 53.0% smaller changes in subsequent memory reads, providing direct evidence that Delta Video selectively suppresses already-represented information. Project page: https://delta-video.github.io/.

Primary area:
generative models
Keywords:
Video Generation, Long Video Generation, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_AJQuOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DeltaVideo adds a fixed-size recurrent memory to causal video generation. A gated Delta rule updates the memory using information that the current state fails to predict, while two readout branches guide attention and supply historical features. Training on repeated histories encourages the memory and its readouts to remain stable under redundant observations. On 30-second videos, the method improves object-revisit and memory-continuity scores over the tested baselines while maintaining comparable overall video quality. Ablations examine the update rule, redundancy-aware training and readout branches, and controlled experiments probe how the memory responds to repeated content and influences generation.

Critical strengths
  1. The interaction between Delta updates and redundancy-aware training is supported by the ablations. RA raises ORS from 24.60 to 32.74 with Delta updates, while it does not improve ORS with additive updates (Table 2).

  2. The evaluation directly tests memory retention. CoM reaches 47.70 versus 41.81 for the strongest baseline (Table 1), and the controlled repetition tests show smaller updates during repeated inputs followed by recovery when natural observations resume.

Critical weaknesses
  1. It is unclear whether the CF++ baseline receives the same additional OpenVid training and long-rollout adaptation. Table 2 controls the memory components but lacks a memory-free variant, so the main CoM gain is not cleanly separated from the additional training.

  2. The practical cost of the memory is not quantified. Section 3.3 re-encodes completed frames through the DiT, but no end-to-end latency or peak GPU memory measurements are reported. Fixed-size state storage alone does not establish the cost of using this method for streaming generation.

Questions for authors
  1. Was the CF++ baseline trained with the same data, additional training budget and long-rollout adaptation? Please report this matched control without the memory module.

  2. What are the latency and peak GPU memory with and without Delta-Memory? Does clean-frame re-encoding reuse the baseline's KV-cache update pass, or require an additional pass?

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Dear Reviewer AJQu,

Thank you very much for taking the time to review Delta Video, for your constructive suggestions, and for your support of our work. We particularly appreciate your recognition of the interaction between Delta updates and redundancy-aware training, as well as our evaluation of long-term memory retention. In your review, you asked whether the performance gains persist under matched additional training and what practical inference overhead the memory introduces. During the rebuttal period, we added a memory-free baseline with matched shared training stages and long-rollout adaptation, measured steady-state generation latency and peak allocated GPU memory, and clarified how memory writing reuses the baseline's KV-cache update pass.

We hope that these additional results address your remaining concerns and further strengthen your confidence in Delta Video. Specifically, we have added the following experiments and clarifications:

  • Added a matched memory-free control under the 30-second evaluation setting. Delta Video improves ORS from 38.35 to 45.02 and CoM from 42.03 to 47.70. We specify the shared training settings and the memory-specific RA stage. (Comment 1)
  • Quantified practical inference overhead under identical hardware and generation settings. Steady-state latency per latent frame increases from 323.49 ms to 351.18 ms (+8.56%), while peak allocated GPU memory increases from 18.562 GiB to 19.042 GiB (+2.59%). (Comment 2)
  • Clarified the implementation of clean-frame memory writing. The memory uses hidden representations from the existing clean-context KV-cache update pass, requiring no additional full DiT forward pass. (Comment 2)

Below, we provide detailed responses to your questions, including the matched comparison and inference-cost measurements.

AuthorsOpenReview ↗

Official Reply for Reviewer AJQu(1/2)

(Weakness #1) It is unclear whether the CF++ baseline receives the same additional OpenVid training and long-rollout adaptation. Table 2 controls the memory components but lacks a memory-free variant, so the main CoM gain is not cleanly separated from the additional training.

(Question #1) Was the CF++ baseline trained with the same data, additional training budget and long-rollout adaptation? Please report this matched control without the memory module.

Re: Thank you for pointing out the issue of attribution. The current paper does not establish that the CF++ baseline undergoes exactly the same additional training and long-horizon generation adaptation. Although Table 2 isolates the memory components under a reduced training schedule, it does not include a memory-free control. Therefore, we agree that these comparisons alone cannot clearly disentangle the contributions of the complete memory design from those of additional training.

To address this concern, we have clarified the training procedure of the CF++ baseline and introduce an additional memory-free baseline under matched training conditions. Both models start from the same publicly released CF++ AR diffusion checkpoint and use identical datasets, optimizer update steps, and applicable causal distillation, DMD, and long-horizon generation settings. The RA stage is exclusively designed to train the newly introduced memory module, with the generation backbone frozen throughout this stage. Therefore, the memory-free CF++ baseline naturally skips RA, as it has no memory module or corresponding RA objectives to optimize.

Table A1. 30-second evaluation of memory-free and memory-enabled models under matched additional training conditions.

ModelORS ↑Camera Control ↑Memory Continuity ↑Quality ↑Subject Consistency ↑Dynamic Degree ↑Imaging Quality ↑
Self Forcing35.8616.7525.3778.4397.6730.1368.46
Causal Forcing++ (2-step per frame)35.1817.3441.8178.3698.2123.2670.24
LongLive 1.036.9315.3632.5579.3998.6119.5170.89
LongLive 2.031.6917.6138.1879.7198.3933.4369.87
Rolling Forcing42.2714.6036.3778.8698.3517.6571.06
Deep Forcing31.6211.7937.6280.4797.8839.9071.40
CF++ w/o Memory (same pipeline)38.3516.4142.0377.9597.3926.0364.66
Delta Video (Ours)45.0216.4947.7080.9996.7458.7171.12
AuthorsOpenReview ↗

Official Reply for Reviewer AJQu(2/2)

(Weakness #2) The practical cost of the memory is not quantified. Section 3.3 re-encodes completed frames through the DiT, but no end-to-end latency or peak GPU memory measurements are reported. Fixed-size state storage alone does not establish the cost of using this method for streaming generation.

(Question #2) What are the latency and peak GPU memory with and without Delta-Memory?

Re: We agree that a fixed storage size for recurrent memory does not necessarily imply low inference overhead. The actual cost includes memory retrieval during denoising, memory writing after clean-frame generation, and any additional computation required to obtain clean-frame features.

Table A2. Actual inference costs under identical hardware and generation settings.

ModelSteady-state Latency per Latent Frame (ms) ↓Peak GPU Memory (GiB) ↓
CF++ w/o Memory323.49 ± 0.69 ms18.562 GiB
Delta Video351.18 ± 1.77 ms (+8.56%)19.042 GiB (+2.59%)

During steady-state generation, the memory-free model takes 323.49 ± 0.69 ms per latent frame, while Delta Video takes 351.18 ± 1.77 ms, corresponding to an additional overhead of 8.56%. These values represent the mean and sample standard deviation of the per-run averages across six generation runs. During the measurement interval, the peak allocated GPU memory reported by PyTorch increase from 18.562 GiB to 19.042 GiB, an increase of 0.480 GiB (2.59%).

Comment:

(Question #2) Does clean-frame re-encoding reuse the baseline's KV-cache update pass, or require an additional pass?

Re: Both configurations perform two denoising forward passes and one clean-context cache-update forward pass per latent frame during steady-state generation. Memory writing reuses the latter and therefore does not introduce an additional full DiT forward pass.

Regarding clean-frame re-encoding, we clarify that the memory-writing features are obtained from the layer-wise hidden representations produced by the clean-context DiT forward pass at (t=0). Importantly, this forward pass is shared with the baseline model's KV-cache update and does not require an additional DiT forward pass. Therefore, the memory module introduces zero additional full DiT forward passes per generated latent frame.

Thank you again for your support and for raising these important practical considerations. We hope that the matched-training comparison and explicit accounting of inference costs address your remaining concerns.

AuthorsOpenReview ↗

Hope to Get Your Reply

Dear Reviewer AJQu,

As the discussion deadline approaches, we wanted to briefly follow up on the two questions you raised. In the 30-second evaluation, Delta Video improved CoM from 42.03 to 47.70 over the memory-free baseline with all shared training stages matched. We also measured an 8.56% increase in steady-state latency per latent frame and a 2.59% increase in peak allocated GPU memory. We hope these results address your concerns.Your feedback is highly valuable to us, and we would appreciate any updates or further guidance you might have regarding our revisions and responses.

Thank you again for your time and constructive feedback.

Reviewer_KNsaOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the challenge of retaining important historical information during long-horizon autoregressive video generation under limited context and memory capacity. The authors argue that repeatedly incorporating already-represented visual information can cause unnecessary memory updates and interfere with previously stored information. To address this issue, they propose DeltaVideo, a fixed-size recurrent memory mechanism that updates memory using gated residuals between observed visual features and features predicted from the existing memory. They further introduce redundancy-aware supervision, which encourages both the memory states and the information retrieved from them to remain stable under redundant observations. Finally, historical information retrieved from memory is incorporated through two pathways. One modifies attention queries, while the other augments attention outputs.

Critical strengths

(1) The redundancy-aware supervision is a strength because it explicitly encourages stable memory states and retrieved information under repeated observations, helping suppress unnecessary changes to stored information. Table 2(a) supports the benefit of this supervision when combined with the Delta update, showing improvements in both ORS and CoM.

(2) The dual-path memory design is a strength because it enables historical information to guide attention over the available local context through the Q pathway and directly supplement attention outputs through the O pathway. Table 2(b) supports the benefit of combining these pathways, showing higher ORS and CoM than either pathway alone.

Critical weaknesses

(1) The lack of separate ablations for L_state and L_read is a weakness because it leaves their individual contributions and the benefit of combining them unclear. Table 2(a) evaluates only the combined RA objective against a model without RA. Comparing state-only, read-only, and combined variants under matched training conditions would clarify whether the two losses provide complementary benefits.

(2) The lack of controlled evaluation distinguishing appearance variations that preserve object identity and state from meaningful state changes is a weakness because it leaves the selectivity of the proposed memory updates unclear. Figure 4 demonstrates stability under exact frame repetition, while Figure 5 shows an association between visual changes and memory corrections. However, these results do not directly establish whether the model avoids unnecessary memory changes while incorporating important new state information and preserving it in subsequent generation. Controlled comparisons between these cases would help substantiate the claim of selective memory updating.

Questions for authors

(1) Clarification on the RA ablation: Were L_state and L_read evaluated individually under matched training conditions? If so, could you report the results and clarify whether combining them improves performance over either loss alone?

(2) Clarification on selective memory updates: Beyond exact frame repetition and the correlation between visual changes and memory corrections, what evidence supports the model’s ability to remain stable under incidental appearance variations while retaining meaningful new state information? Do the current evaluations distinguish these cases, and does RA preserve sensitivity to meaningful changes compared with the model without RA?

8 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Dear Reviewer KNsa,

Thank you very much for taking the time to review Delta Video and for your detailed and constructive feedback. We appreciate your recognition of redundancy-aware supervision and the complementary roles of the Q and O readout pathways. In your review, you asked about the individual contributions of the two RA losses and whether the memory remains stable under incidental appearance variations while retaining meaningful new state information. During the rebuttal period, we conducted separate ablations of the state and readout consistency losses, a comparison of readout dynamics on 24 real-world videos, and a delayed state-recovery experiment covering 10 scenarios.

We sincerely hope that these additional experiments will address your concerns and encourage you to reconsider your assessment of Delta Video and support its acceptance. Specifically, we have added the following results and discussions:

  • Separated the contributions of the two RA losses. We compare no RA, state-only, readout-only, and combined training. The combined objective achieves the highest ORS and CoM, improving ORS from 30.02 to 32.74 and CoM from 31.81 to 32.04 over readout-only training. We also explain the different roles of the two objectives. (Comment 1)
  • Evaluated stability and sensitivity under natural observations. Using identical video inputs and fixed probes, we compare models with and without RA across stable periods and object-state transitions. RA reduces readout variation while increasing the transition-to-stable contrast from approximately 26% to 88%. (Comment 2)
  • Tested whether new states are retained and used in subsequent generation. After the target remains out of view for 24 latent time steps, exceeding the 21-step local window, RA improves unchanged-state recovery from 20% to 60% and new-state recovery from 10% to 60%. (Comment 3)

Below, we provide detailed responses to your questions and present the settings and results of these additional experiments.

AuthorsOpenReview ↗

Official Reply for Reviewer KNsa(1/3)

(Weakness #1) The lack of separate ablations for L_state and L_read is a weakness because...

(Question #1) Clarification on the RA ablation: Were L_state and L_read evaluated individually under matched training conditions? If so, could you report the results and clarify whether combining them improves performance over either loss alone?

Re: Thank you for pointing out this missing comparison. To disentangle the contributions of the two RA loss terms, we compare four variants: neither loss, state consistency only, readout consistency only, and both losses. All variants are initialized from the same checkpoint before RA training and use identical data, training steps, and subsequent training procedures.

Table K1. Ablation of the state and readout consistency losses in RA.

State ConsistencyReadout ConsistencyORS ↑CoM ↑Quality ↑
OffOff24.6030.6480.02
OnOff26.3730.5279.82
OffOn30.0231.8180.04
OnOn32.7432.0480.16

State consistency alone improves ORS but slightly decreases CoM, while readout consistency alone improves both memory metrics. Combining the two losses yields the highest ORS and CoM, with Quality remaining comparable across all four variants.

AuthorsOpenReview ↗

Official Reply for Reviewer KNsa(2/3)

(Weakness #2) The lack of controlled evaluation distinguishing appearance variations that preserve object identity and state from meaningful state changes is a weakness because it leaves the selectivity of the proposed memory updates unclear.

(Question #2) Clarification on selective memory updates: Beyond exact frame repetition and...

Re: Thank you for pointing out that stability under completely repeated frames is insufficient to characterize how the model handles redundancy in natural videos. To address this, we provide additional video analyses involving natural appearance variations and object state changes, examining memory update behavior across three phases: a stable period before an event, the state transition period, and a stable period after the event.

In this evaluation, we use 24 real world videos and feed the RA-enabled and RA-disabled models identical teacher forcing inputs. We select 6-second stable windows before and after each event and use a fixed probe within each video. The per-step relative readout change is defined as the L2 norm of the difference between consecutive readouts, normalized by the L2 norm of the preceding readout.

Table K2. Memory readout dynamics under fixed probes and teacher-forcing inputs.

MetricPre-event stableState transitionPost-event stable
Mean per-step relative readout change (with RA)0.01450.0282 (+87.67%)0.0155
Mean per-step relative readout change (without RA)0.05960.0763 (+25.66%)0.0618
Mean optical flow magnitude3.21933.15693.2204

We use the average of the pre-event and post-event mean per-step relative readout changes as the stable-period baseline, and calculate the percentage increase during the state transition relative to this baseline. Although the per-step relative readout changes are smaller with RA across all three phases, the contrast between state-transition and stable periods is substantially larger. Specifically, the state-transition value exceeds the average of the two surrounding stable-period values by 87.67% with RA, compared to 25.66% without RA.

These results extend our original analysis under completely repeated frames to natural video scenarios. Under the current evaluation conditions, the model exhibits memory update behavior that varies according to the event phase.

Reviewer_HCKrOpenReview ↗
Rating
3(+2)
Confidence
2(−1)
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Delta Video, a recurrent memory mechanism for improving long-term consistency in autoregressive video generation. It uses frame-wise Delta Rule updates to suppress redundant information while preserving new observations, and introduces dual-path memory readout and redundancy-aware training to improve memory retention. Experiments on long-video generation benchmarks demonstrate improvements in object revisiting and memory consistency over existing methods. Ablation studies and memory analysis further validate the effectiveness of the proposed components.

Critical strengths

Reasonable motivation and empirical improvements. The paper explores a relevant problem in long-term video generation and demonstrates improvements in memory consistency over existing baselines. The ablation and memory analysis experiments provide additional support for the proposed approach.

Critical weaknesses
  1. Overly complicated and insufficiently motivated method. The proposed method combines gated Delta Rule updates, dual memory readout branches, and additional state and readout consistency losses within a multi-stage training pipeline. While the paper provides component-level ablations, the motivation for this particular combination of designs is not sufficiently convincing. Notably, the Delta Rule alone performs worse than a simple additive update on ORS (24.60 vs. 29.46 in Table 2), and its advantage only emerges when combined with redundancy-aware training. Overall, the method feels unnecessarily complicated relative to the insights it provides.

  2. Insufficiently convincing experimental validation. The main comparisons in Table 1 involve models with different training pipelines, making it difficult to attribute the improvements specifically to the proposed memory mechanism. The controlled ablations in Table 2 use a shorter generation horizon and reduced training schedule. Furthermore, the redundancy analysis in Figure 4 relies on exact frame repetitions, which does not establish whether the memory can effectively distinguish meaningful new information from naturally redundant observations. These limitations weaken the evidence supporting the central claims.

Questions for authors
  1. Attribution of performance gains. Table 1 compares methods with different training pipelines, while Table 2 uses a reduced training schedule and shorter generation horizon. Can the authors provide a comparison with a no-memory baseline under the same training pipeline and 30-second evaluation setting? This would help clarify how much of the improvement comes from the proposed memory mechanism.

  2. Redundancy in realistic videos. Figure 4 demonstrates that the memory suppresses updates when identical frames are repeated. However, redundancy in real videos typically involves small changes in viewpoint, motion, or appearance. Does the proposed mechanism remain effective under such naturally redundant observations, while still preserving genuinely new information?

7 comments
AuthorsOpenReview ↗

Rebuttal by Authors

Dear Reviewer HCKr,

Thank you very much for taking the time to review Delta Video and for your valuable feedback. We appreciate your recognition of the motivation and empirical improvements of our work. In your review, you raised concerns about the motivation for combining the memory components, the attribution of the performance gains, and the effectiveness of the method under naturally redundant observations. During the rebuttal period, we conducted a 30-second comparison with a memory-free baseline under matched shared training stages, and additional experiments on natural videos. We also clarify the roles of the memory components and their relationship to the adopted training pipeline.

We sincerely hope that these additional results and clarifications will address your concerns and encourage you to reconsider your assessment of Delta Video and support its acceptance. Specifically, we have added the following experiments and discussions:

  • Clarified the training pipeline and the role of RA. The generation-training stages follow Causal Forcing++ and LongLive-style adaptation, while RA is the additional stage introduced to train the memory module. (Comment 1)
  • Explained the motivation for the combined memory design. We clarify the roles of residual updates, RA supervision. (Comment 2)
  • Added a matched memory-free control at the full 30-second horizon. Delta Video improves ORS from 38.35 to 45.02 and CoM from 42.03 to 47.70 over this control. (Comment 2)
  • Extended the redundancy analysis to 24 real-world videos. We compare readout behavior with and without RA during natural appearance variations and meaningful object-state transitions. (Comment 3)

Below, we provide detailed responses to each of your concerns, together with the additional experimental results.

AuthorsOpenReview ↗

Official Reply for Reviewer HCKr(1/3)

(Weakness #1) Overly complicated and insufficiently motivated method. The proposed method combines gated Delta Rule updates, dual memory readout branches, and additional state and readout consistency losses within a multi-stage training pipeline.

Re: Our central motivation is that effective video memory should selectively update its contents when repeatedly observing the same information, preserving useful historical information while remaining capable of incorporating new information. Delta Video achieves this through three complementary components: a residual-based memory update mechanism, supervision specifically designed to promote stability under redundant observations, and interfaces that allow the generator to access stored historical information.

(a) Multi-stage Training Pipeline

We appreciate your concern regarding the complexity of the training pipeline. In fact, the multi-stage training adopted in our paper follows a widely adopted forcing-style training paradigm in few-step autoregressive video generation. Specifically, this paradigm first performs autoregressive diffusion training (AR training) to adapt the model to generation under causal conditioning; subsequently, ODE initialization or causal consistency distillation (causal CD) is employed to provide an initialization for few-step generation; finally, Distribution Matching Distillation (DMD) further optimizes the model on its own generated sequences, followed by LongLive-style DMD to improve its long-horizon rollout capability. Causal Forcing++ specifically adopts the causal CD branch of this paradigm. These stages serve distinct purposes:causal generation adaptation, few-step initialization, and autoregressive generation optimization,and together constitute an established training pipeline.

Our training follows this framework and further adopts LongLive style DMD to extend long-horizon rollout capability. Therefore, the complexity of the main multi-stage training pipeline originates from the existing training paradigm we adopt. The only additional training stage introduced by our work is the RA stage between causal CD and DMD, which provides direct supervision for the memory mechanism under redundant observations. Both the state consistency loss and the readout consistency loss are applied exclusively in this stage, while the remaining stages retain their respective generation and distillation objectives.

(b) Related Work on the Delta Rule and Its Synergy with RA

(Weakness #1) Notably, the Delta Rule alone performs worse than a simple additive update on ORS (24.60 vs. 29.46 in Table 2), and its advantage only emerges when combined with redundancy-aware training. Overall, the method feels unnecessarily complicated relative to the insights it provides.

Re: We build upon the Delta Rule associative memory mechanism introduced in prior work, including DeltaNet and Gated DeltaNet (Yang et al., 2025c, 2025b), which are also cited in our Introduction. This mechanism updates a fixed size state based on the residual between the input features and the features predicted by the current memory state, providing a structural basis for reducing redundant writes of previously stored information. VideoSSM (Yu et al., 2025) has also adopted gated Delta Rule updates for global memory updates in long video generation. In our adaptation to Delta Video, we aggregate spatial tokens corresponding to the same preceding state and perform one recurrent update after processing each frame.

We agree with your observation that, in Table 2(a), the Delta update without RA performs worse than the additive update in terms of ORS (24.60 vs. 29.46). This observation is, in fact, central to our experimental findings. The issue is not that the Delta rule itself is inherently ineffective, but rather that existing training paradigms do not provide sufficiently effective supervision for the memory module.

In our ablation experiments, general-purpose video post-training paradigms such as Causal Forcing++ primarily optimize overall video generation quality, and are not specifically designed to train the memory module. Consequently, without dedicated supervision, the memory module may not receive sufficient training signals to improve either general video quality or long-term memory performance. This motivates us to introduce redundancy-aware supervision, which provides dedicated training signals for the memory module initialized in Stage 2.

As shown in Table 2(a), incorporating RA improves the Delta update from 24.60 to 32.74 in ORS and from 30.64 to 32.04 in CoM. In contrast, for the additive update, RA changes ORS from 29.46 to 29.02 and CoM from 27.30 to 27.96. These results demonstrate a strong synergy between the Delta rule and RA supervision.

AuthorsOpenReview ↗

Official Reply for Reviewer HCKr(2/3)

(Weakness #2) Insufficiently convincing experimental validation. The main comparisons in Table 1 involve...

(Question #1) Can the authors provide a comparison with a no-memory baseline under the same training pipeline and 30-second evaluation setting? This would help clarify how much of the improvement comes from the proposed memory mechanism.

Re: We agree that Table 1 alone does not sufficiently isolate the contribution of memory. To address this concern, we have added a matched comparison between a memory-free CF++ model and Delta Video. Both models start from the same official Causal Forcing++ AR diffusion checkpoint and use identical training data, optimizer update steps, causal distillation/DMD configurations, and long-horizon generation adaptation for all shared training stages. The RA stage is exclusively designed to train the newly introduced memory module, with the generation backbone frozen throughout this stage. Therefore, the memory-free CF++ baseline naturally skips RA, as it has no memory module or corresponding RA objectives to optimize.

Table H1. 30-second evaluation under matched training conditions.

ModelORS ↑Camera Control ↑Memory Continuity ↑Quality ↑Subject Consistency ↑Dynamic Degree ↑Imaging Quality ↑
Self Forcing35.8616.7525.3778.4397.6730.1368.46
Causal Forcing++ (2-step per frame)35.1817.3441.8178.3698.2123.2670.24
LongLive 1.036.9315.3632.5579.3998.6119.5170.89
LongLive 2.031.6917.6138.1879.7198.3933.4369.87
Rolling Forcing42.2714.6036.3778.8698.3517.6571.06
Deep Forcing31.6211.7937.6280.4797.8839.9071.40
CF++ w/o Memory (matched additional training)38.3516.4142.0377.9597.3926.0364.66
Delta Video (Ours)45.0216.4947.7080.9996.7458.7171.12

Comments on the paper

1 comment
AuthorsOpenReview ↗

A heartfelt general comment from the authors

Dear Reviewers and Area Chair,

We sincerely thank Reviewers HCKr, KNsa, and AJQu for their time and constructive feedback. We are pleased that the reviewers recognized the motivation for addressing long-term video memory and the empirical improvements of Delta Video (Reviewer HCKr), the value of redundancy-aware supervision and the complementary roles of the Q and O readout pathways (Reviewer KNsa), as well as the interaction between Delta updates and redundancy-aware training and our evaluation of long-term memory retention (Reviewer AJQu).

We have meticulously addressed each of the reviewers' suggestions one by one and incorporated these feedbacks into the revised version.During the rebuttal period, we conducted numerous additional experiments, and all extended experiments are consistent with the conclusions of the submitted version. The main updates in the revised version include:

(Reviewers HCKr and AJQu) Added a 30-second comparison with a memory-free baseline under matched shared training stages, showing improvements of 6.67 points in ORS and 5.67 points in CoM. [Table 1; Section 4.2]

(Reviewer KNsa) Added separate ablations of the state and readout consistency losses, showing that combining both achieves the highest ORS and CoM under matched training conditions. [Table 2(d); Section 4.3]

(Reviewers HCKr and KNsa) Extended the redundancy analysis to 24 real-world videos, showing reduced readout variation with RA and a stronger transition-to-stable contrast—approximately 88% versus 26% without RA. [Section 4.4]

(Reviewer KNsa) Added delayed state-recovery experiments in 10 scenarios beyond the local attention window. RA improves unchanged-state recovery from 20% to 60% and new-state recovery from 10% to 60%. [Section 4.4]

(Reviewer AJQu) Quantified inference overhead across six runs: +8.56% latency and +2.59% peak GPU memory. Clarified that memory writing reuses the existing KV-cache update pass, requiring no additional full DiT forward pass. [Section 4.2]

We sincerely appreciate the reviewers’ time and welcome further discussion. During the rebuttal period, we have made our best efforts and shown our utmost sincerity in addressing the reviewers' concerns.We hope that the reviewers can see our efforts and continue the discussion.If this round has resolved the reviewers' concerning, we earnestly and sincerely request the reviewers to consider increasing the scores and supporting the acceptance of our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.