acceptodds
Under review as a conference paper at ICLR 2027

Recast the Past: Stable Long Video Generation with Full Context Memory

Abstract

Long-horizon video generation requires retaining an ever-expanding history, but conditioning on its growing token sequence is costly. Existing methods restrict or compress historical context, potentially losing information needed for future generation. This work proposes ReCam, which groups local spatiotemporal blocks of historical latents into vectors and projects them directly into the high-dimensional hidden space of a Diffusion Transformer. The model learns to use the resulting memory tokens through native self-attention, reducing token count while retaining block-level access to the full history. To mitigate temporal drift during autoregressive rollout, we further introduce dual-pointer denoising with separate noise schedules for recent context and target frames. Among the evaluated methods, ReCam achieves the highest VBench-Long overall score of 80.1 and competitive static-scene revisitation performance on R2M-Bench. It also leads on all four consistency metrics for off-screen subject reappearance, including clothing consistency of 4.09 versus 3.03 for the best baseline.

Primary area:
generative models
Keywords:
Video Generation, World Model, Memory Mechanism

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xrfvOpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReCam, a memory method for autoregressive long video generation. It is built on Wan2.2-I2V-A14B. The full video history is split into local spatiotemporal blocks. Each block is flattened into a vector and projected by a small MLP into the hidden space of the DiT. The paper also introduces dual-pointer denoising, which gives recent context frames and target frames separate noise schedules to reduce drift.

Critical strengths
  • The architecture has some clear advantages. The model learns how to represent and read its memory end to end, and no memory labels are needed.
  • The hyperparameter analysis in the appendix is careful and thorough.
  • The subject reappearance results are reported separately for subjects that are present and absent in the first frame. This split directly shows the value of accessing distant history.
Critical weaknesses
  • The videos in the supplementary material show highly repetitive motion and simple structure. Many models built on Wan2.1-1.3B can already produce this kind of result (such as echo-infinity). The paper uses the much larger Wan2.2-I2V-A14B, yet the method is not real-time and cannot follow complex action instructions. This makes the result feel incremental. If possible, I would like to see more complex videos of at least two minutes, in which a person leaves the scene for a while and then comes back.
  • The novelty is limited. The core idea of FramePack is to compress history frames with larger patchify kernels to reduce the token count. ReCam flattens each block and applies an MLP, which is essentially a larger patch embedding. The dual-pointer design is also close to methods that add separate noise to history and target frames, such as Diffusion Forcing [1] and History-guided Video Diffusion [2].
  • The ablation study misses a key comparison. The memory mechanism of ReCam should be compared with other designs under the same backbone and the same data. Possible options include a sliding window only, FramePack-style hierarchical compression, the full uncompressed history, and a simple keyframe retrieval baseline. These do not depend on any released code, and they would show whether the gains come from the memory mechanism itself.

[1] Boyuan Chen et al. Diffusion Forcing: Next-Token Prediction Meets Full-Sequence Diffusion. NeurIPS 2024.

[2] Kiwhan Song et al. History-guided Video Diffusion. arXiv 2025.

Questions for authors
  • When the history grows to several minutes, how do GPU memory, speed and memory performance change?
8 comments
AuthorsOpenReview ↗

Response to Reviewer xrfv (Part 1/2): Complex Actions and Methodological Distinctions (W1–W2)

Thank you for your positive assessment of our architecture and experimental analysis, and for your constructive feedback.

W1. The videos in the supplementary material show highly repetitive motion and simple structure. The paper uses the much larger Wan2.2-I2V-A14B, yet the method is not real-time and cannot follow complex action instructions.

Thank you for suggesting more challenging demonstrations. The original supplementary videos come from benchmarks designed primarily to assess visual quality and memory, which typically use simple prompts, such as “a panda bear is eating a piece of bamboo” from VBench. Their simple activity structure reflects these evaluation prompts rather than demonstrating an inability to follow complex action instructions.

We evaluate ReCam against strong baselines with comparable or larger parameter counts. These comparisons provide quantitative evidence for the effectiveness of our approach. For more qualitative results, please see the newly added demonstrations in the supplementary material for more complex action sequences and subject reappearance after extended absences.

W2. Limited novelty compared with FramePack’s history compression and the history-noising strategies in Diffusion Forcing and History-guided Video Diffusion.

We appreciate the opportunity to clarify the two contributions and their differences from these prior methods.

1. Learning to use the full history. Our contribution is to identify and demonstrate the importance of providing the full, continuous history to the video generator, rather than the flattening and MLP operations themselves. Reducing token count alone does not determine what historical information remains available to the generator. While FramePack allocates finer representations to recent history, ReCam applies the same block representation across the full history, letting the generator learn which historical information to use. This preserves access to both historical content and its evolution over time, while high-dimensional block representations make reading the full history computationally practical. FramePack’s increasing compression also limits its usable history length: the official F1 implementation retains 19 historical latent frames, corresponding to approximately 4.75 seconds at 16 fps. Our controlled comparison in W3 (Part 2/2) directly tests this distinction: with the same backbone, training data, and training budget, ReCam achieves better overall character and clothing consistency than FramePack-style compression.

2. Dual-pointer denoising. We clarify that the contribution of dual-pointer denoising is to ease the optimization difficulty introduced by adding noise to historical context. The key design is to provide two views of the same historical context with shared noise but different noise levels, rather than simply assign different noise levels to historical context and current content.

This distinction matters because context noising weakens the appearance and motion cues available for predicting subsequent content, increasing prediction uncertainty and task difficulty. Dual-pointer denoising provides additional information through the relationship between its two views. For context OO and shared noise ϵ∼N(0,I)\epsilon\sim\mathcal{N}(0,I), we construct

\qquad \tilde O_{\mathrm m}=(1-\sigma_{\mathrm m})O+\sigma_{\mathrm m}\epsilon.$$ At unequal noise levels, their normalized difference directly exposes the context velocity: $$\frac{\tilde O_{\mathrm m}-\tilde O_{\mathrm c}} {\sigma_{\mathrm m}-\sigma_{\mathrm c}} =\epsilon-O =v_{\mathrm{ctx}}^{*}.$$ This relation provides **an input-level shortcut aligned with the velocity-prediction objective**, giving the generator additional information for interpreting noisy context and predicting subsequent content. **Our single-pointer ablation already assigns different noise levels to historical context and current content.** Compared with this setting, dual-pointer denoising achieves lower training loss (Fig. 4(b)) and better results on all four temporal-drift metrics (Table 4(b)). Its final training loss approaches that of clean-context training, supporting the optimization benefit, while the rollout visualization shows less accumulated color drift than clean context (Fig. 5). Part 2/2 presents the controlled memory ablations (W3) and the history-length scaling results (Q1).
AuthorsOpenReview ↗

Response to Reviewer xrfv (Part 2/2): Controlled Ablations and History Scaling (W3, Q1)

Following the methodological distinctions in Part 1/2, we provide the controlled comparisons and scaling measurements.

W3. Missing controlled comparisons of memory mechanisms under the same backbone and training data.

Thank you for suggesting controlled memory ablations. We compare no-memory, sliding-window memory, keyframe retrieval, FramePack-style hierarchical compression, and ReCam on the subject-reappearance benchmark. ReCam achieves the best overall character and clothing consistency, with face distance comparable to keyframe retrieval, without an external retrieval model.

The memory-enabled variants use the same Wan2.2 backbone and training data, at 368p and 16 fps, with 10k training steps and a global batch size of 16. All receive the original video’s first frame as an additional anchor and use the same dual-pointer denoising scheme, differing only in how they represent and access historical context. The configurations are as follows:

  • No-memory: We use the trained ReCam model to generate from the continuation’s starting frame, with historical memory disabled.
  • Sliding window: We retain the most recent 2 seconds of historical latents using the native patch embedding.
  • Keyframe retrieval: During training, we uniformly sample one historical latent frame. At inference, we compute the CLIP similarity between the current prompt and all historical frames, select the top-ranked frame, and use its corresponding latent as the memory input.
  • FramePack-style hierarchical compression: Following the hierarchical packing strategy in FramePack, we divide historical latents into four temporal tiers. From newest to oldest, these tiers contain 1, 2, 16, and 32 latent frames, with compression factors k=1,2,4,8k=1,2,4,8, respectively. We cap the compression factor at k=8k=8, as further compression at our evaluation resolution would collapse the spatial token grid.
  • ReCam: We retain the full history for the generator.

Note that using the full history without compression causes computation and GPU memory requirements to grow rapidly with history length, making this baseline infeasible with available resources.

Each cell lists All / Present / Absent, with the best displayed value for each metric and group shown in bold, including ties.

METHODFACE ↓CHARACTER ↓CLOTHES ↑
No-memory0.77 / 0.78 / 0.778.56 / 8.14 / 8.861.31 / 1.67 / 1.04
Sliding window0.66 / 0.57 / 0.787.22 / 5.43 / 8.452.50 / 3.83 / 1.58
Keyframe retrieval0.65 / 0.57 / 0.746.91 / 6.53 / 7.183.00 / 4.38 / 2.04
FramePack-style0.68 / 0.59 / 0.807.30 / 6.77 / 7.662.66 / 4.33 / 1.50
ReCam0.65 / 0.57 / 0.745.56 / 5.67 / 5.483.45 / 4.39 / 2.80

The gains are largest in the initially absent split, where identity information must be recovered from intermediate history. Compared with keyframe retrieval, ReCam reduces character distance from 7.18 to 5.48 and improves clothing consistency from 2.04 to 2.80. We also observe that keyframe retrieval sometimes produces frames closely resembling the selected reference, with unnatural motion. A single retrieved frame provides appearance cues but omits the temporal dynamics available in ReCam’s full-history representation.

Q1. When the history grows to several minutes, how do GPU memory, speed and memory performance change?

We extend the efficiency measurements in Fig. 4(a) to two minutes of history, using the same profiling protocol on a single H100 80GB GPU at 448×832 resolution with BF16 and FlashAttention-3.

History lengthPeak GPU memory (GiB)Time per forward (s)
0 s31.631.320
5 s33.451.503
10 s33.761.579
20 s34.381.760
30 s34.991.898
45 s35.922.192
60 s36.852.449
90 s38.703.065
120 s40.563.726

Doubling the history from one to two minutes increases peak GPU memory by 10.1% and per-forward latency by 52.2%, while keeping both costs within a practical range.

Regarding memory performance, the model exhibits comparable performance for history lengths of 30–45 seconds, with degradation appearing at longer lengths. We attribute this degradation to the distribution shift beyond the maximum training history of 30 seconds. Training on longer videos is expected to help mitigate this issue. Generalization to out-of-distribution memory settings is also acknowledged as a limitation in our paper (Lines 484–486).

Could you report the history coverage, memory-token counts, and computational costs of each variant? The keyframe baseline also uses random sampling during training but CLIP retrieval at inference; could you clarify whether this affects the comparison?

More importantly, could you report subject-reappearance performance at longer history lengths, particularly when the subject has been absent for over a minute? The current results show that two-minute histories are computationally feasible, but do not yet establish effective memory at that length. Please also provide absolute GPU memory usage and end-to-end generation time.

Reviewer_Agb5OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ReCam, a long-term memory mechanism for autoregressive video generation. It compresses historical latents into memory tokens and introduces dual-pointer denoising to reduce drift. Experiments on long-video generation, scene revisitation, and subject reappearance show competitive results.

Critical strengths
  1. The design is refreshingly simple. History is packed into tokens and read through the model's own attention, no retriever, no extra loss, and it stays cheap: a one-minute history costs about as much as a vanilla 5-second configuration.

  2. I like the initially-absent split of the reappearance benchmark. When identity evidence is in neither the first frame nor the recent window, only mid-history access can help, and that is exactly where the gap shows up (FramePack-F1 face distance 0.745 vs ReCam 0.644).

  3. The ablations are clean and the discussion of the gap to HY-WorldPlay is candid.

Critical weaknesses
  1. The experiments never isolate the memory mechanism. Every baseline differs in backbone, data and training budget, and the ablations only vary block shape and the denoising scheme. So the paper shows the full system works, not that packing history into tokens is what makes it work. A same-backbone, same-data comparison of no-memory / sliding-window / keyframe retrieval / ReCam is the obvious missing experiment.

  2. R2M results use simplified custom trajectories instead of the official ones, and ReCam gets a camera-control adaptation (App. A) that the baselines apparently don't.

  3. The evaluation relies solely on automatic metrics, with clothes consistency judged by a single VLM and no user study on the 64 self-constructed cases. Moreover, cases where a method fails to generate the subject are excluded from the affected metrics, which biases the comparison toward methods that fail more often.

  4. The novelty is limited, as the main contribution is an architectural design without a substantial methodological advance.

Questions for authors
  1. Can you run a same-backbone (Wan2.2), same-data, same-budget comparison of no-memory, sliding-window, keyframe retrieval, and ReCam?

  2. Per method, how many of the 64 cases actually generated the subject? Does the ranking hold if failures are included?

  3. What are the results with the official R2M trajectories, and did all baselines get the same camera-control adaptation as ReCam?

  4. How well does the Gemini clothes-consistency judge agree with humans, even on a small sample?

8 comments
AuthorsOpenReview ↗

Response to Reviewer Agb5 (Part 1/5): Controlled Memory Ablations (W1, Q1)

Thank you for your positive assessment of our method and experimental design, and for your constructive feedback.

W1 & Q1. The ablation study on the memory mechanism.

Thank you for suggesting controlled memory ablations. We compare no-memory, sliding-window memory, keyframe retrieval, FramePack-style hierarchical compression, and ReCam on the subject-reappearance benchmark. ReCam achieves the best overall character and clothing consistency, with face distance comparable to keyframe retrieval, without an external retrieval model.

The memory-enabled variants use the same Wan2.2 backbone and training data, at 368p and 16 fps, with 10k training steps and a global batch size of 16. All receive the original video’s first frame as an additional anchor and use the same dual-pointer denoising scheme, differing only in how they represent and access historical context. The configurations are as follows:

  • No-memory: We use the trained ReCam model to generate from the continuation’s starting frame, with historical memory disabled.
  • Sliding window: We retain the most recent 2 seconds of historical latents using the native patch embedding.
  • Keyframe retrieval: During training, we uniformly sample one historical latent frame. At inference, we compute the CLIP similarity between the current prompt and all historical frames, select the top-ranked frame, and use its corresponding latent as the memory input.
  • FramePack-style hierarchical compression: Following the hierarchical packing strategy in FramePack, we divide historical latents into four temporal tiers. From newest to oldest, these tiers contain 1, 2, 16, and 32 latent frames, with compression factors k=1,2,4,8k=1,2,4,8, respectively. We cap the compression factor at k=8k=8, as further compression at our evaluation resolution would collapse the spatial token grid.
  • ReCam: We retain the full history for the generator.

Each cell lists All / Present / Absent, with the best displayed value for each metric and group shown in bold, including ties.

METHODFACE ↓CHARACTER ↓CLOTHES ↑
No-memory0.77 / 0.78 / 0.778.56 / 8.14 / 8.861.31 / 1.67 / 1.04
Sliding window0.66 / 0.57 / 0.787.22 / 5.43 / 8.452.50 / 3.83 / 1.58
Keyframe retrieval0.65 / 0.57 / 0.746.91 / 6.53 / 7.183.00 / 4.38 / 2.04
FramePack-style0.68 / 0.59 / 0.807.30 / 6.77 / 7.662.66 / 4.33 / 1.50
ReCam0.65 / 0.57 / 0.745.56 / 5.67 / 5.483.45 / 4.39 / 2.80

The gains are largest in the initially absent split, where identity information must be recovered from intermediate history. Compared with keyframe retrieval, ReCam reduces character distance from 7.18 to 5.48 and improves clothing consistency from 2.04 to 2.80. We also observe that keyframe retrieval sometimes produces frames closely resembling the selected reference, with unnatural motion. A single retrieved frame provides appearance cues but omits the temporal dynamics available in ReCam’s full-history representation.

Part 2/5 addresses the R2M evaluation protocol and camera-control training.

AuthorsOpenReview ↗

Response to Reviewer Agb5 (Part 2/5): R2M Evaluation and Camera Control (W2, Q3)

W2 & Q3. The evaluation protocol on R2M-Bench.

We would like to clarify three points about the evaluation setup. First, we observed that most baselines’ scores on the official R2M trajectories were affected by camera-control accuracy, consistent with R2M-Bench’s own discussion of this issue. Second, our rotation-and-return trajectory follows a commonly used protocol for evaluating scene memory. Third, all four baselines have already undergone dedicated camera- or action-control training.

First, R2M-Bench acknowledges that camera-execution errors can affect scene-memory evaluation, noting that “execution errors can remain entangled with generated-world persistence” (Section 7). We observed this issue in our experiments: inaccurate camera execution prevented most models from returning to the intended viewpoints, affecting their consistency scores. For example, Sana-WM obtains negative MG and NMR values on several metrics under the official trajectories.

Second, rotation-and-return is a common protocol for evaluating scene memory (e.g., Context as Memory and MemLearner). Our trajectory requires two returns to the initial viewpoint, extending their single-return setup to test repeated-revisit consistency. This protocol reduces the influence of camera-control differences, enabling a more focused comparison of scene memory.

To fully address your concern, we report the results with the official R2M trajectories below. ReCam achieves the second-best results on 12 of the 16 reported scores, while HY-WorldPlay shows a clear advantage, aided by its stronger camera-control capability.

Each cell reports MG / NMR; higher is better for both. The best results are shown in bold, and the second-best results are italics.

MethodPSNRSSIMLPIPSDINO-simBoQ-simMVPR-simApp. PersistSem. Persist
HY-WorldPlay2.357 / 0.4460.055 / 0.3080.155 / 0.4880.146 / 0.7340.208 / 0.5890.069 / 0.6260.087 / 0.6020.029 / 0.752
Sana-WM0.393 / 0.0940.008 / 0.0710.009 / 0.051−0.004 / 0.0020.003 / 0.012−0.019 / −0.049−0.001 / 0.016−0.003 / −0.007
LingBot-World-Fast0.283 / 0.0470.018 / 0.0830.007 / 0.0300.029 / 0.0730.041 / 0.0800.027 / 0.1030.022 / 0.1260.012 / 0.245
LingBot-World-2-Fast0.271 / 0.1980.009 / 0.1440.011 / 0.0450.083 / 0.0970.013 / 0.0400.057 / 0.1150.031 / 0.1120.014 / 0.311
ReCam1.025 / 0.2450.038 / 0.2800.063 / 0.2370.103 / 0.2900.111 / 0.2300.052 / 0.2590.026 / 0.1820.007 / 0.231

Third, all four baselines have already been explicitly trained for camera or action control, with dedicated conditioning mechanisms and annotated training data. ReCam’s adaptation uses 36K pose-annotated clips to acquire the camera-control capability needed for R2M-Bench evaluation. The baselines’ training setups are summarized below:

  • HY-WorldPlay: Its training corpus contains 320K video clips with camera annotations, and its dedicated Action Control stage runs for 60K iterations (Appendices A–B, Table 7).
  • Sana-WM: It uses approximately 213K clips with metric-scale camera-pose annotations. Its Minute-Scale Video + Camera Control stage runs for 31K steps (Section 4 and Appendix C.2, Table 7).
  • LingBot-World-Fast: It includes a dedicated action-conditioned fine-tuning stage that trains camera/action adapters using Plücker embeddings and AdaLN on data with camera annotations (Sections 2 and 3.3.2).
  • LingBot-World-2-Fast: It explicitly incorporates camera poses into its causal pretraining objective through Plücker embeddings and AdaLN, followed by consistency and distribution-matching distillation (Sections 2 and 3.2–3.3).

Parts 3/5 and 4/5 address subject-reappearance evaluation and human validation, respectively.

AuthorsOpenReview ↗

Response to Reviewer Agb5 (Part 3/5): Subject Reappearance Evaluation (W3, Q2)

W3 & Q2, Q4. The evaluation protocol on the subject-reappearance benchmark.

Thank you for highlighting the need to account for failed reappearances and validate the automatic metrics. We provide success rates, failure-inclusive and common-subset comparisons, and a human study. ReCam retains the best overall scores on all four automatic consistency metrics when failures are included. It also achieves the highest human scores on all three dimensions, both overall and in the initially absent split. Human and Gemini clothing scores have a Spearman correlation of 0.702.

1. Subject reappearance success rates.

MethodPresentAbsentAllSuccess rate
FramePack-F118/3225/3243/6467.2%
Helios12/3228/3240/6462.5%
LongLive 2.032/3232/3264/64100.0%
Sana-WM Streaming31/3229/3260/6493.8%
LingBot-World-Fast30/3229/3259/6492.2%
LingBot-World-2-Fast31/3232/3263/6498.4%
ReCam31/3232/3263/6498.4%

Reappearance failures are concentrated in FramePack-F1 and Helios, with 21 and 24 failures out of 64 cases, respectively; each remaining method has at most five failures.

2. Consistency including reappearance failures.

The original results average consistency scores over successful reappearances, since some metrics are undefined when the subject does not reappear.

To assess the effect of including failures, we assign explicit penalties: 1 for face and background cosine distances, 44 for character L2 distance, and 0 for clothing consistency. We report results across all 64 cases below. Each cell lists All / Present / Absent, with the best displayed value in each metric and group shown in bold, including ties.

METHODFACE ↓CHARACTER ↓BACKGROUND ↓CLOTHES ↑
FramePack-F10.73 / 0.66 / 0.8022.94 / 24.77 / 21.110.36 / 0.45 / 0.261.60 / 2.22 / 0.98
Helios0.84 / 0.87 / 0.8025.23 / 32.24 / 18.210.41 / 0.66 / 0.161.12 / 1.01 / 1.23
LongLive 2.00.63 / 0.55 / 0.7113.19 / 13.03 / 13.340.24 / 0.25 / 0.222.23 / 2.73 / 1.73
Sana-WM Streaming0.67 / 0.64 / 0.7113.87 / 11.90 / 15.830.13 / 0.10 / 0.162.16 / 2.75 / 1.57
LingBot-World-Fast0.62 / 0.60 / 0.6413.44 / 12.90 / 13.970.15 / 0.13 / 0.172.80 / 2.84 / 2.76
LingBot-World-2-Fast0.66 / 0.61 / 0.7012.61 / 12.32 / 12.890.12 / 0.10 / 0.131.56 / 1.64 / 1.47
ReCam0.58 / 0.52 / 0.6410.44 / 10.97 / 9.910.05 / 0.05 / 0.043.97 / 4.36 / 3.57

3. Consistency comparison on a common subset

We also report consistency results on a common subset where all compared methods succeed. Including FramePack-F1 and Helios reduces this subset to 27 cases. We therefore exclude these two baselines and report results on the 53 cases (28 Present, 25 Absent) where all remaining methods succeed.

Each cell lists All / Present / Absent, with the best displayed value for each metric and group shown in bold, including ties.

METHODFACE ↓CHARACTER ↓BACKGROUND ↓CLOTHES ↑
LongLive 2.00.62 / 0.55 / 0.7113.42 / 13.06 / 13.830.24 / 0.23 / 0.252.21 / 2.75 / 1.60
Sana-WM Streaming0.65 / 0.62 / 0.6811.79 / 10.87 / 12.820.08 / 0.07 / 0.092.34 / 3.04 / 1.55
LingBot-World-Fast0.58 / 0.56 / 0.6010.57 / 10.39 / 10.780.06 / 0.03 / 0.093.02 / 3.07 / 2.96
LingBot-World-2-Fast0.64 / 0.58 / 0.7012.10 / 11.36 / 12.930.11 / 0.08 / 0.141.51 / 1.64 / 1.36
ReCam0.56 / 0.49 / 0.649.93 / 10.10 / 9.750.04 / 0.02 / 0.054.01 / 4.42 / 3.56

Our response to W3 and Q4 continues in Part 4/5 with human evaluation and validation of the VLM clothing scores.

Reviewer_CXMYOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

ReCam packs historical video latents into high-dimensional DiT memory tokens and introduces dual-pointer context noising for autoregressive generation. It aims to preserve access to the full history while improving recall and reducing temporal drift.

Critical strengths
  • Simple, end-to-end memory design, supported by useful granularity, capacity, and efficiency analyses
  • Strong reported reappearance results: best aggregate scores across all four metrics, including clothing consistency of 4.09 versus 3.03. Testing subjects absent from the initial frame meaningfully probes intermediate-history recall.
Critical weaknesses
  • Failed subject reappearances are excluded without reporting per-method success rates or valid counts; consistency scores may therefore compare different successful subsets
  • At unequal noise levels, the two shared-noise views algebraically recover the original context, including its errors. This weakens the proposed error-masking explanation, although empirical benefits remain possible
Questions for authors

N/A

4 comments
AuthorsOpenReview ↗

Response to Reviewer CXMY (Part 1/2): Reappearance Evaluation (W1)

Thank you for recognizing our simple memory design and strong subject-reappearance results, and for your constructive feedback on evaluation and dual-pointer denoising.

W1. Failed subject reappearances are excluded without reporting per-method success rates or valid counts; consistency scores may therefore compare different successful subsets

Thank you for highlighting the potential bias from excluding failed reappearances. We report per-method success rates, scores with explicit failure penalties, and a comparison on a common successful subset. ReCam achieves a 98.4% reappearance success rate and retains the best overall scores on all four consistency metrics under the specified failure penalties and on the common subset of five methods.

1. Subject reappearance success rates.

MethodPresentAbsentAllSuccess rate
FramePack-F118/3225/3243/6467.2%
Helios12/3228/3240/6462.5%
LongLive 2.032/3232/3264/64100.0%
Sana-WM Streaming31/3229/3260/6493.8%
LingBot-World-Fast30/3229/3259/6492.2%
LingBot-World-2-Fast31/3232/3263/6498.4%
ReCam31/3232/3263/6498.4%

Reappearance failures are concentrated in FramePack-F1 and Helios, with 21 and 24 failures out of 64 cases, respectively; each remaining method has at most five failures.

2. Consistency including reappearance failures.

The original results average consistency scores over successful reappearances, since some metrics are undefined when the subject does not reappear.

To assess the effect of including failures, we assign explicit penalties: 1 for face and background cosine distances, 44 for character L2 distance, and 0 for clothing consistency. We report results across all 64 cases below. Each cell lists All / Present / Absent, with the best displayed value in each metric and group shown in bold, including ties.

METHODFACE ↓CHARACTER ↓BACKGROUND ↓CLOTHES ↑
FramePack-F10.73 / 0.66 / 0.8022.94 / 24.77 / 21.110.36 / 0.45 / 0.261.60 / 2.22 / 0.98
Helios0.84 / 0.87 / 0.8025.23 / 32.24 / 18.210.41 / 0.66 / 0.161.12 / 1.01 / 1.23
LongLive 2.00.63 / 0.55 / 0.7113.19 / 13.03 / 13.340.24 / 0.25 / 0.222.23 / 2.73 / 1.73
Sana-WM Streaming0.67 / 0.64 / 0.7113.87 / 11.90 / 15.830.13 / 0.10 / 0.162.16 / 2.75 / 1.57
LingBot-World-Fast0.62 / 0.60 / 0.6413.44 / 12.90 / 13.970.15 / 0.13 / 0.172.80 / 2.84 / 2.76
LingBot-World-2-Fast0.66 / 0.61 / 0.7012.61 / 12.32 / 12.890.12 / 0.10 / 0.131.56 / 1.64 / 1.47
ReCam0.58 / 0.52 / 0.6410.44 / 10.97 / 9.910.05 / 0.05 / 0.043.97 / 4.36 / 3.57

3. Consistency comparison on a common subset

We also report consistency results on a common subset where all compared methods succeed. Including FramePack-F1 and Helios reduces this subset to 27 cases. We therefore exclude these two baselines and report results on the 53 cases (28 Present, 25 Absent) where all remaining methods succeed.

Each cell lists All / Present / Absent, with the best displayed value for each metric and group shown in bold, including ties.

METHODFACE ↓CHARACTER ↓BACKGROUND ↓CLOTHES ↑
LongLive 2.00.62 / 0.55 / 0.7113.42 / 13.06 / 13.830.24 / 0.23 / 0.252.21 / 2.75 / 1.60
Sana-WM Streaming0.65 / 0.62 / 0.6811.79 / 10.87 / 12.820.08 / 0.07 / 0.092.34 / 3.04 / 1.55
LingBot-World-Fast0.58 / 0.56 / 0.6010.57 / 10.39 / 10.780.06 / 0.03 / 0.093.02 / 3.07 / 2.96
LingBot-World-2-Fast0.64 / 0.58 / 0.7012.10 / 11.36 / 12.930.11 / 0.08 / 0.141.51 / 1.64 / 1.36
ReCam0.56 / 0.49 / 0.649.93 / 10.10 / 9.750.04 / 0.02 / 0.054.01 / 4.42 / 3.56

Our response to W2 on dual-pointer denoising follows in Part 2/2.

AuthorsOpenReview ↗

Response to Reviewer CXMY (Part 2/2): Dual-Pointer Denoising (W2)

Following the reappearance evaluation in Part 1/2, we address the interpretation of dual-pointer denoising.

W2. At unequal noise levels, the two shared-noise views algebraically recover the original context, including its errors. This weakens the proposed error-masking explanation, although empirical benefits remain possible

Thank you for pointing out this algebraic recoverability. We agree that two shared-noise views at unequal noise levels allow exact recovery of the original context, including its errors. Our justification rests on the optimization shortcut and equal-noise training samples, with empirical evidence of easier optimization and reduced drift.

First, our design provides a shortcut directly aligned with the velocity-prediction objective. The difference between the paired views, normalized by the noise-level difference, gives the exact context velocity. Once this velocity is predicted exactly, the corresponding prediction loss is zero and provides no further gradient signal. This shortcut may allow the model to satisfy this part of the prediction objective without learning an additional context-reconstruction step.

Second, we explicitly mix in additional equal-noise samples during training (Eq. 18), increasing the proportion of this special case of dual-pointer denoising. At the same nonzero noise level, the two views are identical, so the original context cannot be recovered through the paired-view inversion. This can be viewed as a form of regularization that encourages the model to directly use noisy context when predicting the current chunk.

The experiments provide separate evidence for optimization and rollout benefits. Dual-pointer denoising reaches a lower training loss than single-pointer denoising and approaches the clean-context loss (Fig. 4(b)). It also achieves the best results on all four temporal-drift metrics (Table 4(b)) and exhibits less accumulated color drift than clean context in the rollout visualization (Fig. 5).

I do not have many problem with this paper and intend to keep my score.

Thank you for your follow-up and for your positive assessment of our work. We are glad to hear that you have no remaining concerns, and we appreciate your time and constructive feedback throughout the review process.

Comments on the paper

1 comment

Dear Reviewers,

Thank you again for your thoughtful review. Your comments helped us improve the paper and guided the additional experiments we reported in our response.

As the discussion period closes on November 18 (AoE), we would appreciate it if you could take a look at our response and let us know whether it addresses your concerns. We would be happy to clarify any remaining points.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.