- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
LaST decouples persistent object memory from local video generation. A recurrent writer maintains separate scene, geometry, and appearance records in fixed-size object slots; a learned reader uses stored geometry to retrieve appearance at queried space–time locations and corrects the denoising predictions of a frozen generator (Wan2.1/2.2), with parameter-free rescaling regulating spatial contrast. Training proceeds in stages: state pretraining via future reconstruction with teacher-supplied geometry, reader warm-up via appearance replacement (residual ranking + keep consistency), and joint optimization over multi-window generated histories with a detached long-history support mask. On the full WRBench set, LaST improves all five metrics over both Short-KV and Full-KV baselines at 1.3B and 5B scales; a capacity study on MOSEv2 relates reconstruction error and object coverage to slot count, and efficiency measurements show 70.0% lower history storage and 21.5% lower continuation latency (1.3B, seven scenes).
Critical strengths
A well-motivated and cleanly designed decomposition. Separating scene/geometry/appearance records and interfacing the memory to a frozen generator purely through denoising corrections is an appealing design; the staged training pipeline (future reconstruction → teacher-guided reads → reader warm-up → joint optimization on generated histories) is careful and principled.
Diagnostics go beyond average scores. History-content interventions, state-update comparisons, and repeated-continuation experiments provide evidence that the retained history is actually being used (e.g., object similarity 0.840 vs. 0.331 for Full-KV on MOSEv2; 4th-round continuation similarity 0.700 vs. 0.176/0.133).
Critical weaknesses
The headline gains on the main benchmark are modest and possibly within noise. At 1.3B, improvements over Short-KV are ≈0.02–0.05 on most metrics (e.g., re-observed spatial 0.613→0.659; visible state 0.565→0.604), and WRBench is an automated LLM-judge metric without a human study; the large 5B integrity jump (0.364→0.741) is unexplained relative to the modest gains elsewhere.
High system complexity relative to demonstrated benefit, supported by very small samples. The pipeline is intricate (staged pretraining, teacher-guided reconstruction, replacement-based warm-up, joint training with support masks), yet much of the supporting evidence rests on 4 sequences per dataset (interventions) and 2 sequences per dataset (repeated continuation). Stronger evidence — harder baselines (e.g., retrieval-based memory such as WorldMem/Context-as-Memory rather than KV windows) and human evaluation would be needed to establish that the complexity pays off.
Questions for authors
How robust are the WRBench gains? e.g., which prompts or scenes drive the improvements, and do the margins hold consistently or concentrate on a subset of the benchmark?
How do the five WRBench metrics compare against stronger memory baselines such as retrieval-based history (WorldMem / Context-as-Memory) rather than only local-KV windows? What explains the large 5B integrity gain (0.364→0.741) while other metrics improve modestly is integrity dominated by a subset of prompts or scenes?
The reader corrects the generator's clean prediction; what happens when memory content is stale or wrong (e.g., appearance changed off-screen)? Are there failure analyses beyond the mismatch-state experiments?
8 comments
Additional WorldMem results with the public Minecraft checkpoint
Thank you for highlighting WorldMem as a relevant memory-based approach. Following our earlier responses, we provide the additional evaluation below, together with the checkpoint and evaluation settings.
Q. What does the released WorldMem checkpoint achieve on our natural-scene inputs?
A. We evaluated its public Minecraft checkpoint without fine-tuning on 25 fixed WRBench T0 camera-revisit cases. The authors clarify that their RealEstate10K experiments use a separate model (author clarification). As of November 16, 2026, we could not locate an official public release of that model. We therefore used the released Minecraft checkpoint.
Each case uses one initial image and a 60-degree turn-and-return camera trajectory: 81 frames at 640 × 360, 16 fps, and seed 42. We retain WorldMem's history retrieval and memory attention, with 20 sampling steps, an eight-frame generation window, and eight memory conditions. No future reference images are supplied; task text is used for evaluation rather than as an input to WorldMem.
Automatic scores on all 25 videos
| Metric | Score ↑ | Evaluated videos |
|---|---|---|
| Visual integrity (D2) | 0.0558 | 25/25 |
| Visible spatial fidelity (D3) | 0.6102 | 25/25 |
| Visible state fidelity (D4) | 0.4604 | 25/25 |
These scores use the current WRBench evaluator. Its D2 uses global/local DINOv2 features, whereas the LaST main table uses crop-based integrity; the two integrity columns should not be directly compared.
Return evidence across all 25 outputs
AI-assisted frame inspection covered all 81 frames of every video, with ambiguous late frames additionally examined at their original resolution.
| Observed return evidence | Videos | Fraction of all cases |
|---|---|---|
| Clearly identifiable return to the original region | 0 | 0% |
| Late scene fragments; region return remains ambiguous | 5 | 20% |
| No identifiable late return evidence | 20 | 80% |
These are qualitative inspection counts, not blinded human ratings or official D5/D6 scores. The generated videos showed substantial loss of subjects and scene structure. The automatic evaluator found no eligible videos for conditional D5/D6 scoring; the table above retains all 25 cases and describes the affected outputs explicitly.
This experiment characterizes zero-shot transfer of the released Minecraft checkpoint to natural scenes. We report it separately from the matched-backbone comparisons in our earlier responses, since the training domain, backbone, and input interface differ.
Could you provide a comparison with existing open-source memory-based methods for video world models, such as LSM-World or GIM-World? My primary concern is the lack of a comprehensive comparison against methods for similar directions; most of your baselines consist of foundational models or methods developed by the authors themselves. While such experiments are sufficient for an ablation or analysis study, they fail to demonstrate that your proposed method offers a significant advantage over existing memory-based techniques. I would like to see a comparison with LSM-World and GIM-World included in the main tables. If you could incorporate both quantitative and qualitative results, including visualizations in the main text, the paper will be better. The current version is not yet convincing enough.
LaST improves identity recovery after reappearance and preserves updated states across revisits.
Thank you for the thoughtful review and for recognizing our memory decomposition and training design. Your questions motivate a closer look at where the gains arise, how they vary across tasks, and whether the same object is recovered after reappearance or a memory update.
A1. Integrity gains persist after excluding the largest contributors, with trained correction contributing substantially.
At 5B, zeroing persistent records while retaining the reader (S=0) gives integrity 0.718 and revisit spatial 0.667. Training without persistent history gives 0.714 and 0.674, respectively.
5B integrity differences. Columns exclude the top one or five contributors, ranked by LaST minus Short-KV; all rows use the same exclusions.
This sample set differs from the eight-task CI analysis below. Excluding scene groups gives LaST-minus-S=0 revisit spatial differences of 0.021–0.039 (1.3B) and 0.058–0.091 (5B).
Paired differences and 95% CIs. Differences are LaST minus the baseline. We use 20,000 source resamples with seeds 20260924/20260925 for 1.3B/5B. The first three metrics use eight tasks; revisits use three/six.
1.3B
5B
The paired 5B LaST revisit means are 0.732099 (spatial) and 0.621734 (state) on six tasks; the main table uses eight. Intervals describe task variation. Against Short-KV at 5B, integrity and revisit spatial intervals exclude zero; the other three include zero.
A2. LaST leads in revisit scores; Flat is slightly higher in integrity and visible-state consistency.
An asterisk marks paper results; other rows are new measurements. I: integrity (paper crop); VS/VT: visible spatial/state; RS/RT: revisit spatial/state.
1.3B
5B
Flat matches persistent-state size, trainable parameters, training data and budget. Retrieval uses the same frozen generator. WorldMem and Context as Memory have their own trained interfaces and different backbones. Context as Memory's official code is not yet public. WorldMem results will follow with backbone details.
A3. LaST's identity advantage over Flat increases after reappearance at 5B.
The advantage is six percentage points after reappearance versus one in ordinary cases. In blinded, two-choice comparisons without ties, preference for LaST is 0.71 against Short-KV, 0.65 against S=0 and 0.56 against Flat. Visual preference and instance identity are evaluated separately. Human identity judgments accompany automated scores, with agreement of 0.81 in ordinary cases and 0.74 after reappearance.
A4. At 5B, event correctness remains 0.72 at the second revisit after a reliable update.
Unchanged-object identity retention is 0.79 for LaST and 0.75 for Flat. Removing keep or local-preservation reduces it to 0.65 or 0.62, testing preservation during updates.
With incorrect history and a reliable new observation, event correctness is 0.72 for LaST and 0.64 for Flat. Without reliable new evidence, incorrect history is adopted in 0.12 and 0.19 of cases.