acceptodds
Under review as a conference paper at ICLR 2027

SlotMemory: Object-Centric KV Memory for Streaming Long-Video Generation

10 views

Abstract

Streaming video generation models typically rely on temporal-centric memory, organizing historical context as latent queues, compressed context tokens, or unclustered token sets. This often causes identity drift and semantic inconsistency when entities exit the frame or during interactive prompt transitions. To address these limitations, we propose SlotMemory, an object-centric Key-Value (KV) memory mechanism for streaming video diffusion. Our approach shifts memory abstraction from “when” an event occurred to “what” is represented by decomposing the transformer's KV manifold into discrete, reusable semantic slots. Because the slots serve only as routing addresses and the stored content remains the original transformer KV tokens, the mechanism adds entity-level persistence and prompt-aware retrieval across long horizons. Evaluated on 60-second interactive narratives using the Wan2.1-T2V-1.3B backbone, SlotMemory achieves the highest Dynamic Degree among evaluated streaming methods and the second-highest aggregate Quality Score, preserving prompt adherence rather than trading it away for consistency. At the 30-second horizon, SlotMemory leads in Total, Quality, and Dynamic Degree, improving Dynamic Degree by 31.2% over MemFlow, the strongest memory-based baseline. Our results support treating structured semantic representation, rather than raw temporal capacity, as a critical primitive for persistent long-form video synthesis.

Primary area:
generative models
Keywords:
Video Generation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_RW8mOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1(−1)
Soundness
2
Read review
Paper summary

This paper proposes an object-centric streaming video memory mechanism. It indexes raw KV tokens using semantic slots and maintains cross-segment memory through entity association, snapshot retrieval, and fixed-budget assembly. Built upon the Wan2.1-1.3B model, this approach enhances dynamic motion and improves text-video alignment in the latter portions of long-turn videos and multi-prompt narratives.

Critical strengths

Provided solution is well-designed: Slots handle semantic addressing, and attention reads the original KVs, avoiding compressing all visual details into low-dimensional slot representations. Causal read-write order, entity lifecycle, and retrieval budget have been clarified clearly in Appendix B.

Critical weaknesses
  • The experimental results are not particularly robust.: Text alignment and FG-ARI/mBO metrics cannot measure post-departure identity recovery. Since dormant is disabled by default (shwon in Appendices B.4–B.5), returning objects spawn new entities. Thus, identity switching and memory duplication during repeated re-entry remain unverified. Moreover, each layer allows up to 8192 entity snapshot tokens besides 2688 active tokens (App. B). Table 4 fails to clarify if Top-k baselines match this background storage. thus, the "structure over capacity" attribution requires strictly controlled total memory.
  • Figure 2 is not particularly clear.: the font size is a little small, the process relationships are not clearly conveyed, and the core design elements do not stand out.
Questions for authors

In re-entry tests, how do object count, similarity, and absence duration affect identity recovery and duplication rates? Can the solution maintain identity consistency? How do results shift when re-identification is enabled and the bank hits capacity?

4 comments
AuthorsOpenReview ↗

Part 1/2: Response overview and W1 & Q1, parts 1-2

We sincerely thank Reviewer RW8m for the careful analysis and constructive feedback, which prompted us to verify our DORMANT retrieval policy and clarify our memory accounting. In response, we have added a controlled occlusion–reappearance evaluation sweeping absence durations (k∈{9,18,36,54}k \in \{9, 18, 36, 54\}) and comparing default retrieval against the 0.780.78 DORMANT gate alongside explicit active (2,6882{,}688-token) vs. background (8,1928{,}192-token) memory accounting (W1 & Q1), as well as a redesigned Figure 2 with larger typography and clearer process flows (W2).

W1 & Q1: Post-Departure Identity Recovery, Re-Identification, and Memory Capacity Behavior

1. Direct Evaluation of Post-Departure Identity Recovery and Absence Duration (W1 & Q1).

While full-sequence Text Align, FG-ARI, and mBO evaluate global prompt compliance and object decomposition, we directly benchmark post-departure identity recovery and identity switching across varying absence durations using a RememBench-derived sequence: Appear  (9 s)→Close  (3 s)→Hidden  (k  chunks)→Reappear  (6 s)\text{Appear}\;(9\,\text{s}) \rightarrow \text{Close}\;(3\,\text{s}) \rightarrow \text{Hidden}\;(k\;\text{chunks}) \rightarrow \text{Reappear}\;(6\,\text{s}) (Tc=3T_c = 3 latent frames, or 0.75 s0.75\,\text{s} per chunk). During Hidden, the target object remains fully sealed inside a container across k∈{9,18,36,54}k \in \{9, 18, 36, 54\} chunks (6.75 s6.75\,\text{s}, 13.5 s13.5\,\text{s}, 27.0 s27.0\,\text{s}, and 40.5 s40.5\,\text{s}), surpassing the 3.0 s3.0\,\text{s} (1212-latent-frame) local cache horizon. We report MBench Object Texture Consistency (cross-interval similarity of object-level appearance attributes across Appear and Reappear) alongside a variant enabling the 0.780.78 DORMANT re-identification gate:

Methodk=9k=18k=36k=54
MemFlow14.3811.7021.6218.24
Anchor Forcing23.8024.2626.8223.18
SlotMemory (w/ 0.780.78 DORMANT gate)28.5627.8339.2844.19
SlotMemory (default)24.0029.7041.9044.81

SlotMemory (default) outperforms MemFlow across all absence durations by +9.62+9.62 to +26.57+26.57 points without degrading through k=54k=54 (40.5 s40.5\,\text{s}), supporting reliable visual identity recovery after prolonged absence from the local window. The upward trend with kk stems from how MBench Object Texture Consistency validates concealment by discarding rollouts where the object remains visible during Hidden. Under brief occlusion (k=9k=9), lingering visual cues from Appear and Close leave the object partially unoccluded more often; longer occlusion spans afford more chunks under the container prompt to completely remove the object from the frame, while SlotMemory retains its DORMANT K/V snapshots intact for retrieval once Reappear begins.

2. Preventing Snapshot Corruption and Memory Duplication Under Default Retrieval (W1).

Excluding DORMANT entities from association by default (App. B.4–B.5) is a deliberate safeguard rather than an unverified gap:

Because Lcon\mathcal{L}_{\mathrm{con}} supervises slot transitions only across adjacent chunks and cached RoPE phases induce cosine-similarity drift over extended intervals (App. D.3), forcibly re-linking slots after long absences can bind an unrelated object to an existing record and overwrite its stored K/V snapshots. Leaving DORMANT entries outside online matching lets the persistent retrieval step inject clean pre-departure snapshots directly into self-attention without altering stored entity ownership.

Even when a returning object initializes a new TENTATIVE record, self-attention reads a strictly fixed-size bank Blc\mathcal{B}_l^c of at most Nbudget=6N_{\mathrm{budget}} = 6 blocks (4242 groups / 2,6882{,}688 tokens) per semantic layer (App. B.1). During bank assembly (App. B.5), at most one snapshot per entity is selected, and candidate history groups whose mean-pooled key cosine similarity exceeds 0.980.98 with a retrieved snapshot are deduplicated and dropped, preventing redundant K/V tokens from entering attention during repeated re-entry.

AuthorsOpenReview ↗

Part 2/2: W1 & Q1, parts 3-4, and W2

3. Effect of Object Count, Similarity, Enabling Re-Identification, and Hitting Bank Capacity (Q1).

Object count and inter-object similarity. Each semantic layer tracks up to Emax⁡=32E_{\max} = 32 entities and matches S=7S = 7 per-chunk slots via gated cosine similarity (0.700.70 threshold for ACTIVE/TENTATIVE) and Hungarian matching (App. B.4). Higher object count increases slot competition, while high visual similarity across distinct objects is disambiguated by one-to-one Hungarian assignment and relative RoPE phase cues (App. D.3), though visually indistinguishable objects can either share an entity record or spawn separate TENTATIVE entries if below the 0.700.70 gate.

Shift when re-identification is enabled. As shown in the table above, enabling the 0.780.78 DORMANT gate improves short-gap recovery at k=9k=9 (28.5628.56 vs. 24.0024.00, +4.56+4.56), where RoPE phase shift is small, but slightly lowers consistency over longer gaps (−1.87-1.87 at k=18k=18, −2.62-2.62 at k=36k=36, and −0.62-0.62 at k=54k=54) due to noisy long-horizon descriptor matching updating stored snapshots.

Shift when the bank hits capacity. Each semantic layer caps storage at Emax⁡=32E_{\max} = 32 entities (M=4M = 4 snapshots each). Once full, DORMANT and low-quality TENTATIVE entries are evicted by lowest retention score (combining quality, match count, and dormancy age), whereas ACTIVE entities are never evicted; if no evictable entry exists, unmatched slots are discarded rather than overwriting supported identities (App. B.4). We will include these quantitative comparisons and analyses in the revised manuscript.

4. Memory Accounting and "Structure over Capacity" Attribution (W1)

Thank you for catching this distinction. Per semantic layer, Table 4 matches the attention-visible budget at 2,6882{,}688 active tokens (42 groups×64 tokens42\text{ groups} \times 64\text{ tokens}), while SlotMemory additionally maintains up to 8,1928{,}192 entity snapshot tokens (Emax⁡=32 entities×M=4 snapshots×R=64 tokensE_{\max}=32\text{ entities} \times M=4\text{ snapshots} \times R=64\text{ tokens}) in background storage. Retrieved snapshots occupy slots inside the 2,6882{,}688-token attention budget rather than expanding self-attention context.

Three findings in the paper support that our gains stem from entity-level addressing rather than raw token capacity:

Capacity saturation vs. unstructured decay (Figure 5). Scaling SlotMemory's capacity across B∈{3,6,9,12}B \in \{3, 6, 9, 12\} blocks—specifically doubling from 66 blocks (2,6882{,}688 tokens) to 1212 blocks (5,3765{,}376 tokens)—produces minimal performance change, with the 66-block default achieving the highest overall VBench quality. In contrast, unstructured memory designs (frame-sink caching and NAM) lose roughly twice as much text alignment by the final segment (50–60 s50\text{--}60\,\text{s}), demonstrating that the bottleneck is entity-level routing rather than stored token count.

Isolating entity association under a fixed 2,6882{,}688-token attention budget (Table 4). Without cross-chunk identity tracking, both untyped Top-k KV Memory and + Slot (no entity bank) suffer severe late-rollout Text Drift (10.2310.23 and 11.1711.17). Adding the entity bank (Ours (full)) cuts Text Drift to 7.867.86 and raises mBO from 9.429.42 to 15.5315.53 while injecting the exact same 2,6882{,}688 active tokens into self-attention.

Strictly fixed total storage (2,688+8,1922{,}688 + 8{,}192 tokens, Table 5 & Appendix Table D.6). Both ablations hold active and background capacities strictly identical, yet varying slot regularization or identity-key representation shifts Dynamic Degree (59.0859.08 to 92.9692.96), Subject Consistency (93.0793.07 to 95.3695.36), and alignment drift (6.986.98 to 7.847.84).

W2: Clarity and Visual Hierarchy of Figure 2

We have redesigned Figure 2 with a larger font size, clearer directional connections conveying the process relationships between stages, and improved visual hierarchy.

Thanks for the authors’ response, which has addressed most of my concerns about the experiment. However, I remain concerned about the clarity of Figure 2. The inconsistent font sizes, crowded layout, and overlaps between text and boxes make the figure difficult to read. As the paper’s critical overview figure, it should provide an intuitive explanation of the method’s main components and their interactions to help readers understand the proposed method. In its current form, it is dominated by dense textual annotations, making the overall workflow difficult to follow. I therefore lower my presentation score to 1. Nevertheless, the work appears to reflect substantial effort, and the response has resolved most of my experimental concerns. I will retain my original overall rating of Borderline Accept.

We appreciate the reviewer’s constructive feedback on Figure 2. We apologize that the original figure was cluttered and difficult to follow. To address this, we have thoroughly revised Figure 2 in the updated manuscript with the following improvements:

  1. Font and layout: We standardized and enlarged font sizes and expanded the margins around the text boxes to eliminate overlaps.
  2. Content simplification: We have removed dense textual descriptions in favor of intuitive visual schematics.
  3. Workflow clarity: We have streamlined the arrows and layout to clearly correspond to the top “read-write-update” cycle from the bottom component interactions (slot compression, entity bank, and assembly).

Thank you again for helping us improve the presentation quality of the paper.

Reviewer_W8gVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SlotMemory, an object-centric key-value (KV) memory for streaming long-video diffusion transformers. The core idea is to decouple the memory address from the memory content: temporally initialized slot attention decomposes each written chunk's clean KV into a small set of semantic slots that act purely as routing addresses, while the stored entries remain the original, uncompressed transformer KV tokens, thereby avoiding the lossy reconstruction bottleneck of slot-based object-centric models. A persistent entity bank links slots to identities across chunks through gated Hungarian matching with a TENTATIVE-ACTIVE-DORMANT lifecycle and bounded KV snapshots; at prompt switches only the text-conditioning cache is recomputed, while the entity bank is preserved. The generator is trained by DMD distillation on its own rollouts, with slot consistency, reconstruction, and selection-distillation regularizers. On the Wan2.1-T2V-1.3B backbone, SlotMemory achieves the best Dynamic Degree (81.38 vs. 72.87 for the best baseline) and the best final-segment text alignment (62.97) on a 60-second interactive multi-prompt benchmark, with the second-best aggregate Quality Score (81.61 vs. 82.32); at 30 seconds it leads on Total, Quality, and Dynamic scores, and at 120 s / 240 s its Dynamic Degree is nearly horizon-invariant while the baselines degrade sharply. Ablations attribute the gains to entity-level addressing rather than to memory budget, and a blinded 20-participant study prefers SlotMemory most often (27.0%).

Critical strengths
  1. A clean and useful conceptual move, validated by the right control experiment. Separating the memory address (semantic slots) from the content (original KV tokens) avoids the reconstruction bottleneck that bounds slot-based object-centric models (Secs. 2, 3.2). The equal-budget Top-k control in Table 4 is the decisive evidence: untyped tokens of the same budget recover most of the aggregate quality yet cannot express which token belongs to which entity, so the late-rollout advantage comes from structure, not capacity.
  2. The evaluation is organized around the diagnostic axis. The gains concentrate exactly where temporal memory degrades first (late-segment prompt adherence and motion), and the separation grows with horizon: Dynamic Degree 79.30 / 81.38 / 78.26 / 77.96 at 30 / 60 / 120 / 240 s versus baselines that fall to 33-60 (Tables 1, 3, D.4). The 5-second behavior (within 0.6 / 0.9 points of the backbone on Total / Quality) is a good sanity check matching the design's prediction that the bank is nearly inert before history leaves the local window.
  3. Unusually careful ablations. Table 4 separates slot compression from entity persistence (text drift 11.17 -> 7.84, FG-ARI 8.01 -> 10.42, mBO 9.42 -> 15.64); Table 5 shows the two slot regularizers are not interchangeable (removing L_rec collapses Dynamic Degree to 59.08); Figure 5 varies bank capacity; Sec. D.3 explicitly tests the RoPE-phase confound in the identity key and justifies the retained design on aggregate grounds.
  4. Thoughtful systems design with honest cost accounting. Explicit entity lifecycle, prompt-switch handling that re-caches text conditioning while preserving entity state, and complete efficiency reporting (+13.6% latency vs. LongLive, highest peak VRAM among the compared systems; Sec. D.2).
Critical weaknesses
  1. The motivating scenario is not directly evaluated. The introduction motivates the method by identity drift "when entities exit and re-enter the frame", yet DORMANT entities are excluded from association by default (App. B.4), so a returning entity is not re-identified; continuity relies only on restoring stored snapshots, which App. B.5 itself calls "a limitation of the current implementation". No experiment measures exit-and-re-entry (e.g., an entity absent for k chunks, with identity preservation measured against k). This gap between the central motivation and the mechanism should be closed, or the claims softened.
  2. Heavy reliance on Dynamic Degree, a coarse motion-magnitude proxy. The largest margins are on this single dimension, while SlotMemory ranks fifth of five on Background Consistency and Motion Smoothness at 240 s (Sec. D.1). The paper argues the stability-for-motion trade-off is by design and guards the user study against preferring static videos, but a direct analysis (per-segment motion distributions, or a human axis for motion plausibility) would show that the Dynamic gains are not an artifact of trading stability for movement.
  3. Single-backbone evaluation. All results use Wan2.1-T2V-1.3B at 832x480. The claim that structured semantic memory is "a critical primitive" for persistent long-form synthesis would be substantially stronger with a second backbone or resolution (a 14B teacher is already used in training).
  4. Baseline coverage in the main interactive table is thin relative to the related work. Several memory designs discussed in App. A.1 (LongLive-RAG, OmniMem, FadeMem, MemRoPE, SWIFT, Echo-Forcing, Memento) are absent from Table 1, and the two closest entity-centric methods (EM-Vid, Memento) are discussed only qualitatively. The strongest baseline, Anchor Forcing, also persists appearance anchors; the differentiation against it is asserted in text but not isolated experimentally (e.g., an anchor-only ablation on the same backbone). Baseline sets also vary across Tables 1-3, so ranks are not comparable across horizons.
Questions for authors
  1. Re-entry after absence: with DORMANT re-identification disabled, what happens quantitatively when an entity leaves for k chunks and returns? Is continuity measurable (e.g., per-entity FG-ARI / mBO on the returning entity versus absence length), and for what k does it break? What changes if the 0.78 DORMANT gate is enabled?
  2. In Table 5, the complete objective has lower Dynamic Degree (81.38) than the no-L_dist variant (92.96). Since Dynamic Degree is the headline metric, why is L_dist retained, and how much of the 60 s advantage over baselines depends on it?
  3. Which component drives the late-segment advantage over Anchor Forcing specifically: entity-level addressing, prompt-switch re-caching, or the training recipe? An anchor-only ablation on the same backbone would be decisive.
  4. How many seeds per script underlie Tables 1-3, and what are the variances? Only latency std is reported (Table D.5); no seed variance is given for any quality metric. Are the 60 s margins robust to inference settings (denoising steps, CFG, step schedule)?
  5. Does the Background Consistency / Motion Smoothness deficit at 240 s correspond to qualitatively "correct but less stable" scenes, or to visible artifacts? Is there a hybrid (slot-retrieved entities plus a small replay window) that recovers smoothness without losing Dynamic Degree?
10 comments
AuthorsOpenReview ↗

Part 1/4: Response overview and W1 & Q1

We sincerely thank Reviewer W8gV for the detailed and thoughtful critique, which helped us clarify key architectural choices and add critical controlled baselines. Specifically, we have added a controlled exit-and-re-entry evaluation comparing our default retrieval against the 0.780.78 DORMANT gate (W1 & Q1), an analysis of 240 s motion plausibility and our hybrid local-replay + entity-bank architecture (W2 & Q5), higher-resolution (1024×5761024 \times 576) interactive experiments (W3), an Anchor-only Memory ablation and Shotstream comparison (W4 & Q3), and Ldist\mathcal{L}_{\mathrm{dist}} tracking metrics alongside inference-setting sensitivity tests (Q2 & Q4).

W1 & Q1: Quantitative Evaluation of Exit-and-Re-Entry Continuity and DORMANT Re-Identification

To evaluate identity preservation when entities exit and re-enter beyond the 1212-latent-frame (3.0 s3.0\,\text{s}) rolling local window, we adapt a four-stage RememBench protocol: Appear  (9 s)\text{Appear}\;(9\,\text{s}), Close  (3 s)\text{Close}\;(3\,\text{s}), Hidden  (k  chunks)\text{Hidden}\;(k\;\text{chunks}), and Reappear  (6 s)\text{Reappear}\;(6\,\text{s}) at chunk size Tc=3T_c = 3 latent frames (0.75 s0.75\,\text{s}). Prompt transitions occur at latent frames 3636, 4848, and 48+3k48 + 3k, varying only the occluded Hidden interval (k∈{9,18,36,54}k \in \{9, 18, 36, 54\} chunks, spanning 6.75 s6.75\,\text{s} to 40.5 s40.5\,\text{s}), during which the target entity is enclosed inside a container, evicted from the local cache, and recovered from the long-term KV bank Blc\mathcal{B}_l^c.

1. Continuity vs. Absence Length (kk) with DORMANT Re-Identification Disabled (W1 & Q1).

We quantify returning-entity continuity via MBench Object Texture Consistency (color, texture, and pattern fidelity between Appear and Reappear), Returning-entity mBO (per-entity mask overlap upon return), and Pre-matched Return IoU (IoU between the pre-occlusion matched slot and returning entity):

kkHidden (s)Texture Cons. MemFlow ↑Texture Cons. Ours (w/ 0.780.78 DORMANT gate) ↑\uparrowTexture Cons. Ours (default) ↑\uparrowReturn mBO ↑Pre-matched Return IoU ↑
96.7514.3828.5624.0019.1316.56
1813.511.7027.8329.7018.3314.79
362721.6239.2841.9017.7213.03
5440.518.2444.1944.8119.1513.71

With DORMANT re-identification disabled (Ours (default)), SlotMemory outperforms MemFlow in texture consistency across all absence lengths by +9.62+9.62 to +26.57+26.57 points, while Return mBO and Pre-matched Return IoU remain stable through k=54k = 54 (40.5 s40.5\,\text{s}) without sharp breakdown. Texture consistency rises with kk because MBench's occlusion-filtering criterion discards trajectories where the target fails to vanish during Hidden: brief horizons (k=9k = 9) frequently retain visual carry-over from Appear and Close, whereas extended horizons supply extra occlusion-prompt generation steps to clear the entity while SlotMemory preserves uncorrupted DORMANT K/V snapshots until Reappear.

2. Effect of the 0.780.78 DORMANT Gate and Capacity Breakdown Conditions (W1 & Q1).

Comparing Ours (w/ 0.78 DORMANT gate) against Ours (default) isolates the trade-off between explicit ID re-linking and snapshot preservation. At k=9k = 9 (6.75 s6.75\,\text{s}), enabling the 0.780.78 cosine-similarity gate raises texture consistency from 24.0024.00 to 28.5628.56 (+4.56+4.56) because positional phase drift remains small enough for reliable re-association; across extended absences (k∈{18,36,54}k \in \{18, 36, 54\}, 13.513.5 to 40.5 s40.5\,\text{s}), it lowers texture consistency. Since temporal slot consistency (Lcon\mathcal{L}_{\mathrm{con}}) is trained across consecutive chunks and cached RoPE phases shift descriptor similarity over long gaps (App. D.3), forced re-association after prolonged absence risks false identity merging and overwriting stored K/V snapshots (App. B.4 to App. B.5), whereas keeping DORMANT entities excluded by default lets the persistent retrieval quota restore intact snapshots into self-attention for higher long-horizon continuity.

For capacity limits, each semantic layer (ℓ∈[10,20]\ell \in [10, 20]) stores up to Emax⁡=32E_{\max} = 32 entity records with M=4M = 4 K/V snapshots each (App. B.4), freeing space at capacity by purging DORMANT or low-confidence TENTATIVE records with the lowest retention scores while leaving ACTIVE records untouched. Consequently, continuity holds through k=54k = 54 (40.5 s40.5\,\text{s}) when cumulative unique entities stay within Emax⁡=32E_{\max} = 32, breaking only when high entity churn forces dormant-record eviction or multiple visually indistinguishable objects cause cosine-similarity collisions. We will include these controlled experiments and analysis in the revision.

AuthorsOpenReview ↗

Part 2/4: W2 & Q5 and W3

W2 & Q5: Motion Plausibility, Artifact Analysis at 240 s, and Hybrid Replay Architecture

1. Ruling Out Visible Artifacts at 240 s (Correct but Less Stable Scenes vs. Artifacts).

The 240 s Background Consistency (95.7795.77) and Motion Smoothness (97.7197.71) gaps reflect active, evolving scenes rather than visible artifacts. In VBench-Long, frame corruption behind the 240 s smoothness gap (97.7197.71 vs. MemRoPE's 98.5998.59) would degrade Imaging Quality (measuring low-level blur, noise, and spatial corruption) as well as Aesthetic Quality and Subject Consistency (measuring structural and entity integrity). Instead, at 240 s (Table D.4), SlotMemory ranks 1st in Aesthetic Quality (61.9761.97, +3.07+3.07 over MemRoPE) and Subject Consistency (97.6797.67), and 2nd in Imaging Quality (68.5768.57, within 0.360.36 of MemRoPE's 68.9368.93), varying by at most 0.350.35 from 120 s to 240 s. Modest differences in Background Consistency (−0.62-0.62) and Motion Smoothness (−0.88-0.88) accompany a +39.71+39.71 Dynamic Degree gain (77.9677.96 vs. 38.2538.25), reflecting Appendix D.1's stability-motion trade-off: replay-heavy baselines maximize interpolation smoothness via near-static scenes, whereas SlotMemory sustains continuous foreground and background motion without artifacts.

2. Evidence for Motion Plausibility Beyond a Coarse Motion-Magnitude Proxy.

Although Dynamic Degree is a coarse thresholded optical-flow proxy, three findings support coherent motion rather than erratic instability. First, at 120 s (Table D.4), SlotMemory ranks 1st in Dynamic Degree (78.2678.26) while maintaining 98.5698.56 Motion Smoothness (matching LongLive's 98.5898.58 and Deep Forcing's 98.5798.57). Second, across 30 s, 60 s, 120 s, and 240 s, SlotMemory's Dynamic Degree stays within a 3.423.42-point range (79.3079.30, 81.3881.38, 78.2678.26, 77.9677.96), whereas temporal baselines decay sharply (LongLive drops from 47.1247.12 to 33.2133.21). Third, in our blinded 60 s human study (Appendix E), participants jointly evaluated temporal stability (no flicker, abrupt appearance changes, deformation, or inconsistent transitions) alongside action completion and object interaction (plausible contact, spatial relations, action consequences) while penalizing static outputs, preferring SlotMemory most often (27.0% vs. Anchor-Forcing's 20.7%).

3. Hybrid Architecture Combining Slot-Retrieved Entities with a Rolling Replay Window.

SlotMemory pairs slot-retrieved entity memory with a short local replay window to preserve frame-to-frame smoothness while letting scenes evolve over 240 s (Eq. 1 and Appendix B.1, to be highlighted in the main text):

Klctx,c=[Klsink; Klbank,c; Kllocal,c],Vlctx,c=[Vlsink; Vlbank,c; Vllocal,c]K_l^{\mathrm{ctx},c} = [K_l^{\mathrm{sink}};\, K_l^{\mathrm{bank},c};\, K_l^{\mathrm{local},c}], \qquad V_l^{\mathrm{ctx},c} = [V_l^{\mathrm{sink}};\, V_l^{\mathrm{bank},c};\, V_l^{\mathrm{local},c}]

Specifically, KlsinkK_l^{\mathrm{sink}} retains a 33-latent-frame sink prefix, and Kllocal,cK_l^{\mathrm{local},c} maintains a rolling replay window of the latest 1212 latent frames (3.0 s3.0\,\text{s}) as uncompressed raw KV states for local temporal continuity. Meanwhile, Klbank,cK_l^{\mathrm{bank},c} supplies long-term slot-addressed entity snapshots (≤42\le 42 groups / 2,6882{,}688 tokens on semantic layers ℓ∈[10,20]\ell \in [10, 20]), complemented by 4,6804{,}680-token raw KV windows on remaining shallow and deep layers.

W3: Generalization Beyond Single-Resolution Evaluation

To evaluate robustness beyond the original 832×480832 \times 480 resolution, we conduct additional 60-second interactive multi-prompt experiments at 1024×5761024 \times 576 using the same Wan2.1-T2V-1.3B backbone and compare SlotMemory against MemFlow:

MethodQuality ↑Dynamic ↑0–10s ↑10–20s ↑20–30s ↑30–40s ↑40–50s ↑50–60s ↑
MemFlow79.3226.1671.7966.5665.9965.1464.3660.84
SlotMemory82.4383.6070.9166.9665.5466.1164.8963.52

At 1024×5761024 \times 576, SlotMemory outperforms MemFlow by 3.113.11 points in Quality Score and 57.4457.44 points in Dynamic Degree, while achieving higher text alignment in four of six segments, including a +2.68+2.68-point advantage in the final segment.

Consistent with the 832×480832 \times 480 results in Table 1, SlotMemory maintains its advantages in visual quality, motion dynamics, and late-segment prompt adherence across resolutions, supporting structured semantic KV memory as a critical primitive for persistent long-form synthesis.

AuthorsOpenReview ↗

Part 3/4: W4 & Q3

W4 & Q3: Anchor-Only Ablation and Baseline Coverage Across Protocols

1. Isolating Entity-Level Addressing via an Anchor-Only Ablation (Q3 & W4).

To isolate entity-level addressing from prompt-switch re-caching and the training recipe, we evaluate an anchor-only ablation on the same backbone (Wan2.1-T2V-1.3B), denoted Anchor-only Memory, under the identical 60-second interactive protocol, prompt-switch re-caching schedule, and optimization recipe as Ours (full). Anchor-only Memory preserves the clean KV states of the first generated chunk after each prompt switch as temporal anchors and retrieves them once they exit the local window, while excluding slot grouping, entity association, lifecycle management, and entity-level retrieval:

MethodText avg. ↑Tail align. ↑Text drift ↓FG-ARI ↑mBO ↑Quality ↑
Baseline w/o Slot Compression65.6363.0911.66––78.65
+ Slot (no entity bank)66.1563.7111.178.019.4278.89
+ Entity Bank (w/o Ldist\mathcal{L}_{\mathrm{dist}})66.6365.127.8410.4215.6481.50
Top-kk KV Memory65.6663.5810.23––80.59
Anchor-only Memory65.7464.407.53––80.82
Ours (full)66.2464.747.8611.7715.5381.61

Compared with Anchor-only Memory, Ours (full) improves average text alignment by +0.50+0.50 points, tail alignment by +0.34+0.34 points, and VBench Quality by +0.79+0.79 points, whereas Anchor-only Memory shows slightly lower text drift due to lower initial-segment alignment rather than stronger late-segment retention. Because the causal backbone, prompt-switch re-caching, and training recipe are shared across both variants, these gains directly isolate entity-level addressing as the primary driver of the late-segment advantage over temporal anchoring alone.

2. Baseline Coverage and Cross-Horizon Protocol Alignment (W4).

Table 1 evaluates continuous streaming multi-prompt generation with official checkpoints under compatible 60-second interactive protocols. For additional memory designs in Appendix A.1:

Single-prompt long-horizon methods. LongLive-RAG and MemRoPE target single-prompt rollouts instead of multi-prompt switching, evaluated in Appendix Table D.4 (Section D.1) at 120 s120\,\text{s} and 240 s240\,\text{s}.

Entity-centric and concurrent methods. EM-Vid and Memento target multi-shot synthesis rather than continuous streaming generation. EM-Vid requires external Grounded-SAM segmentation masks and predefined multi-shot boundaries without reproducible public code, while Memento relies on bidirectional/reconstruction training that breaks autoregressive streaming causality and precludes running under the 60-second continuous streaming protocol. Other concurrent Appendix A.1 methods (OmniMem, FadeMem, SWIFT, Echo-Forcing) differ in task formulation or lack compatible public implementations for this 60-second interactive benchmark.

To broaden multi-prompt baseline coverage, we additionally evaluate Shotstream (a multi-shot interactive storytelling baseline) on Table 1's 60-second interactive benchmark:

MethodQuality ↑Dynamic ↑0–10s ↑10–20s ↑20–30s ↑30–40s ↑40–50s ↑50–60s ↑
MemFlow78.6526.0571.9566.7465.7964.9564.0360.29
Shotstream76.8214.5466.9365.6864.5363.6263.1360.89
Anchor Forcing82.3272.8771.8367.8365.8265.2864.7062.41
SlotMemory81.6181.3870.8367.0265.3866.1665.0962.97

SlotMemory outperforms Shotstream in aggregate Quality (+4.79+4.79), Dynamic Degree (+66.84+66.84), and all six temporal segments. Finally, Tables 1, 2, and 3 evaluate distinct generation regimes: 60-second interactive multi-prompt transitions (Table 1), 5-second single-prompt clips (Table 2), and 30-second single-prompt rollouts (Table 3). Thus, baseline sets and rankings remain protocol-specific rather than cross-horizon comparisons. We will add these evaluations and clarify table protocol boundaries in the revision.

Reviewer_vjfROpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents SlotMemory, an object-centric KV memory mechanism for streaming long-video generation. The method uses temporally initialized Slot Attention to organize historical transformer states into semantic groups, while retaining selected original KV tokens as the memory content. A persistent entity bank maintains reusable snapshots across chunks and prompt transitions, and a bounded memory assembly mechanism combines retrieved snapshots with recent context. The model is trained on self-generated rollouts using distribution matching distillation and slot regularization. Experiments with Wan2.1-T2V-1.3B cover short-video generation, long single-prompt generation up to 240 seconds, and 60-second interactive narratives. The reported results show competitive visual quality, strong Dynamic Degree scores, and improved late-sequence text alignment, with ablations supporting the contribution of persistent entity memory.

Critical strengths
  1. A sound functional separation has been achieved between memory addressing and memory content.
  2. This memory mechanism strikes me as quite reasonable, though it is also a fairly common approach.
  3. Ablation studies provide useful evidence for the role of entity persistence.
  4. Significant results have been achieved in long-video generation, both in terms of motion performance and overall quality.
Critical weaknesses
  1. I am puzzled as to why the authors did not prioritize ID consistency as a primary evaluation metric, especially in the main results table. The paper’s key improvements are reported based on metrics such as motion dynamics, VBench quality, and text alignment—none of which directly measure the preservation of identity. Incorporating evaluations of identity switching, object reproduction, and long-term appearance consistency would provide much stronger support for the paper’s core claims.
  2. Limited evidence of genuine object-centric identity preservation. The proposed slots are learned without explicit object-level supervision, and the paper does not sufficiently demonstrate that individual slots consistently correspond to the same objects across video chunks.
    3.In Tables 4 and 5, removing the selection-distillation loss improves Dynamic Degree (92.96 vs. 81.38) and tail text alignment (65.12 vs. 64.74), while the full model achieves only a marginal quality improvement (81.61 vs. 81.50). These results raise questions about the effectiveness and necessity of the proposed loss.
  3. Actually, there is a bit too much text in Figure 2.
Questions for authors
  1. Could the authors at least provide a comparison with other mature, open-source works in the same category, such as Shortstream and longlive-RAG? Shotstream: Streaming multi-shot video generation for interactive storytelling ECCV 2026 LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation arXiv:2606.02553

  2. Could the authors add more quantitative experiments regarding ID consistency?

6 comments
AuthorsOpenReview ↗

Part 1/3: Response overview and W1 & Q2

We sincerely thank Reviewer vjfR for the thorough review and constructive suggestions, which helped us strengthen both our empirical validation and presentation. In response, we have added MOTA tracking and controlled occlusion–reappearance evaluations (W1 & Q2), slot-level identity metrics (IDF1, IDTP/frTP) extending Table 4 (W2), tracking and visual analyses of Ldist\mathcal{L}_{\mathrm{dist}} (W3), comparisons with LongLive-RAG and Shotstream (Q1), and a streamlined Figure 2 (W4).

W1 & Q2: Direct Quantitative Evaluation of ID Consistency, Identity Switching, Object Reproduction, and Long-Term Appearance Consistency

Table 1 follows the standard 60-second interactive evaluation protocol (VBench Quality, Dynamic Degree, and segmented Text Align) to enable direct comparison against prior streaming baselines. To directly evaluate ID consistency, identity switching, object reproduction, and long-term appearance consistency, we add two quantitative evaluations on subsets selected by using an LLM to analyze the benchmark prompts and identify scripts suitable for evaluating these metrics.

1. Identity Switching and Spurious Object Reproduction (MOTA).

On the selected subset of the 60-second interactive benchmark, we evaluate Multi-Object Tracking Accuracy (MOTA) via TrackEval. Because generated videos lack ground-truth tracking annotations, we extract frame-level pseudo-ground-truth masks using ReferDINO and obtain cross-frame identity trajectories using Grounding DINO with ByteTrack under an identical pipeline across all methods. We report MOTA rather than identity switches (IDSW) alone because MOTA jointly penalizes identity switching, false positives (spurious object reproduction), and false negatives (dropped entities):

MethodMOTA ↑
MemFlow16.97
Anchor Forcing17.47
SlotMemory25.28

SlotMemory improves MOTA by 8.318.31 points over MemFlow and 7.817.81 points over Anchor Forcing, demonstrating stronger ID consistency with fewer combined identity switches, spurious object reproductions, and dropped entities during interactive prompt transitions.

2. Long-Term Appearance Consistency After Occlusion and Reappearance.

To isolate long-term appearance consistency when an entity exits the visible context and returns, we construct a four-stage sequence adapted from RememBench: Appear  (9 s)→Close  (3 s)→Hidden  (k  chunks)→Reappear  (6 s)\text{Appear}\;(9\,\text{s}) \rightarrow \text{Close}\;(3\,\text{s}) \rightarrow \text{Hidden}\;(k\;\text{chunks}) \rightarrow \text{Reappear}\;(6\,\text{s}), where each chunk contains Tc=3T_c = 3 latent frames (0.75 s0.75\,\text{s}). The target entity is initially visible, becomes fully occluded inside a closed container, remains hidden for k∈{9,18,36,54}k \in \{9, 18, 36, 54\} chunks (6.75 s6.75\,\text{s}, 13.5 s13.5\,\text{s}, 27.0 s27.0\,\text{s}, and 40.5 s40.5\,\text{s}—all exceeding the 1212-latent-frame / 3.0 s3.0\,\text{s} rolling local window), and subsequently reappears. We measure MBench Object Texture Consistency between the pre-occlusion (Appear) and post-return (Reappear) segments to evaluate the preservation of object-specific colors, textures, and patterns:

Methodk = 9 (6.75 s)k = 18 (13.5 s)k = 36 (27.0 s)k = 54 (40.5 s)
MemFlow14.3811.7021.6218.24
Anchor Forcing23.8024.2626.8223.18
SlotMemory24.0029.7041.9044.81

SlotMemory outperforms MemFlow across all hidden durations by 9.629.62 to 26.5726.57 points without deteriorating as the occlusion interval increases, verifying long-term appearance consistency after prolonged absence from the visible context. Notably, scores trend upward at larger kk because the evaluation protocol gates on successful concealment—filtering out trajectories where the subject remains visible despite the occlusion prompt. Brief concealment windows (k=9k=9) suffer from visual carry-over, making incomplete disappearance more frequent; extending the hidden phase affords more generation steps to completely remove the subject from the frame, while the entity bank holds its pre-occlusion K/V snapshots uncorrupted until the container opens. We will include both evaluations in the revised manuscript.

AuthorsOpenReview ↗

Part 2/3: W2 and W3

W2: Evidence of Object-Centric Slot-to-Object Correspondence Across Chunks

Although slots are learned without explicit object-level supervision, temporally initialized Slot Attention (Uˉlc−1\bar{U}_l^{c-1}) encourages consistent object assignments across consecutive chunks, while the persistent entity bank (Elc\mathcal{E}_l^c) associates slot descriptors with historical identities across longer horizons.

Within SlotMemory, slots serve as semantic routing addresses rather than immutable entity IDs.

To directly evaluate object-centric identity preservation and verify whether individual slots consistently correspond to the same objects across video chunks, we extend Table 4 with two tracking-based metrics evaluated via TrackEval against ReferDINO reference object tracks, treating each slot index as a predicted tracking identity:

  • IDF1: measures global identity association accuracy across the full sequence.
  • IDTP/frTP: measures the fraction of frame-level true-positive matches that preserve globally consistent identities.
  • FG-ARI & mBO: existing Table 4 metrics evaluating video-level object decomposition with temporal assignment consistency and object-mask overlap, respectively.
MethodFG-ARI ↑mBO ↑IDF1 ↑IDTP/frTP ↑
Baseline w/o Slot Compression––––
+ Slot (no entity bank)8.019.429.6362.59
+ Entity Bank (w/o Ldist\mathcal{L}_{\mathrm{dist}})10.4215.6418.2766.31
Ours (full)11.7715.5318.9871.37

Compared with slot-only memory (+ Slot), Ours (full) improves IDF1 from 9.639.63 to 18.9818.98 (+9.35+9.35) and IDTP/frTP from 62.5962.59 to 71.3771.37 (+8.78+8.78), alongside gains in FG-ARI (8.01→11.778.01 \rightarrow 11.77) and mBO (9.42→15.539.42 \rightarrow 15.53). These results demonstrate that coupling slot-based KV compression with entity-level memory substantially improves cross-chunk identity consistency without explicit object-level supervision.

W3: Effectiveness and Necessity of Selection-Distillation Loss (Ldist\mathcal{L}_{\mathrm{dist}})

Ldist\mathcal{L}_{\mathrm{dist}} aligns each slot's token-selection distribution with the clean KV states attended by self-attention, weighted by attention mass and value magnitude (Appendix C.2). Without Ldist\mathcal{L}_{\mathrm{dist}}, token selection relies solely on prototype similarity and assignment mass, admitting ungrounded background or transient tokens that induce spurious motion, object hallucination, and identity switching. Visually, Row 3 of Figure F.4 (+ Entity Bank (w/o Ldist\mathcal{L}_{\mathrm{dist}})) supports that its inflated Dynamic Degree (92.9692.96) reflects spurious optical flow from abrupt entity disappearance ("Missing Character", 14s) and sudden appearance shifts ("Attribute Drift", 24s).

To verify that Ldist\mathcal{L}_{\mathrm{dist}} suppresses spurious motion and hallucinations, we evaluate multi-object tracking (IDF1, IDTP/frTP) alongside slot–object alignment and consistency under the same architecture and remaining objectives (Lcon+Lrec\mathcal{L}_{\mathrm{con}} + \mathcal{L}_{\mathrm{rec}}):

MethodIDF1 ↑IDTP/frTP ↑FG-ARI ↑mBO ↑Subject Consistency ↑Background Consistency ↑Dynamic Degree ↑Tail Text Alignment ↑VBench Quality ↑
w/o Ldist\mathcal{L}_{\mathrm{dist}}18.2766.3110.4215.6493.0792.7692.9665.1281.50
Full18.9871.3711.7715.5395.0093.8681.3864.7481.61

Adding Ldist\mathcal{L}_{\mathrm{dist}} improves IDTP/frTP (+5.06+5.06) and IDF1 (+0.71+0.71), alongside gains in FG-ARI (+1.35+1.35), Subject Consistency (+1.93+1.93), and Background Consistency (+1.10+1.10), supporting fewer false-positive hallucinations and stronger cross-chunk identity continuity (trading −0.38-0.38 in tail text alignment for entity stability). Aggregate VBench Quality increases slightly (81.6181.61 vs. 81.5081.50) because its composite average offsets reduced spurious motion against consistency gains. Thus, Ldist\mathcal{L}_{\mathrm{dist}} is a better identity–motion trade-off to ground memory writes in attention-relevant KV tokens while preserving a Dynamic Degree of 81.3881.38—still exceeding the strongest baseline, Anchor Forcing (72.8772.87).

AuthorsOpenReview ↗

Part 3/3: Q1 and W4

Q1: Comparisons with LongLive-RAG and Shotstream (Shortstream)

We provide comparisons against both mature, open-source works in the same category—LongLive-RAG and Shotstream (Shortstream)—under their respective evaluation protocols.

1. Comparison with LongLive-RAG (120s Single-Prompt Generation).

We have included LongLive-RAG in Appendix Section D.1 (Table D.4), which evaluates 120-second single-prompt generation under VBench-Long. We reproduce the results below:

MethodSubject Cons. ↑Background Cons. ↑Motion Smooth. ↑Dynamic ↑Aesthetic ↑Imaging ↑Avg. Rank ↓
LongLive96.9395.6498.5847.1257.9066.954.08
Deep Forcing97.1795.7298.5746.0356.9866.034.58
Infinity-RoPE96.8195.6598.4853.5956.7366.195.00
MemRoPE97.5496.2998.73–59.2569.432.17
LongLive-RAG97.2295.8898.6250.2557.5766.952.92
SlotMemory97.5995.7298.5678.2662.2768.922.25

SlotMemory outperforms LongLive-RAG in Subject Consistency, Dynamic Degree (a +28.01+28.01-point improvement), Aesthetic Quality, and Imaging Quality, achieving a stronger average rank.

2. Comparison with Shotstream (60s Interactive Multi-Prompt Generation).

Although Shotstream primarily targets multi-shot interactive storytelling rather than continuous streaming generation, we additionally evaluate it on the 60-second interactive multi-prompt benchmark (Table 1) for a broader comparison:

MethodQuality ↑Dynamic ↑0–10s ↑10–20s ↑20–30s ↑30–40s ↑40–50s ↑50–60s ↑
Infinity-RoPE79.9936.3369.5866.2065.3864.5163.8159.98
LongLive79.0928.3071.8867.9767.4166.3665.9162.29
MemFlow78.6526.0571.9566.7465.7964.9564.0360.29
Shotstream76.8214.5466.9365.6864.5363.6263.1360.89
IAMFlow78.8224.3072.4166.9165.6964.6264.2659.87
Anchor Forcing82.3272.8771.8367.8365.8265.2864.7062.41
SlotMemory81.6181.3870.8367.0265.3866.1665.0962.97

SlotMemory outperforms Shotstream in Quality Score (81.6181.61 vs. 76.8276.82), Dynamic Degree (81.3881.38 vs. 14.5414.54), and text alignment across all six 10-second segments, while maintaining the highest Dynamic Degree and final-segment (50–60s) text alignment among all evaluated methods.

W4: Readability of Figure 2

To address the issue of too much text in Figure 2, we have revised Figure 2 by removing redundant text descriptions and simplifying the component annotations, improving readability while emphasizing the core read--write--update memory cycle.

Reviewer_gjU2OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SlotMemory, a semantic entity-level KV memory management method for streaming long-video generation. Semantic slots serve as addresses for selecting and retrieving original KV tokens, while a persistent entity bank maintains historical information across chunks. The method is evaluated on long-video and multi-prompt interactive generation, showing improvements in motion preservation and late-stage prompt adherence.

Critical strengths

The main strength is the shift from temporally organized history to semantically structured KV memory. This is particularly well motivated for multi-prompt interaction: a change in instruction may introduce new actions or interactions without changing the identities and attributes of previously established characters and objects. Managing memory around semantic entities therefore provides a reasonable mechanism for retaining relevant content across prompt transitions.

Critical weaknesses

The experimental evaluation does not fully cover the problem setting that motivates the method. The introduction identifies identity drift when entities leave and re-enter the scene, alongside semantic inconsistency during prompt transitions. However, the main interactive benchmark directly evaluates prompt switching, which does not necessarily require entities to leave the scene or disappear from the local context window.

Although the paper provides qualitative examples and aggregate consistency metrics, these do not isolate identity preservation after an entity’s absence and subsequent return. Consequently, the evidence for this capability remains less direct than the evidence for handling prompt transitions. A dedicated, controlled evaluation would strengthen the connection between the paper’s motivation and its experimental conclusions.

Questions for authors

Could the authors provide a controlled evaluation in which an entity appears, leaves the scene for longer than the local context window, and subsequently returns? How well are its identity and distinctive attributes preserved as the absence duration increases?

3 comments

We sincerely thank Reviewer gjU2 for recognizing the motivation of our work and for the insightful feedback. To directly address your shared concern across W1, W2, and Q1, we have added a strictly controlled four-stage occlusion–reappearance evaluation (Appear→Close→Hidden→Reappear\text{Appear} \rightarrow \text{Close} \rightarrow \text{Hidden} \rightarrow \text{Reappear}) measured via MBench Object Texture Consistency across absence durations from 6.75 s6.75\,\text{s} to 40.5 s40.5\,\text{s}.

W1 & Q1: Controlled Evaluation of Entity Absence and Re-entry Across Varying Durations

To directly evaluate the problem setting that motivates the method—identity drift when entities leave and re-enter the scene beyond the local context window—we conduct a dedicated, controlled evaluation adapted from RememBench using a four-stage Appear→Close→Hidden→Reappear\text{Appear} \rightarrow \text{Close} \rightarrow \text{Hidden} \rightarrow \text{Reappear} protocol.

In SlotMemory, self-attention attends to a 33-latent-frame sink, a 1212-latent-frame (3.0 s3.0\,\text{s}) rolling local window, and the long-term KV bank Blc\mathcal{B}_l^c governed by the persistent entity bank Elc\mathcal{E}_l^c. Each chunk spans Tc=3T_c = 3 latent frames (0.75 s0.75\,\text{s}). The four-stage sequence isolates entity absence from general prompt transitions as follows:

  1. Appear (9 s9\,\text{s}, 3636 latent frames): The target entity is visible in the scene.
  2. Close (3 s3\,\text{s}, 1212 latent frames): The target entity is enclosed inside a container and disappears from view.
  3. Hidden (kk chunks, 3k3k latent frames): The closed container remains in the scene with the target entity fully occluded for a controlled interval exceeding the 3.0 s3.0\,\text{s} rolling local window, evicting the entity from the local cache and requiring recovery from Blc\mathcal{B}_l^c.
  4. Reappear (6 s6\,\text{s}, 2424 latent frames): The container opens and the target entity returns to the scene.

The first two prompt switches are fixed at latent-frame indices 3636 and 4848, and the third switch occurs at 48+3k48 + 3k, ensuring that only the absence duration varies. We evaluate k∈{9,18,36,54}k \in \{9, 18, 36, 54\}, corresponding to absence durations of 6.75 s6.75\,\text{s}, 13.5 s13.5\,\text{s}, 27.0 s27.0\,\text{s}, and 40.5 s40.5\,\text{s}—all exceeding the 3.0 s3.0\,\text{s} local context window. Identity and distinctive attributes are measured via MBench Object Texture Consistency between the pre-occlusion (Appear) and post-return (Reappear) segments:

Methodk=9k=9 (6.75 s6.75\,\text{s})k=18k=18 (13.5 s13.5\,\text{s})k=36k=36 (27.0 s27.0\,\text{s})k=54k=54 (40.5 s40.5\,\text{s})
MemFlow14.3811.7021.6218.24
Anchor Forcing23.8024.2626.8223.18
SlotMemory24.0029.7041.9044.81

Across all four settings where the entity disappears from the local context window and subsequently returns, SlotMemory outperforms MemFlow by 9.629.62 to 26.5726.57 points and exhibits no degradation as the absence duration increases from 6.75 s6.75\,\text{s} to 40.5 s40.5\,\text{s}. Scores rise with kk because MBench Object Texture Consistency verifies whether the target entity effectively disappears during Hidden and excludes sequences where the entity remains visible: shorter Hidden intervals (k=9k=9) retain residual visual context from Appear and Close, whereas longer Hidden intervals provide more chunks conditioned solely on the occlusion prompt to clear the entity from the scene.

W2: Isolating Entity Identity Preservation from General Prompt Transitions

To complement aggregate consistency metrics and isolate identity preservation after an entity's absence and subsequent return, we pair the dedicated, controlled Appear–Close–Hidden–Reappear\text{Appear}\text{--}\text{Close}\text{--}\text{Hidden}\text{--}\text{Reappear} protocol with MBench Object Texture Consistency (results reported in W1 & Q1).

Unlike full-sequence aggregate metrics (e.g., VBench Subject Consistency), which average similarity across all frames and can be dominated by uninterrupted visibility or low-motion intervals, MBench Object Texture Consistency evaluates the preservation of distinctive object attributes—color, texture, and complex patterns—exclusively between the pre-occlusion (Appear) and post-return (Reappear) segments while requiring the entity to disappear during Hidden. Holding the visible stage durations fixed and keeping the prompt-switch structure constant while varying only the occluded Hidden interval (6.75 s6.75\,\text{s} to 40.5 s40.5\,\text{s}) decouples long-term entity recall from scene-level temporal stability and general prompt transitions. SlotMemory's sustained performance across all absence horizons (outperforming MemFlow by up to 26.5726.57 points) directly strengthens the connection between the paper's motivation and its experimental conclusions.

Dear Reviewer gjU2,

Thank you once again for your insightful and constructive review, which has greatly contributed to improving the quality and clarity of our paper SlotMemory. We sincerely hope that our rebuttal has satisfactorily addressed your questions and concerns.

if you have any further concerns or questions, we would very like to respond immediately, and hopefully consider to raise the score. We truly appreciate your valuable feedback and the time and effort you dedicated to reviewing our work.

Best regards,

Authors of Submission 2037

Area_Chair_54FoOpenReview ↗

Reminder: Please Follow Up on Authors’ Responses

Dear Reviewer gjU2,

A friendly reminder to check the authors’ responses and follow up with any comments or remaining concerns. Your feedback would be greatly appreciated.

Thank you for your time and effort!

AC

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair for organizing and overseeing the discussion phase, and all the reviewers for their constructive feedback. Your suggestions have substantially strengthened the statistical rigor, cross-backbone generality, and clarity of our manuscript. Sepcifically, Reviewer RW8m comfirmed our rebuttal. Reviewer W8gV said "I would be willing to raise my score if seed-level variance for the main quality metrics and second-backbone validation were provided." and we have provided such experiments.

1. Core Strengths Recognized by Reviewers

  • Principled Memory Decoupling (All Reviewers): Unanimous recognition of shifting from temporal to object-centric KV memory and using semantic slots solely as routing addresses over uncompressed KV tokens—avoiding lossy slot reconstruction bottlenecks.
  • Horizon-Invariant Motion & Late-Rollout Alignment (vjfR, W8gV): Praised for maintaining strong Dynamic Degree across 30s–240s (79.30–81.38 vs. baselines falling to 33–60) and superior late-segment prompt adherence while adding minimal overhead at 5s.
  • Rigorous Ablations & Transparent Systems Cost (W8gV, RW8m): Commended for equal-budget Top-kk controls, thorough loss/RoPE ablations, and honest latency/VRAM reporting.

2. Key Issues Resolved in Rebuttal & Revision

  • Direct Identity Continuity & Reappearance (gjU2, vjfR, W8gV, RW8m): Added a controlled Appear → Close → Hidden → Reappear benchmark (6.75s6.75\text{s}–40.5s40.5\text{s} occlusion, exceeding the 3.0s3.0\text{s} local window) and MOTA tracking diagnostics. SlotMemory outperforms MemFlow by +9.62 to +26.57 in Object Texture Consistency without decay up to 40.5s40.5\text{s}, improves MOTA to 25.28 (vs. 16.97 MemFlow / 17.47 Anchor Forcing), and empirically justifies disabling DORMANT re-ID to avoid long-gap RoPE snapshot corruption.
  • Statistical Robustness & Generalization (W8gV): Reported 5-seed mean ± std across Tables 1–3 (81.61 ± 0.14 on 60s interactive; 83.68 ± 0.26 / 83.79 ± 0.21 Total on 5s/30s), confirming every metric's lower bound (mean - 1 std) beats MemFlow and LongLive. Validated generalization across 1024×5761024\times576 resolution, 3-step/shift-4.0 schedules, and a second backbone (LongSANA / SANA-Video).
  • Controlled Baselines & Memory Accounting (vjfR, W8gV, RW8m): Added an Anchor-only Memory ablation under the identical training/recaching recipe plus comparisons against LongLive-RAG and Shotstream. Explicit per-layer accounting (2,688 active vs. 8,192 background tokens) alongside equal-budget Top-kk and capacity-scaling controls confirms gains stem from entity structure rather than raw capacity.
  • Unsupervised Slot Tracking, Ldist\mathcal{L}_{dist}, & 240s Plausibility (vjfR, W8gV): Added slot tracking metrics (IDF1: 9.63 → 18.98, IDTP/frTP: 62.59 → 71.37) proving cross-chunk object correspondence without object supervision. Verified that Ldist\mathcal{L}_{dist} filters ungrounded KV writes to suppress hallucinations and spurious motion, and confirmed 240s motion plausibility via Subject/Aesthetic/Imaging stability and a blinded 20-user study (top preference at 27.0%).
  • Presentation Clarity (vjfR, RW8m): Redesigned Figure 2 with standardized larger fonts, expanded margins, and cleaner workflow schematics.

With all experimental, statistical, and presentation concerns addressed, we respectfully ask the Area Chair and Reviewers to consider these revisions in the final decision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.