acceptodds
Under review as a conference paper at ICLR 2027

LOCI: Spatial Linear Memory for Streaming World Models

Abstract

When a camera revisits a previously observed region, a video world model should reproduce what was there before. This requires both remembering past observations and retrieving the right one for the current viewpoint. Key–value caches preserve visual detail but grow with video length; recurrent memory is compact but compresses history into a fixed-size state, making individual past observations no longer directly accessible. We introduce LOCI, a hybrid spatial-memory architecture that maintains both representations. In half of the transformer blocks, main attention keeps a key–value cache of past observations; in the other half, it is restricted to the current chunk and complemented by a recurrent linear-attention memory whose reads and writes are conditioned on projective camera geometry, so that viewpoint enters both memory addressing and stored content. Recurrent readouts flow into subsequent cache-backed blocks, supplying their queries with accumulated scene context. On the public MIND memory benchmark and on held-out recorded trajectories, LOCI reproduces revisited content more faithfully than representative world models and a same-recipe full-softmax model; with full history, it lowers peak memory at equal length by about 30% relative to full softmax. With a bounded bank of retained observations, which keeps memory constant, it streams long videos and remains more faithful than full softmax under the same budget.

Primary area:
generative models
Keywords:
World Models, Video Generation, Linear Attention, Memory Retrieval, Long-Horizon Consistency

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bsxrOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LOCI, a hybrid spatial-memory architecture for camera-controllable streaming world models. It combines explicit historical KV access with a fixed-size recurrent KDA memory. The recurrent path is conditioned on camera geometry through PRoPE, and its readout influences later historical-attention layers. The method also introduces per-chunk retention and a bounded bank of retained observations for long-horizon streaming. Experiments on MIND and held-out trajectories show better revisit fidelity than a same-recipe full-softmax baseline, including under the same bounded history budget, while reducing memory usage and enabling longer generation.

Critical strengths
  1. The controlled comparison with full softmax is convincing. The two models use the same backbone, training data, recipe, and number of updates, and the bounded setting also uses the same retained observations. The consistent gains therefore provide reasonably strong evidence that the overall hybrid memory design is useful.
  2. The paper evaluates both memory quality and practical efficiency. Besides revisit fidelity, the paper analyzes long-range recall, recurrent-state behavior, historical attention, and memory/runtime cost. The bounded version supports long streaming generation with essentially constant memory while maintaining better revisit fidelity than the corresponding full-softmax baseline.
Critical weaknesses
  1. The contribution of the key architectural components is not sufficiently isolated. The main controlled comparison changes several factors at once, including the recurrent KDA path, the distribution of local versus historical-attention layers, additional parameters, and PRoPE conditioning. In particular, the paper does not report end-task revisit metrics for an otherwise matched recurrent model without PRoPE. The yaw-decoding analysis mainly shows that camera information is encoded in the state, but does not establish how much projective conditioning contributes to the final generation quality.
  2. The long-horizon streaming claim is not fully validated under the deployment setting. During training, recurrent states are formed by scanning noised ground-truth chunks, whereas at inference they are persistently committed from the model’s own generated chunks after denoising. This train–inference mismatch is directly relevant to long-horizon memory, where generation errors may accumulate in the state. The existing rollout results are encouraging, but the effect of the commit strategy is not directly ablated, and the empirical scope is further limited to one 5B backbone and mostly rendered held-out trajectories.
Questions for authors
  1. Could the authors report end-to-end MIND and/or held-out revisit results for the trained recurrent variant without PRoPE camera conditioning? This would directly clarify how much of LOCI’s gain comes from projective conditioning itself rather than from the recurrent hybrid architecture more generally.
  2. Could the authors quantify the impact of the train–inference mismatch in recurrent-state construction? In particular, how sensitive are long-horizon results to the commit strategy and the choice of t=100 in the bounded setting, and does performance change when memory is accumulated from generated rather than clean/noised history?
3 comments

Thank you for your thoughtful review. Following your questions, we added experiments on commit sensitivity, generated-history effects and long-horizon revisit fidelity. The global response summarizes the shared results; intervals below are paired cluster-bootstrap 95% CIs.

W2 / Q2 — train–inference mismatch, commit strategy, generated history. We agree that training scans noised ground-truth chunks in one forward pass, whereas inference commits the model's own denoised chunk in a separate pass. In our tests, committing with light noise (t=100) helps both models, and LOCI keeps its revisit advantage at both commit timesteps and with generated history.

  • Commit timestep: on 16 recorded 40 s clips with three seeds, crossing full/bounded history with t=0/100 shows that LOCI leads in all four settings (+0.64 to +1.15 dB) and that t=100 helps both models in both modes (+0.78 to +1.30 dB; all unadjusted intervals exclude 0), consistent with a slightly noised commit being closer to training inputs for both state and KV cache. The value was not tuned on any evaluation set; here we compare t ∈ {0, 100}. The 300 s test uses t=100; rerunning its 5 pre-existing scenes with t=0 also keeps LOCI ahead (+1.46 dB [+1.23, +1.65], 5/5 scenes; coarse with 5 scenes; global G3; App. G.2, Table 17).
  • Generated history: at the same revisit instants on set A, absolute quality declines for both models as more history is generated, but LOCI's advantage stays positive at every tested level (+0.81 dB [+0.29, +1.45] with all ground-truth history, +0.99 dB [+0.37, +1.67] with all generated history); its change is uncertain (+0.175 dB [−0.715, +1.052]; global G1(b), App. G.1, Table 16).
  • Long horizon: across 19 held-out scenes and 164 returns in 300 s generated continuations, LOCI gains +2.09 dB PSNR [+1.40, +2.82] (133/164 returns), with the same bounded budget of 29 frames. It leads in every elapsed-time bin, and its absolute revisit PSNR stays at 11.3–11.7 dB while full softmax falls from 10.2 to 9.1 dB; absolute fidelity remains low, so this is a sustained relative advantage. Global G1(a), the keHq reply and App. G.1, Table 15 cover late returns, anchor specificity and secondary metrics.

These tests change KV history and recurrent state together; they do not isolate recurrent-state mismatch alone. This and the single 5B backbone remain limitations. Beyond rendered trajectories, generating from 10 real photographs, LOCI still leads (+0.96 dB [+0.64, +1.30], 49/60 cases; App. G.5, Table 20); recorded real-video revisits remain open.

W1 / Q1 — PRoPE versus the recurrent hybrid architecture. The evidence attributes the gain over full softmax to the complete hybrid design. With UCPE, PRoPE helps early in training, but by 5,000 updates there is no consistent benefit (revisit LPIPS +0.009 [−0.001, +0.020] on 60 designed cases; −0.004 [−0.016, +0.006] on held-out recorded set A, bounded). Because UCPE already carries camera information into the state, we also replaced it with a frame-level embedding; there, adding PRoPE gives −0.018 revisit LPIPS [−0.035, −0.004] at 5,000 updates (37/60; one run per arm). These comparisons do not separate PRoPE's contribution at convergence (global G2 and App. G.3, Table 18 give all checkpoints).

To isolate the split, we trained a half-softmax control with LOCI's 15 chunk-local layers and no state. At the same 2,000-update budget, on two separate evaluation sets of 60 revisits from different maps (same checkpoints), it is not significantly different from full softmax (+0.004 and −0.015 LPIPS; both CIs include 0). LOCI improves over this control by −0.071 [−0.096, −0.049] and −0.048 [−0.065, −0.034] LPIPS (50/60 and 49/60), so at 2,000 updates the layer split alone does not account for the gain (App. G.4, Table 19).

The recurrent branch adds 90.7M parameters (1.7% of 5.38B); the comparison is not parameter-matched, and alternative layer arrangements were not evaluated. We have revised the abstract, introduction and contributions to reflect this attribution, and frame Table 2 as a comparison of the complete hybrid design against the standard design under identical backbone, data, schedule and bounded budget.

Thank you for the detailed response and additional experiments.

The generated-history rollouts and commit-timestep controls largely address my main concerns. The half-softmax comparison also helps clarify the contribution of the recurrent memory.

My remaining concerns are mainly about the limited independent benefit of PRoPE under the original UCPE setting and the lack of reliable real-world revisit evaluation. However, the authors have clarified these limitations, and the new results provide stronger support for the overall hybrid design.

I appreciate the revisions and maintain my weak-accept rating.

We sincerely appreciate your positive assessment and the time you spent reviewing our paper and responses. Your questions regarding the generated history and the commit timestep prompted valuable experiments that strengthened the revised paper, and we will continue to refine it for the final version.

Reviewer_op9wOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LOCI, a hybrid spatial-memory architecture for camera-controllable streaming video world models. In half of the blocks, it performs historical softmax attention that retains the keys and values (KV) of past observations, and the remaining half combines intra-chunk softmax attention with a KDA-based recurrent memory whose queries, keys and values are conditioned on camera geometry via PRoPE, with retention applied once per chunk. By feeding the readout of this memory into the queries of subsequent historical-attention blocks, the method outperforms a same-recipe full-softmax baseline in revisit fidelity on the MIND memory benchmark and on held-out recorded Unreal Engine trajectories, while streaming long videos at constant memory under bounded sparse access.

Critical strengths
  • The paper demonstrates a significant improvement in a controlled comparison with a full-softmax baseline trained under the same recipe, and since the improvement is also obtained under an identical bounded KV budget, the effect of the architecture itself is isolated.
  • Since the paper compares revisits directly against recorded ground truth and also verifies camera following, the evaluation of revisit fidelity is reliable. It is also easy to trace the correspondence between the claims and the evidence, in part because the appendix documents the evaluation protocol and how each external baseline receives the memory segment.
Critical weaknesses
  • The central claim of this paper is that conditioning the recurrent memory on projective camera geometry with PRoPE makes what the memory writes and reads depend on viewpoint, and thereby improves fidelity at revisits. However, all 30 blocks keep a UCPE camera-attention branch, and the token features written to the recurrent memory already carry camera information. Therefore, it is possible that comparable fidelity could be obtained even without PRoPE in the recurrent memory. This point should be verifiable with the recurrent memory without camera encoding shown in Fig. 5; however, this model is used only to compare the linear readout of camera yaw, and no results on generation fidelity (revisit PSNR/LPIPS on the MIND memory test or on the held-out recorded trajectories) are reported for it. At present, it is impossible to distinguish whether the gain over full softmax comes from PRoPE or simply from adding the recurrent memory itself. Generation fidelity with and without PRoPE should be compared under the same recipe.
  • I understand the primary motivation for pairing a fixed-size recurrent state carried over the full history with observation-level KV. However, many of the specific design choices are adopted from existing methods or appear arbitrary, and the rationale for these choices is not sufficiently explained. For example, the layer layout is adopted from ARL2, a text-to-video hybrid without camera control, and no justification is given for why it is suited to spatial memory. Furthermore, the ablations cover the recurrent state itself and the retention schedule, but not the other design choices (layer layout, PRoPE, the choice of KDA, the bank selection rule), so it is impossible to determine which components contribute to the gain. For the key design choices, the rationale or ablations should be provided.
Questions for authors
  • On the recorded trajectories, bounded sparse access scores higher than full history. While the authors attribute this to the training window (about 20 s), which places the full self-generated history further from the training distribution, this does not explain why full softmax does not benefit from bounded access as well (Table 13). Note also that the two settings differ in the commit timestep. Does the advantage remain under the same commit timestep, and would full history improve if the model were trained with longer training windows?
  • Most of the training data of the main models is rendered in Unreal Engine, and the authors' held-out trajectories are also rendered in Unreal Engine; since MIND is itself rendered in Unreal Engine 5, no evaluation with ground truth is on real-world captures. If evaluated on real-world captures, would the improvement over full softmax still hold?
3 comments

Thank you for a perceptive review; your points on attribution and the commit timestep prompted new experiments and revised claims. Intervals are paired cluster-bootstrap 95% CIs.

W1 — PRoPE vs. adding recurrent memory. The evidence attributes the gain over full softmax to the complete hybrid design, whose recurrent readout enters historical attention (see W2), and we have revised the abstract, introduction and contributions accordingly. With UCPE, PRoPE's revisit benefit is concentrated early in training (global G2); by 5,000 updates there is no consistent benefit (revisit LPIPS +0.009 [−0.001, +0.020] on the 60 designed cases; −0.004 [−0.016, +0.006] on held-out recorded set A, bounded). We agree that UCPE injects per-token camera geometry into every model's residual stream, so the variant without PRoPE is not camera-free.

When we replace UCPE and its camera KV with a frame-level camera-motion embedding, adding PRoPE gives −0.018 revisit LPIPS [−0.035, −0.004] on the designed cases at 5,000 updates (37/60; set A: −0.025 [−0.054, +0.004]), a modest effect from one run per arm (App. G.3, Table 18). At 5,000 updates on the same 60 cases, this effect differs from the one with UCPE by −0.027 [−0.048, −0.007] (interval over test cases), though these comparisons still do not separate PRoPE's contribution at convergence.

Q1 — Table 13 and the commit timestep. The original rows indeed compared full history at t=0 with bounded history at t=100. Across three seeds, once the commit timestep is matched, bounded history is 0.4–0.8 dB below full history for both models (one interval excludes 0), whereas committing at t=100 improves both (+0.78 to +1.30 dB; all unadjusted intervals exclude 0), and LOCI leads full softmax in all four settings (+0.64 to +1.15 dB). Full softmax is therefore not an exception: bounded history itself helps neither model, and the commit timestep accounts for LOCI's bounded-over-full gain in Table 13, which was not significant (+0.46 [−0.17, +1.20]; interaction with the model +0.47 [−0.40, +1.31]). We updated Sec. 5.5 and App. C accordingly. The modes still differ in history range and temporal positions; longer training windows are future work (App. G.2, Table 17).

W2 — design choices. Our contribution is a directly trained, camera-controlled hybrid memory that improves on full softmax under the same KV budget. At 2,000 updates, the half-softmax control shows that the layer split alone does not explain this gain (added parameters are not controlled). We did not search over alternatives; the rationale and evidence for each choice follow.

  • Layer layout: hybrid readouts reach later historical-attention blocks and enter their queries. Zeroing only this readout at inference lowers co-visible attention enrichment downstream (+0.56 [+0.21, +0.77], on minus off), most strongly right after a hybrid block (+0.86 [+0.45, +1.04]; Table 5). A half-softmax control with this split and no state is not significantly different from full softmax, while LOCI improves on it by −0.071 and −0.048 LPIPS (App. G.4, Table 19). We adopt ARL²'s released 50% layout (designed for text-to-video, not revisits) and did not test others.
  • KDA: its delta rule can revise stored key–value associations when a place is seen again, unlike additive accumulation (e.g., Hybrid Forcing); we have not tested other recurrences under our recipe. The trained state is used: 6.4–11.1% of channels in three layers have mean retention > 0.9, and resetting it at every chunk worsens local revisit error (+0.021 [+0.013, +0.028], 15 of 16 segments; Sec. 5.3).
  • Bank: a coverage heuristic, not tuned on evaluation sets, using only commanded poses so both models retain identical frames. With most-recent or random selection at the same capacity (set A, 3 seeds), LOCI still leads (coverage / recency / random: +0.85 / +0.59 / +0.73 dB revisit PSNR; the recency interval includes 0) and the rule does not significantly change its advantage; coverage performs similarly to random selection with about 6 bank frames instead of 17 (App. G.6).
  • PRoPE: see W1.

Q2 — real-world captures. We now state explicitly in the main text that all multi-frame ground-truth trajectories are engine-rendered. Starting from 10 real daytime photographs without people, we generate 60 designed revisits. Scored against the real photograph at returns, LOCI leads by +0.78 dB PSNR [+0.35, +1.25] and −0.033 LPIPS [−0.047, −0.018]; including returns to later poses, scored against the model's own earlier frame, it leads by +0.96 dB [+0.64, +1.30] (49/60; App. G.5, Table 20).

This evaluates real image content; recorded real-world camera paths remain an open limitation: in our tests on recorded real video, neither model reliably followed the commanded returns, so revisit fidelity could not be separated from camera following. Moving pedestrians also remain a limitation.

Thank you for your rebuttal. I have read the additional experiments, and I appreciate your responses to each point I raised. In particular, I appreciate that you identified the commit-timestep confound through experiments with matched commit timesteps, revised the main text accordingly, and added ablations on PRoPE and some design choices, as well as an evaluation using real-world images.

However, some concerns remain. The comparison with and without PRoPE showed no consistent effect when PRoPE was used with UCPE. I appreciate that you revised your claims to match this result, but the original central claim is not supported, and the justification for including PRoPE in the final model is now weaker. You also acknowledge that you have not isolated PRoPE’s contribution at convergence. Regarding the choice of KDA and the layer layout, you explained the rationale, but as you note, you did not compare against alternatives.

Although the paper has improved overall, these concerns remain, so I maintain my current score.

Thank you for carefully reading our paper and responses, and for your thorough review and constructive comments. They have helped us improve the revised paper, and we will continue to refine it in the final version.

Reviewer_keHqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LOCI introduces a hybrid memory architecture for streaming, camera-controlled video generation. It combines a fixed-size recurrent state, updated through camera-conditioned delta rules with chunk-level retention, with explicit attention to retained historical observations. A bounded-history configuration maintains a sink frame, recent frames, and a coverage-based bank of older views. The method is evaluated using a Wan2.2 5B backbone, matched-training full-softmax baselines, MIND memory evaluation, rendered revisit trajectories, and long-horizon resource measurements.

Critical strengths
  1. The matched-training comparisons provide substantive evidence for the hybrid architecture. Table 2 controls the backbone, training data, recipe, update count, and retained historical frames. Under the bounded-history setting, LOCI improves MIND PSNR from 12.13 to 13.02 dB, with a paired improvement of 0.89 dB and a 95% interval of [0.64, 1.15], and improves 44 of 50 segments. Improvements also occur under the uniform data mixture and on the common completed full-history subset. These controls make the empirical contribution substantially more convincing than comparisons between independently trained systems.

  2. The paper demonstrates a concrete quality–memory tradeoff with a genuinely bounded-history implementation. On the reported 300-second H200 rollout, bounded LOCI uses at most 23.6 GiB versus 27.6 GiB for the bounded full-softmax baseline, at approximately the same generation speed. The paper distinguishes fixed recurrent storage from explicit historical caches and acknowledges that camera-attention KV remains in every block. The state-reset experiments also provide useful evidence that the recurrent state contributes to revisit fidelity.

Critical weaknesses
  1. The specific contribution of projective conditioning is less clearly established than the benefit of the complete hybrid architecture. Camera-yaw decodability is similar for an untrained branch with the same conditioning, so this probe primarily establishes that camera information is encoded. It does not establish that projective conditioning improves learned scene recall. The no-camera comparison emphasizes decodability, whereas a matched comparison on revisit quality would better distinguish geometry-aware memory from generic recurrent capacity. The evidence for chunk-level rather than token-level retention is encouraging, but its longer-horizon metric was defined after the initial comparison and is appropriately described as exploratory.

  2. Evidence for persistent scene fidelity over extended autonomous generation remains limited. The additional multi-minute recall experiment uses four trajectories with 68–245 seconds of ground-truth history followed by 45–60 seconds of generation. This is informative about recall from an observed prefix, but does not establish comparable behavior after several minutes of self-generated history. The 300-second experiment establishes resource stability rather than sustained factual scene consistency. Evaluation is also restricted to synthetic/rendered environments and one backbone, limiting the demonstrated scope of the spatial-memory claim.

Overall assessment: The controlled improvements and measured memory savings support acceptance, despite incomplete attribution of the geometry-specific contribution and limited evidence for sustained scene fidelity during extended autonomous generation.

Questions for authors
  1. Could you report revisit-quality results for the existing recurrent variant without camera encoding, under the same training recipe and retained-frame budget? The key clarification is whether PRoPE improves generation and recall, beyond making camera pose decodable.

  2. Do you have aggregate revisit-quality measurements for long rollouts in which the historical prefix is generated by the model? Comparing these with the ground-truth-prefix setting would help determine whether the demonstrated recall gains persist under accumulated generation errors.

1 comment

Thank you for your thoughtful review and constructive experimental suggestions. Our new experiments directly evaluate recall after self-generated history; the global response summarizes the shared protocols and results. Intervals below are paired cluster-bootstrap 95% CIs.

Q2 / W2 — revisits after self-generated history. After a recorded sweep, LOCI's advantage persists through 300 s of generated continuation: +2.09 dB PSNR [+1.40, +2.82] across 19 scenes and 164 returns (133/164 improved). Both models use the same retained frames (including 6–17 recorded sweep frames at every return) and commit t=100. The advantage remains after 150 s (+2.27 dB [+1.41, +3.19], 75/89 returns).

To test whether the improvement is specific to the correct anchor rather than generic scene similarity, we also score each return against the scene's other recorded anchors (at least 60° away) as a mismatched-anchor floor. LOCI's margin over its floor exceeds that of full softmax by +1.79 dB [+1.19, +2.45] (18/19 scenes), so most of the advantage is specific to the recorded anchor. On the 132 returns whose anchor frame was never retained, LOCI still leads by +1.80 dB [+0.99, +2.68] (103/132; retention is fixed by the sweep geometry, so we do not compare the two subsets). App. G.1 (Table 15 and text) gives the floors, DINO, LPIPS and absolute PSNR trends.

For your suggested comparison at identical revisit instants (on 40 s set-A clips), LOCI's PSNR advantage is +0.81 dB with ground-truth history and +0.99 dB with all generated history. Although absolute quality declines for both models as more of the history is generated, LOCI's advantage stays positive at every tested level; its change from all ground truth to all generated history is uncertain (+0.175 dB [−0.715, +1.052]; intervals and clip counts for every condition in global G1(b) and App. G.1, Table 16).

Q1 / W1 — does PRoPE improve generation and recall beyond decodability? We agree that decodability alone does not establish better recall. With UCPE, PRoPE helps early in training, but not consistently at 5,000 updates. On 60 pre-specified designed revisits, it gives −0.044 LPIPS [−0.060, −0.023] at 1,000 updates (47/60 cases), whereas by 5,000 updates there is no consistent benefit (revisit LPIPS +0.009 [−0.001, +0.020]; held-out recorded set A, bounded: −0.004 [−0.016, +0.006]).

Because UCPE already carries camera information into the state, the variant without PRoPE can learn viewpoint indirectly. When we replace UCPE with a frame-level camera embedding, adding PRoPE gives −0.018 LPIPS [−0.035, −0.004] at 5,000 updates (37/60; one run per arm). Global G2 and App. G.3, Table 18 give all checkpoints and secondary metrics. Together with the half-softmax control (global G2), the evidence attributes the gain over full softmax to the complete hybrid design, whose recurrent readout enters historical attention, and we have revised the abstract, introduction and contributions accordingly.

W1 — Table 14. We agree: because the long-range metric was defined after the short-range comparison, we now label Table 14 as exploratory and state the related sentence in the introduction as a hypothesis.

Scope. We acknowledge that our recorded evaluations are rendered and use one backbone. Within that setting, 17 of the 19 scenes of the 300 s test lie on maps not used in training; on them LOCI gains +2.16 dB [+1.39, +2.98] (App. G.1, Table 15). Starting from real photographs, LOCI also retains an advantage; our reply to op9w (Q2) and App. G.5, Table 20 report it together with the remaining limitation of recorded real-video revisits.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of new experiments and revisions

Thank you for the constructive reviews. All three reviewers regarded the matched comparison with full softmax (same recipe and bounded KV budget) as substantive evidence for the hybrid design; the shared questions are whether the gain holds after self-generated history (G1) and what drives it (G2). The MIND results (Tables 1–2) are unchanged; held-out recorded and designed trajectories add timed returns to recorded poses and matched-history tests. Intervals are paired cluster-bootstrap 95% CIs.

G1. Revisits after generated history (keHq W2/Q2, bsxr W2/Q2).

(a) 300 s continuations. Across 19 held-out scenes and 164 returns, LOCI gains +2.09 dB PSNR [+1.40, +2.82] (133/164 returns) and leads in every elapsed-time bin, including after 150 s: +2.27 dB [+1.41, +3.19] (75/89 returns). After a 40 s recorded sweep, both models generate controlled excursions (≤ 1.5 m), returning about every 30 s to four recorded anchor poses; they retain identical frames (at most 29; 18–29 at returns, including 6–17 recorded sweep frames) and commit at t=100. Further checks: App. G.1, keHq reply.

(b) Same revisit instants, different histories. On set A with bounded history:

HistoryΔPSNR (LOCI − full softmax), dBClips improved
All ground truth+0.81 [+0.29, +1.45]11/16
Last 7.5 s generated+1.70 [+0.81, +2.80]14/16
Last 17.5 s generated (15 long-enough clips)+1.90 [+0.77, +3.15]13/15
All generated+0.99 [+0.37, +1.67]13/16

Absolute quality declines for both models, but LOCI's advantage stays positive in every condition; its change from all ground truth to all generated history is uncertain (+0.175 dB [−0.715, +1.052]; App. G.1, Table 16). These tests change KV history and recurrent state together, so they do not isolate state mismatch.

G2. PRoPE attribution (keHq Q1, op9w W1/W2, bsxr W1/Q1). The evidence across controls attributes the gain over full softmax to the complete hybrid design, whose recurrent readout enters historical attention. A half-softmax control with LOCI's layer split but no state is not significantly different from full softmax at 2,000 updates, while LOCI improves on it by −0.071 and −0.048 LPIPS on two case sets (App. G.4). We have revised the abstract, introduction and contributions accordingly. We compare recurrent variants with and without PRoPE trained with the same recipe and updates on the uniform mixture (not the main models' later UE-weighted stage). On 60 pre-specified designed revisits, PRoPE gives −0.044 LPIPS [−0.060, −0.023] (47/60 cases) at 1,000 updates and −0.030 [−0.043, −0.018] pooled over 1,000 and 2,000 updates (planned secondary test; 2,000 alone: −0.017 [−0.035, +0.002]); by 5,000 updates there is no consistent benefit (revisit LPIPS +0.009 [−0.001, +0.020] on the 60 designed cases; −0.004 [−0.016, +0.006] on held-out recorded set A, bounded).

As op9w points out, all models share UCPE, whose per-token camera conditioning enters the residual stream, so the variant without PRoPE is not camera-free. Since revisits require following commanded camera paths, the question is whether PRoPE in the memory helps beyond the backbone's camera conditioning. To remove per-token camera conditioning from the backbone, we replaced UCPE and its camera KV with a frame-level camera-motion embedding and retrained both variants. In this setting, PRoPE gives −0.018 LPIPS [−0.035, −0.004] at 5,000 updates (37/60; set A: −0.025 [−0.054, +0.004]), with no reliable gap at 1,000; this is a modest effect from one run per arm (App. G.3, Table 18). Within the hybrid design, PRoPE is thus parameter-free camera conditioning of the recurrent queries, keys and values that helps early in training and, without per-token camera conditioning, also at 5,000 updates; these comparisons do not separate its contribution at convergence.

G3. Commit timestep and Table 13 (op9w Q1, bsxr W2/Q2). As op9w noted, the original rows also differed in commit timestep. When we cross full and bounded history with t=0/100 on 16 clips × 3 seeds, LOCI leads in all four settings (+0.64 to +1.15 dB), and commit t=100 helps both models in both history modes (+0.78 to +1.30 dB; all unadjusted intervals exclude 0). Once the commit timestep is matched, bounded history is 0.4–0.8 dB below full history for both models (one interval excludes 0), the cost of constant memory. Table 13's bounded-history gain therefore comes from the commit timestep, and we updated Sec. 5.5 and App. C (App. G.2, Table 17).

Revisions. App. G.1–G.6 document all new experiments, including real-photo and bank-rule tests (op9w reply). Table 14 is now marked exploratory. The limitations state that recorded real-video revisits remain open, that we use one backbone, and that layer layout and parameter count are not separately controlled.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.