acceptodds
Under review as a conference paper at ICLR 2027

Anamnesis: Training-Free Episodic Recollection for Autoregressive World Models

Abstract

Autoregressive (AR) world models generate interactive worlds by conditioning on the recent past, yet they systematically forget it: scenes revisited after minutes of exploration are regenerated as if seen for the first time, and spatial memory collapses within the span of a single attention window. We argue that this failure has a precise phenomenological signature: current world models implement retention (the KV cache) and protention (action conditioning), but possess no recollection: the deliberate reactivation of earlier experience when the present re-encounters it. We introduce , a training-free episodic recollection mechanism for frozen AR world models: a buffer stores denoised latent states tagged with exact camera poses; a pose-gated detector recognizes revisits; and recognized revisits are re-rendered by re-noising the stored latent to a low level and granting the model a single refinement step. No gradients are updated. A KV-cache alternative (writing the anchor into the recency slot) leaves the output unchanged: recollection must enter through the generative channel, not the retention channel. To measure recollection we propose a closed-loop revisit benchmark with analytically exact pose ground truth from the model's own action interface, and an Excess Revisit Consistency metric that controls for the slow-motion shortcut. On the open-source LingBot-World model (one 4-GPU node), the baseline's excess is negligible (mean ERC-PSNR  dB; negative on the pan loop) while lifts it to  dB on all four loops (), improving PSNR, LPIPS, and SSIM throughout, with a random-anchor control showing the effect is carried by pose-gated detection. Widening the KV window up to does not restore what the excursion erased: recollection, not longer retention, is the missing capability.

Primary area:
foundation or frontier models, including LLMs
Keywords:
episodic memory, world models, training-free methods, autoregressive video generation, diffusion models, closed-loop evaluation, benchmark, revisit consistency, inference-time adaptatio

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aPNAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper argues that autoregressive world models implement retention (KV cache) and protention (action conditioning) but lack recollection: deliberate reactivation of earlier experience on revisit. It proposes Anamnesis, a training-free wrapper for frozen AR world models: store denoised latents with camera poses, detect revisits by pose gating, and reinject memory by re-noising the stored latent for one refinement step (SDEdit-style). A KV-cache override alternative leaves outputs unchanged, suggesting recollection must enter through the generative channel. The authors also introduce a closed-loop revisit benchmark with analytic pose ground truth from the model’s action DSL, plus Excess Revisit Consistency (ERC) against a novel-view control. On LingBot-World-Fast, Anamnesis raises mean ERC-PSNR from about +0.35 dB to +2.24 dB across four loops, with random-anchor and window-widening controls supporting pose-gated generative recollection rather than longer retention.

Critical strengths
  1. The problem diagnosis is crisp and useful: revisit failure is framed as missing recollection, not merely a short KV window.
  2. The method is simple, training-free, and scientifically clean as a probe on a frozen checkpoint.
  3. Evaluation design is thoughtful: exact poses, ERC vs. novel-view control, random-anchor control, KV-null result, and “not just copying / teleport-restart” checks.
  4. Empirical gains are consistent across loops, with useful ablations.
Critical weaknesses
  1. Narrow experimental scope. Results are mainly on one open model (LingBot-World-Fast) and a small set of release scenes/seeds. Cross-model transfer and broader scene diversity are needed for stronger claims.
  2. Exact pose gating limits realism. Detection assumes analytically exact camera poses from the action DSL. Free exploration would need SLAM-quality tracking; gate jitter helps, but is not a full substitute.
  3. Phenomenology framing is interesting but unevenly necessary. The retention/protention/recollection story is memorable, yet some sections read more philosophical than technical. The core ML contribution stands without heavy Husserlian packaging.
  4. Recollection is still short-horizon / single-anchor limited. Long excursions and multi-chunk revisits remain weak (semantic ERC mixed; single-anchor coverage issues). The paper acknowledges this, but “world memory” claims should stay bounded.
Questions for authors

Please see weaknesses.

Reviewer_jAYrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training-free memory method for world models. It stores historical latent variables and camera poses, retrieves the corresponding latent variables upon revisiting a location, and applies a single denoising step after injecting a small amount of noise. The authors demonstrate improved consistency during revisits on the LingBot-World benchmark.

Critical strengths
  • The method is straightforward and can be directly applied to frozen models, making it quite practical.
  • In the primary experiments, it consistently achieve improvement in both PSNR and LPIPS.
  • The paper tests random anchors, KV override, and different noise levels. It also checks whether the gains come from generating nearly static videos. These tests help explain why the method works.
Critical weaknesses
  • The method primarily combines pose retrieval with existing SDEdit-style noise-addition and denoising processes; its novelty is relatively limited.
  • The method improves PSNR and LPIPS; however, the DINO metric declines on two longer trajectories, and noticeable jumps occur when memory is injected.
  • The experiments are relatively limited, covering only one model and four scenarios, thus limiting the scope of validation.
Questions for authors
  • Ablation studies on noise intensity indicate that lower noise levels yield better results. What happens if no noise is added at all? Is a single denoising step necessary?
  • Why does the KV-cache method choose to overwrite the most recent context rather than appending historical content? Since the overwriting process itself might disrupt the current context, could this lead to an underestimation of the effectiveness of injecting memory via context?
Reviewer_8AVmOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a training-free wrapper for the frozen LingBot-World-Fast model that stores each generated latent chunk with its exact camera pose and detects revisits by pose proximity. On a revisit, the stored latent is lightly re-noised and denoised for one step; an alternative that writes it into the KV cache is also tested. Evaluation uses four scripted out-and-back camera loops on 4 scenes × 2 seeds, scoring revisit similarity to the departure frame relative to a novel-view control. The approach is framed in phenomenological terms (retention / protention / recollection).

Critical strengths
  • The paper targets a real and widely reported failure of AR world models (forgetting on revisit) with a simple, training-free mechanism.
  • Exact loop-closure poses from the action interface, plus a novel-view control, make the intended evaluation protocol clean in principle.
Critical weaknesses
  • The experiments do not appear to have run correctly, so the results cannot be verified. The generated frames in Fig. 1, including those of the unmodified baseline, are dominated by blob and mosaic artifacts and do not look like normal video generation output. This is consistent with the degraded setup (a 3-latent-frame attention window, the fp32 checkpoint cast to bf16, 4×RTX-4090). All reported revisit PSNRs are around 11–14 dB, a range in which the frames are barely related to the reference. Differences of 1–2 dB at that level do not indicate meaningful scene consistency, so neither the claimed "recollection gap" nor the improvement from Anamnesis can be assessed.
  • The method and evaluation are too narrow to support the claims. The detector fires only within 0.15 m / 10° of a stored pose, and the stored latent is then lightly refined. The metric measures similarity to that same departure frame, so the method essentially reuses the stored content at the identical pose.
  • The philosophical framing replaces technical substance. The Husserlian retention / protention / recollection trichotomy and the Ship of Theseus discussion mostly rename existing components (KV cache, action conditioning, pose-gated retrieval) without adding testable content. The underlying technique (pose-based nearest-neighbour retrieval plus SDEdit-style re-noising) is standard, and the framing overstates the contribution.
Questions for authors

See weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.