acceptodds
Under review as a conference paper at ICLR 2027

NARRA: Self-Adapting Video World Models at Test Time

17 views

Abstract

Long-horizon video world models often suffer from visual drift: as generation progresses, the appearance and geometry of a scene can gradually change, even when the camera returns to a previously observed viewpoint. We introduce NARRA, a self-adaptive framework that enables video world models to learn and remember at test time. NARRA uses an initial generation as self-supervised context to specialize the model to each generated world. Inspired by Nested Learning, adaptation occurs across interacting learning processes with different update frequencies, allowing new information to be incorporated without overwriting pretrained knowledge. The framework can further enhance video quality by leveraging channel-wise memory of earlier temporal segments and known camera trajectories that identify revisited viewpoints. We show that NARRA significantly improves video quality, with consistent gains across VBench and other evaluation metrics on the SANA-WM benchmark, and particularly strong improvements on challenging trajectories that require the model to return consistently to previously observed viewpoints.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video world models, long-horizon video diffusion, test-time adaptation, test-time learning, memory, Nested Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tmW3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes NARRA, a test-time adaptation method for long camera-controlled video generation on top of SANA-WM. Given an image, a caption and the full camera trajectory, the frozen model first generates a video, which is then used as self-supervision to fine-tune the generator before a second, final generation. The adaptation is nested: a fast weight update on the last block's feed-forward projection runs inside slower updates of the Gated DeltaNet projections and of the fast update's starting point, which is meta-learned offline. Two more components target long-range scene memory: a camera-return loss that uses the known trajectory to pull the prediction at a revisited view towards the matching earlier frame of the first pass, and a channel-weighted retrieval of memory states from earlier segments. Experiments use a new 45-scene departure-and-return benchmark and a 50-video subset of the SANA-WM benchmark, with revisit-consistency metrics (including new time-stratified ones), VBench and camera errors, plus a 16-second comparison with other world models. At 60 seconds NARRA improves revisit consistency and VBench over the frozen backbone.

Critical strengths
  1. The gain over the frozen backbone is seen in most individual scenes and shows up again on a second benchmark. On the 45-scene benchmark at 60 seconds (Table 1), NARRA raises PSNR from 15.50 to 16.33 and N-PSNR from 16.21 to 17.16, with higher PSNR on 34 and higher N-PSNR on 39 of the 45 scenes. On the SANA-WM subset (Table 3), all five adapted variants beat the frozen backbone on PSNR, MSE, N-PSNR, N-SSIM, VBench and N-LPIPS, on both simple and hard trajectories. But the only 60-second comparison is with the frozen backbone, since the other models the paper tried often drifted or left the return path at this length (Sec. 3.1, App. A.4).

  2. Sec. 3 points out a real problem with the existing revisit protocol: SANA-WM averages over the five closest revisit pairs, but in 90.4% of these pairs both frames lie within 1 s of another selected pair, so the five pairs mostly measure the same few returns. The departure-and-return benchmark and the time-stratified metrics are a sensible fix, and neither is tied to NARRA.

Critical weaknesses
  1. I consider the source of the main gain unclear. The adaptation target is mainly the first-pass video, which already contains the drift the method should remove. The flow-matching loss covers all of it, drifted return frames included, without filtering. Why would fitting it make the regenerated video more consistent rather than reproduce its errors? The conclusion leaves "selecting reliable self-supervision to avoid reinforcing generation errors" to future work. Yet plain fine-tuning with this loss already gives 0.60 of NARRA's 0.87 dB N-PSNR gain (Table 3). The camera-return loss, the only component that explicitly uses revisit pairs, gives only 0.04 dB more when added alone to nested adaptation. Larger gains at 60 s than at 16 s hint at memory, but no controlled test separates remembering this scene from a general gain. So I consider the claim that NARRA lets video world models "learn and remember at test time" not yet supported.

  2. NARRA's additions to plain fine-tuning give under a third of the N-PSNR gain (0.27 of 0.87 dB), each with a small or mixed effect. NARRA needs two full generations per video plus K slow iterations, each backpropagating through R fast updates. Without step counts or per-video time, I cannot check whether the variants had equal budgets or compare NARRA with the frozen model at equal compute. On hard trajectories, camera rotation error rises from 4.73° for the frozen model to 7.32°–8.28° for every adapted variant (Table 3), which the text never addresses. The step behind most of the gain, test-time fine-tuning on the generator's own output, was already done during generation by SlowFast-VGen (ICLR 2025) and in a recall test by CoME (ICLR 2026). Neither is cited or discussed.

Questions for authors

Q1. What happens if the model is adapted on an unrelated video, on a temporally shuffled first pass, or on a first pass cut before the return segment? Does the final generation reuse the first pass's noise?

Q2. NARRA is tested only on SANA-WM, and App. A.3 links the effect to how its Gated DeltaNet blocks construct and propagate their recurrent state. Does the gain, at least the part from plain fine-tuning, carry over to a backbone whose long-range memory does not rely on such a state?

1 comment

Thank you for the rebuttal, the three controls, and the DreamX-World results. The new controls do not fully resolve my concerns about the main claim. Adapting on an unrelated scene already gives 0.74 of the 1.53 dB N-PSNR gain over the frozen model, the first 30 s of the target scene is on par with the unrelated video (15.93 vs. 15.95), and the shuffled first pass gets the highest score with noticeably less motion. At matched time, NARRA is also only about 0.3 dB ahead of plain fine-tuning.

For DreamX-World, the 16 s result reflects the full pipeline with earlier pose-matched views blended into the attention cache, so the 11 dB gain does not show how much comes from adaptation alone, which was my question.

I appreciate the additional experiments, but I still consider the scene-memory claim only partially supported, so I am keeping my score.

Reviewer_Ra6HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

NARRA specializes SANA-WM to a generated scene using an initial complete rollout as self-supervision. It combines fast adaptation of an additive feed-forward parameter memory, slower recurrent-projection and memory-initialization updates differentiated through the fast updates, an offline meta-learned initialization, camera-return consistency, and channel-wise retrieval from recurrent segment summaries. The adapted generator then samples the final video with a frozen refiner. This is adaptation before final sampling, interleaving adaptation with generation is future work. Evaluation includes a new 45-scene departure-and-return benchmark, a 50-video SANA-WM benchmark, 16-second comparisons with external models, and component ablations.

Critical strengths
  1. Consistent improvements on the intended long-horizon task. On the 60-second NARRA benchmark, Table 1 reports N-PSNR improving from 16.209 to 17.157 dB, N-MSE decreasing by 21.2%, and VBench increasing from 80.521 to 81.920. N-PSNR improves on 39/45 scenes, giving stronger support than an aggregate alone. Table 3 also shows a 0.870 dB N-PSNR gain on SANA-WM, while Figure 5 illustrates preservation of recognizable scene details. These results make the underlying self-specialization idea worth pursuing.

  2. A useful evaluation of persistence across extended absences. The new benchmark deliberately requires departure and return, with revisit gaps of 26–60 seconds (Section 3). Temporal bins and pair-spacing constraints reduce concentration on near-duplicate return events. The inclusion of ordinary fine-tuning, nested adaptation, camera regularization, and memory variants (Table 3; Appendix A.3) helps establish which mechanisms contribute. Appendix A.4 also candidly explains the difficulties in comparing external pipelines at 60 seconds.

Critical weaknesses
  1. The central consistency claim is not adequately separated from camera-control degradation. In Table 3's hard split, rotation error rises from 4.733° to 8.284° (75.0%), translation error from 1.409 to 1.618, and Cam-MC from 1.437 to 1.688, despite improved revisit scores. The new 60-second benchmark does not report camera errors in Table 1. Section 3 and the revisit evaluator select pairs using prescribed benchmark poses, without checking that generated frames achieve those poses. Thus increased agreement can coexist with changes in the actual excursion or return, and the current evidence does not isolate persistent scene memory under faithful control. This is a possible confound, not an established explanation of the gains; simple-trajectory camera errors improve, and estimated poses also have limitations. Nevertheless, it directly affects the paper's claimed world-model capability.

  2. The benefit of the nested procedure is insufficiently established relative to its cost and training requirements. Table 3's nested variant improves over basic TTT by only 0.101 dB N-PSNR and 0.057 VBench. Appendix A.3 provides useful ablations, but no runtime or comparison at a matched computation budget. The available configuration and nested implementation specify 40 ordinary TTT updates versus 40 nested slow iterations with two fast updates each, plus final adaptation; update counts do not establish equal cost. The PDF also leaves the offline meta-training data and split insufficiently specified. The supplementary README states that the required learned initializer and its companion checksum file are unavailable in the release. These gaps limit assessment of the nested contribution and independent verification, even though many inference settings are available in code.

Questions for authors
  1. Decision-relevant evidence request: Can you provide camera-adherence results for the 60-second NARRA benchmark and show whether revisit gains remain on trajectories with comparable achieved motion? How do you interpret the large hard-split rotation-error increase in Table 3?
  2. Decision-relevant evidence request: What are the end-to-end time and peak memory costs, including the initial rollout and adaptation, and does nested adaptation retain an advantage over ordinary TTT at a comparable wall-clock budget?
  3. Decision-relevant clarification request: Which source videos train the meta-initialization, how are they separated from the 12-video design-selection subset and both evaluation sets, and can the initializer or sufficient training details be supplied to verify this part of the method?
Reviewer_eDHROpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper attempts at solving a common problem in world models, the fact that content drifts especially when objects exit and re-enter the frame. This paper approaches this problem through a test-time solution. By generating a video, then essentially fine-tuning the video model with a combination of a diffusion loss, and camera return loss that encourages frames with similar camera positions have similar appearance. They evaluate their results against relevant baselines on their own defined benchmark, measuring frame reconstruction metrics on camera revisitation.

Critical strengths
  • The method is quite interesting. Generating a video and then fine-tuning it on the fly in order to make a better aligned sample is quite different from conventional methods of retrieval.

  • The paper is well written and clear to understand.

Critical weaknesses

There are some glaring flaws to this paper that I don't think can be ignored:

  • There is a very simple baseline here that isn't being compared to. Rather than having to fine-tune the model on the fly, you could simply use a stronger conditioning signal on relevant frames to guarantee consistency. A very simple solution to the revisit consistency problem given a known camera trajectory is to simply condition on the frames that you know are relevant to the generation. There is a vast amount of existing literature on frame retrieval for autoregressive video generation that is relevant here.

  • This method is incredibly inefficient as each generation requires an adaptation phase that includes actual weight updates to the model. A properly trained conditional model can bypass this training phase completely. However, there is no comparison to baselines that simply use conditioning as a signal.

  • The training objective itself is already flawed. The camera return loss essentially encourage revisited camera positions to generate similar frames. However, in world modelling use cases, we are often modelling dynamic scenes. By assuming revisiting the same camera trajectory will yield the exact same frame is quite a flawed assumption and greatly limits the model.

  • Evaluations are also purely done on reconstruction metrics. This assumes all scenes are static, which is not a very safe assumption in video generation in general.

Questions for authors
  • Is there a baseline that can be run where instead of retraining the model for every sample, you can simply use the first sample as a conditioning signal for a subsequent generation?

  • Are there any non-reconstruction metrics that can be used to evaluate your method? For example, WorldScore [1] is an example of a benchmark that could fit this task.

[1] Duan, H., Yu, H., Chen, S., Li, F., & Wu, J. (2025). WorldScore: A Unified Evaluation Benchmark for World Generation. 2025 IEEE/CVF International Conference on Computer Vision (ICCV), 01-12.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.