acceptodds
Withdrawn from ICLR 2027

Closing the Loop: Training-Free Revisit Consistency for Autoregressive Generative Rendering

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent conditional video generation models have shown promising potential to transform 3D engine renderings, such as depth maps and untextured geometry, into photorealistic videos for diverse gaming and immersive content creation. These applications require long-horizon autoregressive generation that continuously synthesizes new frames while preserving a persistent 3D world. Autoregressive generators synthesize video chunk by chunk with a bounded KV cache, so when the camera revisits a location after its context has been evicted from the KV cache, the model often regenerates inconsistent appearance, even though the conditioning renderings (e.g., depth) remain perfectly aligned since they originate from the same underlying 3D geometry. We address this revisit inconsistency without any post-training by exploiting correspondences the 3D engine already provides: temporal correspondence retrieves pose-matched historical latent chunks into the KV cache as loop-closure memory, while spatial correspondence from camera pose and depth reprojection biases token-level attention toward geometrically corresponding regions of the retrieved chunks. We demonstrate our method on loop-closure trajectories mined from the TartanAir and TartanGround datasets to mirror complex real-world application scenarios, where it outperforms existing training-free baselines on revisit consistency without losing video quality.

Primary area:
generative models
Keywords:
Video Generation, Generative Rendering

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_aJ3DOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address revisit inconsistency, this paper proposes a training-free framework. Its temporal correspondence retrieves pose-matched historical latent chunks into the KV cache as loop-closure memory, while spatial correspondence from camera pose and depth reprojection biases token-level attention toward geometrically corresponding regions of the retrieved chunks.

Critical strengths

The proposed method is simple and training-free, exploiting camera poses and metric depth already provided by a 3D engine. It retrieves pose-matched historical latent chunks and guides attention toward geometrically corresponding tokens without post-training.

Critical weaknesses
  1. This paper stated that clean latents and poses of evicted chunks are retained outside the attention cache so that any historical chunk can be retrieved (Line 269), while the memory and retrieval costs are not reported. Moreover, despite being training-free approach, the proposed is evaluated on Wan-VACE. It is unclear how its generalizability to other video diffusion models.

  2. The ablation study is not sufficiently comprehensive, as table 3 adds the proposed components cumulatively without comparing them against simple alternative components. For example, random retrieval and other simple history retrieval methods. These comparisons are necessary to justify the proposed components.

Questions for authors

In Table 3, adding GB yields marginal improvements in keypoint, while DINO similarity slightly decreases and L1 error remains unchanged on both benchmarks. Since the GB is one of the core components, its effectiveness is not sufficiently supported by the current ablation results.

Reviewer_BT6POpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses revisit inconsistency in long-horizon autoregressive generative rendering. It retrieves pose-matched historical latent chunks into a bounded KV cache and uses depth- and camera-based reprojection to guide attention toward geometrically corresponding regions. The method requires no additional training once a causal Wan-VACE student is available and improves revisit consistency on the proposed benchmarks.

Critical strengths
  1. It addresses an important practical problem in long-horizon generative rendering.
  2. The method is intuitive and effectively exploits camera poses and metric depth already available from 3D engines.
  3. The method improves quantitative consistency metrics without noticeably degrading overall video quality.
Critical weaknesses
  1. The attention cache is bounded, but the total memory footprint is not. The method retains the clean latent, metric depth, and camera pose of every evicted chunk in an external historical store. Consequently, storage and potentially retrieval time grow with video duration. The paper does not report the memory footprint, indexing strategy, retrieval latency, host-to-device transfer cost, or total generation throughput as the history grows. This weakens the claim of scalable long-horizon streaming.

  2. The paper’s cache retrieval, RoPE reassignment, and attention-bias mechanisms indeed require no additional training. However, the overall system relies on a Causal Wan-VACE student obtained through causalization and Self-Forcing distillation. Therefore, the proposed method is not a training-free modification that can be directly applied to an arbitrary pretrained Wan-VACE model. More precisely: The proposed revisit-consistency module requires no additional training once a specially post-trained causal autoregressive student has been obtained. This distinction is important because training the causal student itself may be computationally expensive and is a necessary prerequisite for the overall system to function.

  3. The novelty relative to the closest retrieval-based memory approaches is not fully established. The main experiments compare against general training-free cache-management methods such as Infinity-RoPE, Deep Forcing, and MemRoPE. However, the dominant component in the ablation is pose-based retrieval, which is conceptually closer to Context-as-Memory, LongLive-RAG, Matrix-Game 3.0, and related view-retrieval methods. The paper discusses these works but does not experimentally compare against them or clearly isolate the substantive methodological difference.

  4. The evaluation is limited in scope and lacks uncertainty estimates. The experiments are restricted to synthetic environments. TartanGround-Revisit contains only 71 loops, while the larger TartanAir-Revisit benchmark mainly consists of scripted rotations and zooms rather than translation-heavy trajectories with substantial parallax. No confidence intervals, paired significance tests, or per-sequence distributions are reported. The method is also not evaluated under pose/depth noise, dynamic objects, repeated revisits, changing illumination, or very long generation horizons.

  5. The incremental contribution of the geometry-guided attention bias is weakly supported. In Table 3, adding the geometry bias on top of pose-retrieved memory increases keypoint matches only from 47.84 to 49.08 on TartanGround-Revisit and from 283.30 to 284.92 on TartanAir-Revisit. DINO similarity slightly decreases on both datasets, while L1 remains nearly unchanged. Therefore, most of the improvement appears to come from retrieving the historical latent chunk rather than from the proposed spatial attention prior.

  6. In the castle example, the lighting conditions in the final frames differ from those in the initial visit. Therefore, although the method appears to improve geometric consistency to some extent, the example does not demonstrate corresponding improvements in finer appearance details such as lighting, materials, and textures. I also wanted to verify whether the first wall retained the same three-window configuration shown during the initial visit. However, the video ends before this part of the scene is fully revisited, making it impossible to assess such fine-grained consistency. More generally, the demonstrated camera trajectories are relatively simple, consisting mainly of left and right turns and lacking more complex motions or substantial viewpoint changes. Consequently, the qualitative results alone are not sufficiently convincing evidence of robust revisit consistency.

Questions for authors

Please refer to the weaknesses.

Reviewer_uXw8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary
  1. This works aims to solve a meaningful problem to enable realtime long video generation with memory consistency.

  2. The paper writing is easy to understand.

Critical strengths
  1. The soundness and effectiveness of the proposed work is not clear to me. It is a training-free work via a cache split for achor/retrieved/recent slots. This is not a new thing, many existing works like helios have adopt it as a regular strategy. The reviewer can not identify the effectiveness and novelty from the manuscript.

  2. I am doubt this training-free frame work's generalization ability. Please show more diverse benchmark evaluation results

  3. Retrieval is pose proximity, not visibility overlap; one retrieved slot; the anchor is always the first chunk. Trajectories that revisit several places aren't handled or discussed.

4.Gains over Deep Forcing are small on most metrics

Critical weaknesses
  1. The soundness and effectiveness of the proposed work is not clear to me. It is a training-free work via a cache split for achor/retrieved/recent slots. This is not a new thing, many existing works like helios have adopt it as a regular strategy. The reviewer can not identify the effectiveness and novelty from the manuscript.

  2. I am doubt this training-free frame work's generalization ability. Please show more diverse benchmark evaluation results

Questions for authors

Please address the concerns raised above. Provide clarification about novelty and generalization ability.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.