acceptodds
Withdrawn from ICLR 2027

CamCanvas: Absolute Camera Placement in Video World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Interactive video world models generate long sequences under camera commands and sustain extended exploration of a generated world. Their camera interface comes from relative pose encodings: a command specifies motion with respect to the current context window. As a result, when the camera is commanded back to a place visited long ago, these models fail to return to the commanded viewpoint. We present CamCanvas, which stores the model’s own generated content in a panoramic canvas, retaining the first observation of each location. When the camera returns, the model is conditioned on the rendering of this canvas at the commanded pose. The model thus receives the target view directly, so it has neither to infer its absolute pose from relative motion nor to recover that view from a limited context window. The return test assesses absolute camera placement by commanding the camera away and back and estimating, from the frames, the viewing-angle difference between the return and the model’s own first visit, with no external alignment. Return success rises from 0% to 91% on 90◦ returns and from 18% to 95% on real video. CamCanvas also generalizes to a video world model with a different architecture.

Primary area:
generative models
Keywords:
video world models, camera control, interactive world simulation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_UZzfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents CamCanvas, a framework for improving interactive video world models. The method uses depth estimation and camera poses to project generated content into a panoramic canvas, preserving the first observation of each location. When the camera returns to a previously visited pose, the canvas is rendered at the commanded pose and used to guide video generation. A return test is also introduced to measure the viewing-angle difference between the return frame and the model's own first-visit frame.

Critical strengths

The experimental results show improvements in return success rates on the evaluated benchmarks, with ablation studies providing some support for the proposed design.

Critical weaknesses
  1. The paper introduces a number of Theorems, Lemmas, and Propositions, many of which merely formalize intuitive observations or established mathematical properties. These analyses offer limited non-trivial theoretical insights and do not adequately explain the core mechanisms of the proposed method. Instead, the excessive formalization makes an otherwise relatively straightforward approach unnecessarily difficult to understand and even obscures important implementation details.

  2. The core design of the proposed panoramic canvas is insufficiently justified. It remains unclear how a single panoramic canvas accommodates observations from different camera positions and how the First-Write-Only strategy prevents discontinuities between existing and newly generated content. Furthermore, since the panoramic canvas itself relies on depth estimation and reprojection, it is unclear what advantages it offers over existing 3D cache-based methods, such as GEN3C. These concerns make it difficult to assess the necessity and technical soundness of the proposed framework.

Questions for authors
  1. How does a single panoramic canvas accommodate observations from different camera positions, particularly when the camera undergoes translation? How are the corresponding depth and visibility handled?

  2. Under the First-Write-Only strategy, how does the method ensure continuity between previously stored content and newly generated content, especially at their boundaries?

  3. Given the use of a panoramic canvas, a natural question is why not directly generate or complete a panorama and render the desired views, particularly for rotation-based scenarios.

Reviewer_bs36OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CamCanvas aims to make video world models return accurately to previously visited viewpoints. It stores the first observation of each scene region in an external canvas and renders this canvas at the requested return pose. The rendered view is supplied as a clean, spatially aligned condition through a small attention adapter. Experiments on panoramas and RealEstate10K show large improvements in return accuracy, with additional results on SANA-WM.

Critical strengths
  1. A useful test of camera return. The paper clearly explains why image similarity alone can give a misleading result. For example, a model may preserve similar content simply because its camera barely moves. Comparing the return orientation with the model’s own first visit helps reveal this problem without relying on alignment to an external trajectory.

  2. Large and consistent improvements in the tested setting. Tables 1–3 show clear gains over the base model. For example, success on 90-degree returns rises from 0% to 91%, while RealEstate first-frame return success rises from 17% to 99%. The component ablations and results on a second backbone provide useful evidence that the conditioning method works in practice.

Critical weaknesses
  1. The contribution over existing geometric conditioning is not clearly isolated. Rendering a stored scene at the requested camera pose is already used in GEN3C, and Section 2 discusses related streaming methods such as GeoStream and WorldWarp. CamCanvas mainly differs through first-observation storage and clean, aligned conditioning. However, Table 3 does not isolate first-write storage versus updating the cache, or clean versus partially noised conditions under the same alignment. The context-frame comparison changes the injection method as well. The results therefore establish the value of rendered content more clearly than the advantage of these particular design choices.

  2. The evidence for camera placement under translation is limited. Table 4 reports full canvas coverage for every panorama return, so these tests mainly examine reproduction of already stored views. This is useful, but does not test completion of uncovered regions. More importantly, the translation-cycle results in Table 10 still report orientation error rather than camera-position error. A camera can face the correct direction while being at the wrong location. Section 3.3 also obtains depth from the first frame, while Equation (3) updates only color and coverage. How newly observed regions receive valid depth remains unclear.

Questions for authors
  1. Design choices. Do you have a comparison that changes only first-write versus updated storage, or clean versus partially noised aligned conditioning? Which result best separates CamCanvas from prior geometric-cache methods?

  2. Translation accuracy. How is depth assigned to regions first observed after the initial frame? Can the existing translation experiments provide camera-position errors as well as orientation errors?

Reviewer_RaypOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CamCanvas, a method that helps interactive video world models return to an earlier view when the camera is commanded back, a failure the authors trace to relative camera encoding combined with a limited context window. CamCanvas keeps the model's own generated content in a panoramic canvas and, on a return, renders the canvas at the commanded pose and feeds it to the model as a clean, pose-aligned condition.

Critical strengths
  • The paper tackles a well-motivated and challenging problem that would be quite useful if solved, especially for camera-sensitive applications such as world-model-based policy learning.
  • Diagnosing the issue as relative camera poses that only rely on the limited history context fed to the DiT is reasonable.
  • Designing the 3D-aware memory of generated frames as a panoramic canvas is a novel choice from existing memory approaches.
Critical weaknesses
  • The main scope is limited to the go-and-return setting. Returning to a previous view is already achievable with warping/reprojection-based camera control (e.g., GEN3C), and the primary difference here is the use of a panoramic canvas, which limits the practical contribution.
  • The paper emphasizes the shift from offline methods that require the full trajectory in advance to a streaming setup as a key distinction. However, this stems from applying the idea to a causal model rather than from the proposed method itself. The contribution on this point appears marginal.
  • The panoramic canvas mainly handles camera rotation. As an RGB-D panorama, it can accommodate some translation, but its limitations are clear in translation-dominant scenarios such as orbiting around an object.
  • The method supports only static scenes, which falls short of the robot policy training motivation stated in the introduction.
  • Overall, apart from the panoramic memory design, the novelty is limited, and the scope is narrow (static scenes, small camera translation, and mostly revisits).
Questions for authors
  • How would the proposed approach handle translation-dominant trajectories, such as orbiting around an object?
  • How could the proposed approach be extended to dynamic scenes, which are important for diverse applications?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.