acceptodds
Withdrawn from ICLR 2027

PanoWorld: Real-World Panoramic Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

In this work, we aim to address the challenge of controllable panoramic world modeling by leveraging the rotation-equivariant properties of omnidirectional representations, where camera rotation can be naturally modeled as an implicit geometric transformation. Building on this insight, we propose PanoWorld, which simplifies camera trajectories into translations via fixed headings for both current-action modeling and long-range memory through Dense Panoramic Ray-Conditioning (DPRC) and Geometry-aware Memory Augmentation (GMA). Then, a three-stage training pipeline is introduced to progressively optimize each component. To better evaluate physical consistency under large-scale spatial variations and diverse real-world environments with varying lighting scenarios, we construct World360, a large-scale dataset consisting of both real-world video clips collected via panoramic unmanned aerial vehicles and high-quality simulated clips generated by AirSim360. Extensive experiments on World360 demonstrate the effectiveness of PanoWorld, achieving superior generation quality while maintaining strong controllability. Our models, training code, and dataset will be publicly available.

Primary area:
generative models
Keywords:
Panoramic Video Generation;World Models;Trajectory-Controlled Generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_bznEOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces PanoWorld, along with the World360 dataset. The authors fine-tune a video generation model to adapt to panoramic video generation, and propose DPRC and GMA to realize camera control and memory retrieval. The authors also conduct experiments, including both qualitative and quantitative evaluations, to demonstrate the effectiveness of the proposed method.

Critical strengths

The proposed World360 dataset is valuable for the community.

Critical weaknesses
  1. The main claim of "rotation-equivariance" is exaggerated. Rotating an ERP along the yaw axis is simple, i.e., horizontal translation (or circular padding), which means that the main visual content still lies within the ERP grid. However, when it comes to pitch/roll, the situation becomes more complex. In this case, trivial visual content (e.g., sky or ceiling) takes over the ERP grid, and severe distortion appears. Building on this fact, the technical contribution of the paper is significantly undermined, as the remaining technical designs seem to be a combination of existing works, such as PRoPE, CubeDiff, and WorldPlay.

  2. The authors should carefully review the manuscript before submitting it to the conference. There are many problems regarding writing. For example, in line 069: "To stably optimize each component, we employ a progressive, three-stage training strategy." — and the paragraph ends there, which seems to be an incomplete sentence. Likewise, in line 161: "...despite variations in velocity. Finally," seems to be cut off by the next paragraph. (Italics can be used to indicate sub-headings within a paragraph, such as Uniform Spatial Resampling and Exposure Quality Filtering.)

  3. I checked the supplementary videos, and I didn't find a remarkable improvement in PanoWorld compared to Matrix-3D, though the latter method is not fine-tuned on the World360 dataset.

Questions for authors

How well does PanoWorld perform in terms of camera control that includes pitch/roll?

Reviewer_B1qtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PanoWorld is a controllable panoramic (360°) video world model. It exploits the rotation-equivariance of the equirectangular projection to decouple rotation from translation, and proposes ray-wise SE(3) attention conditioning plus geometry-aware memory, trained in three progressive stages. It contributes a large curated dataset of real UAV and synthetic panoramic trajectories with precise poses, and demonstrates strong camera-control and revisit-consistency results.

Critical strengths
  1. World360, combining real UAV and synthetic panoramic data with precise multi-altitude poses and scene-disjoint splits, is a useful dataset resource, and the appendix matched-backbone comparison supports the method rather than the alternative.
  2. A reasonably clean system design integrating ray-wise conditioning and geometric memory for panoramic video.
Critical weaknesses
  1. The central motion-decoupling idea overlaps heavily with PanFlow (AAAI 2026), which also decouples camera rotation from translation in equirectangular video and additionally curates a 150k pose-annotated dynamic panoramic dataset; this concurrent work is neither cited nor discussed, which weakens both the novelty of the decoupling design and the framing of World360 as the first large motion-rich panoramic dataset.
  2. The matched-backbone result is buried in the appendix rather than surfaced in the main tables, and coverage of matched baselines is thin: only OmniRoam is retrained at a matched size and fine-tuned on the dataset; Matrix-3D and Imagine360 backbone sizes are unspecified, and Imagine360 reports no trajectory-control metrics.

Reference: C. Zhang, H. Liang, D. Y. Chen, Q. Wu, K. N. Plataniotis, C. Cruz Gambardella, and J. Cai. PanFlow: Decoupled Motion Control for Panoramic Video Generation. In AAAI, 2026.

Questions for authors
  1. Could the authors add PanFlow (AAAI 2026) to the related work and clarify the difference between their rotation/translation decoupling and PanFlow's derotation-based decoupling (ray-wise SE(3) trajectory conditioning vs optical-flow derotation)?
  2. It would help to move the matched-backbone comparison (Table 11) into the main tables and state the backbone sizes of all compared methods in the main text.
  3. Would the authors clarify the backbone sizes of Matrix-3D and Imagine360, and would fine-tuning these baselines on World360 change the main conclusions?
  4. Would the authors consider promoting the ViPE ATE/RPE results (currently in the appendix) alongside the frame-PSNR control metric?
Reviewer_rTTMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

PanoWorld generates 360° videos from an initial panorama and a camera trajectory. It separates yaw from translation, uses ray-based camera conditioning to guide motion, and retrieves historical features to improve scene consistency without explicit 3D reconstruction (DPRC and GMA). Built on Wan2.2-5B, it adds camera and memory modules trained in stages. The authors introduce the World360 dataset and report improved camera control and revisit consistency. A separate Causal Forcing variant accelerates generation.

Critical strengths
  • DPRC is well-suited to ERP panoramas, providing a strong geometric prior for camera-controlled panoramic video generation
  • GMA helps maintain long-horizon consistency through geometry-aware memory retrieval and confidence-guided feature fusion.
  • Causal Forcing provides a useful complementary extension, demonstrating the framework’s potential for faster, interactive generation.
Critical weaknesses
  • I would like to see a comparison with other methods such as EvoWorld (https://arxiv.org/abs/2510.01183), since it also explores panoramic generation with memory. If this comparison is not feasible, the authors should explain why.
  • The tests focus on horizon-leveled cameras and static scenes. It is unclear how well the method would handle tilted cameras or moving objects.
  • It is still unclear what GMA offers over existing memory methods. The ablations show that memory helps, but not why this particular retrieval and gating design is better.
  • The method adds trainable modules and a fairly involved training pipeline. Reporting the extra parameters and compute, and comparing against simpler alternatives with a similar parameter budget, would make the gains more convincing.
Questions for authors
  • What does GMA offer over existing geometry-aware memory methods? Could you compare against a simpler memory-attention module using the same historical frames, and separately test the contribution of confidence-guided gating?
  • Could you include more baselines (eg EvoWorld or other method that contain the memory modules)?
  • How well does the method handle cameras with pitch/roll changes or scenes containing moving objects? Are these unsupported settings?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.