acceptodds
Withdrawn from ICLR 2027

SPW-Nav: A Streaming Panoramic World Model for Language-Guided Navigation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

11 views

Abstract

Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views. We propose SPW-Nav, a streaming panoramic world model that understands movement instructions and streams one minute of 2K 360 video in real time from a single panorama. SPW-Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose-aligned conditioning keeps translation inputs bounded over long streams, and a multi-term memory with a distilled few-step generator continues the scene as instructions change. We also build SPW-NavSet, panoramic videos with camera trajectories and verified instruction annotations. Driven by language, SPW-Nav outperforms existing panoramic generators in camera following and video quality, and supports on-the-fly instruction switching.

Primary area:
generative models
Keywords:
world model, video generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pJiWOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SPW-Nav, a semantic-aware streaming panoramic video generation framework. The authors also construct SPW-NavSet, a 233-hour dataset paired with verified explicit and goal-based movement instructions, and SPW-NavBench, a 100-clip evaluation benchmark with ground-truth trajectories and dual instruction types.

Critical strengths

The writing is easy to follow.

Critical weaknesses

The performance is really poor. I carefully reviewed the supplementary video and found that it does not appear to accurately follow the navigation prompts. For example, at timestamp 00:23, among the three scenes sharing the instruction history that includes "move forward three meters," consider the leftmost one: after turning left by 90 degrees, the camera's movement becomes nearly static. Even if there is slight movement, the displacement is far smaller than that produced by the earlier "move forward three meters" instruction. As a result, the subsequent instruction "walk forward five meters" appears to be ineffective. Similarly, the middle video also fails to comply with the instruction "Keep walking until you reach the statue." The same issue persists in nearly all subsequent scenes. Based on this, I believe the experiments do not support the paper's claim in line 048 that "SPW-Nav understands the semantics of movement instructions."

Questions for authors

I had originally intended to ask about the generalization ability of text instructions; however, I would instead suggest that the authors first ensure the model can accurately follow navigation instructions.

Reviewer_zNqMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SPW-Nav proposed a streaming panoramic world model for language-guided navigation. A VLM (Qwen-VL-8B, LoRA) parses instructions into motion targets, while a video backbone (Wan2.2-TI2V-5B) generates translation only results. Rotation is applied as exact spherical resampling (SRD), poses are anchored to the prior chunk (PAC) to keep translation bounded, and multi-term memory with few-step distillation sustains 26.6 FPS streaming with multiple GPUs. The paper also contributes the SPW-NavSet dataset and the SPW-NavBench benchmark.

Critical strengths
  1. The system is complete and reasonable for an interesting problem, which include understanding, trajectory generation, memory, generation, which are carefully designed for the final goal.
  2. The idea of translation-only pano generation is reasonable and well-motived, and achieve effective results.
  3. Experiments show the effectiveness of the proposed method.
Critical weaknesses
  1. Several claims need qualification or stronger evidence:

a) Camera-following accuracy: The abstract claims superiority over prior methods, but Sec. 5.2 reports worse ATE than PanoWorld-A [1]. Specify the metrics on which SPW-Nav improves.

b) Novelty: The “first” claim is overstated given PanoWorld-Action’s text-to-motion presets [1]. Clarify the distinction and accurately describe prior work’s rotation capabilities.

c) Real-time performance: The reported 26.6 FPS requires 8×H200 GPUs and sparse attention. State these conditions and clarify whether end-to-end latency includes VLM inference and instruction switching.

d) Convergence: Final-quality ablations do not demonstrate faster convergence. Provide training curves or remove the claim.

  1. Limited external validation: All experiments use the self-built SPW-NavBench, with training and test data are from MUGEN. Evaluation on public open benchmarks, e.g., the benchmark released with PanFlow [2], is needed to assess camera control and loop consistency.

  2. Reproducibility: No release commitment is provided for the data, benchmark, code, or weights. Since SPW-NavBench claim the uniqueness for the proposed language-navigation task, its release is essential for independent verification and community adoption.

  3. Insufficient long-horizon evaluation: Quantitative consistency results cover only ~25 seconds (12 chunks), despite the one-minute streaming claim, and memory gain trails Matrix-Game 3.0 [3] (0.173 vs. 0.249). Longer-route experiments should quantify how consistency degrades with route length.

References:

[1] Haoyuan Li et al. PanoWorld: Real-World Panoramic Generation. arXiv:2607.09661, 2026. https://arxiv.org/abs/2607.09661

[2] Cheng Zhang et al. PanFlow: Decoupled Motion Control for Panoramic Video Generation. Proceedings of the AAAI Conference on Artificial Intelligence, 40(15):12385–12393, 2026. https://doi.org/10.1609/aaai.v40i15.38231

[3] Zile Wang et al. Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory. arXiv:2604.08995, 2026. https://arxiv.org/abs/2604.08995

Questions for authors
  1. Can authors report closed-loop consistency and memory-gain over the full one-minute stream, including how they vary with route length and revisit interval? The current quantitative evaluation test covers only several seconds.

  2. What are the main failure modes behind the reported 62% goal-localization success rate? How does the system handle localization failures, and how do these failures affect end-to-end navigation success?

  3. Does the reported 26.6 FPS on 8×H200 GPUs include VLM inference and instruction switching? What are the end-to-end response latency and sustained frame rate with these components included?

  4. Have you evaluated camera control and loop consistency on an independent public benchmark, such as PanFlow? If so, how do the results?

  5. Will you release SPW-NavBench, including its evaluation and annotations, together with the code and model weights needed to reproduce the results?

Reviewer_VrQLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces a text-based control interface for streaming 360-degree video. This lets a user say thing like "walk towards the door", etc. Technically, this is based on a VLM understanding of the current frame and the text prompt, and is combined with system tools to make the movement smooth and continuous. This is definitely a useful paradigm for user interaction, and can be especially beneficial in VR applications, where head rotation becomes "free". That said, I do not feel the paper makes substantial contributions to video generation or understanding.

Critical strengths

The paradigm introduced is clearly practically useful. I can imagine this to be true especially in VR, but also as a general approach to streaming "world model" type video generation. The 360 video has clear advantages for text control, because all directions are visible, and are available to the VLM interpreting the text; a perspective view would make it hard to refer to things out of view by text. Overall, the system presented seems nicely designed.

Critical weaknesses

I believe the main issue is that this is fundamentally a systems paper; that by itself would not be a problem, but then it should be written as a good systems paper, clearly identifying the system design choices and tradeoffs (see "What Makes a (Graphics) Systems Paper Beautiful" by Fatahalian).

Instead, the paper is written in somewhat hard to read "AI-ese" that appears to try to play up the design choices as fundamental contributions to streaming video generation and/or text instruction understanding, which I believe is not quite convincing.

Finally, I personally already believe that 360-video generation is very useful, and can imagine its applications, but I think the paper should make those more explicit, from VR to guided 3D scene generation.

Questions for authors

Could you make the benefits of 360 video for this type of text control explicit, compared to standard perspective video?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.