acceptodds
Under review as a conference paper at ICLR 2027

WorldNoise: Making Video Diffusion Models See the 3D World through Geometric Noise

Abstract

Recent advances in camera-controlled video diffusion models have enabled interactive world modeling, where an initial noise is progressively transformed into 3D scenes that can be explored along arbitrary camera trajectories. Yet, surprisingly, the source of these 3D worlds—randomly sampled diffusion noise—is agnostic to scene geometry. This raises a fundamental question: can the diffusion noise itself serve as a structured representation of the 3D world? In this paper, we introduce WORLDNOISE, a novel framework that brings 3D structure directly into the noise space of video diffusion models. Our key insight is to reinterpret diffusion noise not merely as unstructured randomness, but as a latent representation of the world that can be structured and transformed according to 3D geometry. Specifically, WORLDNOISE first constructs a 3D noise space over the observable world region jointly covered by the camera frusta along the trajectory and distributes randomness throughout this space. Then, it projects the world-space randomness into view-specific 2D noise through a carefully designed whitening and covariancecompensation process that recovers the native spatial noise prior expected by the video diffusion model. Different views therefore become different observations of the same underlying noise world, rather than being generated from unrelated noise. The resulting WORLDNOISE can be used through the existing diffusion noise interface without modifying the denoiser architecture. We further extend WORLDNOISE to long-horizon exploration by allowing the noise world to persist and grow as the camera moves through the scene. Extensive experiments demonstrate the effectiveness of WORLDNOISE, revealing a previously overlooked role of diffusion noise as a latent carrier of 3D structure and opening a new direction for 3D-aware generative world models.

Primary area:
generative models
Keywords:
Camera Control Generation, World Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Km1DOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WorldNoise, a method that aims to add camera control to video diffusion models and improve scene consistency via structured 3D noise. The motivation is that most generative video models start from entirely random noise that is uncorrelated across frames even when frames view the same content. The paper's solution is to instead construct a 3D noise space that is shared across all views.

The method first looks at each camera frustum along a predefined trajectory to determine the 3D area that needs to be filled with noise. To keep the number of noise specks manageable, the method scatters a fixed number from each camera viewpoint, sampling uniformly in image coordinates and logarithmic depth. The specks are then sized according to the expected local spacing derived from the sampling density. That density drops with the cube of depth, such that specks are small and capture fine detail up close and are larger far away. Each speck is given a random Gaussian value that is fixed across views.

The 3D noise cloud is then converted into 2D frame noise via EWA splatting (similar to 3DGS but without alpha blending). This leads to blotchiness that is different from the iid Gaussian noise diffusion models typically see, as each speck's Gaussian value is spread across the pixels it occupies. A noise residual is therefore added that cancels this within-frame correlation, variance is then normalized, and the resulting noise is spatially iid within a frame. As the residuals are drawn independently for each view, cross-view correlation remains intact.

WorldNoise can add camera control capabilities to a pre-trained video model by LoRA-finetuning it to align with the structured noise distribution (the architecture and training objectives remain unchanged). The paper also proposes a long-horizon generation extension that partitions noise into "temporary" specks (the 3D noise previously described) and "history" specks that reside on surfaces that the camera has already observed (as determined by DA3). The initial history comes from the first frame's depth and later surfaces are added from generated keyframes if they pass a confidence and consistency check. To generate a new chunk, temporary specks fill the upcoming camera views and are filtered by geometric carving rules. The generation is also conditioned on the first frame, 4 retrieved past frames with good view overlap, and the most recent 16 frames. After a frame chunk is generated, the temporary noise is discarded.

The method is integrated into Wan 2.1-14B and LTX-2.5-22B and LoRA-finetuned on DL3DV-10K. The paper evaluates 81-frame sequences from 100 RealEstate10K and 100 DL3DV-10K scenes by measuring camera control (via GLOMAP reconstruction compared to the requested trajectory), geometric consistency (TSED), and visual quality. The main baselines are camera-control methods (SCoPE, Gen3C, UCM), other noise methods (CameraNoise and UniCaMo Wan 2.1-14B reimplementations trained on the same data as WorldNoise), and the base Wan and LTX models. Compared to the camera-control methods, the method generally has the lowest rotation error but higher translation error than SCoPE (both datasets) and UCM (RealEstate10K) and worse TSED score than UCM. Compared to the noise-based methods and the base models, it does much better. Long-horizon generation is evaluated on sequences from DL3DV-10K longer than 300 frames and compared to SANA-WM and Lyra 2.0. WorldNoise performs best on most camera and geometry metrics (the margins are small) with comparable visual quality. The point sampling budget is ablated (more points generally gives better metrics) as is the log-depth sampling (which matters more for translation error and less for rotation error).

Critical strengths

I find the core idea of the paper (how to structure noise in 3D in a manner that improves video diffusion) to be an interesting and neat topic. Although there is prior work that discusses how to use 3D consistent noise generally and for camera-control specifically, this is the first that I'm aware of that does so across multiple depths without needing the underlying 3D scene geometry (at least in short-horizon settings), which is a valuable contribution. More generally, how the noise is constructed seems principled and well-defined. The idea is also relatively simple to implement and doesn't require model architecture changes, which is practical.

The evaluation section does a convincing job of showing that the method works well in comparison to some other noise-based schemes and baselines, validating the need for principled 3D consistent noise, along with evidence of generalization (training on DL3DV-10K and evaluating on RealEstate10K).

Critical weaknesses

Some of the novelty in the paper is overstated such as the abstract's claim to reveal "a previously overlooked role of diffusion noise as a latent carrier of 3D structure." ConsistDreamer [1], CFD [2], and MultiDiff [3] (none of these are cited) structure noise via 3D scene geometry, with MultiDiff doing so to enhance camera control (although it also uses warped-image conditioning). The core novelty of this submission then becomes deriving 3D noise across multiple depths without known scene geometry (and the novelty of the long-horizon approach, which does require it, diminishes). This is still valuable, but the authors should take care to properly attribute prior work and contextualize their claims accordingly (How I Warped Your Noise [4] should also be cited as its Eq. 6 and Eq. 2 are very similar to how this paper derives its noise residual). In the same vein, the "conclusion and limitations" section does not contain any discussion of limitations. Several immediately come to mind (the camera trajectory has to be known beforehand and the 3D noise construction assumes the scene is static). The runtime overhead of the noise construction strategy should also be discussed.

The evaluation protocol has some gaps that are important for the authors to address:

  • Table 2 is missing a crucial baseline - the depth-warped noise construction strategy used by MultiDiff is essential to compare against.
  • Several important metrics are missing. Standard video (FVD/FID) and reconstruction metrics (PSNR/SSIM/LPIPS) are not provided. TSED also only compares consecutive frames and nothing measures whether a revisited area looks the same. VMem [5], WorldMem [6], and Context-as-Memory [7] discuss protocols for evaluating revisiting, and this paper should do something similar.
  • Key components of the method aren't ablated, such as the impact of the noise residual and normalization step and the LoRA-finetuning (it's a shame that the method isn't training-free, but it would still be an interesting ablation). As the long-horizon generation is more of a systems-level contribution as opposed to something theoretically novel, it is especially important to ablate its components (such as the carving rules and the memory frames).

The writing is generally ok, although terminology like "world-space stochastic support" could be made simpler. The figure captions should be expanded to describe what the figures are actually conveying. For example, Figure 2 doesn't explicitly say which noise strategy the paper is actually using and why it's better than the other approaches. Figure 3's caption is quite jargon-heavy ("noise projection with covariance completion and normalization") and is a missed opportunity to allow readers to better understand the method.

References:

  • [1] Jun-Kun Chen et al. ConsistDreamer: 3D-Consistent 2D Diffusion for High-Fidelity Scene Editing. CVPR 2024.
  • [2] Runjie Yan et al. Consistent Flow Distillation for Text-to-3D Generation. ICLR 2025.
  • [3] Norman Müller et al. MultiDiff: Consistent Novel View Synthesis from a Single Image. CVPR 2024.
  • [4] Pascal Chang et al. How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Models. ICLR 2024.
  • [5] Runjia Li et al. VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memory. ICCV 2025.
  • [6] Zeqi Xiao et al. WORLDMEM: Long-term Consistent World Simulation with Memory. NeurIPS 2025.
  • [7] Jiwen Yu et al. Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval. SIGGRAPH Asia 2025.
Questions for authors

My tentative positive rating reflects that I find the noise construction method to be valuable, but is contingent on the authors addressing my evaluation concerns, especially the depth-warped baseline for Table 2 and revisit experiments that go beyond TSED.

Also:

  • It would be helpful for the authors to re-articulate their own claims of novelty in the context of the prior work I mentioned.
  • What is the runtime and memory overhead of the 3D noise construction?
  • Can you confirm that there is no overlap between the DL3DV-10K train and test scenes?
5 comments

Q1: Claims and Novelty.

A1:

We thank the reviewer for highlighting these relevant works. Accordingly, we have toned down the original claim in the revised abstract. The revised statement now reads: "Extensive experiments demonstrate the effectiveness of WorldNoise in improving camera controllability and cross-view consistency, highlighting the potential of shared 3D noise construction for generative world models."

Besides, we would like to emphasize that our key novelty lies in constructing a shared 3D noise space throughout the observable world region, rather than restricting noise to scene surfaces. Specifically, ConsistDreamer [1], CFD [2], and MultiDiff [3] establish noise correspondences based on scene surfaces, where noise is effectively supported on surface geometry rather than distributed throughout the 3D space. In contrast, WorldNoise distributes world-space noise primitives throughout the region jointly covered by the camera frusta along the trajectory. This volumetric construction naturally supports multiple depths without requiring prior knowledge of the geometry of the scene to be generated.

We appreciate the reviewer for bringing these additional references to our attention. We have accordingly refined our novelty claims and expanded the Related Work section to properly acknowledge and discuss these studies.


Q2: Limitations.

A2: We thank the reviewer for the constructive suggestion. We have further clarified the following limitations in the revised Conclusion and Limitations section. (1) Camera Trajectory Requirement. WorldNoise requires a predefined camera trajectory to determine the observable world region for noise construction. (2) Dynamic Scene Modeling. Our current formulation focuses on static scenes, while extending the stochastic world representation to dynamic scenes remains an interesting direction for future work. (3) Limitations of Video Generation Models. Despite recent progress, video generation models still face challenges in maintaining physical consistency and long-term temporal coherence, particularly in complex scenes and extended generation.


Q3: Computational Overhead of Noise Construction.

A3: We measured the runtime and memory overhead of 3D noise construction on a single NVIDIA A100-SXM4-80GB GPU, using a budget of 4M points for 81 RGB frames (21 latent frames) at 480×832480 \times 832 resolution. Across the RealEstate10K and DL3DV datasets, the average noise construction time is 0.533 s, with approximately 0.95 GiB of incremental peak GPU memory allocation. Noise construction accounts for only 1.01%1.01\% of one training step (forward, backward, and optimizer update) and 0.162%0.162\% of one inference run (15 denoising steps).


Q4: Compare with MultiDiff.

A4:

We thank the reviewer for this valuable suggestion. We have added the depth-warped noise construction strategy from MultiDiff as an additional baseline. The result are as below. The corresponding quantitative results and qualitative visualizations have also been updated into the revised manuscript.

Comparison of noise construction strategies.

RealEstate10K

MethodCamera: RotErr ↓Camera: TransErr ↓Camera: RPE-t ↓Camera: RPE-r ↓Geometry: TSED ↑Geometry: Reg. ↑VBench: MS ↑VBench: DD ↑VBench: AQ ↑VBench: IQ ↑
MultiDiff4.344‾\underline{4.344}0.37360.0228‾\underline{0.0228}0.4180.9331‾\underline{0.9331}0.99600.99030.5600‾\underline{0.5600}0.51990.7310
CameraNoise7.2490.23810.02470.347‾\underline{0.347}0.90450.9973‾\underline{0.9973}0.99030.68000.51730.7349‾\underline{0.7349}
UniCaMo7.3160.48250.03000.3880.92860.99530.9904‾\underline{0.9904}0.55000.5204‾\underline{0.5204}0.7330
WorldNoise1.9400.2422‾\underline{0.2422}0.02160.1280.97790.99960.99070.68000.52420.7448

DL3DV

MethodCamera: RotErr ↓Camera: TransErr ↓Camera: RPE-t ↓Camera: RPE-r ↓Geometry: TSED ↑Geometry: Reg. ↑VBench: MS ↑VBench: DD ↑VBench: AQ ↑VBench: IQ ↑
MultiDiff28.324‾\underline{28.324}0.2103‾\underline{0.2103}0.1021‾\underline{0.1021}1.813‾\underline{1.813}0.23710.9311‾\underline{0.9311}0.96211.0000.47930.6633
CameraNoise36.3530.34260.14652.7060.16480.92790.95100.9844‾\underline{0.9844}0.47840.6756
UniCaMo37.7010.37120.13462.4860.2715‾\underline{0.2715}0.91920.9635‾\underline{0.9635}0.95310.49260.6962
WorldNoise7.6640.18060.07781.0330.48650.96660.96361.00000.4805‾\underline{0.4805}0.6770‾\underline{0.6770}

The relatively limited performance of this strategy may be attributed to its reference-view-based noise construction. Specifically, MultiDiff warps noise from the first view to subsequent views using depth information and fills uncovered regions with independent Gaussian noise. Under large camera rotations, the overlap with the reference view becomes limited, leaving large regions filled with independent noise and weakening cross-view correlations. In contrast, our method constructs a shared 3D noise field covering the camera trajectory, allowing different views to share consistent noise even in regions not visible from the first view. This provides stronger geometric correlations across views, particularly under challenging camera motions.

--

Q5: Additional Metric.

A5:

We thank the reviewer for this valuable suggestion. We have added the missing evaluation metrics in the revised manuscript, including PSNR, SSIM, and LPIPS for reconstruction quality, as well as FVD for overall video quality. These additional results provide a more comprehensive evaluation of both video generation quality and scene consistency. All these results have also been updated into the revised manuscript.

Additional evaluation results on RealEstate10K and DL3DV using reconstruction metrics and standard video quality.

RealEstate10K

MethodPSNR ↑SSIM ↑LPIPS ↓FVD ↓
SCoPE16.81300.54540.3121156.466
Gen3C16.83610.56230.2931149.856
UCM15.75080.52190.3382197.988
Wan12.79130.44820.5283435.162
Wan + Ours16.33630.54170.3100220.588
LTX16.05210.52220.4001276.290
LTX + Ours17.27330.56780.2800141.315

DL3DV

MethodPSNR ↑SSIM ↑LPIPS ↓FVD ↓
SCoPE12.69690.37980.5232233.823
Gen3C13.34870.39420.5387279.080
UCM12.85880.39270.5308435.646
Wan10.55320.31460.6175574.857
Wan + Ours13.71450.38860.5272195.275
LTX12.36370.38250.5781531.800
LTX + Ours14.15210.43490.5187203.681

Q6: Revisitation experiments.

A6:

We thank the reviewer for this valuable suggestion. Following this recommendation, we have added a revisitation evaluation in the revised manuscript (Sec.4.4). Specifically, we follow the evaluation protocol of Context-as-Memory[7] and consider both ground-truth and generated historical observations as memory.

We evaluate the revisit results against the corresponding ground-truth or previously generated observations at identical camera poses using PSNR, SSIM, and LPIPS. We further compare WorldNoise with SANA-WM and Lyra 2.0 under the same evaluation protocol. Since the released SANA-WM inference pipeline does not support ground-truth history conditioning, its ground-truth comparison is marked as N/A. The results are as below:

Quantitative comparison of revisitation consistency on DL3DV.

MethodGT: PSNR ↑GT: SSIM ↑GT: LPIPS ↓History: PSNR ↑History: SSIM ↑History: LPIPS ↓
SANA-WMN/AN/AN/A11.62510.23270.6326
Lyra 2.013.3907‾\underline{13.3907}0.3855‾\underline{0.3855}0.531012.8718‾\underline{12.8718}0.34780.5914‾\underline{0.5914}
WorldNoise13.45260.39670.5431‾\underline{0.5431}12.99050.3477‾\underline{0.3477}0.5803

The quantitative results have been added to Figure 8, and more cases are provided in the Appendix E.4 of the revised manuscript. We kindly refer the reviewer to Sec.4.4 and the corresponding results in the revised manuscript for a detailed evaluation of long-term scene consistency.


Q7: More Ablations.

A7:

We thank the reviewer for this helpful suggestion. We add more detailed Ablations studies i

Noise Construction. We first conduct ablation about noise construction as below:

Ablation of the noise residual and normalization on DL3DV.

Noise ResidualNormalizationRotErr ↓TransErr ↓TSED ↑Reg. ↑
--✓7.6880.18130.48370.9645
✓--7.7010.18210.48090.9587
✓✓7.6640.18060.48650.9666

Lora. In our experiments, directly injecting our constructed noise into a pretrained diffusion model fails to produce coherent images, instead yielding drifting, blob-like color patterns. This suggests that the diffusion model needs to be adapted to the noise distribution induced by our construction strategy to generate meaningful visual content. We further investigate the impact of LoRA rank by fine-tuning the baseline model with ranks of 32, 256, and 512, respectively. Increasing the LoRA rank brings modest improvements in generation performance, but also incurs substantially higher computational costs. We therefore adopt rank 32 as a practical trade-off between performance and computational efficiency. Detailed quantitative results are provided in Appendix F.3, and we kindly refer the reviewer to this section for further details.

long-horizon generation. We have added a progressive ablation of the long-horizon generation components, including visual memory, persistent noise identities, and geometry updates and carving. Notably, visual memory provides substantial improvements: compared with the baseline, it reduces RotErr from 19.214 to 15.181 (a 21.0% reduction) and TransErr from 0.2528 to 0.2116 (a 16.3% reduction), while improving PSNR from 12.7331 to 13.5230 and LPIPS from 0.5633 to 0.5521. The additional ablation results and detailed analysis are provided in Appendix F.4. We kindly refer the reviewer to this section for further details.


Q8: Writing.

A8: We thank the reviewer for this helpful suggestion. We have replaced the term "world-space stochastic support" with the simpler term "world-space noise field" throughout the revised manuscript to improve readability. We revised the captions of Figures 2 and 3 to make them more informative and accessible. Specifically, Figure 2 now explicitly identifies (d) as our proposed WorldNoise and explains its advantages over other noise construction strategies. Figure 3 now provides a clearer step-by-step explanation of the noise construction and projection process, including how the residual noise and normalization preserve the Gaussian noise distribution while maintaining cross-view correlations. We have also revised the corresponding terminology in the manuscript to improve clarity.


Q9: Dataset.

A9: Yes, we confirm that there is no overlap between the DL3DV-10K train and test scenes.

Reviewer_aQeROpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

WorldNoise constructs shared 3D noise and uses projection and covariance completion to obtain spatially independent, cross-view-correlated Gaussian noise for Wan2.1 and LTX-2.5. A long-video extension incorporates historical geometry and visual memory.

Critical strengths

1)The paper provides a principled way to introduce shared 3D structure through the diffusion noise interface. Under the stated assumptions, covariance completion preserves within-view spatial independence while retaining cross-view correlations. This offers a useful geometric inductive bias without a separate camera-control network.

2)Same-backbone comparisons provide convincing evidence of improved camera control. In Table 2, WorldNoise reduces RealEstate10K rotation error from 7.249/7.316 for CameraNoise/UniCaMo to 1.940 and improves TSED from 0.9045/0.9286 to 0.9779. Results on two backbones further support applicability. These findings are the main basis for my positive recommendation.

Critical weaknesses

The long-horizon experiments do not isolate persistent noise from the accompanying visual-memory and geometry-update components. Moreover, TSED measures adjacent-frame consistency rather than consistency when revisiting previously observed regions. The results therefore provide limited evidence for the specific contribution of persistent noise to long-term world consistency. This weakens the long-horizon claim, but does not outweigh the supported short-video contribution.

Questions for authors
  1. Are the DL3DV training, short-video evaluation, and long-video evaluation sets strictly scene-disjoint? Please clarify how these splits were constructed.

  2. Which components and training strategies are retained in the Table 2 adaptations of CameraNoise and UniCaMo, particularly mixed-noise training and the 3D-track branch? Were training budgets matched across variants?

  3. What evidence distinguishes the contribution of persistent noise identities from that of visual memory and geometry updates in long-horizon generation? If available, please provide a controlled ablation or revisitation-consistency results; otherwise, clarify which conclusions the current experiments support.

2 comments

Q1: Dataset.

A1: Yes, the DL3DV training and evaluation sets are strictly scene-disjoint. For evaluation, we use the full videos for long-video generation and the first 81 frames of the same videos for short-video evaluation.


Q2: Training Setting.

A2:

We thank the reviewer for this question. In Table~2, our goal is to isolate the effect of different noise construction strategies under the same training setting. Therefore, we only replace the noise construction strategy, without retaining mixed-noise training or the 3D-track branch, as these components are orthogonal to our method.

For a fair comparison, we strictly control the training budget and keep all other settings identical across variants. Specifically, all variants use the same Wan2.1-I2V-14B backbone, construct noise at the same 832×480832 \times 480 resolution, and are trained on the same DL3DV-10K dataset. We also use identical training configurations, including LoRA rank 32, a learning rate of 1×10−51 \times 10^{-5}, and 10K training iterations.


Q3: Ablation.

A3:

We thank the reviewer for raising this important question. In long-horizon generation, WorldNoise constructs a shared stochastic world to encode camera-motion-dependent spatial correlations in diffusion noise, enabling camera-controllable video generation. To better reuse this noise world across chunks and maintain long-term consistency, our framework incorporates three complementary mechanisms: (1) Visual Memory, which provides historical appearance context; (2) Persistent Noise Identities, which help maintain consistent stochastic representations across chunks; (3) Geometry Update and Carving, which support the maintenance and expansion of the 3D noise.

To examine their contributions, we conduct progressive ablations, starting from WorldNoise with none of these mechanisms, the results are shown as below.

Long-horizon ablation on DL3DV.

Visual MemoryPersistent IdentitiesGeometry Update & CarvingRotErr ↓TransErr ↓PSNR ↑LPIPS ↓
------19.2140.252812.73310.5633
✓----15.1810.211613.52300.5521
✓✓--14.9070.210413.66160.5391
✓✓✓14.5700.191513.79870.5383

Thank you for the clarifications and for the additional long-horizon ablation.

The dataset and training-setup questions are addressed. I appreciate the confirmation that the DL3DV training and evaluation scenes are disjoint, and that the Table 2 adaptations of CameraNoise and UniCaMo replace only the noise construction strategy under a matched training budget, without mixed-noise training or the 3D-track branch.

The new progressive ablation is a useful step toward isolating the long-horizon components. Visual memory accounts for most of the measured gain, while persistent noise identities and geometry update/carving provide smaller additional improvements. This supports a complementary role for persistent identities, but the incremental effect is modest, and the ablation still uses standard error and quality metrics rather than a revisitation test of previously observed regions. I therefore continue to view the long-horizon claim as only partly supported, while the short-video camera-control evidence that motivated my original positive rating remains convincing.

I will keep my rating of 3 (Weak acceptance).

Reviewer_1BpCOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

he research studies show that diffusion noise can serve as a representation of a 3D world. The author proposes the WorldNoise framework that introduce correlations on a sequence of noise frames to modeling the implicit 3D relationship and camera movement between frames. The author validates the proposed methods on diverse experiments to validate their effectiveness.

Critical strengths
  • The idea proposed to introduce constructed noise frames conditioned by the 3D relationship, inspired by the limitation of independent noise sampling between video frames. The idea overall seems quite natural and simple.
  • The method describes in detail the implementation of world noise construction, making it easy to follow.
  • The results shows significant improvement compared to baseline methods.
Critical weaknesses
  • On DL3DV, some metrics are far behind ScoPE. Can the author explain the reason of it?
  • The Fig.5 shows little improvement compared to Wan baseline. I am concerned that LoRA-rank32 might not be enough to fully demonstrate the upper bound of the methods.
Questions for authors
  • Complex trajectory visualization. As the paper claims worldnoise benefit long-term generation, I am glad to see more visualizations on more complex and longer trajectories. Maybe construct a 360 degree rotation and some revisit camera trajectory to demostrate the upper bound of the proposed method to help us better understand the methods.
3 comments

Q1: Performance.

A1:

We thank the reviewer for raising this concern. We would like to clarify that TransErr is the only metric on which our method underperforms SCoPE on DL3DV. Considering the results across both backbones, our method achieves better or comparable performance to SCoPE on all remaining evaluation metrics.

We believe this gap may be partly attributed to differences in training data. Specifically, SCoPE is trained on four datasets (RealEstate10K, DL3DV, PanShot, and OmniWorld). Beyond the commonly used RealEstate10K and DL3DV, SCoPE further incorporates PanShot, a carefully designed synthetic panoramic dataset that provides accurate camera trajectories. It also leverages OmniWorld, a large-scale dataset containing over 600K sequences and 300M frames across simulation, robotics, human activities, and in-the-wild videos, with rich geometric annotations. Together, these datasets expose SCoPE to substantially more diverse scene content and camera-motion patterns, as well as richer geometric supervision. This broader training coverage may help SCoPE better generalize to diverse camera translation patterns, potentially contributing to its lower TransErr.

However, both original backbones perform poorly in terms of camera translation control on DL3DV, with TransErr values of 0.4672 for Wan2.1 and 0.3955 for LTX-2.5. Incorporating our method substantially reduces these errors to 0.1806 and 0.1514, respectively, corresponding to reductions of 61.3%61.3\% and 61.7%61.7\%. These substantial and consistent improvements across two different backbones demonstrate the effectiveness and generality of our noise construction strategy.


Q2:Lora.

A2:

We thank the reviewer for this comment. At first glance, we agree that the Wan baseline produces visually plausible results. However, a closer comparison with the ground-truth (GT) sequence in the first row of Fig. 5 reveals noticeable deviations in camera viewpoints and angles. In contrast, our method follows the target camera trajectory more faithfully, resulting in better viewpoint alignment with the GT.

As suggested, we conducted additional experiments with LoRA ranks of 256 and 1024. As shown in the table below, increasing the LoRA rank brings some improvements across multiple evaluation metrics, suggesting that our method can benefit from increased fine-tuning capacity. Nevertheless, we adopted the widely used LoRA rank of 32 in the original paper as a practical trade-off between performance and computational cost. Importantly, rank 32 already demonstrates substantial improvements in camera controllability and geometric consistency over the Wan baseline, supporting the effectiveness of our method.

Comparison of the Wan baseline and LoRA training with ranks 32, 256, and 1024 on DL3DV.

TrainingCamera: RotErr ↓Camera: TransErr ↓Camera: RPE-t ↓Camera: RPE-r ↓Geometry: TSED ↑Geometry: Reg. ↑VBench: MS ↑VBench: DD ↑VBench: AQ ↑VBench: IQ ↑
Baseline43.8150.46720.16062.6610.25860.93630.95550.92190.47130.6613
LoRA 327.6640.18060.07781.0330.48650.96660.96361.00000.48050.6770
LoRA 2567.1330.12720.06940.0990.52370.98770.97161.00000.48620.6783
LoRA 10247.0980.11880.06910.0980.52440.98810.97251.00000.48700.6881

Q3:Complex trajectory visualization.

A3:

We thank the reviewer for this valuable suggestion. We have added visualizations of 360$^\circ$ camera rotations and revisitation trajectories to demonstrate WorldNoise under more challenging and longer camera motions. Please refer to Appendix E for these additional visualizations.

We have also added a revisitation experiment in Sec. 4.4 to further evaluate long-term scene consistency. Please refer to this section for detail.

Reviewer_1BpCOpenReview ↗

Response

Thanks for the rebuttal, the author has addressed most of my concern. I will kepp my rating.

Thanks for the helpful comments and suggestions. They have been valuable in improving our experiments and the overall presentation of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.