acceptodds
Under review as a conference paper at ICLR 2027

Generating Parallel Worlds of a Flight

Abstract

We study parallel worlds of a flight: visually distinct videos that faithfully reproduce a prescribed trajectory's shape, orientation, and temporal progression. Pre-trained video models provide rich visual priors for these worlds, but can produce convincing camera motion that drifts from the prescribed flight. We present WorldSlider, a generative pipeline that addresses this control challenge. Existing camera controls require models to translate geometric instructions into visual motion, while source-video references also carry the original world's appearance and geometry. WorldSlider's FlightGrid representation bridges this gap by automatically rendering calibrated camera poses as an appearance-free control video of the prescribed trajectory and its surrounding flight corridor. We distill the generator to four inference steps and refine flight alignment through reinforcement learning with feedback from a camera-motion verifier. WorldSlider improves trajectory fidelity over existing methods on standard trajectories and diverse flights, with qualitative demonstrations of transfer between simulated and real inputs. Its task-consistent visual diversification provides training data for visuomotor policies, substantially improving task success in simulation. Code and synchronized supplementary videos are available on https://github.com/parallelworldsanon/WorldSlider.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
UAV egocentric video generation, flight trajectory control, controllable video generation, visual data augmentation, video diffusion models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_wimeOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies generating visually different videos that follow the same prescribed camera trajectory in orientation, normalized path geometry, and relative temporal progression. WorldSlider represents the trajectory using FlightGrid, an appearance-free rendered video combining a time-indexed flight trace with a gravity-aligned metric scaffold. A Cosmos Transfer control branch is adapted on 26.0 hours of simulated and physical footage, followed by four-step distribution matching distillation and DA3-based geometric reward weighting of the DMD update. Evaluation uses ViPE, separately from the training verifier, on standard and diverse trajectories, together with cross-world generation, human preference studies, and simulated gate-traversal policy training. Translation fidelity is evaluated up to a global scale, rather than as absolute metric displacement.

Critical strengths
  1. FlightGrid exposes timing, attitude, and translational parallax without carrying source RGB or scene depth. Figure 7(a) tests its components, and Appendices C.3/C.6 address both control initialization and analytic camera encodings on the same backbone and corpus. Even without depth-control initialization, FlightGrid obtains diverse-flight errors of 10.1 degrees and 0.230, versus 18.90/0.345 for Plücker and 19.55/0.315 for PRoPE at the same adaptation-stage NFE. These results support a useful control design beyond simply inheriting a depth-control checkpoint.

  2. Table 1 reduces diverse-flight rotation/translation errors from the best baseline in that comparison, HY-WorldPlay, at 19.93/0.378 to 6.65/0.101 using four NFEs. Importantly, ViPE is not the DA3 training verifier, and Table 6 also improves over plain DMD with equal additional training, from 10.40/0.184 to 6.65/0.101. Table 5 extends evaluation to 12 trajectories in 20 worlds each. The flight-aware human study in Table 7 provides further support, while visual-only preference is comparable to HY-WorldPlay. Together these make a persuasive empirical case for the resulting system; Figure 6 adds useful evidence of simulated policy-training utility.

Critical weaknesses
  1. Appendix C.6 trains the analytic Plücker/PRoPE controls using LoRA, whereas FlightGrid uses a ControlNet branch. The shared backbone, corpus, output length, schedule, and adaptation-stage NFE make this a useful comparison, and the base-DiT initialization experiment addresses the dense-control warm start. The conditioning paths nevertheless differ, and the manuscript does not report trainable parameter counts or the analytic variants' LoRA configuration, leaving capacity comparability unclear. These comparisons support the combined FlightGrid interface; the more specific explanation in terms of a pose-video modality gap remains partly inferential. This is a limitation on attribution, not evidence that the comparison is unfair or that the system gains are invalid.

  2. Section 4.2 and Appendix A.3 reuse waypoint labels after editing the initial image with GPT Image 2, with all gates initially visible. Figure 11 offers encouraging examples, but the manuscript does not quantify gate-layout consistency throughout the generated sequences or describe an acceptance/filtering procedure for policy-training clips. Accurate camera motion alone does not establish that task-relevant objects retain the geometry required by the original labels. The policy gains provide evidence of practical utility in the stated setting; an audit of existing clips would establish how consistently their generated task geometry supports the reused labels. This limits the strength of the general label-preservation claim without negating the reported policy improvements.

Questions for authors
  1. Appendix C.6 already specifies the common backbone, corpus, output length, and optimization schedule. Please supplement this with trainable parameter counts, LoRA ranks, and adapted layers for the analytic variants and the corresponding capacity of the FlightGrid branch. These details would help interpret how much the comparison separates representation from adaptation-path effects; the base-DiT initialization comparison already usefully addresses the warm-start question.

  2. How are edited initial frames and generated gate-traversal clips accepted for training, and what fraction, if any, are rejected? Can an audit of existing examples quantify task-object layout and waypoint-label consistency over time? This would directly substantiate the label-preservation claim beyond camera-pose agreement and selected visualizations.

  3. For the existing 240 cross-world videos in Table 5, please report the distribution of per-world errors, including a high-error quantile and examples from that tail. The 240/240 recovery count establishes estimator availability, while the reported trajectory-level means do not show how often individual worlds depart substantially from the prescribed flight. This distinction matters to the reliability of generating many worlds from one trajectory.

Reviewer_eJNyOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper addresses camera-controlled image-to-video generation for drone views, framed as generating “parallel worlds”, which use the same trajectory under different prompts and first frames. It has three components: i) FlightGird, a rendered camera trajectory video for camera control; ii) fine-tuning the depth-control ControlNet branch of Cosmos Transfer 2.5 on tartan-air and RE10K (~26h); iii) DMD distillation with a trajectory reward. Experiments show that the FlightGrid design is better than other camera-control methods, including Plucker embedding and prope.

Critical strengths

The proposed FlightGrid design is simple and practical. In the ablation study, it outperforms Plucker embedding and Prope, two common camera-control methods.

Critical weaknesses
  1. Positioning of the method

This work is camera-controlled I2V applied to drone trajectories, where the key contribution is the camera representation FlightGrid. The representation is agnostic to the flight scenario and only derived from camera parameters. Though the paper emphasizes “parallel worlds”, all other camera condition methods with different prompts and first frames also yield to “parallel worlds”.

In related works, the video editing part is not very relevant, and the controllable video generation part should focus on more camera control methods. Also, it omits flight video generation entirely; I’d recommend adding this in related work. Comparison with Captain Safari (CVPR2026) is a must.

  1. Experiments

Although the paper targets flight scenarios, it lacks results on real flight datasets for training and evaluation. Public real drone datasets include MotionScape (2604.07991), Sekai (2506.15675), and Captain Safari (2511.22815). Baselines are dated. Most baselines were published two years ago.

Questions for authors
  1. How does the proposed method perform on real-world drone data? For example, on MotionScape or Sekai.

  2. More baselines on camera-controlled I2V: Captain Safari, Wan-Fun-Control-Camera, Voyager, Uni3C, Lyra 2.0.

Reviewer_5cqgOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces WorldSlider for parallel‑world flight video generation. Given a calibrated 6‑DoF camera trajectory, text prompt and optional first frame, it synthesizes visually diverse videos that strictly preserve the input camera path’s geometry, orientation and temporal progression. To mitigate trajectory drift common in pose‑conditioned video generation, the authors propose FlightGrid, an appearance‑free visual proxy rendering flight geometry as control signal instead of raw analytic pose encodings. The pipeline includes three stages: control adaptation via ControlNet, DMD distillation for 4‑step fast inference, and RL fine‑tuning with geometric rewards from Depth Anything 3. Evaluations on standard and challenging flight benchmarks show strong trajectory fidelity against multiple baselines. The authors further demonstrate its utility as visual data augmentation for simulated drone visuomotor policy training.

Critical strengths
  1. FlightGrid is a solid core contribution. It bridges the pose‑to‑pixel modality gap; controlled ablations and fair comparisons against Plücker rays / PRoPE verify its advantages for complex coupled camera motion.
  2. Complete pipeline design: visual control adaptation, few‑step distillation, and geometry‑aware reinforcement learning enable high‑fidelity generation with only 4 inference steps.
  3. Comprehensive experiments: quantitative trajectory metrics with disjoint evaluator ViPE, component ablations, two human preference cohorts, and downstream robot simulation validation. Code and supplementary materials are promised for reproducibility.
Critical weaknesses
  1. “Parallel‑world flight generation” largely overlaps with existing camera motion‑transfer. The naming risks implying task‑level novelty, while actual novelty resides only in the FlightGrid technical solution.
  2. Training reward depends on monocular pose estimator DA3. Reconstruction ambiguities may induce training bias.
  3. The parameters and computational costs of different control methods are not specified.
Questions for authors
  1. Why route FlightGrid via VAE‑ControlNet latent injection instead of simple pixel‑level concatenation as extra context?
  2. What are the amounts of trainable parameters, LoRA rank, and training computational volume for Plücker, PRoPE, and FlightGrid individually? Is it the larger parameter size that brings better results?
  3. Why choose DA3 to obtain the training reward?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.