acceptodds
Under review as a conference paper at ICLR 2027

Skytopia: Monocular Drone Navigation with Action-Conditioned Latent World Models

17 views

Abstract

Monocular drone navigation requires reaching a goal in an unseen environment from a single forward-facing camera, which offers few cues for depth and scale. World models address this by modelling how observations evolve under actions, but they are built to be executed: the prediction is produced at deployment and fed back into action generation at every control step. We argue that what a policy needs from a world model is not the prediction but the representation required to produce it: in flight the executed action explains almost all of the change between observations, so prediction reduces to reprojecting a static scene under a known displacement. We therefore introduce skytopia, a policy built on an action-conditioned latent world model, and the 3D Gaussian Splatting platform on which it is trained. A forward objective predicts the representation of the next observation from the intended motion, and an inverse objective recovers that motion from the predicted transition. Because the prediction never reaches action generation, the predictor is discarded and one policy serves point-goal, image-goal, and goal-free navigation. Simulation experiments show that skytopia outperforms every baseline under all three specifications, attaining 57.8%, 66.0%, and 49.0% success rate, while discarding the predictor removes 59% of the inference cost. The same policy is subsequently deployed on a physical drone without fine-tuning and reaches goals in indoor, open outdoor, and woodland environments.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied AI, Visual Navigation, World Models, Action-Conditioned Representation Learning, Drone Navigation, Sim-to-Real Transfer

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_RnvdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework for monocular drone navigation that supports point-goal, image-goal, and goal-free settings at a scale of 6x6-30x30 meters in both indoor and outdoor environments. It uses a 3DGS-based simulation platform and jointly trained forward/inverse latent dynamics objectives to learn implicit world representations, while discarding the prediction modules during deployment for efficient action generation. Zero-shot real-world experiments were conducted on a DJI Tello with inference running on an external RTX 3090 GPU at approximately 7 Hz and flight speeds limited to 1 m/s.

Critical strengths
  1. The paper proposes a pipeline that leverages 3DGS to reconstruct real-world scenes as training environments, avoiding the need for costly and time-consuming real-world drone deployment. The authors also collect a large-scale dataset consisting of 94 scenes, 94K trajectories, and 32.5M rendered frames, which is a valuable contribution.

  2. The paper demonstrates zero-shot transfer to a physical drone empirically without any policy fine-tuning.

Critical weaknesses
  1. The paper lists “A policy deployed without the predictor” as its third contribution. While this is a desirable property for real-time UAV deployment, the underlying idea is not entirely new. Prior works such as FutureNav and Fast-WAM have explored training-time world-model supervision without requiring future prediction during inference. In particular, FutureNav also combines forward and inverse dynamics objectives to improve a shared navigation representation while maintaining efficient direct action generation. I acknowledge that FutureNav's discrete action space is not directly applicable to drone control, and adapting this framework to UAV navigation is a meaningful contribution. However, predictor-free deployment alone seems insufficient to establish a major methodological contribution.

  2. The paper lists “A world model driven by the intended motion” as its second contribution. The proposed forward and inverse dynamics objectives are reasonable, and I appreciate the authors' shortcut analysis in the Appendix and their efforts to mitigate this issue. However, given the similarities to prior work such as FutureNav, the paper could better clarify what is methodologically distinct about the proposed formulation.

Questions for authors
  1. Clarification of the simulation platform and dataset contributions. The proposed platform integrates real-world scene capture, 3DGS reconstruction, mesh/SDF-based collision geometry, and physics simulation to generate navigation demonstrations. Could the authors clarify which aspects of this pipeline represent the main technical contributions beyond integrating existing components? In particular, are there any UAV-specific adaptations to the standard 3DGS reconstruction pipeline, such as improvements in aerial viewpoint handling, geometric consistency, or rendering quality? It would also be helpful to understand how the collected dataset differs from existing aerial navigation datasets in terms of scene diversity, trajectory coverage, and visual/geometric quality.

  2. The zero-shot transfer results are encouraging, especially given the differences between reconstructed simulation environments and physical UAV deployment. Could the authors elaborate on which factors they believe are most important for achieving successful transfer despite differences in visual appearance, obstacle geometry, and flight dynamics? Were there any notable failure cases or systematic challenges during real-world deployment? Additionally, could the authors clarify whether any calibration, dynamics matching, or other sim-to-real adaptation was performed beyond the described training protocol?

6 comments

Thank you for recognizing the platform and zero-shot flight experiments. Manuscript revisions are highlighted in blue.

W1. Predictor-free deployment. FutureNav [1] and Fast-WAM [2] also avoid future prediction at inference. Fast-WAM retains its video DiT to encode observations for an action expert, while FutureNav retains VGGT spatial fusion. SKYTOPIA removes the auxiliary V-JEPA 2 encoder and both dynamics heads, deploying only the vision-language backbone and flow-matching action head. This saves 59.4% of full-model FLOPs. Table 2 shows improved navigation without increasing deployed policy capacity.

W2. Coupled forward and inverse prediction. FutureNav recovers actions from observed-state features, their difference, and context. Our inverse head instead takes the current state and predicted next state to recover continuous commands. Its gradients pass through the prediction into the forward head and the backbone's action tokens. Forward prediction must therefore both match the observed future and preserve the executed motion.

Our shortcut probe motivates dropping the preceding state with 30% probability so action recovery must use the predicted next state. The forward loss anchors that prediction to the observation, while the inverse loss preserves command information. Objective ablations and geometric probes support this formulation.

We added FutureNav and WorldVLN [3] with matched data, goal guidance, and evaluation. Both use seven-step continuous controls, and FutureNav retains its dynamics losses. SKYTOPIA achieves higher SR/SPL in all three modes (Table 1), with 34.9% fewer FLOPs and 38.6% lower latency per query than adapted FutureNav. It also performs better across route lengths and visual perturbations (Figs. 10–11).

Q1. Platform and dataset contributions. The platform contribution is converting captured scenes into interactive flight tasks with a shared metric basis. Standard 3DGS optimization fits appearance to posed RGB images. LiDAR supplies metric geometry, and the reconstructed mesh supplies a 10 cm SDF. Planning, rendering, clearance-dependent velocity commands, and collision decisions thus refer to the same scene geometry.

When a policy leaves the expert path, the simulator renders its actual camera pose and checks collisions at its new position. This turns the reconstruction into a flight-training environment with feedback beyond recorded routes.

UAV-Flow pairs real pilot trajectories with atomic instructions for fine-grained flight [4]. IndoorUAV builds language-guided trajectories in Habitat indoor scenes [5]. Our collection spans 18 indoor, 48 urban outdoor, and 28 vegetation scenes across 75,657 m², covering rooms, corridors, streets, and woodland within one control interface.

We collect 1,000 routes per scene, yielding 94K trajectories and 32.5M frames. Samples pair RGB images, physical states, and continuous velocity commands, with speed adapted to clearance and curvature. This coverage supports learning visual-motion dynamics and obstacle avoidance under point-goal, image-goal, and goal-free inputs. Appearance and collision geometry come from the same real capture, linking rendering to planning and evaluation.

Q2. Transfer, calibration, and failures. Diverse real-scene reconstructions expose the policy to varied appearance and layouts. The geometric probes and perturbation tests support learning scene structure and motion through dynamics supervision. Simulation and deployment share body-frame velocity and yaw-rate commands, with physical stabilization handled by the drone controller.

Policy weights remain fixed in real flights. Deployment includes velocity-to-command calibration, deadband compensation, coordinate conversion, and inference-delay handling. Translation and yaw are scaled together to preserve their ratio when reducing speed. Training uses ideal velocity tracking.

The main observed failure was overshooting near the goal: the drone failed to see the target and adjust yaw in time, passed the destination, and left the test area. Failures also occurred in dense clutter.

References.

[1] Zhang et al. (2026). FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation. arXiv:2606.30367.

[2] Yuan et al. (2026). Fast-WAM: Do World Action Models Need Test-time Future Imagination? arXiv:2603.16666.

[3] Zhao et al. (2026). WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation. arXiv:2605.15964.

[4] Wang et al. (2026). UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning. arXiv:2505.15725.

[5] Liu et al. (2026). IndoorUAV: Benchmarking Vision-Language UAV Navigation in Continuous Indoor Environments. arXiv:2512.19024.

Thank you again for your constructive feedback. We would appreciate it if you could review our rebuttal and, if our responses have addressed your concerns, consider updating your rating. We would be happy to provide further clarification if needed.

Reviewer_RnvdOpenReview ↗

Remaining Concerns Regarding the Contributions of (1) the Simulation Platform and (2) Predictor-Free Deployment

Thank you for the rebuttal. I appreciate the solid technical work and the additional experimental results.

I have two remaining concerns regarding the core contributions:

  1. 3DGS-based simulation platform. The platform could be a valuable engineering and dataset contribution. However, without a systematic comparison with existing UAV simulation or reconstruction-based training platforms, its contribution has not been sufficiently established. Besides, could the authors clarify whether the platform and dataset will be publicly released or made accessible to the research community?

  2. Positioning of predictor-free deployment. While the technical motivation is reasonable, I feel that the Introduction somewhat overstates the novelty of predictor-free deployment. Given that similar ideas have been explored in prior work, I would encourage the authors to position it primarily as a practical efficiency advantage rather than a standalone methodological contribution.

Reviewer_kKNPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

SKYTOPIA is a monocular drone navigation model trained with action-conditioned latent representations. Specifically, the model is trained (with action conditioning in the latent space) on data generated from a 3D Gaussian Splatting model. It is efficient since explicit future latent prediction is not required for policy decisions during inference. The method is shown to outperform the chosen baselines on the SKYTOPIA dataset generated for this work.

Critical strengths

Using 3DGS to render 3D scenes from monocular RGB data and physics guidance is ingenious. The proposed SKYTOPIA dataset is therefore useful.

Removing the latent predictor and using only action prediction for policy decisions helps improve inference efficiency.

Closed-loop navigation results on OOD scenes from the proposed dataset show improvement over the hand-picked baselines.

Critical weaknesses

The validation is limited to the SKYTOPIA dataset. Testing on more benchmarks such as UAV-Flow and IndoorUAV would strengthen the work.

The two closest methods are not included in the performance comparison. WorldVLN is directly comparable, while FutureNav has a similar design.

A major weakness (from a deployment perspective) is the remote decision-making. The video is sent over a network to a ground-based computer where the model carries out inference. Further, the WebSocket connection makes communication slower than, say, UDP-based protocols. Furthermore, since the solution is aimed at drones, the wireless communication channel between the drone and the computer is going to be stochastic, which in turn makes performance unreliable. Models such as WorldVLN also suffer from this shortcoming, making them impractical for outdoor deployment.

Questions for authors

Why were the WorldVLN and the FurtueNav models not considered in the performance comparison?

Why were the UAV-Flow and IndoorUAV datasets not used for evaluation?

How can you overcome the weakness identified above?

Why can the policy decisions not be computed onboard (even with the predictor module removed)? Models such as OmniVLA provide explicit evaluation on edge devices.

3 comments

Thank you for the review. We address the related weaknesses and questions together. Manuscript revisions are highlighted in blue.

W1/Q2. Evaluation scope and benchmarks. Our task is monocular flight control under point-goal, image-goal, and goal-free settings. UAV-Flow [3] evaluates language-conditioned maneuvers and trajectory alignment through nDTW. IndoorUAV [4] includes language-guided routes and short-horizon instruction-conditioned actions, with trajectory alignment and final heading among its criteria. These instructions specify intermediate maneuvers and spatial relations, not only destinations.

Using these original protocols would require language-conditioned training and a new goal interface. Substituting endpoint coordinates or images changes the task: two routes can reach the same endpoint while satisfying different instructions. Extracting point-goal or image-goal episodes would define a new protocol, without direct comparability to published benchmark results. We therefore retain the three goal settings and adapt the requested methods to identical training data, goal guidance, and evaluation criteria.

Our evidence includes comparisons in five held-out scenes, objective ablations with the same deployed backbone and action head, and zero-shot physical flights in three unseen environments. These assess navigation learning and physical transfer, with the closest methods compared under a shared protocol.

W2/Q1. FutureNav and WorldVLN. These are recent arXiv preprints [1,2]. The ICLR reviewer guidelines do not require arXiv-only comparisons. We have included both because of their close relevance.

Both adaptations use seven-step continuous control heads, and FutureNav retains its auxiliary dynamics losses. SKYTOPIA achieves higher SR and SPL in all three modes (Table 1), with 34.9% fewer FLOPs and 38.6% lower latency per query than adapted FutureNav. The advantage also holds across route lengths and visual perturbations (Figs. 10–11).

FutureNav recovers actions from observed-state features. Our inverse loss supervises predicted transitions and backpropagates through the forward predictor, requiring predictions to preserve executed motion. FutureNav retains VGGT spatial fusion at inference, while our auxiliary video encoder and dynamics heads are removed. WorldVLN decodes actions from generated future representations, whose errors can affect action generation. Our action head avoids this dependency. These differences offer possible explanations without isolating their individual effects.

W3/Q3. Communication and inference. The wireless Tello link uses UDP over Wi-Fi for commands, state, and video [5]. WebSocket connects the control bridge and policy server locally through the ground computer's loopback address. We clarified this distinction in the manuscript.

Inference runs asynchronously at approximately 7 Hz with a separate command loop. Only the latest pending observation is retained. Action selection accounts for elapsed time, expired chunks are discarded, and zero velocity is sent when no valid action remains. This avoids blocking command updates or repeating stale actions. Wireless variability remains a limitation, and real flights demonstrate transfer under the tested conditions.

Q4. Onboard inference and OmniVLA. OmniVLA-edge [6] uses a separate 50M-parameter ViNT-based model. Our policy retains a Qwen3-VL-2B backbone and a 160M action head. Removing the auxiliary modules saves 59.4% of full-model FLOPs, but the deployed model remains larger.

Our preliminary 16 GB Jetson Orin test loaded and ran the model, but inference was too slow for the intended control loop. We therefore use workstation inference. Table 2 shows that dynamics supervision improves navigation with the same deployed backbone and head as action-only training. The benefit is improved navigation without increased deployed policy capacity. Practical onboard operation still requires optimization.

References.

[1] Zhang et al. (2026). FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation. arXiv:2606.30367.

[2] Zhao et al. (2026). WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation. arXiv:2605.15964.

[3] Wang et al. (2026). UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning. arXiv:2505.15725.

[4] Liu et al. (2026). IndoorUAV: Benchmarking Vision-Language UAV Navigation in Continuous Indoor Environments. arXiv:2512.19024.

[5] Ryze Tech. Tello SDK 2.0 User Guide.

[6] Hirose et al. (2026). OmniVLA: An Omni-Modal Vision-Language-Action Model for Robot Navigation. arXiv:2509.19480.

Reviewer_kKNPOpenReview ↗

Appreciate responses but concerns remain

The responses and additional comparisons with the WorldVLN and FutureNav models are appreciated. However, they do not fully address the concerns. Specifically, the following concerns remain: limited experimental validation and impractical system.

Validation on one dataset is not convincing enough. Even if training on the suggested datasets is not feasible, a discussion about them could improve the literature coverage.

As shown by the latest experiments, the high communication and inference costs render the approach unsuitable for practical deployment. WebSocket is not ideally suited for the low-latency requirements of this application. This is further exacerbated by the high inference costs which forces the need for a server (and communication with it). In my opinion, this limits the utility of the method and is a major weakness.

Therefore, I choose to retain my rating.

Thank you for the follow-up and for considering the additional comparisons. We have added references to UAV-Flow and IndoorUAV in the revised manuscript and discussed their task differences in our response. To clarify, WebSocket is used only locally between the control bridge and policy server on the same ground computer. The wireless drone link uses UDP over Wi-Fi. We acknowledge that off-board inference remains a deployment limitation. We respect your decision to retain the rating and thank you for your time and consideration.

Reviewer_captOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper builds a photorealistic simulation environment from 94 real scenes reconstructed with Gaussian splatting, and uses it to train a monocular drone navigation policy that learns the depth it needs by training a world model, without having to run that world model in flight. Next to predicting the next view from the intended motion, an inverse objective recovers that motion, so the predictor cannot just extrapolate the past. One policy handles point-goal, image-goal and goal-free navigation. It beats seven baselines in simulation by at least 15 points success rate and flies zero-shot on a real drone in three environments.

Critical strengths
  • real-world representative successful experiments
  • zero-shot real flights, no fine-tuning, in 3 unseen sites: indoor room, open outdoor area, woodland
  • new training platform with 94 real scenes, photorealistic Gaussian splatting + physics + collision geometry, millions of frames -> useful for others too
  • well-motivated contribution: training-only dynamics losses -> depth for monocular flight; inverse loss blocks 'extrapolate the past'
  • 7 baselines from 3 families (imitation, vision-language-action, world models), all retrained on same data -> fair comparison
  • ablations: both losses help, together best; linear probes: depth + camera displacement easier to read from features
  • clear gain: success at least 15 points above strongest baseline in all 3 modes, no scene-specific tuning
Critical weaknesses
  • Main concern: requires more investigation: possible goal leak: 13-dim state holds relative goal vector -> goal-free 'never shown' goal may still be in state; would explain 49% success (inference)
  • image-goal = subgoal images along expert path in sim: this means it becomes a visual path follower and not "fly to a target view"
  • Off-board! -> motivation 'drone cannot pause' to motivate ground RTX 3090 over WebSocket at ~7 Hz: drone CAN hover!
  • 1 m/s speed limit
  • 'monocular' questionable: velocity in state; its source in real flights could not be found (drone has downward vision? motion capture indoors)
  • 59.4% compute saving is by construction: removed encoder, forward + inverse heads never feed the action head
  • real-world statistics are "medium" quality: 20 trials per site, no baseline flown, point-goal indoors only, operator judges goal-free arrival
  • simulated drone model, velocity tracking, camera tilt with attitude could not be found -> detailed sim-to-real claim hard to judge: it just shows: it still works in the real world, although the video seems to show the wind has a big influence and it might have gotten lucky
  • 'fewer collisions' but out-of-bounds rate second highest in point-goal + goal-free -> failures moved, not removed
  • no standard deviations in Tables 1 + 2
Questions for authors
  • First outdoor location looks like indoors in the video
  • is the time to success of about 130 steps (~2.7 s) faster than the expert mean episode 346 steps (~6.9 s for 17.2 m)?
  • sim camera 256 x 256, field of view could not be found; drone: 720p 82.6 deg resized to 224 x 224 -> aspect ratio? Please give more details. -sim flight speed could not be found, only 'adjusts speed to clearance'; please elaborate. and this varying speed is also described in literature already: see e.g. MAVRL: Learn to Fly in Cluttered Environments With Varying Speed
  • real image-goal landing on cosine similarity threshold: value could not be found
  • release of platform, data or code could not be found, while platform is listed as main contribution
  • simulated drone model + velocity controller + camera tilt could not be found
  • Figure 3: 'Three Deployable Navi Modes' -> Three Navigation Modes
1 comment

Thank you for the positive assessment and detailed comments. Manuscript revisions are highlighted in blue.

W1. Goal inputs. All three relative-goal state entries are zeroed during training and evaluation in image-goal and goal-free modes. The reported 66.0% and 49.0% SRs already use this mask. Goal-free has no goal vector or goal image, but retains an initial heading toward the endpoint with up to 45° yaw perturbation and a visible goal marker.

W2. Image-goal guidance. All baselines receive identical subgoal sequences and update rules. In open scenes, a fixed destination image may provide little guidance before the target is found. Our protocol evaluates local flight control and obstacle avoidance with shared visual route guidance.

W3/W4/W7. Flight conditions and evidence. Multirotors can hover. We revised the motivation to latency during continuous flight, which increases travel before the next correction. Ground-GPU inference runs asynchronously at approximately 7 Hz with a separate command loop. The 1 m/s limit is an upper bound, with lower limits in some flights. We report 20 trials per evaluated site and mode without fine-tuning. Point-goal flights use indoor motion capture, and an operator judges goal-free arrival. These tests establish transfer under this setup. Baseline comparisons are in simulation, without claims of high-speed flight or quantified wind robustness.

W5. Sensing and state inputs. The policy receives one forward-facing RGB image and proprioception. Real-flight velocity channels contain the previous command scaled to simulation units, not motion-capture velocity. Point-goal uses motion capture for goal and heading. Image-goal and goal-free use IMU yaw. The Tello retains built-in stabilization, including downward vision [2], but downward images are not policy inputs.

W6. Compute saving. The 59.4% saving comes from removing the video encoder and dynamics heads. Table 2 shows that their training supervision improves navigation over action-only training with the same deployed backbone and action head.

W8/Q6. Dynamics and camera attitude. Isaac Sim uses a Crazyflie rigid-body asset. Body-frame planar velocity and yaw rate are set directly, with gravity disabled and zero vertical command. Physics and control run at 100 and 50 Hz. This gives ideal velocity tracking without a motor controller. The body-mounted camera does not reproduce acceleration-induced roll and pitch. Real flights execute commands through the physical flight controller.

W9. Failure rates. Lower collision rates are not fully offset by out-of-bounds failures. CR+OB is 42.2% for SKYTOPIA versus 52.3% for the added FutureNav baseline [3] in point-goal, and 51.0% versus 66.2% in goal-free. Higher SR therefore reflects fewer failures overall.

W10. Standard deviations. Table 9 adds three-seed SR/SPL means and standard deviations for SKYTOPIA and all nine baselines. Table 1 retains means for readability.

Q1. Outdoor site. The site is inside a campus building open on all sides, exposed to natural daylight and ambient conditions. Its lighting is uncontrolled, unlike the indoor room. Flight restrictions motivated this outdoor test location.

Q2. Episode length. Each step is 0.02 s, so 134 steps equal 2.68 s. TTS covers successful test episodes and excludes inference wall-clock time. The 346-step expert mean covers the full training scene collection. Different routes prevent a matched speed comparison.

Q3. Cameras and simulated speed. Simulation renders 256×256 images with approximately 82.6° horizontal FOV. The Tello streams 720p with nominal 82.6° FOV. Both are directly resized to 224×224, stretching rectangular images. Simulated speed is capped at 3 m/s and yaw rate at 1 rad/s. The expert slows near obstacles and turns. Varying speed was studied in MAVRL [1] and is a demonstration-collection choice here.

Q4. Image-goal termination. Image-similarity weights yield a mean subgoal index, filtered and constrained to advance monotonically. Landing occurs within 0.15 subgoal intervals of the final image. This uses visual route progress without external position measurements, rather than a fixed cosine threshold.

Q5. Release. Company confidentiality restricts platform, code, and full-data release. A data subset is under consideration, subject to approval. Its scope and release date remain unconfirmed.

Q7. Figure 3. The label is corrected to “Three Navigation Modes.”

References.

[1] Yu et al. (2024). MAVRL: Learn to Fly in Cluttered Environments with Varying Speed. arXiv:2402.08381.

[2] Ryze Tech. Tello User Manual, v1.4.

[3] Zhang et al. (2026). FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation. arXiv:2606.30367.

Reviewer_6PyPOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SKYTOPIA, a framework for monocular drone navigation that combines a 3D Gaussian Splatting simulation platform with action-conditioned latent world-model training. The method employs forward latent prediction and inverse dynamics supervision to improve representations for navigation, while discarding the prediction branches during deployment. Experiments across 94 reconstructed scenes demonstrate improvements over seven navigation baselines, with additional zero-shot evaluations on a physical drone and substantial reductions in inference cost.

Critical strengths

The paper addresses monocular drone navigation in challenging environments where depth estimation, obstacle avoidance, and real-time control are tightly coupled. Supporting three navigation modes within one policy is practically valuable, and the physical drone experiments provide encouraging evidence of sim-to-real transfer.

The 3DGS-based platform comprises 94 reconstructed scenes and 32.5 million rendered frames. The evaluation includes seven baselines, three navigation modes, objective ablations, geometric representation probes, and robustness experiments. These provide useful evidence of the framework's effectiveness.

The combination of forward and inverse dynamics objectives consistently improves navigation performance over action-only training. Removing the prediction branches during deployment reduces inference FLOPs by 59.4%, demonstrating a practical benefit of separating representation learning from online prediction.

Critical weaknesses
  1. The paper states that the 13-dimensional proprioceptive state includes the relative goal vector, while Eq. (6) separately masks the metric goal input. It is unclear whether goal information already contained in the state is removed in image-goal and goal-free settings. Furthermore, goal-free navigation is evaluated against a sampled but unobserved target, making the reported 49.0% success rate difficult to interpret without specifying how hidden goals are sampled and related to the policy inputs.

  2. The frozen V-JEPA 2 encoder processes an entire eight-frame clip to produce temporally indexed latent states, while only the forward prediction head is explicitly constrained by causal attention. If earlier latent states already incorporate later-frame information through the video encoder, the prediction objective may exploit future observations without learning genuinely causal action-conditioned transitions. The paper should clarify the temporal attention structure of the target encoder and how future information is excluded from prediction inputs.

  3. Image-goal evaluation dynamically updates visual subgoals along a reference path supplied by an expert planner, rather than relying solely on a goal image specified once per episode. Real-world point-goal navigation also uses motion capture, while inference runs on an external GPU. These settings demonstrate useful capabilities but make the autonomous monocular navigation claims and comparisons across goal specifications less straightforward.

Questions for authors
  1. Does the 13-dimensional proprioceptive input contain the relative goal vector during image-goal and goal-free evaluation? How is this information masked, and can the authors verify performance when all goal-dependent state channels are explicitly removed?

  2. How are the unobserved evaluation goals sampled? What makes reaching a randomly selected hidden goal a meaningful success criterion for a policy that receives no target specification?

  3. Are the V-JEPA 2 latent states computed using causal or prefix-only temporal contexts? Can the authors demonstrate that the forward predictor cannot access future information through its input latent states?

  4. How are intermediate image subgoals selected and updated, and do all image-goal baselines receive identical guidance? Can the authors evaluate navigation using only one fixed destination image?

  5. Can the authors compare with training-time world-action modeling baselines under matched conditions and provide action-shuffling or counterfactual prediction tests? Can real-world performance be further validated without external localization or reference trajectories?

2 comments

Thank you for the careful review. We address the related weaknesses and questions together below. Manuscript revisions are highlighted in blue.

W1/Q1. Goal masking and performance. The three relative-goal entries in the 13-dimensional state are zeroed during training and evaluation in image-goal and goal-free modes. The remaining 10 channels contain linear velocity, angular velocity, and orientation. No unmasked goal vector enters the policy. Table 1 already uses this masking: image-goal and goal-free SRs are 66.0% and 49.0%, respectively. At the policy interface, changing raw goal coordinates while holding other inputs fixed leaves the masked state unchanged.

W1/Q2. Goal-free evaluation. Start–goal pairs are sampled from free space within the scene boundary and require a collision-free connecting path. The initial heading points toward the endpoint with up to 45° yaw perturbation, and a visible goal marker remains. Goal-free means no goal vector or goal image is supplied. SR measures collision-free arrival under this initialization and these visual cues, rather than search for an arbitrary hidden target. The simulator retains the sampled goal coordinates to evaluate arrival, although they are withheld from the policy. In real-world goal-free trials, an operator judges arrival at the designated endpoint.

W2/Q3. Encoder temporal context. The frozen V-JEPA 2 encoder uses bidirectional attention within the eight-frame clip. The forward head uses causal attention across windows. We added a prefix-only variant (w/ PO): each input state is encoded separately from the prefix ending at its window, retaining only the last temporal output. Later windows cannot enter that state's encoder context. The objectives and policy architecture remain unchanged.

Prefix-only SRs are 58.3%, 65.3%, and 48.2% across the three modes (Table 6), comparable to SKYTOPIA and higher than action-only training. Thus, dynamics supervision remains effective when future frames are excluded from the input-state encoder context.

W3/Q4. Image-goal protocol. All image-goal baselines receive identical subgoal sequences and update rules. Images are rendered every 0.5 m along the planner path, facing the direction of travel. The evaluator uses simulator position to advance with a 3 m lookahead. The policy receives the selected image without waypoint coordinates.

A fixed destination image can provide local cues in compact indoor layouts. In larger open scenes, the destination may lie outside the current view, introducing target search. Our preliminary fixed-image tests found this setting difficult for all evaluated baselines. We therefore use visual subgoals to evaluate local flight control and obstacle avoidance under shared guidance. We do not report a quantitative fixed-destination-image comparison.

Q5 and W3. Comparisons, action dependence, and deployment. We added FutureNav [1] and WorldVLN [2] under the same training data, goal guidance, and evaluation protocol. Both use seven-step continuous control heads. FutureNav retains its auxiliary dynamics losses. SKYTOPIA achieves higher SR and SPL in all three modes (Table 1), with 34.9% fewer FLOPs and 38.6% lower latency per query than adapted FutureNav.

FutureNav recovers actions from observed-state features. Our inverse loss supervises predicted transitions and backpropagates through the forward predictor, requiring predictions to preserve executed motion. FutureNav retains VGGT spatial fusion at inference, while our auxiliary video encoder and dynamics heads are removed. WorldVLN decodes actions from generated future representations. Our action head does not depend on such predictions. These differences offer possible explanations for the results, without isolating their individual effects.

The existing probe compares aligned state pairs, the preceding state alone, and pairs with a shuffled next state. It identifies inverse-prediction shortcuts and motivates dropping the preceding state with 30% probability. This probes state-pair dependence, not action-shuffling or counterfactual action conditioning.

Outdoor goal-free flights use onboard IMU orientation without motion capture or reference-trajectory inputs. Point-goal flights use motion capture, and inference remains off-board. Monocular refers to the visual input, with proprioception also supplied. The real flights demonstrate zero-shot transfer under these conditions.

References.

[1] Zhang et al. (2026). FutureNav: Unified World-Action Modeling for Vision-and-Language Navigation. arXiv:2606.30367.

[2] Zhao et al. (2026). WorldVLN: Autoregressive World Action Model for Aerial Vision-Language Navigation. arXiv:2605.15964.

Thanks for your feedback. I have no additional questions about this paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and Responses

We thank the Area Chair and reviewers for their constructive feedback. We summarize the main concerns and our responses below.

Methodological novelty. Reviewers questioned the distinction from prior world-action methods and the novelty of predictor-free deployment. We clarified that our inverse loss recovers continuous commands from predicted transitions and backpropagates through the forward predictor. We also revised the third contribution to “Efficient deployment without the predictor,” positioning it as a practical efficiency advantage. Objective ablations demonstrate improved navigation with the same deployed backbone and action head.

Comparisons and statistical evidence. We added FutureNav and WorldVLN using matched training data, goal guidance, and evaluation protocols. SKYTOPIA achieves higher SR/SPL across all three modes, with 34.9% fewer FLOPs and 38.6% lower latency per query than adapted FutureNav. We extended the route-length and visual-perturbation comparisons to include these baselines and added three-seed SR/SPL statistics for all methods in the main comparison. We also discussed the task differences from UAV-Flow and IndoorUAV and explained why evaluating our policy on these language-conditioned benchmarks would require a different protocol.

Goal inputs and temporal context. We clarified that all relative-goal state channels are masked during training and evaluation in image-goal and goal-free modes. We explained the shared visual-subgoal protocol and the initialization and visible goal marker retained in goal-free evaluation. To address possible future-information leakage through the video encoder, we added a prefix-only training control. It achieves comparable performance and exceeds action-only training across all three modes, showing that dynamics supervision remains effective when future frames are excluded from input-state encoding.

Platform contribution and accessibility. We clarified the platform’s engineering and data contribution and explained the controls needed for a systematic cross-platform comparison, which is not included in the present evaluation. We provided lightweight PLY previews of five reconstructed scenes with loading instructions in the supplementary material. Broader release remains subject to confidentiality constraints.

Real-world deployment. We clarified sensing inputs, simulation dynamics, deployment calibration, and observed failure cases. The wireless drone link uses UDP, while WebSocket operates locally on the ground computer. Inference remains off-board, and the physical-flight results demonstrate zero-shot transfer under the reported deployment conditions.

The corresponding manuscript revisions are highlighted in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.