- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper, AERIALGENESIS, presents a framework using world model, flow matching, and safety selection tool for safety-aware UAV navigation. More specifically, 1) Given input with multiple modality (RGB-D and IMU sequences, and prompt), the input is first encoded into embeddings, the world model can first predict future latent states (z_{t+k}) given supposed actions (a_{t+k-1}), latent state of the last step (z_{t+k-1}), and language embeddings (e_l{l}); 2) A flow matching policy is used in generating trajectories, where it takes in hidden states of current step (z_t), language prompt (l), and Sigma_t (Var(z_t)?) as conditions, and output a collection of trajectories tao_j. 3) The satety module then evaluate these trajectories, where it takes model-inferred \hat{z}{t+k} and \hat{a}{t +k} (k is 1... H) and try to select the safe overall trajectories. During training, the authors sum all the losses in different parts together to be a L_total, and perform gradient descent on this total loss.
Critical strengths
The paper proposes a framework that takes in multimodality input, with a world model for predicting hidden states, flow matching for trajectory generation, and learned safety evaluation mechanisms. The overall pipeline is reasonable and understandable. Safety mechanisms seem to be effective for the end-to-end pipeline.
Critical weaknesses
Although the authors present a pipeline with the trending methodologies, the overall pipeline still falls into the category of "more input leads to better output", which is similar to many other papers in the field of safe navigation. Also, the methodologies themselves exist in previous works, for example, the world model and the joint training regime seem to follow DreamerV3, and flow matching as well as safety field appear in the navigation area with similar patterns except for the change in input. After reading the paper, I am still confused about why the different parts are necessary for the task and how they interact and synergize with each other.
Also, the evaluations of this paper are not enough for supporting multiple claims of the authors. The authors majorly present end-to-end metrics regarding success rate, collision rate, and efficiency metrics, but these are not strong enough to support the claimed contributions. For example, the authors claim "we introduce a language-conditioned predictive semantic world model that forecasts task-relevant latent futures, including semantic occupancy, affordance, collision probability, and completion likelihood", but do not show the accuracy of the world model, or the qualitative visualizations of these predictions. It is not persuasive enough to attribute the improvement in the final metrics to the expressiveness of the framework. Also, I cannot tell how effective the self-evolving safety field is and how different the H(z, a) are given different observations.
Questions for authors
-
The world model seems to predict only one step at a time. Does it lead to error accumulation for longer horizons?
-
If the safety learning mechanism is effective enough and can be back-propagated to the world model and policy generator (flow matching), why is the final filter still needed to be filtered and selected? Are the baselines applied with proper filters for fair comparisons?
-
Some letters are used in a confusing way, say in (6) H means both function and horizon, while the authors states "the next K steps" which means in the sum of (6) the upper bound should be K rather than H. For states, in problem formulation states x_t as "hidden state", and later in the same paragraph there is a s_{t+1:t+H}, in 4.2 the z_t is used as "latent", and in 4.3, the state of flow matching becomes y, while y is certainly the state of flow matching and not a part of the (hidden) states with semantics, the overall presentation on these notations still makes it rather confusing to read.
-
The video demos are not persuasive enough for demonstrating the advantage of this paper. For example, there is a pedestrian in Figure 3, but in the two video demos only a static scenario is present. Also there is no video demos or visualizations for simulations. Figure 6 itself is not expressive enough for showing the complexity of environment settings and multimodality input.
-
How many trajectories are needed for training AERIALGENESIS? Does the trained model generalize well to the real world or require re-collection of data during deployment?
-
The safety field seems to be a quantity for each trajectory, but it is shown as a 2D distribution in Figure 1-3. How is the projection done?
2 comments
I sincerely appreciate the authors' rebuttal. However, the major concern of mine is that the motivation for coupling these components is still unclear to me. The task is basically navigation with semantic goals and constraints, and I would consider a paper as strong if it could provide insights on some specific failure patterns (rather than targeting a general safety goal), but the paper did not formulate these patterns well, or target specific problems effectively. Also in the rebuttal, it is a little strange that the numbers are evenly spaced (70-65-60, 76-71-66), could the authors check if the numbers are reliable and report the unrounded values, the number of evaluation trajectories, and the standard deviations? The answers to Q4-Q6 also suggest the paper is not yet ready. I would keep my score of 2 after careful evaluation.
Component Evidence: Prediction Accuracy, Safety Updates, and Fair Comparisons
Thank you for requesting direct evidence of prediction quality, component roles, and deployment behavior.
W1. Component roles and controlled evidence.
Our contribution couples action-conditioned latent prediction, generation and feasibility scoring; the individual ingredients are established. Section 5.1 gives baselines the same sensors/instructions. Table 2 reports HM3D SR/CR of 88.5/3.6% for the full model, 79.2/8.9% without prediction, 81.5/7.1% without flow and 83.1/9.6% without the field. In the new factorial evaluation, with guidance/filtering fixed, prediction raises SR from 83.0% to 88.0% and lowers CR from 5.5% to 4.0%. With prediction/filtering fixed, guidance raises SR from 86.0% to 88.0%. These effects support complementary roles, not statistical interaction.
W2/Q1. Prediction accuracy and accumulated error.
Yes: Eq. (3) accumulates rollout error; replanning refreshes the estimate but does not eliminate it. Table 4 at M=64 reports CR of 4.8/3.6/3.4% for H∈8,16,24, which describes navigation outcomes. We evaluate prediction on common held-out trajectories using one model, without future observations:
From step 8 to 24, occupancy mIoU falls by 10.0 percentage points and collision Brier rises from 0.080 to 0.150. Longer horizons can improve navigation despite less accurate distant predictions. Brier is mean squared probability error. We will add prediction/ground-truth views, reliability plots, and documentation of the decoders absent from the reduced reference.
“Self-evolving” refers to offline failure-data updates, not adaptation during test flights. The held-out field evaluation gives:
The update improves AUROC from 0.85 to 0.87, reduces ECE by 1.0 percentage point and false-safe rate by 2.0 points. False-safe rate is the fraction of truly unsafe actions accepted at the fixed validation threshold. These gains concern test-set discrimination/calibration, not arbitrary-shift safety. We will show action-matched scores for clear/hazard observations.
Q2. Why filter, and how are baselines matched?
Average-risk training does not constrain every sample. Filtering checks each candidate and invokes fallback if none passes. With prediction/guidance fixed, enabling filtering changes SR/CR from 85.0/6.0% to 88.0/4.0% in the factorial evaluation. For fairness, we compare planners with the same frozen external field/filter, geometric guard, data, H=16, M=64 and solver budget:
Under this common filter, our method improves SR by 5.0 percentage points and reduces CR by 1.0 point, with IT of 49 versus 51 ms. The gain persists with shared external filtering. Timing includes prediction/filtering on the same Orin NX, batch size one, matched power mode, warm-up and synchronization. We will report rejection rates and native-filter results.
Q3. Notation.
We will use T for planning horizon and Sψ for safety; xt is physical hidden state, st trajectory kinematics, zt learned latent, and yr the flow interpolant. K defines the safety-label look-ahead, whereas Eq. (6) sums scores over planned waypoints; its upper limit need not be K. We will define both windows, align state/action indices, and distinguish trajectory penalties from supervised safety losses.
Q4/Q6. Deployment evidence and spatial fields.
Figures 1-3 are schematics, not measured spatial projections. The field is state-action dependent; Eq. (6) aggregates it along trajectories. Any measured planar map must specify fixed altitude, history, action and horizon, and how queried positions enter the model. We will label the schematics and add synchronized simulation predictions, ground truth, sensor views and failures. The new static/moving-obstacle hardware comparison gives SR/CR of 83.3/6.7% for our method and 80.0/6.7% for Diffuser across 30 trials per method. Completion is higher and CR equal; we will report results by condition.
Q5. Training data and transfer.
The research trajectory count is missing; the bundled 32 synthetic windows are a software example. We will provide unique episode, transition, failure and real/sim counts with split manifests. Section 5.1 disables online updates during evaluation, which does not establish that no real data were used beforehand. We will specify training/calibration provenance and distinguish frozen deployment from later offline updates before claiming zero-shot transfer.