acceptodds
Under review as a conference paper at ICLR 2027

Benchmarking Interaction-Oriented Aerial Mobility across Heterogeneous Environments

Abstract

Aerial mobile manipulation requires unmanned aerial vehicles (UAVs) to traverse heterogeneous environments such as indoor and outdoor domains, and connect navigation with subsequent physical interaction within the same trajectories. Existing aerial benchmarks are typically confined to single-environment settings, limiting evaluation of agents across continuous environmental transitions. When interaction is evaluated, agents are often initialized near the target, leaving the transition from navigation to interaction largely unassessed. This work introduces CrossFly, a benchmark for long-horizon interaction-oriented aerial mobility across heterogeneous environments. CrossFly contains 15K trajectories across more than 20 reconstructed scenes under a unified simulation and evaluation framework, covering Indoor, Outdoor, and IndoorOutdoor trajectories. It further introduces Interaction Readiness Rate (IRR) to assess whether terminal navigation states provide suitable conditions for downstream interaction without assuming a specific manipulation policy. Experiments with five representative aerial vision-language-action (VLA) agents and the proposed CrossFly-Agent show that IndoorOutdoor settings are more challenging than single-environment settings, with the highest success rate reaching 9.7% on unseen scenes. A downstream manipulation case study further validates the relevance of interaction-oriented arrival to subsequent physical interaction. We hope CrossFly helps advance aerial embodied intelligence toward unified mobility and physical interaction in heterogeneous environments. Related datasets and code will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Aerial Mobile Manipulation, Vision-Language-Action (VLA), Vision-Language Navigation (VLN), Embodied AI, Heterogeneous Environments, Indoor-Outdoor Cross-Domain Navigation, Long-Horizon Robot Tasks, Hierarchical VLM-VLA Agent, 6-DoF UAV Control, Robot Benchmark

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_aAeuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents CrossFly, a benchmark for evaluating interaction-oriented aerial mobility across heterogeneous environments. To address the limitations of existing benchmarks in continuous indoor-outdoor navigation and the transition from navigation to physical interaction, the authors construct a dataset containing 15,263 trajectories across 23 reconstructed scenes. An Interaction Readiness Rate (IRR) is introduced to evaluate whether terminal navigation states satisfy the geometric conditions required for subsequent manipulation. In addition, a hierarchical VLA agent is developed by integrating high-level language-based planning with low-level action generation. Experimental results demonstrate the increased difficulty of cross-environment navigation and show that the proposed agent outperforms several representative baselines.

Critical strengths
  1. Practical significance of the research problem. The paper addresses the important connection between long-horizon aerial navigation and subsequent physical interaction, which is not sufficiently considered in conventional navigation benchmarks. The integration of heterogeneous environments and interaction-oriented evaluation provides a meaningful research setting for aerial embodied intelligence.
  2. Systematic benchmark construction and evaluation. The proposed benchmark incorporates diverse reconstructed environments, structured trajectories, and compositional task instructions. The evaluation across multiple VLA models, together with the controlled comparison of different task settings, provides useful evidence of the challenges associated with cross-environment aerial navigation.
Critical weaknesses
  1. Limited quantitative evidence supporting the generalizability of IRR. Although the authors derive the IRR thresholds from manipulation outcomes across multiple aerial embodiments, the effectiveness of the resulting metric is not sufficiently evaluated on independent test cases. In particular, the paper does not report quantitative results demonstrating how reliably IRR distinguishes manipulation-ready states from states that lead to manipulation failure. Further evaluation of its predictive performance and cross-embodiment generalizability would strengthen the central contribution of the benchmark.
  2. Insufficient experimental evidence for the contribution of hierarchical planning. The proposed CrossFly-Agent achieves improved performance over several representative baselines. However, the paper does not provide controlled ablation studies to isolate the contribution of the high-level VLM planner. As the compared methods differ in their underlying architectures and pretrained models, the current results do not clearly establish whether the improvements primarily arise from hierarchical subtask planning. This limits the experimental support for the effectiveness of the proposed agent architecture.
Questions for authors
  1. Can the authors provide quantitative results evaluating the predictive performance of IRR on held-out manipulation trials? In particular, how do manipulation success rates differ between IRR-satisfied and IRR-violated terminal states, and how well do the derived thresholds generalize across different aerial embodiments?
  2. Can the authors provide an ablation study comparing the same QwenPi-based policy with and without the high-level VLM planner? This would help clarify the contribution of hierarchical subtask planning to the reported performance improvements.
3 comments

We thank reviewer for the valuable time and positive feedback. We provide point-to-point responses below and have revised the manuscript accordingly (please see the revised PDF in the Rebuttal Revision).


W1-1:"...In particular, the paper does not report quantitative results demonstrating how reliably IRR distinguishes manipulation-ready states from states that lead to manipulation failure..."

A-W1-1: Thanks for highlighting the need for quantitative validation. We address this concern through an experimental validation and mechanism analysis.

(1) Experimental validation.

We evaluate 50 independent test cases across 10 scene categories, comprising 25 IRR-satisfied and 25 IRR-violated terminal states. Results are reported below.

Terminal State GroupSuccessful TrialsFailed TrialsSuccess Rate (%)
IRR-satisfied23292
IRR-violated61924

The 68-percentage-point success-rate gap supports IRR’s ability to distinguish interaction readiness beyond target reaching, since both groups already satisfy the navigation success criterion.

(2) Mechanism analysis.

IRR constrains terminal distance and target-facing alignment, both of which affect depth-based target localization. Excessive distance or yaw misalignment can increase target-position errors, impairing downstream manipulation.

Our failure analysis examines terminal distance, yaw offset, and localization error. Figure 6 (§4.4) illustrates this connection, with the same grasping policy succeeding from an IRR-satisfied state and failing from an IRR-violated state.

We include the evaluation protocol, results, and failure analysis in the revised manuscript (Lines 429–469 and 990–995, Figure 6, Table 5), with the revised text highlighted in red.

W1-2: "...Further evaluation of its predictive performance and cross-embodiment generalizability would strengthen the central contribution of the benchmark."

A-W1-2: We agree. To assess the applicability of IRR across embodiments, we selected two representative aerial manipulation platforms for a controlled comparison, the hand-like UAV HI-ARM [1] and a UAV equipped with an SO100 arm [2].

Both platforms use the same visual localization pipeline and each has a dedicated RL grasping policy. Each policy receives the target’s 3D position in the UAV body frame, estimated from the depth at the center of the target’s visible surface (§4.4).

For each platform, we compare grasping success using visually estimated positions and simulator ground-truth positions.

EmbodimentGrasping success with ground-truth position (%)Grasping success with visual position estimate (%)
HI-ARM [1]99.7949.66
SO100-equipped UAV [2]99.8250.23

With accurate target positions, grasping success exceeded 99.7% on both platforms. Using visual position estimates reduces grasping success by approximately 50.13 percentage points on HI-ARM and 49.59 percentage points on the SO100-equipped UAV. These decreases indicate that target-localization accuracy is the main bottleneck on both tested embodiments.

Excessive terminal distance or yaw misalignment can increase target-localization error, impairing downstream grasping. The IRR distance and viewpoint criteria provide an embodiment-independent geometric assessment of terminal-state suitability for target localization.

We report this comparison and discuss how the shared visual localization pipeline supports cross-embodiment evaluation in the revised manuscript (Lines 471–485 and 899–936, Table 6), with the revised text highlighted in red.

[1] Hand-like autonomous flying robot for airborne grasping and interaction. In Nature Communications 2026.

[2] LeRobot: State-of-the-art Machine Learning for Real-world Robotics in PyTorch. GitHub. SO100 documentation.

W2: "However, the paper does not provide controlled ablation studies to isolate the contribution of the high-level VLM planner. As the compared methods differ in their underlying architectures and pretrained models..."

A-W2: Thank you for this question. We add controlled ablations to examine the contribution of hierarchical subtask guidance, including the roles of planner fine-tuning and VLA subtask supervision.

(1) Experimental validation.

We compare four configurations on the Outdoor task suite from the seen-scene split, using the same pretrained QwenPi backbone, training trajectories, VLA visual inputs, action supervision, and fine-tuning budget. All configurations use the same evaluation episodes as Table 2, and the full-model scores match its Outdoor results.

ConfigurationHigh-level PlannerVLA Training SubtaskVLA Inference SubtaskSR (%) ↑IRR (%) ↑
A. VLA only———18.140.4
B. Frozen-planner variantFrozen pretrained VLMGround truthPlanner prediction20.742.5
C. Predicted-subtask variantFine-tuned VLMPlanner predictionPlanner prediction23.145.6
D. Full CrossFly-AgentFine-tuned VLMGround truthPlanner prediction25.550.3

(2) Results and mechanism analysis.

Compared with the VLA-only baseline, the full model (A → D) improves SR by 7.4 percentage points and IRR by 9.9 percentage points. The high-level planner gives QwenPi an explicit local objective for target approach and manipulation invocation, connecting the global instruction to the current stage of execution.

With the same trained VLA checkpoint, planner fine-tuning (B → D) produces differences of 4.8 percentage points in SR and 7.8 percentage points in IRR. This comparison isolates the effect of learning to select the active subtask from the global instruction and visual context.

With the same fine-tuned planner, using ground-truth rather than predicted subtasks during VLA training (C → D) produces differences of 2.4 percentage points in SR and 4.7 percentage points in IRR. Ground-truth subtasks connect the local objective with the corresponding flight actions and invocation decisions during training, helping coordinate final pose adjustments with the navigation-to-manipulation handoff.

We include these ablations and the corresponding analysis in the revised manuscript (Lines 1011–1034, Table 11), with the revised text highlighted in red.

The answers to Q1 and Q2 are the same as those provided above.

AuthorsOpenReview ↗

Looking forward to your further discussion

Dear Reviewer aAeu,

Thank you again for your kind review and comments. We hope our responses and the revised paper have addressed your concerns. If any further clarification would be helpful, we would be glad to provide it. We sincerely hope that we will be able to use the remaining time to engage in further dialogue with domain experts to enhance the quality of our work. We truly appreciate your time and feedback.

Thanks again, Authors

Reviewer_Lc4mOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CrossFly, a benchmark for interaction-oriented aerial mobility across heterogeneous environments. Unlike existing aerial navigation benchmarks that mainly evaluate reaching targets, CrossFly evaluates whether UAVs can navigate through indoor, outdoor, and cross-domain environments while terminating in states suitable for subsequent interaction. The benchmark contains 15K trajectories across 23 reconstructed scenes with language instructions, 6-DoF flight states, and interaction-related annotations. The authors further propose Interaction Readiness Rate (IRR) to measure terminal-state suitability for downstream manipulation and evaluate several VLA agents together with a hierarchical CrossFly-Agent.

Critical strengths
  1. The transition from navigation to interaction is a meaningful evaluation gap. Existing UAV benchmarks often stop at target reaching, while CrossFly explicitly evaluates whether the terminal pose provides a usable starting point for downstream manipulation.

  2. The benchmark combines heterogeneous environments with long-horizon trajectories. The Indoor↔Outdoor setting is particularly useful because it introduces domain transitions within a single task rather than evaluating isolated environments.

  3. The proposed IRR metric provides a simple interface between navigation and manipulation. By evaluating terminal distance and viewpoint alignment instead of a specific manipulation policy, it offers a potentially modular way to compare navigation systems designed for downstream interaction.

Critical weaknesses
  1. The interaction readiness formulation is a valuable abstraction, but the current IRR mainly captures terminal distance and viewpoint alignment. Additional analysis on other factors relevant to aerial manipulation, such as object visibility, approach direction, or embodiment-specific constraints, would further clarify the scope of this metric.

  2. The current benchmark focuses on navigation trajectories that successfully reach interaction regions. Including more challenging cases involving recovery, replanning, or failures during navigation would make the benchmark better reflect the requirements of long-horizon autonomous interaction.

  3. The manipulation validation provides encouraging evidence that IRR correlates with downstream grasping success, but the analysis is mainly based on grasping tasks. Evaluating additional interaction types would help demonstrate that the metric is not tied to a specific manipulation skill.

  4. The CrossFly-Agent combines hierarchical planning, subtask supervision, and VLA control, making it a strong baseline for the benchmark. A more detailed ablation of these components would help understand which parts contribute most to the observed improvements.

Questions for authors

Pls seek weakness

5 comments

Thank you for your careful review and positive feedback. We provide point-to-point responses below and have also re-rendered the results with smoother camera trajectories (please see the revised PDF in the Rebuttal Revision).


W1: "...Additional analysis on other factors relevant to aerial manipulation, such as object visibility, approach direction, or embodiment-specific constraints..."

A-W1: Thanks for this constructive suggestion. We add analyses of target visibility, approach direction, and cross-embodiment applicability to clarify the scope of IRR.

(1) Visibility.

Visibility perturbation. We perturb 0%–80% of the pixels within the target's visible region in the RGB image and apply the same spatial perturbation mask to the corresponding region in the depth map. This jointly degrades the target's visual appearance and the depth information available for localization.

We then use the existing visual localization pipeline to estimate the target's 3D position from the perturbed RGB-D observations and pass this estimate to the grasping policy. The terminal state, target object, and manipulation policy remain fixed. We report 3D target localization error and grasping success to assess how disturbances to the visible target region affect downstream manipulation.

Localization and grasping outcomes. Figure 10 in the revised manuscript plots target-localization error and grasping success rate over 0%–80% perturbation coverage. Between 70% and 80% coverage, target-localization error rises sharply and grasping success declines. Such widespread corruption represents extreme perceptual degradation. These results indicate that most visibility perturbations have a limited effect on IRR’s validity as a measure of interaction readiness.

(2) Approach direction.

During the final approach for aerial manipulation, the target-relative orientation affects visual localization and subsequent interaction. Relative yaw measures the horizontal deviation between the UAV heading and the target-facing direction. We progressively increase the yaw offset from perfect alignment at a fixed terminal distance within the IRR bound, while keeping the relative height, target object, and manipulation policy fixed.

Figure 12 shows grasping SR curves against the magnitude of the yaw offset, with separate thin lines for positive and negative deviations. The vertical lines mark 47.5° for positive yaw and 50.5° for negative yaw, corresponding to the prescribed signed interval [−50.5°, 47.5°]. The success rate is highest near 0°, decreases gradually within this interval, and declines more steeply beyond the corresponding boundary.

(3) Cross-embodiment evaluation.

We clarify that, under the tested conditions, the main bottleneck is the accuracy of visually estimated target positions, rather than embodiment-specific grasping capability. UAV-target distance and yaw alignment affect visual localization accuracy, providing a geometric basis for IRR's distance and yaw criteria. We support this interpretation through the following cross-embodiment comparison.

We evaluate HI-ARM [1] and a UAV equipped with an SO100 arm [2], each using its corresponding RL manipulation policy. For each platform, half of the tested terminal states satisfy IRR and half violate it. From the same terminal states, we compare visually estimated and ground-truth target positions, both expressed in the UAV body frame, while keeping the policy and execution budget fixed.

EmbodimentGrasping success with ground-truth position (%)Grasping success with visual position estimate (%)
HI-ARM [1]99.7949.66
SO100-equipped UAV [2]99.8250.23

With ground-truth target positions, both platforms achieve over 99.7% grasping success, demonstrating that their respective policies can reliably grasp from the tested terminal states. Using visual estimates reduces success to approximately 50% for both. This consistent pattern supports visual localization as the shared bottleneck and the applicability of IRR's geometric criteria across the tested embodiments.

We include these analyses in the revised manuscript (Lines 471–485 and 899–1000), with the revised text highlighted in red.

[1] Hand-like autonomous flying robot for airborne grasping and interaction. In Nature Communications 2026.

[2] LeRobot. SO100 documentation

W2: “Including more challenging cases involving recovery, replanning, or failures during navigation would make the benchmark better reflect the requirements of long-horizon autonomous interaction.”

A-W2: Thanks. We expand CrossFly, adding recovery and replanning cases and preserving failed navigation rollouts with outcome annotations. The expanded dataset and annotations will be publicly released upon acceptance.

(1) Recovery. Following the dynamics-corruption setting in RobustNav [1], we introduce temporary motion disturbances, such as actuation drift, during flight. The additional trajectories cover execution deviations and subsequent corrective maneuvers, evaluating whether the UAV can recover and continue toward the interaction target.

(2) Replanning. Following the unexpected-obstruction setting in R2R-UNO [2], we block the intended route while preserving an alternative feasible path. These cases require the UAV to adjust its route while maintaining the original interaction goal.

(3) Navigation failures. We preserve unsuccessful rollouts and annotate their primary failure modes as collision, stagnation, incorrect termination, or timeout, supporting analysis of where long-horizon execution breaks down.

We evaluate CrossFly-Agent on Indoor↔Outdoor cases from the seen-scene split using a checkpoint trained on the original dataset, without further training on the added data. Original, recovery, and replanning conditions share the same scene, instruction, starting state, and interaction goal. The results are reported below.

SettingSR (%) ↑IRR (%) ↑
Original19.843.0
Recovery10.433.9
Replanning7.628.7

Without further training on the added data, motion disturbances reduce SR and IRR by 9.4 and 9.1 percentage points, respectively, while path obstructions reduce them by 12.2 and 14.3 percentage points. These declines show that CrossFly-Agent trained on the original demonstrations achieves lower navigation success and, among successful arrivals, reduced interaction readiness under both conditions. The expansion provides targeted data for learning and evaluating recovery and online replanning, extending CrossFly to more demanding long-horizon execution.

We include the expansion and evaluation results in the revised manuscript (Lines 1136–1160, Table 18), with the revised text highlighted in red.

[1] RobustNav: Towards Benchmarking Robustness in Embodied Navigation. In ICCV 2021.

[2] Navigating Beyond Instructions: Vision-and-Language Navigation in Obstructed Environments. In ACM MM 2024.

W3: “The manipulation validation provides encouraging evidence that IRR correlates with downstream grasping success, but the analysis is mainly based on grasping tasks. Evaluating additional interaction types would help demonstrate that the metric is not tied to a specific manipulation skill.”

A-W3: We agree that additional interaction types strengthen the validation of IRR. In response, we supplement CrossFly with diverse contact-based interaction tasks and present button pressing and circuit-breaker toggling as two representative examples in Fig. 7. In both examples, the UAV navigates to the target and invokes a dedicated RL policy to perform the interaction. Among episodes that successfully reach the target region, we apply the same IRR criteria to the terminal navigation states and compare downstream interaction success. Results are reported below.

TaskSuccess from IRR-satisfied States (%)Success from IRR-violated States (%)
Button pressing95.231.9
Circuit-breaker toggling93.422.5

Interaction success is 63.3 and 70.9 percentage points higher from IRR-satisfied states for button pressing and circuit-breaker toggling, respectively. These results extend the empirical validation of IRR beyond grasping and support its applicability across different interaction skills.

We include the task descriptions, evaluation protocol, visual examples, and results in the revised manuscript (Lines 486–505 and 1000–1004, Figure 7, Table 7), with the revised text highlighted in red. We will continue expanding the interaction tasks and corresponding data.

W4:"The CrossFly-Agent combines hierarchical planning, subtask supervision, and VLA control, making it a strong baseline for the benchmark. A more detailed ablation of these components would help understand which parts contribute most to the observed improvements."

A-W4: Thank you for this suggestion. We evaluate three configurations on the Outdoor task suite from the seen-scene split, using the same pretrained QwenPi backbone, training trajectories, action supervision, and VLA fine-tuning budget. All configurations use the same evaluation episodes as Table 2, and the full-model scores match its Outdoor results:

(1) VLA only (A). QwenPi directly predicts action chunks and manipulation-invocation signals from the global instruction, visual observations, and gripper state, without subtask conditioning. (2) Pretrained planner (B). A frozen pretrained VLM generates subtasks to condition QwenPi during both training and inference. (3) Full CrossFly-Agent (C). The VLM planner is fine-tuned on annotated subtasks (§§3.2 and 4.1, Appendix A.1). QwenPi is trained with ground-truth subtasks and uses planner-predicted subtasks at inference.

QwenPi is fine-tuned independently for each configuration.

ConfigurationHigh-level plannerVLA training subtaskVLA inference subtaskSR (%) ↑IRR (%) ↑
A. VLA only———18.140.4
B. Pretrained plannerFrozen pretrained VLMPlanner predictionPlanner prediction22.843.9
C. Full CrossFly-AgentFine-tuned VLMGround truthPlanner prediction25.550.3

A → B evaluates the effect of introducing pretrained hierarchical planning and subtask conditioning (4.7 percentage points in SR; 3.5 percentage points in IRR). B → C evaluates the additional benefit of subtask-supervised training for both the planner and VLA (2.7 percentage points in SR; 6.4 percentage points in IRR).

We include these ablations and their analysis in the revised manuscript (§D.2, Table 11).

Reviewer_KvLsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper extends semantic UAV navigation toward aerial manipulation by introducing tasks in which a UAV follows language instructions to autonomously reach a suitable position for grasping. It also introduces the Interaction Readiness Rate (IRR) to assess whether the final configuration supports subsequent manipulation. The proposed Cross Fly-Agent combines high-level subtask decomposition with a Qwen-Pi-based vision-language-action (VLA) executor and achieves the best overall performance among the evaluated navigation agents and VLA architectures. Experiments further show that cross-environment tasks remain substantially more difficult even after matching trajectory length, subtask count, and instruction length, suggesting that environment transitions introduce additional challenges.

Critical strengths
  1. The task formulation is novel, extending UAV navigation toward embodied aerial manipulation by requiring the UAV to reach configurations suitable for subsequent interaction.
  2. The dataset includes tasks spanning indoor and outdoor environments, increasing scene diversity and addressing a limitation of existing benchmarks. This setting better reflects the range of environments relevant to general embodied aerial agents.
Critical weaknesses
  1. The definition of interaction-ready configurations and the corresponding IRR metric is incomplete. The current criteria constrain distance and yaw alignment but do not account for the relative elevation angle between the UAV and the target object. A UAV approaching from a steep elevation angle may satisfy these criteria while remaining unable to grasp the object. Moreover, although the yaw threshold is empirically derived, it may reflect limitations specific to the selected reinforcement learning policy rather than provide a broadly applicable criterion for evaluating loco-manipulation.
  2. IRR is computed only over tasks in which each model successfully reaches the target, and these subsets may differ across models. Consequently, a higher IRR does not directly establish more accurate terminal pose control on the same set of tasks.
Questions for authors
  1. During trajectory collection with EGO-Planner, are the UAV heading, visual sensor viewing direction, and gripper approach direction aligned? For vision-based navigation and search, one would expect the UAV to orient toward the target object as it approaches. However, the reported IRR remains relatively low. Could the authors analyze cases that satisfy the success-rate (SR) criterion but fail the IRR criterion? Assuming that SR and IRR use the same distance threshold, these failures should not be attributable simply to declaring interaction readiness too far from the target.
  2. The agent combines autoregressive VLM reasoning with VLA action inference, and the former can introduce substantial latency. Could the authors report the system’s control frequency and inference latency, and discuss the feasibility of deployment on a physical UAV?
  3. Could the authors explain why the proposed agent architecture improves IRR? Its benefits for long-horizon task execution are apparent, but it is less clear why the design yields substantially better terminal pose control than the π-series baselines. Additional analysis would help establish the mechanism underlying this improvement.
6 comments

We sincerely thank the reviewer for the positive feedback and great suggestions! We provide point-to-point responses below and have revised the manuscript accordingly (please see the revised PDF in the Rebuttal Revision).


W1-1:"... The current criteria constrain distance and yaw alignment but do not account for the relative elevation angle between the UAV and the target object..."

A-W1-1: Sorry for not sufficiently discussing relative elevation in the original manuscript. We conduct an experimental analysis showing that relative elevation has a limited effect on manipulation success over a broad angular range. But we also introduce an elevation-aware extension of IRR, termed EIRR, to account for extreme vertical viewing angles.

(1) Experiment Design. We first denote the vertical angular deviation between the target bearing and the camera optical axis by α. Following the multi-embodiment manipulation evaluation protocol in §4.4 (Lines 450–460), we evaluate multiple aerial embodiments, each with its own manipulation policy. For each embodiment, we randomly sample terminal UAV states around fixed target objects, densely sampling different relative elevation angles while satisfying the original IRR distance and yaw criteria, and record manipulation outcomes.

(2) Analysis and EIRR Extension. Figure 11 plots success rate against relative elevation. Success rate varies little near 0° and decreases gradually as the absolute elevation deviation increases. The decline becomes sharper beyond approximately 60°. For the success-rate curve, we estimate the elevation boundaries using the same maximum-likelihood fitting procedure as in the original manuscript (§4.4), assigning equal weights to the sampled angles. The fitted interval is [−64.04°, 64.32°]. Based on this analysis, we introduce an elevation-aware extension, EIRR, which retains the original IRR distance threshold of 0.626 m and yaw interval of [−50.5°, 47.5°], and adds the corresponding elevation constraint.

To compute EIRR, we replace Xint\mathcal{X}_{\mathrm{int}} in Eq. (2) with the terminal-state set in Eq. (13).

We additionally compare IRR and EIRR for CrossFly-Agent on the seen-scene split. SR and IRR follow the Ours row in Table 2.

Task SuiteSR (%) ↑IRR (%) ↑EIRR (%) ↑
Indoor23.551.050.7
Outdoor25.550.350.1
Indoor↔Outdoor19.843.042.4

The fitted elevation interval spans approximately 128°, compared with 98° for yaw, indicating greater tolerance to elevation deviations under the tested conditions. Consistently, EIRR differs from IRR by only 0.2 to 0.6 percentage points across the three task suites. These results show that adding the elevation constraint has little effect on the reported scores in these settings. EIRR mainly supplements assessment at extreme vertical viewing angles.

The definitions, experimental results, and analysis are added to §D.1 (Lines 956–990, Figure 11, Table 10) of the revised manuscript.

W1-2: "...it may reflect limitations specific to the selected reinforcement learning policy rather than provide a broadly applicable criterion for evaluating loco-manipulation..."

A-W1-2: Thank you for raising this concern. We clarify that the main bottleneck of manipulation is the accuracy of visually estimated target positions, rather than the capability of the policies. This suggests that the IRR criteria primarily reflect geometric requirements for reliable target localization, rather than limitations specific to a particular policy. We support this interpretation through the following experimental and theoretical analyses.

(1) Experimental analysis.

We evaluate HI-ARM [1] and a UAV equipped with an SO100 arm [2], each using its corresponding RL manipulation policy. For each platform, 50% of the tested terminal states satisfy IRR, and 50% violate IRR. Starting from the same terminal states, we provide each policy with two alternative target-position inputs: one estimated from the depth image and the other obtained directly from the simulator ground truth. Both inputs represent the target’s 3D position in the UAV body frame. We keep each platform’s policy and execution budget unchanged, directly testing how target-position accuracy affects grasping outcomes.

EmbodimentGrasping success with ground-truth position (%)Grasping success with visual position estimate (%)
HI-ARM [1]99.7949.66
SO100-equipped UAV [2]99.8250.23

With accurate target positions, both platforms and their corresponding policies achieve over 99.7% success, demonstrating that these policies can execute grasping from the tested terminal states. Grasping success decreases substantially when visually estimated positions are used. The same pattern across both platforms supports inaccurate target localization as their common bottleneck, rather than a grasping limitation specific to one policy.

(2) Theoretical analysis.

We further examine how the UAV–target distance and yaw misalignment affect target localization. Existing depth-error analyses show that, for triangulation-based depth estimation, the same measurement error produces a larger range error as the target distance increases [3, 4]. For horizontally arranged observation positions, viewing the target at a larger horizontal angle can further amplify range error under the corresponding geometric assumptions [3].

Consequently, excessive target distance or yaw misalignment can reduce the accuracy of the target-position input and thereby impair grasping. Together with the position-input comparison above, this provides a localization-based explanation for the distance and yaw constraints in IRR.

We include these experiments and theoretical analyses in §4.4, Table 6, and §D.1 of the revised manuscript (Lines 471–485 and 899–936).

[1] Hand-like autonomous flying robot for airborne grasping and interaction. In Nature Communications 2026.

[2] LeRobot. SO100 documentation

[3] Multiple Fisheye Camera Calibration and Stereo Measurement Methods for Uniform Distance Errors throughout Imaging Ranges. In MVA 2021.

[4] Intel RealSense Stereoscopic Depth Cameras. In CVPR Workshops 2017.

W2: "...Consequently, a higher IRR does not directly establish more accurate terminal pose control on the same set of tasks."

A-W2: Thanks for raising this point. The original IRR is intended to assess whether a UAV's terminal state is ready for further interaction once it has successfully reached the target's vicinity. We therefore compute IRR over navigation-successful episodes, while SR separately measures navigation success. This allows IRR to characterize interaction readiness among successful arrivals without counting navigation success again alongside SR.

To address your concern about comparisons on the same set of tasks, we additionally introduce S-IRR, extending evaluation to all episodes and measuring the fraction in which the agent both reaches the target successfully and satisfies the original interaction-readiness criteria:

S-IRR=SR×IRR.\mathrm{S\text{-}IRR} = \mathrm{SR} \times \mathrm{IRR}.

We retain the original IRR to analyze interaction readiness among successful arrivals and supplement it with S-IRR, allowing all methods to be compared using the same evaluation denominator in terms of their overall rate of successful, interaction-ready arrivals.

In the revised manuscript, we define S-IRR and explain its relationship with SR and IRR in Eq. (3), and Tables 13 and 14 report SR, IRR, and S-IRR for each method to present comparisons using the same evaluation denominator.

Q1-1: "During trajectory collection with EGO-Planner, are the UAV heading, visual sensor viewing direction, and gripper approach direction aligned?"

A-Q1-1: Yes, they are aligned. During the final approach, UAV first rotates to align its heading, camera viewing direction, and gripper approach direction with the target, and then flies toward it. This produces a search-then-approach sequence, analogous to how humans first locate a target before approaching it. We clarify this procedure in §3.2 Lines 208–211, with the revised text highlighted in red.

Q1-2: "...analyze cases that satisfy the success-rate (SR) criterion but fail the IRR criterion? Assuming that SR and IRR use the same distance threshold, these failures should not be attributable simply to declaring interaction readiness too far from the target."

A-Q1-2: Thank you for this question. Most navigation SR typically measures proximity without requiring the agent to face the target[1, 2]. Following the navigation criterion in IndoorUAV [2], our SR considers an episode successful when the UAV’s final 3D distance to the target is within 2 m. In contrast, IRR imposes stricter interaction-readiness criteria, such as requiring a terminal distance to the interaction point of ≤0.626 m (§3.4). SR-successful episodes can therefore fail IRR due to insufficient proximity or excessive yaw misalignment. For example, stopping 1 m from the target satisfies SR but violates IRR's distance criterion, while stopping within 0.626 m with excessive yaw misalignment satisfies SR but violates IRR's viewpoint criterion.

This gap motivates IRR. Reaching an object’s vicinity does not necessarily prepare the UAV for interaction. IRR evaluates whether successful navigation provides the proximity and viewpoint alignment needed for manipulation, capturing the navigation-to-interaction handoff that SR alone overlooks.

In the revised manuscript, we clarify the SR distance threshold and the stricter proximity and alignment requirements of IRR in §3.4 (Lines 294–296 and 305–310).

[1] Habitat: A Platform for Embodied AI Research. In ICCV 2019.

[2] IndoorUAV: Benchmarking Vision-Language UAV Navigation in Continuous Indoor Environments. In AAAI 2026.

Comments on the paper

3 comments

Dear Reviewers,

Thank you for your efforts so far. As we are at the end of the discussion period, I would like to kindly encourage you to review the authors' rebuttals and engage in further discussions, particularly if the authors have addressed your questions or concerns.

Thank you for your time and contributions!

AC

AuthorsOpenReview ↗

Summary of Our Responses & Paper Revisions

We thank the reviewers for their constructive feedback and for recognizing CrossFly’s focus on the transition from aerial navigation to physical interaction, its diverse indoor and outdoor environments, and its systematic benchmark construction and evaluation.

We addressed each comment in our point-by-point responses and revised the manuscript accordingly, with the main changes summarized below.

Interaction Readiness Rate (IRR) Definition and Validation

  1. Clarified how the UAV aligns its heading, camera, and gripper during the final approach and why a successful arrival can still fail IRR’s stricter distance and yaw criteria (Secs. 3.2 and 3.4).
  2. Added S-IRR to compare the fraction of successful, interaction-ready arrivals over the same evaluation episodes for all methods (Sec. 3.4, Eq. 3, Appendix E, Tables 13–14).
  3. Added sensitivity experiments on target visibility, yaw, and elevation, and introduced elevation-aware IRR (EIRR) to account for extreme vertical viewing angles (Appendix D.1, Figs. 10–12, Table 10).
  4. Evaluated IRR on 50 held-out grasping trials, with 92% success from states that met the IRR criteria versus 24% from states that did not (Sec. 4.4, Table 5).
  5. Assessed how IRR applies across embodiments and interaction skills by comparing visual and ground-truth target positions on two aerial platforms and testing button pressing and circuit-breaker toggling (Sec. 4.4, Fig. 7, Tables 6–7).

CrossFly-Agent Ablations

  1. Added controlled QwenPi ablations to assess how hierarchical planning, subtask supervision, and visual history contribute to the agent’s IRR gains (Appendix D.2, Tables 11–12).

Benchmark Extensions and Deployment

  1. Expanded CrossFly with recovery and replanning evaluations under motion disturbances and blocked routes, and retained failed rollouts with failure-mode annotations (Appendix G, Table 18).
  2. Reported inference latency and control frequency, evaluated a distilled model, and discussed onboard and remote inference for physical UAV deployment (Appendix F, Tables 15–17).

The main revisions are highlighted in red in the revised manuscript. We welcome further discussion and would be glad to clarify any remaining questions.

AuthorsOpenReview ↗

Summary to AC

Dear Area Chair,

Thank you for overseeing the discussion of Submission 42435. Below, we summarize each reviewer’s main concerns, the corresponding revisions, and any follow-up.

  1. Reviewer Lc4m asked about the scope of Interaction Readiness Rate (IRR), including its applicability beyond grasping, and requested evaluations of navigation recovery, replanning, and the contributions of individual agent components. We added visibility and yaw analyses, two contact tasks beyond grasping, recovery and replanning evaluations with annotated failures, and controlled ablations. Lc4m was the only reviewer to post a follow-up to our rebuttal. The reviewer stated that most concerns had been adequately addressed and kept the original positive rating.

  2. Reviewer KvLs asked about IRR’s geometric criteria, comparisons across methods, the source of the agent’s IRR gains, and deployment feasibility. We clarified the final approach and readiness criteria, added localization and elevation experiments, introduced a shared-denominator metric (S-IRR), and reported agent ablations, runtime measurements, and distillation results. KvLs has not posted a follow-up.

  3. Reviewer aAeu asked for held-out and cross-embodiment validation of IRR and a controlled comparison with and without hierarchical planning. We added held-out grasping trials, compared target localization across multiple aerial embodiments, and ran controlled QwenPi ablations. aAeu has not posted a follow-up.

We believe the new experiments and clarifications address the concerns raised in the reviews.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.