acceptodds
Under review as a conference paper at ICLR 2027

LeadVLA: Learning Follower-Aware Route Execution for Robot Leading

Abstract

Robot leading requires executing a route while continuously adapting to a designated follower. Existing robot-leading systems often mediate follower understanding and physical route execution through explicit states, planners, or modular interfaces; directly learning this coupling through end-to-end trajectory prediction can instead entangle route execution with follower response. We introduce LeadVLA, a structured multimodal policy that preserves the route as an explicit Path reference while learning a Pace for route progress and a Residual for local follower- and scene-conditioned adaptation. To make this problem trainable and measurable, we develop LeadInfra, a configurable Unreal Engine-based closed-loop data-generation infrastructure from which we instantiate an approximately 4M-sample training corpus, and LeadBench, a standardized benchmark with 108 fully held-out closed-loop episodes. LeadVLA achieves the highest task success (39.6%) among evaluated robot policies on LeadBench while maintaining strong route fidelity and follower responsiveness, and transfers to unseen real-world environments without real-world policy fine-tuning. **Project page**: https://anonymous44paper.github.io/LeadVLA/#overview.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied AI, Robot Navigation, Human-Robot Interaction

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_FaQwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies robot leading along a given route while adapting to a language-designated follower. LeadVLA preserves the nominal route as an explicit Path reference, learns a Pace variable to regulate progress along it, and predicts a Residual for local spatial adaptation. The policy uses temporal front/rear RGB observations, language instructions, and route context, with auxiliary semantic supervision and referent-aware training. The authors also introduce LeadInfra, instantiated with approximately four million training samples, and LeadBench, comprising 108 held-out closed-loop episodes. Evaluation includes comparisons with adapted baselines, controlled ablations, and 80 real-robot trials without real-world policy fine-tuning. The reported results support improved task completion through structured route execution, with a macro-averaged leading success rate of 39.6%, while also revealing limitations in local obstacle handling and difficult multi-person interactions.

Critical strengths
  1. A clear and useful structural inductive bias, supported by informative ablations.

The separation of route geometry, progress along the route, and local spatial correction is well aligned with the task. Table 2 shows that replacing direct trajectory prediction with Path + Pace increases LSR from 0.174 to 0.354, while adding Residual further increases it to 0.396 and improves SCS. The difficulty breakdown in Table 18 is particularly informative: Residual helps under stronger route interference while slightly reducing success under Core conditions. This trade-off is consistent with its intended role and gives the design more explanatory value than an aggregate performance improvement alone.

  1. A coherent framework connecting interaction supervision, closed-loop evaluation, and physical deployment.

LeadInfra and LeadBench make follower-aware route execution trainable and measurable, including language-conditioned target attribution through counterfactual referent pairs and target reassignment. The real-robot experiments provide evidence that some of the learned behavior transfers beyond simulation without policy fine-tuning. Appendix E.1 also states that the downstream controller does not perform obstacle replanning, which helps establish that successful local adaptation originates from the predicted trajectories. Together, these elements constitute a meaningful contribution beyond the policy architecture itself.

Critical weaknesses

The physical experiments establish that transfer is possible, but success falls to 10/20 in Easy-S and 4/20 in Easy-M, and Appendix E.3 identifies contact with introduced obstacles among the observed failure modes. These results limit the strength of the practical claim when the nominal route becomes obstructed. The contribution remains meaningful as a study of follower-aware route execution, but its demonstrated capability should be distinguished from reliable obstacle handling. In particular, the behavior when a blockage cannot be resolved through local deviation remains unclear.

Questions for authors

I understand that the Residual branch is intended to support local deviations while the nominal route remains fixed. Could the authors distinguish the system’s behavior in two situations: (i) an obstacle that can be bypassed locally before rejoining the nominal route, and (ii) a blockage that makes the passage impassable and requires a different global route?

In particular, are the route-interference cases in LeadBench constructed to remain locally bypassable? When a local bypass is not possible, what is the current system’s behavior—for example, stopping or waiting, reporting that it cannot proceed, or requiring intervention from a higher-level planner or a human operator? A description of the benchmark assumptions and current implementation would help clarify the method’s operating scope and failure handling.

1 comment
AuthorsOpenReview ↗

Revised Manuscript Follow-up

Dear Reviewer,

Thank you again for your thoughtful feedback. We have uploaded our revised manuscript, with further clarifications on locally bypassable interference, the scope of Residual-based adaptation, failure modes, and future recovery mechanisms.

Revisions addressing your comments are highlighted in green, with shared revisions marked in orange for easy reference.

As the discussion period approaches its end, we would greatly appreciate it if you could revisit our rebuttal and updated manuscript. We would be happy to clarify any remaining concerns.

Thank you for your time and consideration!

Reviewer_dF4DOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LeadVLA, a Vision-Language-Action model targeted at solving specific robot leading tasks. Given current and historical camera observations (front and rear), a natural language instruction and a predetermined route, LeadVLA predicts Pace and a Path-Residual to modulatie the global route and predict robot waypoints. Training is performed in a behavior cloning fashing based on computer generated data, using the authors proposed LeadInfra toolset. The paper additionally proposes LeadBench, a benchmark for robot leading / following, incl. metrics related to follower interaction. In their experiments, the authors show that LeadVLA compares favorable to other methods in overall success rate, route preservation and follower interaction.

Critical strengths

The paper contains two critical strengths:

Firstly, it is highly appreciated that the authors contribute a new benchmark and training data generation infrastructure to the community. Based on a short browse through the related work, it seems standardized benchmarking has as of now not been present in the robot leading / following task, which makes this contribution even more important.

Secondly, the authors compare to a variety of approaches of different kinds: traditional modular approaches, hybrid approaches and end-to-end foundation model based approaches. They even provide a small-scale human expert baseline, putting results into context. Additionally, the authors also provide real, qualitative evaluation.

Critical weaknesses

A critical weakness is that quantitative comparison only happens on the self-proposed benchmark. Based on a few-minute-long survey of the subfield however this may reasonable if no standardized benchmarking methodology exists for this subfield.

Questions for authors

Apart from above-mentioned critical strengths and weaknesses, other questions come up when reading the paper:

  • For training the policy, the paper proposes causal expert supervision - an expert rule-based policy run in simulation using priviledged information. Why did the authors not evaluate using this policy with state of the art perception models, directly? And what results can the causal expert policy (with priviledged information) achieve on LeadBench (as some of the metrics have upper bounds different from 1, e.g. RF)?
  • Claim of "direct trajectory prediction may distort the nominal route" should be substantiated with quantitative or qualitative evidence in the introduction or linked appropriately towards the evaluation section.
  • How does model performance vary with different backbones?

Things to improve the paper that did not impact the score:

  • Generally, many detailed information can only be found in the appendix. For example, from the paper it is difficult to interpret the metrics. E.g. for LSR - when is it deemed unsuccessful?
  • The main graphic could be mirrored horizontally to follow the 1-2-3-4 step sequence from left to right.
  • In eq. (1), ll should be dependent on ρt\rho_t
4 comments
AuthorsOpenReview ↗

Author Response (1/3): Critical Weakness & Presentation Suggestions

Thank you for your thoughtful review and suggestions. To make our responses easier to follow, we have organized them into three comments addressing the critical weakness and presentation suggestions, Question 1, and Questions 2–3, respectively.

Response to Critical Weakness: Evaluation on a self-developed benchmark

Thank you for raising this concern. We chose to evaluate on LeadBench mainly because, to the best of our knowledge, existing benchmarks in robot guiding and human-aware navigation do not directly provide a unified closed-loop evaluation protocol for the two requirements central to our task: executing a given route and adapting to a designated follower.

Existing resources address different aspects of this problem. For example, GuideData [1] and SAIT/SA-Bench [2] provide resources for assistive guidance and spatial understanding, while SEAN 2.0 [3] and HuNavSim [4] focus on human-aware navigation in dynamic pedestrian environments. General-purpose simulation frameworks such as UnrealZoo [5] and Isaac Lab [6] offer flexible environments, but require task-specific scenarios and evaluation protocols to support robot leading. This motivated us to develop LeadBench as a dedicated benchmark for follower-aware route execution.

Since LeadBench was developed by us, we also took several steps to make the evaluation as fair and reproducible as possible. The benchmark consists of 108 held-out episodes, with environments, routes, human actors, and scene configurations disjoint from training. All methods are evaluated on the same frozen episodes and outcome metrics. The learned policies and perception-driven baselines share the same policy-visible information boundary; the privileged Causal Expert is explicitly reported as an oracle reference. We also ensure that the evaluation metrics are based on observable closed-loop task outcomes rather than LeadVLA-specific action representations, allowing different architectures to be evaluated under the same criteria. These measures aim to reduce benchmark-specific bias and support objective comparisons under a common evaluation protocol.

We hope LeadInfra, LeadBench, and LeadVLA can provide useful resources for further research on robot leading, and plan to publicly release all three along with the evaluation tools.

In the revised paper, we will expand the Related Work discussion to make the motivation for LeadBench clearer and better explain the evaluation design and its limitations.

References

[1] Hochul Hwang et al. GuideNav: User-Informed Development of a Vision-Only Robotic Navigation Assistant for Blind Travelers. ACM/IEEE International Conference on Human-Robot Interaction (HRI), 2026.

[2] ByungOk Han et al. Space-Aware Instruction Tuning: Dataset and Benchmark for Guide Dog Robots Assisting the Visually Impaired. IEEE International Conference on Robotics and Automation (ICRA), 2025.

[3] Nathan Tsoi et al. SEAN 2.0: Formalizing and Generating Social Situations for Robot Navigation. IEEE Robotics and Automation Letters (RA-L), 2022.

[4] Noé Pérez-Higueras et al. HuNavSim: A ROS 2 Human Navigation Simulator for Benchmarking Human-Aware Robot Navigation. IEEE Robotics and Automation Letters (RA-L), 2023.

[5] Fangwei Zhong et al. UnrealZoo: Enriching Photo-realistic Virtual Worlds for Embodied AI. IEEE/CVF International Conference on Computer Vision (ICCV), 2025.

[6] Mayank Mittal et al. Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning. arXiv:2511.04831, 2025.


Response to Presentation and Clarity Suggestions

Thank you for these helpful suggestions. Due to the page limit of our initial submission, we placed the detailed definitions of the LeadBench evaluation metrics in Appendix C.4. With the additional space available in the revised paper, we will bring a clearer and more intuitive explanation of these metrics into the main text to make the evaluation easier to follow.

We will also adjust the reading order of Figure 1 as you suggested. Regarding Eq. (1), you are right, and we appreciate your attention to this subtle notation detail. We will clarify how the route hint is incorporated into the language prompt and correct the corresponding notation in the revised paper.

Thank you again for reading our paper so carefully and providing these thoughtful suggestions.

AuthorsOpenReview ↗

Author Response (2/3): Question 1 (Causal Expert)

Response to Question: Causal Expert with Privileged and Visual Perception

Thank you for this suggestion. We have conducted two additional closed-loop evaluations of the Causal Expert on the complete 108-episode LeadBench, and will include their results in the revised Table 1.

We evaluate two variants:

  • Privileged Causal Expert: The original expert used for LeadInfra data generation, with access to simulator-derived follower states, scene geometry, and privileged local execution references.

  • Perception-driven Causal Expert: We use SAM 3 [1] for language-guided follower identification and tracking, and Depth Anything 3 [2] for geometric estimation, replacing privileged follower and scene information with RGB-based estimates. We retain the original causal interaction rules and local planning framework, while generating local detours from estimated geometry rather than privileged references.

Both variants are evaluated under the same frozen LeadBench protocol. The results below are equally averaged over the six evaluation conditions:

MethodLSR ↑RF ↑TRS ↑SCS ↑
Privileged Causal Expert0.9580.9410.7560.524
Perception-driven Causal Expert0.3130.9200.3070.172
LeadVLA0.3960.8530.3520.250

The Privileged Expert achieves 0.958 LSR, showing that the causal rules and local planning strategy can complete most LeadBench tasks when accurate state information is available. However, its performance drops to 0.313 LSR when relying on visual perception. The perception-driven version essentially follows a modular paradigm: explicitly estimated follower and scene states are passed to leading-specific interaction rules and a local planner. Since these rules were originally designed around accurate simulator states, errors in estimated geometry and visibility can affect state transitions and subsequent planning decisions.

In contrast, LeadVLA learns how visual observations should influence route execution through a shared multimodal representation, rather than relying on an explicit follower-state interface and fixed interaction rules. It achieves higher LSR (0.396 vs. 0.313), TRS (0.352 vs. 0.307), and SCS (0.250 vs. 0.172) than the perception-driven expert. This comparison suggests that learning the coupling between follower understanding and route execution, while retaining an explicit route reference, can be effective for follower-aware route execution.

The results also highlight an important distinction between route fidelity and task completion. The Privileged Expert achieves 0.941 RF despite its high task success, as successful local detours may require deviations from the nominal route. Conversely, our Path + Pace variant achieves 0.988 RF but only 0.354 LSR (Table 2), since staying close to the route does not guarantee successful leading. This is also why LeadBench uses multiple complementary metrics (LSR, RF, TRS, and SCS) to evaluate different aspects of robot leading, rather than relying on a single score.

We will include both Causal Expert variants in the revised Table 1, together with their implementation details and an analysis of these results.

References

[1] Nicolas Carion et al. SAM 3: Segment Anything with Concepts. International Conference on Learning Representations (ICLR), 2026.

[2] Haotong Lin et al. Depth Anything 3: Recovering the Visual Space from Any Views. International Conference on Learning Representations (ICLR), 2026.

AuthorsOpenReview ↗

Author Response (3/3): Questions 2 & 3 (Route Distortion & Backbone Generalization)

Response to Question: Evidence for Route Distortion

Thank you for this suggestion. We should have made the quantitative evidence supporting this claim more explicit.

Our structured route-execution ablation in Sec. 4.3 (Table 2) already provides quantitative evidence for this observation. For convenience, we reproduce the results below:

FormulationLSR ↑RF ↑TRS ↑SCS ↑
Direct Trajectory0.1740.6630.3000.124
Path + Pace0.3540.9880.3490.172
Path + Pace + Residual0.3960.8530.3520.250

RF (Route Fidelity) measures adherence to the nominal route, as defined in Appendix C.4.

As shown above, Direct Trajectory prediction achieves an RF of only 0.663, indicating that its executed trajectories do not consistently preserve the nominal route. In contrast, explicitly anchoring trajectory generation to Path through Pace increases RF to 0.988, while also improving LSR from 0.174 to 0.354. This substantial difference provides quantitative evidence for our claim that direct trajectory prediction may distort the nominal route, whereas explicit route anchoring helps preserve its geometry during execution. Adding Residual then allows local spatial corrections, further improving LSR to 0.396, with a corresponding reduction in RF to 0.853 as the robot gains flexibility to deviate locally from the nominal route.

In our initial submission, this connection was not sufficiently clear in the Introduction. We will explicitly link the claim to the quantitative results in Sec. 4.3 in the revised paper.


Response to Question: Generalization Across Different Backbones

Thank you for this suggestion. To examine whether LeadVLA's performance depends on the choice of VLM backbone, we additionally trained LeadVLA with Qwen3-VL-4B and Qwen2.5-VL-3B, covering different model scales and generations. All three variants use the same LeadInfra training data, Path + Pace + Residual formulation, and LeadBench evaluation protocol, with only the backbone changed.

The results on LeadBench are summarized below (six conditions equally averaged):

BackboneLSR ↑RF ↑TRS ↑SCS ↑
Qwen3-VL-2B (original)0.3960.8530.3520.250
Qwen3-VL-4B0.3750.8380.3680.281
Qwen2.5-VL-3B0.3610.8200.3600.240

We observe comparable performance across all three backbones, with LSR ranging from 0.361 to 0.396. Qwen3-VL-4B achieves slightly lower LSR but higher TRS and SCS than the original 2B model, indicating that a larger backbone does not consistently improve every evaluation metric in these experiments. Overall, these results suggest that LeadVLA's structured route-execution formulation remains effective across the evaluated backbone configurations, rather than depending on a single backbone choice.

We will include these additional results and the corresponding implementation details in the revised paper.

AuthorsOpenReview ↗

Revised Manuscript Follow-up

Dear Reviewer,

Thank you again for your valuable suggestions. We have uploaded our revised manuscript, including the additional Causal Expert comparisons, backbone generalization experiments, and clarifications to the benchmark and evaluation protocol.

Revisions addressing your comments are highlighted in purple, with shared revisions marked in orange for easy reference.

As the discussion period approaches its end, we would sincerely appreciate it if you could take another look at our rebuttal and revised manuscript. We welcome any further questions or feedback.

Thank you for your time and consideration!

Reviewer_whpPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents LeadVLA for robot leading. The robot follows a given route while adapting to a follower specified by language. LeadVLA separates trajectory prediction into a fixed Path, a learned Pace, and a learned Residual. It also uses follower-related supervision and training examples that change the designated follower. The authors introduce LeadInfra to generate about four million training samples and LeadBench with 108 held-out episodes. LeadVLA achieves the highest macro-averaged success rate among the tested robot policies. The paper also reports 80 real-world trials without real-world policy fine-tuning.

Critical strengths
  1. The action design is simple and well motivated. Keeping the route explicit lets the model adjust progress and local motion without predicting the whole route from scratch.
  2. The evaluation covers both controlled simulation and physical deployment. The paper tests held-out environments, routes, and actors. The 80 real-world trials provide useful evidence that the learned behavior can transfer without policy fine-tuning.
Critical weaknesses
  1. The real-world tests do not show whether LeadVLA’s advantage transfers. Only LeadVLA is tested on the physical robot. There is no comparison with StarVLA or Path + Pace under the same conditions. Success also falls from 16/20 in Core-S to 4/20 in Easy-M. These results show that deployment is possible, but leave the benefit of the full design under real-world interference unclear.
  2. The route inputs used in the comparison are unclear. The model receives 10 route points covering about 1.5 m, while LeadVLA’s Path Composer receives 24 points covering about 3.6 m. Does StarVLA also receive the longer route? If it only sees the shorter route, part of the gain may come from more route information rather than the action design itself.
Questions for authors
  1. How do StarVLA and Path + Pace perform in the same real-world settings? Does adding Residual improve success under physical route interference?
  2. Does StarVLA receive all 24 route points or only the 10 points in the prompt? If it receives only 10, how does it perform with the same 3.6 m route input?
  3. How were appropriate follower responses judged in the real-world trials? Were the criteria fixed beforehand, and did multiple annotators check the judgments?
4 comments
AuthorsOpenReview ↗

Author Response (1/3): Real-world Baseline Comparison

First of all, we would like to thank you for your careful review and thoughtful suggestions. To make our responses easier to follow, we have organized them into three parts: real-world baseline comparisons, route input fairness, and the evaluation protocol.

Response to Critical Weakness and Question 1: Real-world Baseline Comparison

Thank you for highlighting the need for more real-world comparisons. To address this concern, we have additionally deployed the Path + Pace ablation from Table 2 and a retrained StarVLA-24 (also evaluated on LeadBench in our second response) on the Unitree G1, following the same four experimental settings and evaluation protocol as LeadVLA, with 20 closed-loop trials per setting. StarVLA-24 uses a 24-point route input during both training and real-world deployment.

The real-world success rates are summarized below:

MethodCore-SCore-MEasy-SEasy-MOverall
StarVLA-2410/204/204/200/2018/80
Path + Pace16/2013/206/202/2037/80
LeadVLA16/2012/2010/204/2042/80

The results show a trend consistent with our simulation ablations. Under the clean Core conditions, Path + Pace and LeadVLA achieve comparable success rates (29/40 vs. 28/40). However, when local route interference is introduced in Easy, LeadVLA achieves 14/40 successful trials, compared with 8/40 for Path + Pace and 4/40 for StarVLA-24.

During the real-world experiments, we observed that adding Residual allowed the robot to make more active local spatial adjustments, including formation corrections and partial obstacle bypasses. Path + Pace maintains higher average RF (0.938 vs. 0.900), while LeadVLA achieves better task completion under interference and higher SCS (0.180 vs. 0.140). Compared with StarVLA-24, LeadVLA also achieves higher overall success (42/80 vs. 18/80) and route fidelity (0.900 vs. 0.625). These observations are consistent with the intended role of Residual in providing local spatial flexibility while retaining an explicit route reference.

Nevertheless, as you pointed out, real-world performance still drops from 16/20 in Core-S to 4/20 in Easy-M. Although Residual improves performance under local interference, reliable obstacle handling and precise formation control remain challenging in physical environments, especially when multiple people are involved. We will make these limitations and the current scope of real-world local adaptation clearer in the revised paper. Further improving the robustness of real-world follower-aware route execution remains an important direction for future work.

We will include the complete SR, RF, TRS, and SCS comparisons in the revised Table 5, together with the deployment details and additional analysis in Appendix E.

For completeness, we also provide the full real-world results below:

MethodSettingSR ↑RF ↑TRS ↑SCS ↑
StarVLA-24Core-S10/200.700.2500.150
Core-M4/200.700.2500.120
Easy-S4/200.550.2000.120
Easy-M0/200.550.2500.120
Path + PaceCore-S16/200.950.3200.180
Core-M13/200.950.3200.120
Easy-S6/200.950.2800.150
Easy-M2/200.900.2800.110
LeadVLACore-S16/200.950.3200.210
Core-M12/200.950.3050.195
Easy-S10/200.900.2850.165
Easy-M4/200.800.2800.150
AuthorsOpenReview ↗

Author Response (2/3): Route Input Fairness

Response to Question 2: Route Input Fairness

Thank you for raising this question. You are right that our original StarVLA baseline received 10 route points through the VLM prompt, while LeadVLA additionally uses a 24-point Path reference in its deterministic Composer. Our initial intention was to keep the VLM-level language and route inputs identical between the two models. However, we did not include a comparison with matched route lookahead, and we appreciate you pointing this out.

To address this, we additionally trained StarVLA-24, which receives 24 route points (approximately 3.6 m) directly through its VLM prompt. We kept the same backbone, training data, optimization budget, and evaluation protocol, changing only the route input length. The model was retrained with this input configuration rather than modified only at inference time.

We evaluated StarVLA-24 on the complete 108-episode LeadBench, with the following results (six conditions equally averaged):

MethodLSR ↑RF ↑TRS ↑SCS ↑
StarVLA-10 (original)0.1740.6630.3000.124
StarVLA-240.2150.7150.3020.136
LeadVLA0.3960.8530.3520.250

Providing a longer route context improves StarVLA's overall LSR from 0.174 to 0.215 and RF from 0.663 to 0.715. The improvements are also visible across different difficulty levels, such as Core-S (0.42 → 0.50), Easy-S (0.33 → 0.417), and Constrained-S (0.08 → 0.167), although the effects vary across conditions.

However, StarVLA-24 still achieves lower LSR and RF than LeadVLA across all six evaluation settings. The extended route input therefore provides measurable benefits, but does not close the overall performance gap (0.215 vs. 0.396 LSR; 0.715 vs. 0.853 RF). These findings suggest that the observed advantage of LeadVLA cannot be attributed simply to the difference in route lookahead.

We will clarify the route-input configurations in the revised paper and include both StarVLA-10 and StarVLA-24 in Table 1 for a more comprehensive comparison, with additional implementation details in Appendix D.1.

For completeness, we provide the full per-setting results below:

MethodSettingLSR ↑RF ↑TRS ↑SCS ↑
StarVLA-10Core-S0.420.780.360.16
Core-M0.170.640.240.07
Easy-S0.330.800.400.18
Easy-M0.000.610.260.10
Constrained-S0.080.740.280.13
Constrained-M0.040.410.270.10
StarVLA-24Core-S0.500.800.360.18
Core-M0.170.640.240.08
Easy-S0.420.810.400.18
Easy-M0.040.590.250.08
Constrained-S0.170.840.300.20
Constrained-M0.000.610.260.10
LeadVLACore-S0.750.960.350.23
Core-M0.460.890.340.25
Easy-S0.500.960.330.23
Easy-M0.130.720.410.24
Constrained-S0.500.920.350.34
Constrained-M0.040.690.320.20

All per-setting results are rounded to two decimal places, consistent with the original submission. Overall results are equally averaged across the six evaluation conditions.

AuthorsOpenReview ↗

Author Response (3/3): Real-world Evaluation Protocol

Response to Question: Real-world Evaluation Protocol

Thank you for raising this question. Our real-world evaluation followed predefined criteria, with two annotators responsible for the evaluation and cross-checking of the results.

For SR, a trial was considered successful only when the robot and designated follower both reached the destination, the robot responded appropriately to the introduced follower-interaction events, and no obstacle collision occurred. The expected responses were defined in advance based on the event types, such as slowing down or waiting when the designated follower stopped, and resuming when the follower continued. These judgments were made and cross-checked by the two annotators.

For metrics requiring interaction-event annotations, such as TRS, the event onsets and response windows were manually annotated from the recorded trials and cross-checked by the same two annotators. RF and SCS, on the other hand, were computed from SLAM-based localization and sensor depth estimates, respectively.

In our initial submission, we did not describe this evaluation procedure clearly enough. We will clarify the predefined criteria and annotation protocol in Appendix E.1 of the revised paper.

AuthorsOpenReview ↗

Revised Manuscript Follow-up

Dear Reviewer,

Thank you again for your constructive feedback. We have uploaded our revised manuscript, incorporating the additional StarVLA-24 comparison, real-world baseline experiments, and evaluation protocol clarifications discussed in our rebuttal.

Revisions addressing your comments are highlighted in blue, with shared revisions marked in orange for easy reference.

As the discussion period approaches its end, we would greatly appreciate it if you could revisit our rebuttal and revised manuscript. We would be happy to address any remaining questions.

Thank you for your time and consideration!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.