- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces RobotPath, a simulation platform for diagnosing and repairing collision-prone warehouse robot paths. RobotPath-Bench contains 20,399 execution records across five robot configurations, seven layouts, and two control regimes. LLM-RobotPath uses structured geometric inputs for collision diagnosis and generates local repairs through supervised learning and GRPO, with geometric verification and planner fallback. Experiments evaluate diagnosis, local repair, configuration transfer, and verifier agreement with simulated execution.
Critical strengths
- Sections 3.1–3.2 provide collision labels, object and segment identification, contact localization, and separate robot/layout transfer settings. Table 2 demonstrates that these metrics reveal failures hidden by collision classification alone.
- Section 4.2 independently executes 1,213 geometrically verified repairs in simulation, with 1,202 successful arrivals. This provides useful evidence for the reliability of the platform’s repair-generation and verification tools.
- Table 3 distinguishes greedy repair, best-of-eight sampling, fallback frequency, and final success. GRPO improves greedy success from 68.00% to 92.50% on the evaluated contexts. The platform–dataset–model organization is understandable.
Critical weaknesses
- Tables 1–3 compare only variants of LLM-RobotPath. No geometric collision-checking baseline, standalone planner, or alternative learned model is evaluated. Since inputs include explicit robot and obstacle geometry and the platform already implements Hybrid A* and deterministic fallback, matched comparisons are important for establishing benchmark difficulty and the learned model’s practical value.
- The novelty discussion needs a clearer relationship to established pre-execution collision checking and trajectory refinement, including FCL (Pan et al., 2012) and CHOMP (Ratliff et al., 2009). These do not invalidate the dedicated execution-annotated benchmark, but they limit the broader characterization of prior work as primarily intervening during or after execution.
- Repair evaluation is restricted to valid Simple-collection contexts and depends strongly on target adjustment. Figure 5 reports context coverage of 97.32% at a 3 m displacement allowance, 58.15% at 0.25 m, and 1.95% with no displacement. It is also unclear whether Table 3 uses predicted diagnosis throughout or whether the complete path continuation is verified after target adjustment. Section 4.2’s 99.09% result evaluates platform-generated repairs rather than the complete learned pipeline.
- Table 3 identifies its GRPO final-success result as separately selected without specifying the selection criterion. The supplied PDF also omits Appendices A–D, preventing assessment of details deferred there; their supplementary availability needs clarification.
Questions for authors
- Are standalone geometric-checking or planner results available under matched constraints?
- Does Table 3 use predicted collision information, and how was its separately reported GRPO result selected?
- After target adjustment, is continuation along the remaining original path verified and executed?
- Are Appendices A–D available separately?
3 comments
W2: Relation to prior works.
We acknowledge that FCL (Pan et al., 2012) supports pre-execution geometric collision checking, while CHOMP (Ratliff et al., 2009) enables collision-aware trajectory optimization. However, FCL focuses on geometric collision queries, whereas CHOMP focuses on trajectory optimization. Neither is designed to provide execution-grounded collision annotations or a unified benchmark for fine-grained path diagnosis and repair. In contrast, our work provides collision labels, first-collision localization, and verified local repairs derived from simulated executions, enabling systematic evaluation across different robot configurations, environment layouts, and execution regimes.
W3.1: Repair-Context Coverage and Target Displacement
The 97.32%, 58.15%, and 1.95% reported in Figure 5 for displacement bounds of 3m, 0.25m, and 0m represent valid repair-context coverage, rather than repair success rates. For example, when obstacles are located near the robot's original target, allowing target adjustment provides additional space for local obstacle avoidance. However, restricting the displacement to 0.25m or even 0m prevents some collision cases from forming valid repair contexts under our current repair protocol. This does not indicate model repair failures, but rather a reduction in the number of repair tasks that satisfy the constraints.
W3.2 & Q3 & Q2.1: Predicted Diagnosis and Path-Continuation Verification
The repair evaluation in Table 3 uses predicted collision diagnoses as the basis for path repair. In the local repair experiments, geometric verification primarily covers the path from the repair start to the adjusted local target. The optional continuation check further verifies the connection to the next original waypoint. In the end-to-end evaluation reported below, the system iteratively performs collision diagnosis and repair over the complete waypoint sequence. The resulting paths are then executed in simulation to evaluate whether they can be completed safely.
W3.3: End-to-End Evaluation of the Complete Pipeline
To evaluate the complete learned diagnosis-and-repair pipeline, we conduct an end-to-end simulation evaluation of LLM-RobotPath w/ PF on 520 paths under the Simple Collection, covering both the in-distribution test set and unseen scenarios. Using model-predicted collision information, the evaluation includes collision prediction and localization, local repair generation, geometric verification, fallback planner (FP), and execution of the complete repaired paths. The results are summarized below:
W4.1 & Q2.2: Clarification of GRPO’s FRS
The GRPO final repair success rate (FRS) reported in Table 3 corresponds to the evaluation run that achieved the highest FRS across multiple repeated evaluations of the LLM-RobotPath SFT GRPO.
W4.2 & Q4.2: Availability of Appendices
Thank you for your attention. The appendix was already included in the Supplementary Material at the time of the initial manuscript submission.
Dear Reviewer,
Thank you for your time and consideration of our submission. We are immensely grateful for your insightful reviews. As the rebuttal period is drawing to a close, please do not hesitate to reach out if you have any further questions or concerns.
Best regards,
Authors
We thank the reviewer for the valuable comments and address each point as follows:
We use the swept-footprint collision-checking procedure underlying the
Geometric Verifier(Sec. 2.2), together with controller-kinematic replay, as our non-LLM collision-diagnosis baseline. This allows us to directly evaluate geometric collision checking on the diagnosis test sets. We further introduce two categories of baselines: (1) two LLMs (GPT-6-AstraandDeepSeek-flash v3); and (2) lightweight MLP- and GNN-based models. Specifically, the MLP processes flattened geometric features, while the GNN employs multi-head self-attention over a fully connected graph of geometric entities, followed by graph-level pooling to obtain a scene representation. Both models jointly predict collision occurrence, the colliding object, the first collision segment, and the contact location. The comparison results are presented below. All metrics are percentages except MCE, which is reported in meters.For the non-LLM local-replanning baseline, we introduce the Fallback Planner (FP), a standalone deterministic planning method. SFT and GRPO refer to supervised fine-tuning and subsequent GRPO optimization, respectively. Greedy (Top-1) uses a single greedily generated candidate, while Bo8 selects from eight sampled candidates. w/ FP invokes the fallback planner when no model-generated candidate passes geometric verification. The evaluation uses 411 collision cases from the Simple test set, of which 400 have valid repair contexts. All methods in this repair comparison use ground-truth collision context, identical repair constraints, and the same geometric verification procedure.
The two tables report collision-diagnosis and local-repair performance, respectively. The geometric detector performs substantially worse than LLM-RobotPath in the Hard setting (50.52% vs. 93.66% BAcc). Although the standalone planner achieves a 97.00% local repair success rate, combining it with GRPO further improves the rate to 99.25% in this evaluation, demonstrating the complementary contributions of the learned policy and deterministic planner.