acceptodds
Under review as a conference paper at ICLR 2027

Learning Efficient Coverage under Hard Energy Constraints: Coordinating Feasibility Estimation and Policy Selection

Abstract

Minimizing task cost alone is insufficient for robotic missions with hard constraints: a single violation can prevent mission completion. How can robots learn to satisfy these constraints while improving task efficiency? We investigate this question through energy-constrained coverage, where a robot seeks to cover a workspace with minimal travel without exhausting its energy away from a charger. We find that high average feasibility-prediction accuracy alone is insufficient: the task policy can disproportionately select targets incorrectly predicted to be feasible. We propose Return-Aware Coverage PPO (RAC-PPO), which coordinates learned return-cost estimation, coverage-target selection, and recovery. The learned estimates support filtering candidate actions and checking the predicted energy reserve of the target proposed by the policy. When the predicted reserve becomes small, a persistent recovery controller maintains return behavior until recharging. The policy learns to advance coverage while reducing unnecessary travel and unproductive recharge cycles. Experiments on ProcTHOR-derived floorplans show that reward-driven PPO without a feasibility mechanism does not reliably satisfy the energy constraint. RAC-PPO achieves 93.92% mission success and 95.89% coverage, with only 6.08% constraint violations. Its normalized-distance score is approximately 25% lower than that of TSP-Split planning. These findings highlight the importance of coordinating feasibility learning, policy selection, and recovery when learning efficient behavior under hard constraints.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Reinforcement Learning under Hard Constraints, Energy-Constrained Coverage, Feasibility Learning, Recovery Control

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_WWHrOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The authors study the problem of coverage path planning for a robot in a navigation setting with a set of charging locations under energy constraints. The authors propose a framework around standard PPO to enforce this constraint in a way that is robust to estimation errors for underlying cost/energy predictors.

Critical strengths

The authors are also commended for a well written and clear draft that studies a very clearly defined problem and reports reasonable results. I like the crisp problem.

Critical weaknesses

That said, there are some questions that may limit the impact and significance of the findings. Primarily, while the improvement over naive PPO seems clear, it is less clear why something like "Exact-mask PPO" or alternative (heuristic or algorithmic) approaches may not suffice or even be more desirable to enforce strict adherence to safety instead of improved empirical adherence.

I am open to bumping up the score if there is satisfactory expository or empirical clarification around the presented question. The strength of the paper is that the problem is very crisp. Covering the bases to strengthen the significance argument will improve the draft.

Questions for authors

Specific comments/questions follow.

Consider related work [1] or similar literature that studies algorithmic or optimization-based views of this category of problem. In the gridworld simplification the authors impose on Procthor, can such methods work? What are the challenges of applying approximate combinatorial/MILP solvers and optimizers to known maps? I.e., beyond improving PPO, the authors should consider discussing and motivating the key challenges in the problem that necessitate the policy and then motivate the benefits of the safety constraint improvements to the learned policy.

Exact-PPO is presented as undesirable because of needing an oracle. For a given map, this oracle should be precomputation. How expensive is this? This relates also to unclear details about the dimensions of the maps (grids). Details about Exact-mask PPO are missing -- is the oracle/masking needed during training or deployment?

Consider also alternative simple/heuristic safety filters that operate strictly locally. Use any black box policy for the primary task -- coverage. This may be the PPO the authors train. From a state you can estimate/compute R*(s). Explicit computation for a single state should be a single start multi goal search on a grid if the map is known. The policy \pi proposes an action/next state s'. You can compute/estimate d_g(s,s') and R*(s'). If the current charge E_t < d_g(s,s') + R*(s'), go to nearest charger by incurring cost R*(s). This forces the black box policy to remain feasible, albeit potentially inefficient and also not complete. Will such a heuristic strategy suffice? What are challenges in such strategies? Are they computational or is some information not known?

Empirical evidence will be strongest but clear exposition or clarification about challenges and limitations in the studied setting may be helpful.

While recovery is active, the controller selects an available charging target with the largest predicted reserve. -> why will the charging target not be available?

What is the dimension of the grids the authors evaluate? Examples of the maps and settings should be included as well to help the reader.

The assumption that traversing a cell is not covering it may be too restrictive in many scenarios.

[1] Sharma, Gokarna, Ayan Dutta, and Jong-Hoon Kim. "Optimal online coverage path planning with energy constraints." In Proceedings of the 18th international conference on autonomous agents and multiagent systems, pp. 1189-1197. 2019.

2 comments

Thank you for these questions and for the recommendation paper. We will cite their work on online energy-constrained coverage and explain how it relates to our known-map benchmark.

  1. In the gridworld simplification the authors impose on Procthor, can such methods work? What are the challenges of applying approximate combinatorial/MILP solvers and optimizers to known maps?

The planning problem is to choose the order of service targets and divide them into charging tours that each fit within the battery budget. Shortest-path search gives the cost of traveling between targets. Choosing the order of visits and deciding where to recharge remain joint routing decisions. Our TSP-Split baseline demonstrates that classical planning is effective in this setting. It completed every evaluated mission without a violation. Our contribution is to examine what happens when a coverage policy makes decisions using estimated return costs under a hard energy constraint. In Table 2, the false-feasible rate was 2.1% among admitted candidates and 4.6% among PPO’s proposals. This distinction matters because feasibility estimates are used on the targets the policy actually favors. RAC-PPO combines checks on the proposed target with a recovery mode that stays active until recharging. The known-map benchmark gives us exact costs against which to measure the estimation errors and study how they interact with those decisions.

2.Exact-PPO is presented as undesirable because of needing an oracle. For a given map, this oracle should be precomputation. How expensive is this? Details about Exact-mask PPO are missing -- is the oracle/masking needed during training or deployment?

Thank you for pointing out that exact return costs can be precomputed. For each static map, charger layout, and motion model, a single multi-source Dijkstra search initialized at all chargers computes the minimum return cost from every traversable cell. With a binary heap, the search takes O((|V| + |E|) log |V|) time, and subsequent decisions can retrieve these costs from a lookup table. The manuscript does not report measured preprocessing times. For RAC-PPO, we use exact return costs to fit the return-cost estimator. During evaluation, candidate admission, proposal checking, and recovery all use the estimator’s predictions. We compute exact reserves after action selection solely to assess constraint violations. The Exact-mask PPO baseline uses precomputed exact return costs to exclude infeasible targets during evaluation. The submitted manuscript does not specify whether this baseline also used the exact mask during training. We will clarify the evaluation-time use of exact costs for both methods in the revised manuscript.

  1. Will such a heuristic strategy suffice? What are challenges in such strategies? Are they computational or is some information not known?

We agree that, on our known grid, the proposed filter preserves return feasibility if the robot starts in a return-feasible state and uses exact, precomputed costs. Coverage can still stall if the policy repeatedly proposes an infeasible target after recharging. RAC-PPO uses a fixed reserve check with a learned return-cost estimate. The known grid lets us measure how estimation errors affect target selection and recovery. Our longer-term goal is to learn feasibility from experience when interacting constraints make exact checks difficult. For example, recharging may keep the robot within its energy budget but leave too little time to meet a deadline.

  1. Why will the charging target not be available?

While coverage is incomplete, the robot’s current charger is excluded from the candidate set. In a single-charger map, this can leave the set empty. Recovery selects the eligible charger with the largest predicted reserve. If none is eligible, it selects the candidate minimizing outbound cost plus predicted return cost. RAC-PPO achieved 93.92% mission success, with violations in 6.08% of episodes overall and 18.23% at the tightest budget. We will clarify the fallback in Algorithm 1.

  1. What is the dimension of the grids the authors evaluate? The floorplans are discretized into 0.5 m × 0.5 m cells, and we use a robot radius of 0.2 m when determining traversability. The manuscript gives the cell resolution but omits the grid dimensions and the distribution of traversable-cell counts. We will report these statistics and include representative maps showing the charger placements and motion settings. Future evaluations will consider a broader range of map sizes.

  2. The assumption that traversing a cell is not covering it may be too restrictive in many scenarios.

We agree that pass-through coverage is appropriate for many applications. Our benchmark marks a cell as covered when the robot reaches it as a selected target, as in tasks that require a measurement at a designated location. We will clarify this assumption and discuss its implications for route choice and travel distance.

I thank the authors for their careful response.

I will repeat my original comment that the problem is very crisp. Its simplicity merits a clear discussion about the aspects that make it interesting.

Going by the numbering in the author response 1-6.

I will group 1, 3 together. The focus on "using estimated return costs" is weakened by the observation in 3 that "exact, precomputed costs" are applicable. The mitigating argument is that "Coverage can still stall". While this does sound somewhat intuitive, this needs some evidence and clear motivation. For example, a probabilistic policy that has support over enough of the action space should return a different action eventually. I think the point to "learn feasibility from experience when interacting constraints make exact checks difficult" may prove interesting but the presented domain does not appear to have this characteristic.

The authors mention "recharging may keep the robot within its energy budget but leave too little time to meet a deadline." Nitpick: I do not agree with this point. Given the standard optimization formulation, charging can be modeled as a constraint. So if the robot charges, it should not, for example, always charge fully. The time spent to charge should be recovered from the optimization for the primary task. This constrained optimization problem should have an optima. If the optimal budget allocation to charging and coverage fails to meet a time deadline this means that that deadline is infeasible given the specification of the optimization. For example, if all coverage locations are farther away than a safe return range.

The response point 4 are handcrafted assumptions within the studied setting. In future drafts, these should be accompanied by some practical motivation.

Overall, a fair set of discussions and comparisons with classical, algorithmic, optimization-based methods, or filters on black box policies will benefit the communication of both the methodological and practical design choices behind designing robotic systems that need to solve navigation-based tasks with charging budget constraints. At this point, I think this may require a fresh resubmission that takes into account all reviewers' comments.

Reviewer_ePJwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RAC-PPO for servicing a known grid map while retaining enough energy to reach a charger. A CNN predicts return costs, an empirical calibration offset reduces underestimation, and a PPO policy selects targets admitted by a feasibility filter. Additional reserve checks can initiate recovery until recharging. The paper reports 93.92 percent mission success and 6.08 percent violation episodes, with approximately 25 percent lower normalized distance than TSP-Split on each method’s successful missions. It also examines how policy selection changes prediction-error exposure and ablates recovery, proposal checking, and recharge-cycle penalties.

Critical strengths
  • The analysis highlights a useful distinction between predictor accuracy and policy-level risk - The false-feasible rate increases from 2.1 percent among admitted candidates to 4.6 percent among PPO-selected proposals (Table 2). This suggests that policy selection concentrates exposure to estimation errors and motivates evaluating the proposals themselves. The paper appropriately distinguishes these proposals from executed violations because the controller may intervene (Appendix B.4). Clarifying the aggregation issue raised below would strengthen this finding.
  • The recovery ablations examine safety and completion together - Conservative stopping produces few violations but only 12.24 percent success (Table 8). Persistent recovery achieves 63.27 percent success, compared with 29.59 percent for reactive recovery, while reducing violation episodes from 53.06 to 7.14 percent (Table 8). These results illustrate why low violation rates alone are insufficient and provide useful evidence about the recovery design.
Critical weaknesses
  • The efficiency and generalization claims need stronger evaluation - Normalized distance is computed separately over each method’s successful missions (Section 5.2, lines 396 to 399). Since TSP-Split succeeds on every scenario and RAC-PPO succeeds on 93.92 percent (Table 3), the reported distance reduction does not establish the same improvement on matched scenarios. Exact-mask PPO achieves 100 percent success with no violations (Table 1), but its travel distance is not reported, leaving a particularly relevant efficiency comparison incomplete (Tables 1 and 3). Furthermore, the 24 maps underlying the headline result were used during method development (Appendix C.3, lines 853 to 860), limiting the evidence for generalization to untouched maps.
  • The methodological contribution and treatment of recovery costs need clearer justification - [1] already combines PPO, map observations, repeated recharging, and return-feasibility action masking (Section IV-A), and provides a greedy comparison method (Section V-A). RAC-PPO studies learned rather than exact return estimates (Section 4.1), but comparison with an adapted version of this prior approach would better establish the additional value of its coordination mechanisms. There is also a direct concern about the efficiency objective: PPO return targets exclude rewards from intervening recovery actions (Section 4.4, lines 345 to 352), whereas the objective counts all mission travel (Equation 3). The paper should explain how earlier policy choices receive credit for recovery costs they cause, or justify this alternative training objective empirically.

References

[1] Learning to Recharge: UAV Coverage Path Planning through Deep Reinforcement Learning, arXiv'23

Questions for authors
  • Can you report matched-scenario travel distances for RAC-PPO, TSP-Split, and Exact-mask PPO, addressing the differing success sets and missing oracle-policy distance (Tables 1 and 3)? Can you also evaluate on maps unused during development and clarify how that evaluation relates to the existing settings (Appendices C.1 through C.3)?
  • How are recovery travel costs and terminal outcomes credited to preceding policy actions when recovery rewards are omitted from return targets (Section 4.4, lines 345 to 352)?
  • Why does mean predicted reserve decrease from 0.184 for all candidates to 0.121 after admission (Table 2)? Removing negative-reserve candidates should increase the mean if observations and weighting are otherwise identical. Please specify the state populations and aggregation weights.
1 comment

Thank you for these questions and for pointing us to Learning to Recharge: UAV Coverage Path Planning through Deep Reinforcement Learning. This is a relevant reference for our discussion of learned coverage and recharging decisions. We will cite it in the revised manuscript and explain how it relates to our work.

  1. Can you report matched-scenario travel distances for RAC-PPO, TSP-Split, and Exact-mask PPO, addressing the differing success sets and missing oracle-policy distance (Tables 1 and 3)? Can you also evaluate on maps unused during development and clarify how that evaluation relates to the existing settings (Appendices C.1 through C.3)?

Thank you for suggesting these evaluations. On matched successful scenarios, RAC-PPO traveled 1.68 m/cell, about 75% less than Exact-mask PPO (6.83 m/cell) and about 9% more than TSP-Split (1.54 m/cell). Both baselines completed all 288 scenarios, and the two RAC-PPO policies completed 269 and 272, respectively. For each RAC-PPO policy, we computed normalized travel distance for all three methods on the same scenarios that the RAC-PPO policy completed. We ran the matched comparison on the 288-scenario panel of Appendix C.3: 24 validation maps, each with two charger placements, three battery budgets, and two motion patterns. These maps were used during method development, so this evaluation varies task conditions on reused map geometry and does not test held-out maps. Appendices C.1 and C.2 report component ablations on 98 validation scenarios drawn from the same validation split.

  1. How are recovery travel costs and terminal outcomes credited to preceding policy actions when recovery rewards are omitted from return targets (Section 4.4, lines 345 to 352)?

As specified in Section 4.4, PPO updates use only transitions in which the policy’s proposed target is executed. Travel incurred during controller-selected recovery steps is counted in the episode’s total travel distance, but the corresponding rewards are omitted from the PPO return targets. We do not assign those rewards to the preceding policy action. Likewise, if the mission terminates during a recovery step, the terminal reward from that step is not assigned to the preceding action. A terminal outcome on an executed PPO step, by contrast, contributes through that step’s reward. If coverage resumes after recharging, rewards from later retained policy transitions can contribute to the return target of an earlier policy action within the same episode. Rewards from the intervening recovery steps are excluded. The rewards from those retained decisions can enter the earlier return target within the same episode, so recovery can affect learning indirectly. We will clarify this distinction in Section 4.4. The training return used for PPO updates omits recovery rewards, whereas the reported episode travel distance includes recovery travel.

  1. Why does mean predicted reserve decrease from 0.184 for all candidates to 0.121 after admission (Table 2)? Removing negative-reserve candidates should increase the mean if observations and weighting are otherwise identical. Please specify the state populations and aggregation weights.

We agree that removing negative-reserve candidates from a fixed pool, with the same weights for the retained candidates, cannot lower the mean. The reported decrease from 0.184 to 0.121 therefore cannot be explained by admission alone. The manuscript does not specify the state populations or aggregation weights for the two rows in enough detail to resolve the discrepancy. We will verify the aggregation used for Table 2, state which decision states are included and how candidates and states are weighted, and correct the reported values if the audit identifies an error.

Reviewer_t3gtOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a coverage policy for robots in a grid-world that need to repeatedly visit various grid cells, while also returning regularly to a charging station. The policy is trained with PPO. The key novelty is a "return-aware" component, where a learned estimator predicts how many steps will be required to return to the charging station. Candidate actions (goals to navigate to) are filtered based on this learned estimate.

Note that "return" in this paper refers to "returning to the charger" not "cumulative reward" as is typical in reinforcement learning. This confused me for a while.

Simulation experiments compare the proposed approach against plain PPO without a learned estimator, PPO with a perfect ground truth estimator, and a classical approach (TSP-split).

Critical strengths

The paper is generally clear and easy to follow.

Critical weaknesses

The contribution is framed around "hard constraints: a single violation can prevent mission completion," but the proposed approach is very much a soft constraint. Underestimates in the learned return estimator do produce failures; this is easily seen in the main results tables.

The problem is contrived. An "action" is to select a goal cell, from which we assume the robot is able to plan a shortest path. But if the robot can plan a path a priori to any cell, there is no need for a learned return estimator at all! Simply plan a path from a goal cell back to a charger, and check whether the battery has enough energy left.

The problem is extremely narrow. Robot navigation in a grid world is a very old problem, and doesn't capture much of the complexity of controlling real robotic systems.

The contribution is extremely problem-specific. I was not able to glean any new insights that would be applicable to problems outside this particular (and fairly contrived) problem.

The baselines are extremely limited. With robot planniing in a grid world being such an old problem, there are many possibile algorithms to choose from, including non-learning-based baselines. The advantage of any learned component, much less a learned return estimator, is not compellingly illustrated.

Questions for authors

The terminology "return aware" is confusing, since "return" has a specific meaning in RL. Is there another name we can use for this algorithm?

What is the role of PPO? It seems that any RL algorithm - or indeed any policy, learned or not - would be compatible with the proposed approach.

What knowledge about the graph/grid world geometry is assumed known? All of it? If so, why not use classical graph planning algorithms, which tend to be quite performant and scalable, rather than learning methods?

Algorithm 1 is quite detailed, and a web of references to elsewhere in the paper make it difficult to follow. Could it be made more concise, direct, and self-contained?

How sensitive is the coverage policy to the coefficients λ\lambda?

Can you provide baselines against more of the constrained RL and similar algorithms from the related work section?

2 comments

Thank you for these comments. We appreciate the opportunity to clarify the role of PPO, the assumptions of the benchmark, and the scope of the evaluation.

  1. The terminology "return aware" is confusing, since "return" has a specific meaning in RL. Is there another name we can use for this algorithm?

We agree that “return-aware” is ambiguous because “return” conventionally refers to cumulative reward in reinforcement learning. We propose renaming the method Reserve-Aware Coverage PPO, retaining the acronym RAC-PPO. Here, reserve means the predicted battery energy remaining after visiting a candidate target and then traveling to a charger. We will define this term when we introduce the method.

  1. What is the role of PPO? It seems that any RL algorithm - or indeed any policy, learned or not - would be compatible with the proposed approach.

PPO trains the policy that ranks admitted targets and proposes the next one, with the objective of completing coverage while limiting total travel. Candidate admission, the reserve checks, and persistent recovery are implemented separately from the policy update. We agree that the controller could use another learned policy or a heuristic that proposes a target from the admitted set. The choice of policy can still affect the behavior of the combined system. In Table 2, for example, 2.1% of admitted candidates were incorrectly predicted to be feasible, compared with 4.6% of PPO’s proposals. These are proposal statistics, measured before any intervention by the controller. We will clarify that PPO is the target-selection policy evaluated in this study and that the results characterize its interaction with the feasibility checks.

  1. What knowledge about the graph/grid world geometry is assumed known? All of it? If so, why not use classical graph planning algorithms, which tend to be quite performant and scalable, rather than learning methods?

Thank you for raising this point. Yes, we assume that the full static grid graph is known, including the traversable cells, permitted moves and their costs, and charger locations. We use shortest paths to reach selected targets, and exact return costs can be precomputed. TSP-Split completed every evaluated mission without violations. Its worst-case approximation bound for the number of charging tours is O(B/(B − 2D)), where B is the battery budget, and D is the farthest shortest-path distance from the charger. The bound grows as B approaches 2D from above. Our goal is to understand whether a robot can learn to choose coverage targets that limit total travel when feasibility decisions rely on estimated costs under a hard energy constraint. The known map gives us exact costs against which to measure those estimates and examine how their errors interact with the policy’s target choices. We view this single-constraint setting as a starting point. We use the known map to measure return-cost estimation errors and study how they affect a learned policy’s target choices under a hard energy constraint. This benchmark is a starting point for studying interacting constraints, such as energy limits and task deadlines, which can require problem-specific planning rules.

  1. Algorithm 1 is quite detailed, and a web of references to elsewhere in the paper makes it difficult to follow. Could it be made more concise, direct, and self-contained?

We will define predicted reserve directly in the pseudocode as remaining energy minus outbound travel cost and predicted return cost. We will state the candidate filter, recovery triggers, and recovery target rule there. Recovery stays active until recharging. Tie-breaking details will go in the appendix.

  1. How sensitive is the coverage policy to the coefficients?

In Table 9, adding the recharge-cycle penalty at a weight of 0.05 increased completed missions from 64 to 70 out of 98 without proposal checking and from 64 to 67 with proposal checking. Violation counts stayed at five and four, respectively. This ablation measures the effect of enabling the penalty at 0.05. Sensitivity to its magnitude and to other reward weights remains open.

  1. Can you provide baselines against more of the constrained RL and similar algorithms from the related work section?

Yes, we agree that comparisons with constrained RL and shielding methods would help position RAC-PPO. Plain PPO, Exact-mask PPO, and TSP-Split provide references for reward-based learning, exact feasibility masking, and classical planning, respectively. We will acknowledge the absence of direct comparisons with the suggested methods in the revision.

Thanks to the authors for their detailed response. However, the revisions described above would not be sufficient for me to change my score. The two main issues are 1) claiming hard constraint satisfaction while not achieving this in practice and 2) the limited scope and complexity of the problem at hand. This simple problem domain prevents any meaningful exploration of "interacting constraints" in my opinion.

While the topic and approach may have merit, I believe it requires a complete redesign, a more complex set of problems (that are not trivially solvable with classical methods), and a complete manuscript rewrite.

Reviewer_dL7aOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies energy-constrained coverage, where a robot must service all required locations and return to a charger while minimizing travel. The proposed RAC-PPO combines a separately trained return-cost model, candidate filtering, a PPO target-selection policy, reserve checks, and a persistent recovery controller. Experiments on ProcTHOR-derived floorplans examine mission completion, constraint violations, prediction errors among policy-selected targets, component contributions, and travel efficiency relative to TSP-Split.

Critical strengths
  1. The paper studies a meaningful issue in constrained decision making: prediction quality across candidate actions may not reflect the risk of the actions a policy actually selects. Table 2 reports a higher false-feasible rate among PPO proposals than among admitted candidates, alongside smaller predicted reserves. This is a useful observation that motivates evaluating feasibility estimates under the policy’s actual choices.

  2. The method has a clear and coherent design, and the experiments provide useful evidence about recovery and failure conditions. Table 8 reports substantially more completed missions and fewer violations with persistent recovery than with one-step recovery. The energy-budget breakdown also shows that the remaining violations are concentrated in the tightest setting. These results help explain both the benefits and the current limits of the approach.

Critical weaknesses
  1. The added value of proposal checking over a simpler action mask remains unclear. Table 9 provides some positive evidence, while mission success remains at 64/98. A natural alternative, however, is to exclude targets below the proposal reserve threshold before PPO selects an action, using the same return estimator and recovery controller. This would allow the policy to choose another target with sufficient predicted reserve when one is available. The current ablations do not establish whether proposal checking offers a better balance of safety, completion, and travel efficiency than this simpler alternative. This comparison is important for assessing the methodological contribution.

  2. The main efficiency result needs a more comparable evaluation. Table 3 computes normalized distance on each method’s own successful missions. RAC-PPO therefore excludes its failed scenarios, whereas TSP-Split includes all scenarios. Since RAC-PPO’s failures occur at the tightest energy budget, this difference may affect the reported distance advantage of approximately 25%. Results on common successful scenarios would help establish its magnitude. The scope of the main performance results also needs clarification. The paper includes validation experiments, but the 24 maps underlying the main results were used during method development. The relationship between this evaluation and the 98-scenario component study, where RAC-PPO achieves 68.37% success, is not sufficiently clear. These issues make it difficult to assess the magnitude of the efficiency advantage and how well the reported performance generalizes beyond the development settings.

Questions for authors
  1. How does proposal checking compare with directly excluding targets whose normalized predicted reserve is below (τπ\tau_\pi), while keeping the return estimator, set-level check, and persistent recovery controller unchanged?

  2. Could the authors report a paired distance comparison between RAC-PPO and TSP-Split on scenarios where both methods succeed? A breakdown by energy budget would also help interpret the reported efficiency advantage.

  3. What accounts for the difference between RAC-PPO’s success rates in the main evaluation and Table 9? Please clarify the differences in map selection, energy budgets, and evaluated policies, as well as which maps were kept separate from method development and model selection.

1 comment

Thank you for these questions. They help us clarify how the reserve checks affect the robot’s decisions and how to interpret the results across the evaluation settings.

  1. How does proposal checking compare with directly excluding targets whose normalized predicted reserve is below (), while keeping the return estimator, set-level check, and persistent recovery controller unchanged?

Thank you for suggesting this comparison. Proposal checking matched or exceeded direct exclusion in the RAC-PPO configuration: in both training seeds, it achieved higher success and fewer violating episodes. We keep proposal checking because the full method gives the fewest violating episodes of the four variants tested. We implemented the suggested alternative: the policy's selection is restricted to admitted targets with m̂_t(a)/B ≥ τ_π (= 0.01, not tuned), and the proposal check is removed; all other components are unchanged. Both variants were retrained from the same initialization (two seeds) and compared per scenario on the 98 scenarios of Table 9.

Methodλ_cycleSuccessViolating episodesRecovery entries
Proposal checking (RAC-PPO)0.05134/1968/19620123
Direct exclusion0.05131/19611/1967362
Proposal checking0128/1969/19616599
Direct exclusion0144/19610/1967887

With the cycle penalty, direct exclusion adds three violating episodes and removes none, all at β = 1.1; the margins are small (paired McNemar p = 0.55 for success, 0.25 for violations). Without the penalty, direct exclusion completes more missions (p = 0.0004), mainly on two-charger layouts in one seed. Direct exclusion enters recovery 51–69% less often. We keep proposal checking because the full method gives the fewest violating episodes of the four variants.

  1. Could the authors report a paired distance comparison between RAC-PPO and TSP-Split on scenarios where both methods succeed? A breakdown by energy budget would also help interpret the reported efficiency advantage.

We agree that a paired comparison would better isolate the difference in travel distance. Table 3 reports normalized distance over each method’s own successful missions, yielding 1.16 for RAC-PPO and 1.54 for TSP-Split. Because TSP-Split completed every scenario, the paired comparison would restrict its distance calculation to the scenarios completed by each RAC-PPO policy. The published aggregates do not provide TSP-Split’s distance on those subsets. We will clarify that the approximately 25% difference refers to the aggregate scores in Table 3. Table 10 provides context for interpreting the budget breakdown. Both RAC-PPO policies completed every scenario at β =1.5 and 2.0, whereas their average success rate was 81.77% at β =1.1. This will allow us to determine whether the reported distance improvement remains consistent when both methods are evaluated on the same scenarios. We will update the reported improvement in the paper if the paired results differ from the current estimate.

  1. What accounts for the difference between RAC-PPO’s success rates in the main evaluation and Table 9? Please clarify the differences in map selection, energy budgets, and evaluated policies, as well as which maps were kept separate from method development and model selection.

Thank you for pointing out that the manuscript could explain the relationship between these two evaluations more clearly. The main evaluation averages the results of two trained RAC-PPO policies evaluated on the same 288 scenarios per policy. As described in Appendix C.3, these scenarios combine 24 validation maps, two charger placements, two motion patterns, and three equally represented battery multipliers. Table 10 shows that the mean success rate across the two policies was 81.77% at β = 1.1 and 100% at both β = 1.5 and β = 2.0. The average across the three budgets was 93.92%. Table 9 reports a separate component ablation in which four PPO variants were evaluated on a common set of 98 scenarios. The RAC-PPO variant completed 67 of the 98 missions (68.37%). The budget breakdown in Table 10 explains why the overall success rate in the main evaluation is higher than the success rate at its tightest budget. The available results do not establish how much of the difference between the two evaluations is attributable to map selection, energy budgets, or the policies evaluated. The 24 validation maps in the main evaluation were used during method development. The 93.92% result therefore characterizes performance across scenario configurations on those maps. The manuscript does not specify the overlap between the Table 9 maps and those 24 maps, the budget settings or RAC-PPO checkpoint used for Table 9, or which maps, if any, were held out from both method development and model selection.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.