- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper formalizes the problem of when to request a plan and how to act while waiting for it and once receiving it, with a specific cost and delay as an MDP. They train RL agents to get modest improvements compared to various heuristic baselines.
Critical strengths
The problem of when to plan and when to act is an interesting one for sure, and the formulation is clean.
Critical weaknesses
The paper overstates its novelty compared to prior literature. The problem is well known and its formulation is not particularly novel. The formulation of when to plan is very similar to the framing in When2Ask (Hu et al. 2024) and Learning when to plan (Paglieri et al. 2025), the first of which is not referenced and the second one is only mentioned at a high-level. As far as I'm aware, the main novelty here comes from the delay term and acting while waiting for the requested plan to arrive, while other methods usually stop acting while waiting for the plan to be formed.
A limitation is that the planner here is a perfect oracle, which to be fair the authors mention, however the training of the planner is as important of a problem as the acting policy.
Finally, the improvements over the heuristic baselines are quite modest.
Questions for authors
Can the authors better compare and contrast their work and the actual novelty with the existing literature, both in the framing and in the baselines where possible? How will AWT work when there is no optimal planner?
3 comments
Hello reviewer CAWo,
As today is the last day of the rebuttal period, we were hoping you would let us know if we have addressed your concerns.
Thank you!
Thank you for the response and clarifications. I will review these points carefully and take them into account during the reviewer–AC discussion period.
Thank you for your feedback!
We have included When2Ask in the revised version. The key novelty of our work in comparison to these papers is the need to 1) strategically ask to plan even though environment change results in a partially stale plan and 2) learning what to do while waiting for the plan. This fills a critical gap in the literature as there are many domains (e.g., emergency response robotics, autonomous vehicles) where the world will not pause while the agent plans.
We agree that training of models for planning is also an important problem and there are interesting questions around how a learned model would interact with meta-reasoning in dynamic environments, however, we chose to first study the meta-reasoning questions which specifically relate to plan delay for this paper. We noted this in the global rebuttal, but we have modified the text to clarify that an oracle planner meant an observation of the full state, not a perfect model of transition dynamics as our previous description was imprecise. If anything, learning agent behavior models/world models would improve the plan if it provided an accurate projection.
As far as limited improvement over baselines, we don't claim that our method is the best possible solution to this problem and instead, our paper provides a formalism for studying this novel problem and a method to train an agent in it. We believe that further improvement is possible and are interested in continuing that direction.