- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigate vision-language models can learn the search process of classical search algorithms. The authors study this problem at two levels: first, by training the model to execute the step-by-step operations of BFS and BFWS from search traces; and second, by asking the model to choose the next node to expand for GBFS and Weighted A* based only on images of candidate states, without access to explicit heuristic scores. The paper evaluates these settings across multiple PDDL planning domains and compares performance under small-scale and larger-scale training.
Critical strengths
- The research question is meaningful. Rather than focusing only on whether a model can produce a correct plan, the paper asks whether it can also learn the search process that leads to the plan. In particular, separating “executing a search algorithm” from “choosing the next node to expand” makes the experimental findings easier to interpret.
- The experimental design is reasonable. The model does not have access to explicit heuristic scores, and the order of candidate nodes is shuffled. Even under these settings, the fine-tuned model clearly outperforms both random selection and the untuned base model. This suggests that the model learns useful information from search traces about which states are more promising to explore, rather than relying on simple positional or formatting cues.
Critical weaknesses
- The BFS/BFWS execution setting relies heavily on the runtime. The runtime handles several important components, including successor generation and some algorithm-specific information. Therefore, the current results seem closer to showing that the model can produce correct operations within a constrained search interface, rather than independently implementing the full search algorithm.
- The conclusions about training data scale are still incomplete. The model fails with 512 training examples but succeeds with tens of thousands of examples, which is an interesting result. However, there are no intermediate data-scale experiments, and the small- and large-data settings are not fully matched. As a result, it is still difficult to determine the actual amount of data required or to make a clear comparison of whether BFS or BFWS is harder to learn.
Questions for authors
Could the authors provide additional experiments to address the weaknesses above
We hope our responses and additional experiments have addressed your concerns. Please let us know if you have any remaining questions or feedback. We would greatly appreciate hearing from you sufficiently ahead of the rebuttal deadline so that we have time to address any outstanding issues.