- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes EgoPathBench, a dataset focus on first-person waypoint decision-making tasks. It includes 31K training data, 1.3K validation data and 1.1K testing data. From the results, this benchmark is quite hard that the best evaluated model can only get 28.3 and by fine-tuning Qwen3.5-4.5 on it, it boosts the performance from 3.9 to 38.9.
Critical strengths
The use of point traversability and embodied traversability are interesting. The samples inside the EgoPathBench is large. Based on the Tab. 1, the EgoPathBench has strong potential to be used for fine-tuning as it helps the Qwen3.5 to improve the performance from 3.9 to 38.9.
Critical weaknesses
- Although the author has mentioned [1] inside related work, it would be better if the author can more explicitly distinguish the difference between this paper and [1] as I found the general idea is very similar.
- I'm wondering the single image setting is insufficient for the task of ego-view navigation. In general robot navigation task, it mainly includes exploration, observation updates, memory and replanning. Using a single ego-view image, due to the viewpoint, will distort the object position in further area, which might make the planned path cannot fully reveal the capability of VLMs.
- The designed benchmark provide limited new understanding of VLM limitations. Although EgoPathBench is so hard that even the best evaluated VLM gets only 28.3, its success jointly depends on visual grounding, spatial reasoning, and route construction, which the results offer limited insight.
[1] Tim Windecker, Manthan Patel, Moritz Reuss, Richard Schwarzkopf, Cesar Cadena, Rudolf Lioutikov, Marco Hutter and Jonas Frey. "NaviTrace: Evaluating Embodied Navigation of Vision-Language Models"
Questions for authors
Similar as Weaknesses.