- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TrackForesight, a VLA with tracking-based visual foresight CoT. TrackForesight analyses different CoT representation, comparing point tracks against dense RGB and semantic segments and demonstrates sparse point tracks offer a favorable balance of richness, predictability, and efficiency and this CoT helps to decision making for planner. In order to improve the policy results, it intriduces the GRPO algorithm, After GRPO training, TrackForesight reaches a state-of-the-art 92.1among VLA-based planners on the NAVSIM v1 benchmark, with only 60 generated foresight. In addition, it achieves a state-of-the-art PDMS 31.7 on Navsim v2-hard. This competitive benchmark results show the effective of tracking-based CoT.
Critical strengths
- TrackForesight analyses different CoT representation, comparing point tracks against dense RGB and semantic segments and demonstrates sparse point tracks offer a favorable balance of richness, predictability, and efficiency and this CoT helps to decision making for planner.
- In order to improve the policy results, it intriduces the GRPO algorithm, After GRPO training, TrackForesight reaches a state-of-the-art 92.1among VLA-based planners on the NAVSIM v1 benchmark, with only 60 generated foresight.
Critical weaknesses
- Lack of close-loop benchmark, such as bench2drive. Navsim is a easy open loop benchmark, it's hard to validate the ability of proposal CoT. It would be better to add a close loop validation, close loop is more similar to real driving environment.
- This paper introduces the grpo to improve the policy results. But the reward is pdm score, which is the same as navsim evalution metrics. It's easy to reward hacking. And lack some analysis about the effectiveness of grpo and tracking-base CoT's quality.
Questions for authors
- It would be better to add close loop experiments, such as bench2drive. Navsim is a easy open loop benchmark, it's hard to validate the ability of proposal CoT.
- Add some analysis about grpo, such as reward hacking. The effectiveness of grpo and CoT should be analysed. Dose the pdm rewad help improve the CoT precision. Table 3 only validates the effectiveness of CoT after SFT by comparing it with the action-only setting. It would be better to also include a comparison after GRPO training.
- Add some comparison baselines, such Recogdrive, wam-diff, wam-flow and so on.
- I'm confused about "early selection, motion-ranked, smooth motion-ranked, final tokenizer" in Table 4, hoping for more explanation.
- It's necessary to add a runtime overhead analysis after introducing the tracking-based visual foresight CoT.