- Rating
- 2
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces AnchorReasoning, a driving dataset built on WOD-E2E with annotations for decision-critical elements, visual grounding, impact ranking, element-level implications, driving rationales, and action/trajectory planning. It also proposes a curriculum supervised fine-tuning strategy and evaluates the resulting supervision across eight VLM/VLA backbones. The main contribution is a structured annotation resource connecting object-level evidence with driving explanations and planning.
Critical strengths
1.The annotation pipeline includes substantial human involvement, including critical-element selection, cross-review, and correction of generated implications, rationales, and action plans.
2.Experiments cover a broad set of backbones and show improvements in both trajectory prediction and the official RFS metrics.
3.The grounding ablation in Table 4 provides useful evidence that grounded supervision improves planning beyond the corresponding nongrounded CoT variant.
Critical weaknesses
1.Table 4 shows relatively modest RFS improvements for some models, such as Qwen3-VL (+0.10) and Cosmos-R2-2B (+0.21). Could the authors provide one or two additional experiments, such as challenging-scenario evaluations or impact-ranking ablations, to further demonstrate the benefits of visual grounding?
2.The proposed annotation pipeline appears similar to DriveLM, nuReasoning. In particular, Table 1 marks DriveLM as lacking visual grounding, although it uses coordinate-based object references, while nuReasoning also incorporates spatial grounding and decision reasoning. Could the authors clarify the key differences, especially how AnchorReasoning uniquely connects visual grounding, object-level reasoning, and trajectory planning?
3.Appendix A.1 states that 456 validation frames were annotated, but their representativeness remains unclear. Could the authors provide a comparison of training and validation distributions, particularly regarding critical object categories, driving scenarios, and action types?
Questions for authors
1.How many of the 456 validation frames contain rare events or interactions involving multiple critical elements, and what is the difficulty distribution of these scenarios?
2.Table 1 marks DriveLM as lacking visual grounding, although it uses coordinate-based object references in perception, prediction, and planning QA. nuReasoning also includes spatial grounding and decision reasoning. Rather than treating visual grounding as a binary distinction, the authors should clarify what AnchorReasoning adds beyond these datasets, particularly in how grounding, object-level reasoning, and trajectory planning are connected.
5 comments
Response to Suggestions for the Rebuttal
Dear Reviewer Y2uP,
Thank you for the clarification, and no apology is needed. As suggested, we focused on the first two weaknesses and have revised the paper accordingly. Our detailed responses to each point are provided below, together with brief answers to the remaining questions.
Best regards,
The Authors of AnchorReasoning
Response to Reviewer Y2uP part 1
We thank the reviewer for the constructive questions on the benefit of visual grounding, the relation to prior datasets, and the composition of our validation set. We have added challenging-scenario evaluations and distribution analyses in response, as detailed below.
W1: further evidence for the benefit of visual grounding
Visual grounding brings the most benefit in challenging long-tail scenes, where the driving decision depends on identifying the right decision-critical elements. The modest overall gain averages these scenes with simpler ones, where the correct action is often evident without grounding. On five challenging subsets of the validation set, VG-CoT improves RFS over its no-grounding ablation by +0.22 to +0.43 on average across three backbones.
Each subset targets a situation in which finding and interpreting the right elements matters: longitudinal transitions (the ego vehicle starts, stops, or decelerates); rare element states or intentions (e.g., turning, lane changing, backing, pulling out); multiple element categories (critical elements from at least two categories); the highest difficulty level (multiple critical elements, rare content, and a longitudinal transition together); and single-lane maneuvers (an official WOD-E2E cluster). We compare VG-CoT with its no-grounding ablation (text CoT without visual grounding) under the same training recipe on Alpamayo-1.5, Qwen3-VL, and Cosmos-R2-2B.
RFS improvement of VG-CoT over the no-grounding ablation.
| Scenario | Frames | Alpamayo-1.5 | Qwen3-VL | Cosmos-R2-2B | Mean RFS [95% CI] |
|---|---|---|---|---|---|
| Longitudinal transitions | 224 | +0.21 | +0.20 | +0.42 | +0.28 [0.11, 0.44] |
| Rare element states/intentions | 100 | +0.34 | +0.32 | +0.20 | +0.29 [0.08, 0.52] |
| Multiple element categories | 112 | +0.17 | +0.25 | +0.23 | +0.22 [0.05, 0.41] |
| Highest difficulty level | 46 | +0.24 | +0.47 | +0.36 | +0.36 [0.08, 0.69] |
| Single-lane maneuvers | 34 | +0.44 | +0.53 | +0.33 | +0.43 [0.10, 0.80] |
The benefit also grows with scene difficulty. Counting three difficulty factors (multiple critical elements, rare content, longitudinal transition), the mean RFS improvement rises from $-$0.04 for frames with none of them to +0.36 for frames with all three. In simple scenes, the correct action is often to keep the current motion, which the no-grounding model already predicts. Some baselines also already reach the maximum rater score, and grounding sometimes yields more cautious trajectories that raters score lower when they prefer continued motion.
These gains are meaningful for WOD-E2E. Published improvements include +0.08 RFS from WOD-E2E fine-tuning, +0.08 from multi-camera inputs, and +0.09 from test-time scaling (Xu et al., Table 3), and +0.08 from RL post-training (Poutine-Base 7.91 vs. Poutine 7.99; Rowe et al.). The gaps between AutoVLA and HMVLM and between HMVLM and Poutine are +0.18 and +0.25 (Xu et al., Table 2). These results use different splits and are not directly comparable, but they put our +0.22 to +0.43 in context. The results are added to Appendix C.8 (highlighted in red).
Reference:
Xu et al. (2026). WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarios. CVPR 2026.
Rowe et al. (2025). Poutine: Vision-Language-Trajectory Pre-Training and Reinforcement Learning Post-Training Enable Robust End-to-End Autonomous Driving. arXiv:2506.11234.
Suggestions for the Rebuttal
I apologize for revising some of my review comments, as a few of my previous questions may have been difficult to address. I hope the authors can focus on the first two weaknesses during the rebuttal. If these concerns are adequately addressed, I would be happy to raise my score.