- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a symbolic-first framework for driving-scene question answering. It constructs a spatiotemporal scene graph, answers supported queries through deterministic operations, and delegates unresolved queries to a retrieval augmented LLM. Experiments evaluate 5,916 frame-specific questions from ten nuScenes-mini scenes using ground-truth annotations.
Critical strengths
-
Selective symbolic execution improves accuracy by 5.48–6.64 percentage points and reduces QA runtime by 30.3–43.5% across two LLM backends, demonstrating useful gains over the LLM-only configuration.
-
Explicit graph facts and deterministic operations make the answering process inspectable and potentially useful for offline driving-data queries and analysis.
Critical weaknesses
-
The methodological novelty is limited. The framework combines established scene-graph representations, symbolic execution, and retrieval-augmented LLM reasoning. The benefit of its spatiotemporal structure is not demonstrated, since evaluation uses only frame-specific questions and lacks a frame-level graph baseline.
-
Practical applicability remains insufficiently validated. Experiments use ground-truth observations from only ten scenes, without predicted perception or robustness tests. The results therefore establish offline QA performance under ideal inputs, but leave reliability under missing objects, incorrect attributes, and tracking errors unclear.
Questions for authors
-
What benefits does the hierarchical STSG provide over independent frame-level graphs on the current benchmark?
-
What is the intended deployment scenario, and how would predicted perception outputs be integrated into the system?
-
How does performance change under missing detections, incorrect attributes, or identity-switch errors?