- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SD-LlaVA, a dual-stream architecture that can help VLMs to understand BEV maps by explicilty extracting their semantics before passing them to lanugage model. It includes two streams: visual stream uses CLIP to convert BEV into visual tokens and semantic stream uses a handcrafted parser to split the image into four regions.
Critical strengths
Treating color as semantic labels is appropriate for maps whose colors have predefined meanings and the proposed semantic stream adds only four tokens, which is compact and interpretable.
Critical weaknesses
- The number of baselines are too limited. This idea of enhancing the understanding of VLM for BEV is not novel as it has been studied by [1] [2]. Thus, the author should include more baselines as comparison for the performance.
- The experiment setting is unclear. The author does not mention the dataset size, train/test partitioning, etc, which further obscures the contribution of this paper.
[1] Tushar Choudhary, Vikrant Dewangan, Shivam Chandhok, Shubham Priyadarshan, Anushka Jain, Arun K. Singh, Siddharth Srivastava, Krishna Murthy Jatavallabhula, and K. Madhava Krishna. "Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving"
[2] Qingyao Xu, Siheng Chen, Guang Chen, Yanfeng Wang, Ya Zhang. "ChatBEV: A Visual Language Model that Understands BEV Maps"
Questions for authors
Similar as weaknesses.