- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SelectionVLM augments a VLM with a pixel-aligned CAD selection buffer, allowing it to associate visible entities with API-addressable pick IDs and structured metadata. Experiments on Fusion 360 Gallery assemblies show improvements over color-keyed, bounding-box, and Set-of-Mark baselines on entity identification and metadata-based reasoning tasks.
Critical strengths
The central idea is practical and well motivated. Using the CAD selection buffer directly connects visual entities with programmatic references and associated metadata, providing a useful interface for future CAD agents.
The method consistently improves performance across the evaluated task families. The assembly-level split and architectural ablations provide encouraging evidence that the approach can generalize to unseen assemblies with a relatively lightweight additional encoder.
Critical weaknesses
The baseline comparisons could be better controlled. SelectionVLM is task-trained, whereas the bounding-box and Set-of-Mark baselines are not. Fine-tuning the color-keyed baseline decreases performance; the authors attribute this to distribution shifts, but the explanation is not experimentally validated. Comparably trained alternatives would help establish how much improvement comes specifically from the selection-buffer modality.
The quantitative evaluation focuses on synthetic questions at the body/occurrence level. Face/edge selection and downstream CAD operation success are not quantitatively evaluated. These acknowledged limitations leave the practical scope of the method less established than its entity-identification capability.
Questions for authors
Can you provide comparably trained bounding-box and Set-of-Mark baselines and further investigate why fine-tuning degrades the color-keyed baseline?
How are ambiguous color references and duplicate color-keyed metadata entries handled when colors are reused above 20 visible entities?
How robust is SelectionVLM to changes in rendering appearance, such as using original CAD materials or a uniform color instead of the distinctive training palette? This would help establish its applicability to typical CAD views.
Thanks the authors for detailed experiments and replies. I also appreciate the effort on spotting the baseline bugs and revise the results which make the tables much more convincing.
I still think "providing a list of 2d boxes and asking VLM to choose" is fundamentally different from "asking VLM to directly write a point location", in the first option the given list can affect the implicit model prediction, and box is fundamentally different from point. Asking it directly is much closer to the proposed SelectionVLM here in terms of input / output.