SplatScout: Query-Driven Part-Level Referring Segmentation in Gaussian Scenes

Existing referring segmentation methods on Gaussian scenes primarily target whole objects, relying on segmenter-generated candidates from which a vision- language model (VLM) selects. Such pre-determined fixed candidates may omit fine-grained parts needed as targets or cues for identifying objects. We intro- duce SplatScout, a query-driven, closed-loop approach that places perception un- der VLM control. The model actively selects views, directs segmentation, and inspects rendered 3D results to refine its predictions. For part-level evaluation, we introduce PartRef-ScanNet++, comprising 261 manually verified queries across 12 ScanNet++ scenes with mesh-derived ground truth. On PartRef-ScanNet++, SplatScout achieves 0.538 mIoU, substantially outperforming existing methods. Ablations demonstrate the benefits of iterative refinement, while comparisons across VLM backbones highlight the importance of effective tool-feedback uti- lization. SplatScout also achieves higher scores on four existing referring bench- marks.