- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces a query-based closed-loop method called SplatScout, which is used for "part-level referential segmentation" in Gaussian scenarios. Existing referential segmentation methods usually rely on predefined or one-time segmentation candidates, making it difficult to capture fine-grained object parts and lacking the flexibility to adjust the segmentation precision according to specific query requirements. To address this limitation, the authors propose a perception framework driven by a visual-language model (VLM), which consists of three core components: a "navigator" for selecting informative viewpoints, a "segmenter" for generating masks for specific queries, and an "updater" for constructing 3D Gaussian instances. The VLM iteratively optimizes its predictions by evaluating the intermediate segmentation and rendering feedback. Additionally, the authors have proposed the PartRef-ScanNet++ benchmark dataset, which contains 12 ScanNet++ scenes, covering 261 manually verified queries, and provides ground truth annotations from the mesh.
Critical strengths
- This paper points out a limitation of the candidate-based anaphora resolution method: when the target component is not included in the candidate set, the visual language model (VLM) cannot retrieve it solely through candidate selection. Allowing the VLM to actively control the perception process is a reasonable and well-grounded solution to this limitation. The focus on "component granularity dependent on queries" and "different roles of components in referential expressions" in the paper is particularly notable.
- The PartRef-ScanNet++ dataset covers various types of anaphora, including those for complete objects, those for component targets, discriminative descriptions based on components, relationship anchors, and referential expressions for multiple targets. This dataset uses manually verified query statements and ground truth generated based on meshes to provide an effective evaluation environment for fine-grained 3D anaphora segmentation tasks.
Critical weaknesses
1.The paper claims that SplatScout is superior to methods based on fixed candidate objects, but this comparison may conflate two distinct concepts: query-adaptive candidate generation and closed-loop perception. Although SplatScout can dynamically generate segmentation candidates based on the query, its Navigator retrieves views only from a pre-existing, fixed image pool. Therefore, the ability to recover previously missed regions or objects may primarily come from the query-conditioned SAM3 prompting, rather than from the proposed VLM-controlled iterative perception process. This raises an important question: if all available images were exhaustively traversed and the corresponding object points were simply identified/localized, followed by a single SAM3 invocation, would such a much simpler pipeline already achieve comparable performance?
2.The overall pipeline involves multiple interactions with both the VLM and SAM3, yet the paper does not provide sufficient details regarding the computational cost, inference time, number of VLM/SAM3 calls, or overall resource consumption. This makes the comparison with other methods potentially unfair, especially if those baselines operate under substantially different computational budgets. More importantly, how does SplatScout perform when compared with other methods under a comparable computational/resource budget?
3.The Navigator, Segmenter, and Lifter modules are largely built upon existing techniques, including geometric view selection, SAM3, and FlashSplat. The paper would therefore benefit from a clearer discussion of what fundamentally distinguishes SplatScout from existing agentic grounding systems, such as OpenGround. In particular, the authors should clarify which components or capabilities are genuinely novel, and how SplatScout differs conceptually and technically from these prior agent-based grounding frameworks beyond the specific combination of existing modules.
4.As the paper also introduces a benchmark, the benchmark itself should be described in substantially more detail. In particular, the paper currently provides limited information about the benchmark composition, task distribution, and difficulty levels, and lacks sufficient qualitative visualization of the benchmark examples. Providing representative visualizations would make it easier to understand what the benchmark actually evaluates and how challenging the proposed tasks are. In addition, the failure cases deserve a more thorough analysis, particularly for part-level targets.
Questions for authors
see above
2 comments
Thank you again for your detailed review. As the discussion period is closing soon, we would like to kindly follow up on our response. On your two main questions:
- One-pass pipeline (W1): we built the pipeline you proposed. It is a strong baseline, but it stays within 0.39–0.46 mIoU across five backbones, while the closed loop overtakes it with the stronger ones (0.538 vs. 0.458 with GPT-5.6; Table wUop-1).
- Cost (W2): under the same GPU and VLM, SplatScout takes 64 s per query, versus 89 s for OpenGround and 208 s for ZeroSplat (Common Response, Table G-2).
W3 and W4 are addressed in the revision (Sec. 2, Fig. 4, Appendix B.4 and G).
We would be grateful to know whether this addresses your concerns, and we are happy to clarify anything further.
Response to Reviewer wUop
Response to Reviewer wUop
We sincerely thank the reviewer for the detailed review, which covers the method, its cost and the benchmark. The one-pass pipeline proposed in Weakness 1 is now in our ablations (SplatScout, Table 4). We answer each point below, and would be happy to discuss further if any concern remains.
Re Weakness 1 (Query-adaptive candidate generation vs. closed-loop perception)
We thank the reviewer for decoupling two parts of our contribution: segmentation at query time, and the loop that controls it. This helped us understand the position of our paper more clearly. Following the suggestion, we built this pipeline. The VLM gives a phrase from the query, SAM3 segments the frames with it (up to 200 frames per query, since segmenting every frame would take far longer), the masks are grouped and lifted in 3D with our tools, and the VLM selects once among the candidates. We ran it with five VLMs (Table wUop-1).
Table wUop-1: mIoU on PartRef-ScanNet++ (one-pass: phrases from Qwen3.5-397B, selecting VLM varied).
Analysis Query-time segmentation alone is a strong baseline: under every backbone it is better than all the other baselines (SplatScout, Table 3). However, it is exactly the two-stage process described in C1, and it is limited by its first stage. It stays within 0.39–0.46 however strong the VLM is, whereas SplatScout rises from 0.128 to 0.538 and overtakes it with the stronger backbones (significantly with Qwen3.8-Flash and GPT-5.6).
Conclusion Query-time segmentation and the loop therefore complement each other. Segmenting at query time provides candidates that a fixed table lacks, and the loop lets the VLM decide, from what it has observed, what to segment next, where, and with which phrase. It is this control that takes the result beyond what the first stage provides. We also agree that the Navigator only selects among captured frames. This is the common setting of this line of methods, most of which work from the captured images.
Re Weakness 2 (Computational cost and comparison under a comparable budget)
We thank the reviewer for pointing this out. The time, the VLM and SAM3 calls, and the tokens per query of all methods under GPT-5.6 are reported in the Common Response (C2, Table G-2). In short, SplatScout runs under a budget comparable to the baselines: it takes less time per query than OpenGround and ZeroSplat, and makes fewer VLM calls and segments far fewer frames than ZeroSplat.
Re Weakness 3 (What distinguishes SplatScout from agentic systems such as OpenGround)
We thank the reviewer for this question. The key difference is who makes each perception decision (Common Response, C1).
Our contributions are:
Re Weakness 4 (Benchmark details, visualizations, and failure analysis)
We thank the reviewer for these suggestions. We have added to the paper example queries of each role with their ground-truth targets (Fig. 4), the results for each role (Appendix B.4, Table 9), and a failure analysis for part targets (Appendix G).