- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies premature candidate filtering in interactive 3D visual grounding, where the target may be incorrectly discarded before sufficient clarification is available. The authors propose a trajectory learning framework that uses interaction history, candidate transitions, and trajectory-level preference optimization to improve candidate retention and subsequent disambiguation. A trajectory-aware adapter is introduced to refine candidate features and grounding scores. Experiments on AmbiRefer3D and a newly constructed challenging subset show consistent improvements across four grounding backbones.
Critical strengths
-
Well-motivated problem. The paper identifies an important issue in interactive grounding: early candidate filtering can affect all subsequent interactions. I find the motivation convincing, and the idea of optimizing candidate selection based on future interaction outcomes is reasonable.
-
Solid experimental validation. The method is evaluated on four different grounding backbones, with consistent improvements in target survival and final accuracy. The comparison between ordered trajectories and shuffled pairs is particularly useful, as it demonstrates the benefit of modeling sequential dependencies rather than simply adding more supervision. The ablations also provide reasonable evidence for the proposed components.
Critical weaknesses
-
The method combines several components and training objectives, including the trajectory-aware adapter, retention supervision, gating, and preference optimization. While the ablations demonstrate their effectiveness, the overall framework is somewhat complicated, and it is unclear how much additional training and inference overhead is introduced.
-
Limited improvements in final accuracy. Although the gains in target survival are substantial, the improvements in final grounding accuracy are relatively modest for some backbones. This suggests that better candidate retention does not always translate into successful
Questions for authors
What is the additional training and inference overhead of the proposed framework compared to direct training? In particular, how much overhead comes from the trajectory-aware adapter?
Thank you for this important question and for recognizing our motivation and multi-backbone evaluation. To evaluate the training and inference overhead, we analyze adapter parameters, training cost, peak GPU memory usage, and module-wise inference latency across Direct, Shuffled pairs, and Ordered trajectory on an NVIDIA A40 GPU.
As shown in Table, compared with Direct, our framework introduces only 0.397M additional parameters, while training cost and peak GPU memory increase by 2.18 GPU-h and 8.76 GB, respectively. Compared with Shuffled pairs, Ordered trajectory requires comparable training time, with an additional 2.50 GB of peak GPU memory usage.
During inference, the trajectory-aware adapter introduces only 1.88 ms per query, accounting for approximately 1.6% of the total inference latency. Despite this additional overhead, the overall inference latency decreases from 120.60 to 118.19 ms per query, demonstrating comparable inference efficiency.
Thank you for bringing up this phenomenon. For some backbones, the limited accuracy gains may be attributed to more severe ambiguity in the newly retained instances, making subsequent disambiguation more challenging. Another factor is that the extracted features are not sufficiently exploited by the downstream disambiguation module to distinguish the intended target from similar candidates.