- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
AnchorSplat predicts referring masks and a renderable Gaussian scene from sparse, uncalibrated images and a text query. It combines DINOv2 semantics with DA3 geometry, cross-view feature transport, and owner-excluded rendering. On MVRefer, it improves global mIoU from 39.9 to 41.0 over MVGGT. Additional experiments evaluate novel-view rendering.
Critical strengths
- Predicting a renderable, language-selected Gaussian scene from sparse uncalibrated views is a useful extension beyond view masks and lifted points. It allows the selected object to be inspected from other viewpoints.
- Owner-excluded rendering addresses a concrete supervision shortcut: a Gaussian could otherwise copy its source pixel’s label without learning agreement across views. This is a reasonable design worth investigating.
Critical weaknesses
-
The main components are not sufficiently isolated in the ablations. Table 5 mainly compares backbone substitutions with the full model. It does not show a DINOv2-only baseline or separately remove the geometry ladder and cross-view transport. I therefore cannot tell whether the gains come from the proposed geometry mechanisms or the added semantic features.
-
The benefit of the Gaussian readout is not directly established. The main segmentation metrics evaluate view masks and lifted points, while Table 4 measures rendering quality. Object-region PSNR does not tell us whether the selected Gaussians exclude background or preserve the entire target. A comparison against direct mask assignment and owner-inclusive rendering, using the same geometry and held-out-view mask IoU, is needed to support this contribution.
-
Poor figure quality makes the qualitative results difficult to assess. The pipeline in Figure 3 is overcrowded, with tiny labels and visibly compressed text. The result panels in Figures 6–9 are also too small, and many details remain unclear even when enlarged. The comparisons make it difficult to inspect object boundaries, background leakage, and fine-grained differences between methods. This is not just a cosmetic issue: the presentation limits my ability to verify the claimed improvements. Larger panels, readable labels, and clear close-ups of corresponding regions are needed.
Questions for authors
-
Can you provide matched ablations for the DINOv2-only baseline, geometry ladder, and cross-view transport? These would help establish which components account for the improvement.
-
Do any reported segmentation scores directly evaluate the final Gaussian readout? On the same DA3 Gaussians, how does owner-excluded rendering compare with direct mask assignment and owner-inclusive rendering in held-out-view mask IoU? Please include clear visual comparisons with ground truth.
3 comments
Comment 2 of 3. Direct evaluation of the Gaussian readout (W2, Q2)
A distinct output. The readout produces what view masks and lifted points cannot provide, a referred Gaussian scene that can be rendered from any viewpoint. Its gradients reach only the readout head, so it never alters the reported view masks (Section 4.6). We therefore added a dedicated Gaussian-level evaluation in exactly the requested setting, with the same DA3 geometry and held-out-view mask IoU (Section 5.4, Table S6b).
Protocol. All variants share identical DA3 Gaussians. Gaussian mIoU scores primitives matched within 2 cm to held-out GT sensor-depth surfaces over 21 queries. Any selected background primitive enters the union and any missed referent primitive lowers the intersection, so the metric measures background exclusion and target completeness directly, which object-region PSNR cannot. Referred masks rendered with opacity are scored by IoU on 28 target-visible held-out frames that are never input views.
Results on identical Gaussians (Gaussian mIoU and held-out mask mIoU).
- Direct mask assignment from the MVGGT+DA3 state, 17.82 and 13.58.
- Direct mask assignment from the AnchorSplat state, 22.95 and 15.05.
- Owner-inclusive readout head, 23.02 and 15.21.
- Owner-excluded readout head (ours), 23.02 and 15.21. The anchored referring state accounts for the largest gain, +5.13 Gaussian mIoU and +1.47 mask mIoU over MVGGT+DA3, and the readout adds 0.07 and 0.16 over direct assignment at a cost of 0.015 s per pass (Table 3b). Absolute values are lower than view-mask mIoU because every primitive is judged against GT surfaces in unseen frames.
Owner-excluded versus owner-inclusive rendering. The two heads were trained for the same 400 updates with identical seed and sample order, and both refine the same anchored owner logit, which explains their equal held-out accuracy. They differ in the signal they learn from. Under owner-inclusive rendering, source-owner primitives carry 44.41% of rendered opacity and 31.46% [26.74, 36.56] of primitive-logit rendering gradients, and this share can be lowered by copying each pixel's own label. Exclusion sets the share to zero, so every gradient enforces agreement with other views, which is the condition for Proposition 3 (Section S2.4). Negating one view's owner logits changes the owner-inclusive render by 54.89 percentage points and the owner-excluded render by exactly zero in all 50 cases.
Visual comparison with ground truth. Figure 7 now places GT beside the full-scene, selection, and referred renders, with matched insets on object boundaries. Figure S13 shows two expressions selecting different Gaussians from one shared reconstruction, illustrating that only opacity depends on the query.
Comment 3 of 3. Figure quality (W3) and summary of contributions
Figure 3. We re-laid out the pipeline figure with a minimum font size of 7 pt and removed the vertical compression that distorted its text. All figures are now included at their native aspect ratio and full resolution.
Figures 6 to 9. We added local magnifications to Figures 6 to 9 to enhance fine details. Each inset enlarges an identical region across methods, so object boundaries, background leakage, and fine differences between methods can be inspected directly against ground truth.
Supplementary views. The two enlarged single-case comparisons (Figures S9 and S10) and the nine-scene gallery (Figure S11) provide full-page views with point IoU on a shared canvas.
Summary of contributions and evidence.
- Diagnosis. We identify why reconstruction backbones fail as referring backbones. A decoder-free probe shows object identity in DA3 falling to near chance with depth, and the source-owner shortcut is measured directly, carrying 31.46% of rendering gradients and shifting owner-inclusive renders by 54.89 percentage points.
- Output. AnchorSplat returns view masks and a referred Gaussian scene from sparse uncalibrated views in one forward pass, now evaluated at the Gaussian level and in held-out views.
- Mechanisms. Under matched budgets, the semantic anchor, geometry ladder, and transport each improve on the preceding stage, while owner exclusion makes primitive supervision fully cross-view at no cost in held-out accuracy. \item Results. AnchorSplat improves 17 of 20 MVRefer entries and all nine ScanRefer entries over MVGGT, whereas every direct reconstruction substitution loses 7.1 to 9.0 points.
We hope these results resolve the reviewer's concerns. Should any question remain, we would welcome further comments and will promptly clarify or provide additional experiments.
Comment 1 of 3. Matched ablations for DINOv2, the ladder, and transport (W1, Q1)
We thank the reviewer for the detailed assessment and the concrete experimental requests. We trained every requested control on all 9,508 MVRefer test queries with matched budgets and added them to Table 5a (Section 5.5) and Table S1 (Section S2.1). Throughout our responses, pairs report global mIoU / [email protected] (%), and brackets give paired 95% scene-bootstrap intervals.
Matched protocol. All referring controls use identical data, views, evaluator, four-head decision, and 20,000 referring updates. Transport variants are calibrated from one shared ladder for 100 updates (Table S7). Table 5a now separates these component controls from backbone controls.
DINOv2-only baseline. With every geometry gate and transport parameter fixed at zero, the decoder reads DINOv2 tokens alone and reaches 39.53 / 39.87. This is below the released MVGGT (39.9 / 41.5) and below MVGGT continued for the same 20,000 updates (40.14 / 43.74). The added semantic features therefore do not explain our improvement.
Removing the geometry ladder. Adding gated DA3 features to the same anchor raises the result to 40.88 / 44.28, a gain of 1.35 [0.78, 1.91] / 4.41 that holds on Hard, Easy, Unique, and Multiple. Geometry contributes cross-view evidence, raising Neg mIoU from 73.72 to 80.21 and View mIoU from 66.00 to 69.75. A standard aligned DA3 interface trained with the same budget reaches only 34.29 / 35.77, 6.59 [5.64, 7.55] points below the ladder, so the gain depends on how geometry enters the decoder.
Removing cross-view transport. From the same ladder, full transport reaches 41.01 / 44.64, against 40.88 / 44.35 with transport disabled and 40.62 / 43.73 with uniform aggregation over the same four neighbors. Full transport improves [email protected] over both alternatives on all four splits. Uniform aggregation lowers Neg mIoU from 80.19 to 78.46 because it spreads referent activation into views without a matching surface, whereas our confidence and distance weights keep Neg mIoU at 80.17. The transport graph sends 72.91% of its weight to target patches against a 3.80% prevalence baseline (Section S2.3).
Attribution. Under identical budgets, each mechanism improves on the stage that precedes it (39.53, 40.88, and 41.01 global mIoU, with 39.87, 44.28, and 44.64 [email protected]). On the same frozen DA3 features, the complete design exceeds direct substitution by 10.08 / 15.40, and its margin over MVGGT grows with views, reaching 71.56 against 65.63 at 32 views (Table 3a). A decoder-free probe explains why the semantic anchor is needed. Same-object ranking in DA3 falls by 7.93 points [3.50, 12.69] from layer 4 to layer 39, close to chance, while DINOv2 stays 18.97 points above deep DA3 (Section 5.3, Table S2b).
We address W2, Q2, and W3 in the following comments.