- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper addresses 3D Gaussian Splatting from unconstrained photo collections, arguing that transient distractor removal and camera pose errors are coupled problems that should be solved jointly. The proposed method, PAGauss, builds on RobustSplat and adds two modules. First, a spatial aware adaptive mask uses Depth Anything 3 (DA3) per pixel confidence residuals aggregated over SAM instances with a learnable threshold to identify distractors, complemented by a context aware expansion based on DA3 feature similarity. Second, a spatial aware pose alignment regresses per image SE(3) residuals toward poses estimated by DA3 from rendered reference views. Training follows a three stage schedule. The authors also contribute WildArena, a dataset of 82 phone captured scenes with distractor mask annotations. Results are reported on five benchmarks with rendering, pose, and segmentation metrics.
Critical strengths
-
The masking module introduces a simple and effective signal based on DA3 multi view confidence residuals. Plug in experiments (Table 5b, Table 12) show that this signal improves several existing methods, suggesting it captures genuinely useful information beyond what photometric or semantic residuals provide. This is the most transferable contribution of the paper.
-
WildArena provides 82 scenes with distractor mask annotations and a unified evaluation protocol. The field currently lacks standardized mask level evaluation for in the wild reconstruction, and this dataset could serve as a valuable community resource if fully documented and released.
Critical weaknesses
-
The pose refinement mechanism is underspecified and appears to have a circularity problem. Renderings produced at the current pose estimate are used to obtain DA3 poses, which are then used as supervision targets for that same pose. The paper does not explain how DA3 can recover information about the true pose that is not already encoded in the rendering, nor does it describe how poses from DA3's arbitrary similarity frame are aligned to the SfM coordinate frame before the loss in Eq. 10 is computed. Without resolving this, the theoretical basis for the pose component is unclear.
-
The reported improvements over the strongest baselines are modest (within 0.25 dB on most established benchmarks without pose optimization), and the reliability of the results is undermined by numerous table inconsistencies. Memory values appear permuted between Tables 23 and 24, timing values are duplicated across rows in Tables 20 and 21, per scene results are duplicated in Tables 18 and 19, and several averages in Table 4b do not match the per scene values. These issues need to be resolved before the numerical claims can be properly evaluated.
Questions for authors
Clarifications that could materially change the assessment:
-
In the source refinement step, exactly which images are input to DA3: rendered images only, or the real target image together with rendered references? Please explain why DA3 applied to renderings produced at pose estimate T_hat_i provides information about the true pose of I_i rather than simply recovering T_hat_i.
-
How are the DA3 estimated poses, which live in an arbitrary similarity frame for each forward pass, aligned to the SfM coordinate frame before computing the loss in Eq. 10? Is a Sim(3) or Umeyama alignment solved, and if so over which point correspondences?
-
How were the ground truth poses for Table 4b obtained? If WildArena includes clean captures (as Table 9 indicates), could you report pose errors against ground truth computed independently from those clean images, rather than from SfM on prior methods' renderings?
-
Could you clarify the duplicated values across tables (for example, the shared memory numbers in Tables 23 and 24, copied timing rows in Tables 20 and 21, and the averages in Table 4b that do not match the per scene values)?
Suggestions for strengthening the paper in a revision:
-
Adding RefineSplat to the WildArena comparison (Table 2) and citing NeRF HuGS, which also combines multi view consistency heuristics with SAM instance masks, would help clarify the novelty of the masking component.
-
A controlled quantitative analysis of DA3 confidence residuals as a distractor detection signal (for example, AUROC against ground truth masks compared with photometric and DINO residuals) would strengthen the motivation beyond the qualitative evidence in Fig. 4.