- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes W3D, a query-centric framework for diagnosing errors in multi-view 3D fusion through three related tasks: Where, which localizes unreliable fused geometry; Who, which identifies views whose removal improves local reconstruction quality; and Why, which predicts four predefined observation factors: Bias, Residual Outlier, Noise, and Multi-view Conflict. Each fused-surface query is linked back to its multi-view observations using an 11-dimensional point-normal-residual (PNR) representation. A Set Transformer encodes the observation set, followed by gated Where→Who→Why message passing with stop-gradient connections between task heads. Supervision is constructed using reference geometry and leave-one-view-out re-fusion, while inference requires only the fused surface, observations, and camera calibration. Experiments on Skoltech3D show substantial gains from the PNR representation and smaller additional gains from the proposed detached-gated architecture. The paper also evaluates different view counts, four fusion procedures, local view-removal utility, DTU, and ETH3D.
Critical strengths
-
The Where/Who/Why formulation links a fused-surface error to both its supporting observations and interpretable observation-pattern labels. This is more informative than simply predicting a scalar uncertainty or confidence value and constitutes the most distinctive contribution of the paper.
-
The progressive PNR ablation shows that relative-depth features provide a substantial contribution, while the paper additionally evaluates multiple fusion procedures, different input-view counts, frozen cross-count transfer, geometric baselines, per-seed results, coverage statistics, factor-level performance, and an explicit negative ETH3D boundary case. The authors also appropriately acknowledge that the Who and Why targets do not establish physical causality.
Critical weaknesses
-
At identical PNR inputs, W3D improves the parallel Base on all six reported metrics, but the model introduces additional adapters and parameters while simultaneously changing gating, cross-task conditioning, and gradient routing. The manuscript itself states that the comparison is not parameter matched and that the architectural factor evaluates gating and gradient decoupling jointly. Consequently, the current results demonstrate an advantage for the complete larger architecture, but do not establish that the proposed Where→Who→Why detached hierarchy is the cause. A parameter-matched baseline and path-specific ablations are important because the architectural gains are relatively modest for most metrics compared with the much larger gains obtained from PNR.
-
Who requires a Bad query, complete leave-one-view-out evidence, and a positive removable effect, so only a fraction of constructed queries are Who-valid; this becomes particularly restrictive for Screened Poisson. In addition, the Bias and Noise Why labels are almost always positive on seven-view TSDF, making their AP values close to one largely because of class prevalence. The paper acknowledges these issues in the appendix and limitations, but they should be integrated into the main evaluation because they materially affect how the reported Who and Why performance should be interpreted.
Questions for authors
-
Can the authors provide a parameter-matched Base+PNR comparison, or an adapters-only control, to determine whether the fixed-PNR improvements remain after controlling for the additional parameters introduced by W3D? This would materially affect my assessment of the claimed architectural contribution.
-
Please provide targeted ablations separating the effects of cross-task message passing and stop-gradient, for example adapters without messages, messages without stop-gradient, and removal of individual Where→Who or Who→Why paths? The current experiment evaluates these mechanisms jointly, so it is not possible to determine whether the proposed hierarchy itself is responsible for the improvements.
-
Please report Who coverage alongside Who Top-1/Mass in the main results, particularly for the four fusion methods? The method appears to provide valid Who supervision for only a subset of reconstructed queries, and I would like to understand how much of the reconstructed surface can actually receive meaningful view attribution.
-
If Why performance is evaluated using only the less-degenerate Residual Outlier and Conflict factors, does W3D retain a clear advantage over Base+PNR? This would help determine how much of the reported Why AP is driven by the extremely high positive prevalence of Bias and Noise.