- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
Matisse reuses a frozen SAM3D generator and Stream3D's canonical evidential memory to guide active reconstruction and keyframe selection. It interprets attention-derived weights as Gaussian observation precisions, obtains token uncertainty alpha/(alpha + accumulated evidence), and approximates information gain by summing uncertainty over visible predicted occupied tokens. Object-balanced aggregation and viewpoint-diversity weighting extend the criterion to multi-object acquisition, while sparse-structure decoding reduces planning cost. The main paper reports GSO30 and YCB-V reconstruction results and a GSO30 keyframe experiment; additional validation is provided in the appendices.
Critical strengths
- Using one spatially shared evidence representation for both acquisition and retention is an interesting conceptual direction. It connects two decisions that are often treated separately and gives a simple diminishing-return rationale for directing observations toward weakly supported regions. The Gaussian derivations are internally consistent under their stated assumptions.
- The implementation makes candidate evaluation inexpensive without uncertainty-specific training. Planning uses sparse-structure decoding rather than full structured-latent generation, and candidate scores are batched on the GPU. This offers practical value for active reconstruction with a costly frozen generative backbone.
Critical weaknesses
- The comparisons do not establish the full acquisition-and-retention claim. Figure 3 compares selections from a spiral trajectory with Stream3D's chronological processing, without equal-budget random, uniform, or pose-diversity selections from the same candidate pool. The additional YCB-V keyframe study appears only in Appendix C.6 and repeats the selected-frames-versus-prefix comparison. Thus, the 14% view-count result does not isolate the scoring contribution. The abstract also highlights a 9.2% Replica CD reduction, whereas Section 6.3 states the gain and places the complete quantitative comparison in Appendix C.1. Part of the headline cross-dataset evidence is consequently not directly assessable from the main-paper comparisons. The GSO30 and YCB-V results deserve credit, but they cannot substitute for evidence supporting the broader claims. The appendix also does not report reconstruction uncertainty on method differences, and MAGICIAN's beam search is disabled on Replica. These limitations concern evidence sufficiency and comparability; supplementary experiments are not inherently invalid or prohibited.
- The main paper does not supply sufficient direct validation of the central uncertainty interpretation. Theorems 1 and 3 assume independent Gaussian observations and define their precision to equal the attention-derived weight. They do not establish that actual attention scores measure geometric reliability or that correlated views provide independent evidence. The practical score further substitutes binary visibility for future evidence and uses a first-order entropy approximation. Section 6.4 announces visibility, symmetry, and model-confidence analyses but places all corresponding evidence in Appendix D; numerical pseudo-evidence sensitivity is also deferred to Appendix C.4. These analyses concern the central formulation, rather than peripheral detail. Because ICLR explicitly states that reviewers are not required to read appendices, the essential validation should be sufficiently presented in the nine-page main paper. I examined the appendix, and its examples still do not test posterior calibration or whether predicted gain ranks actual reconstruction improvements. Moreover, covariance over tokens in a decoded completion does not establish uncertainty that is global in scene-hypothesis space. The issue therefore extends beyond presentation to the validity and scope of the interpretation.
Questions for authors
- What direct evidence validates attention-derived precision and predicted information gain against geometric reliability and actual improvement from acquiring a view? Please identify the support in the main paper and explain how repeated or nearly identical views affect the independence and uncertainty assumptions.
- Does the keyframe advantage persist against random, uniformly spaced, and pose-diversity selections at the same view budget and from the same full candidate pool? Please clarify whether selection has access to the entire sequence's poses or must operate causally.
- What paired uncertainty estimates support the headline 9.2% Replica gain, and how sensitive is the comparison to the disabled MAGICIAN beam search? The complete comparison is currently confined to Appendix C.1; clarifying the evidence and baseline configuration could materially change the assessment of cross-dataset generality.