- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Generating views in sparse-view scene reconstruction is not reliable because the generated content may conflict with the original observation. This paper GenAnchor, solves this issue by preserving reliably observed content while selectively applying the generative info to uncertain and occluded regions. GenAnchor surpasses other baseline methods on various public datasets.
Critical strengths
-
This paper identifies a real failure case of generative reconstruction and solves it directly. Though reconstruction via generation solution is not new, this paper proposes a clear tri-state idea, which is a neutral response.
-
Re-estimating geometry after generative repair is conceptually strong. Once diffusion changes object boundaries or generates new content, the old gs depth is actually no longer valid. This paper recognises this and anchors the depth to the real observation.
-
They do more exps rather than simply report global psnr at novel views. The observed/unobserved region analysis is valuable, and it meets the core claims in the paper: improve the missing areas w/o destroying observed areas.
Critical weaknesses
-
It is hard to fully prove that the attribute gains to GenAnchor because the representations and backbones varies across baselines.
-
The choice of 2dgs is insufficiently justified and limits evidence of generality. This paper proposes a framework-level contribution, demonstrating the same idea on vanilla 3dgs and its variations would make the contributions substantially stronger.
-
The exact integration with WorldStereo is not fully transparent from the main content and the appendix. This paper gives the formulations for masking and anchoring operations, but it is not fully clear where exactly does this modified conditioning enter the pretrained pipeline.
-
The camera planner considers whether content can be rendered, rather than the content is verified by real observations. This may allow trajectories with substantial provisional content but little grounded content. This seems somewhat disconnected from the paper's observation-anchored motivation.
Questions for authors
-
Why the full pipeline is based on 2dgs rather than vanilla 3dgs?
-
How much of the performance gain is from the pipeline itself vs the choice of the pretrained models?
-
How much computational overhead does this pipeline introduce?
-
Will the pipeline crash if the generative models introduce more noise rather than correct and precise content?
5 comments
[W3] Exact Integration with Pretrained WorldStereo
We thank the reviewer for pointing out the need for further implementation details. We use the pretrained four-step WorldStereo Memory-DMD pipeline based on Wan2.1-I2V-14B, without introducing additional learnable layers or fine-tuning the pretrained model. We have revised Appendix C.2 to clarify the integration as follows.
Condition construction and integration.
The tri-state partition is incorporated through multiple conditions rather than a single three-class mask. The conditional RGB sequence (render_video) contains projected real-input colors in Grounded regions, rendered colors in Provisional regions, and black pixels in Uncovered regions. The binary editing mask (render_mask) specifies editable regions, while (render_rgb_hole_mask) controls an additional feature-gating operation.
The conditional RGB, editing mask, and camera information are incorporated through WorldStereo's pretrained conditioning pathways. We additionally introduce feature gating to suppress unreliable rendering features in Uncovered regions, while retaining the original camera and reference-image conditioning mechanisms. The resulting conditions guide the pretrained denoising Transformer without modifying its learnable parameters.
Observation-based latent replacement.
After each scheduler update, we apply Eq. (A5):
Here, selects latent cells whose pooling regions lie entirely within Grounded, and is encoded from a color-extended Grounded reference image, distinct from the original real-image references. The replacement uses reference latents at the corresponding noise level, with clean reference latents in the final step. The updated latent is passed to the next denoising step, and exact projected real-input colors are restored within the original Grounded mask after decoding.
These modifications operate during inference, while all pretrained model weights remain unchanged.
[W4] Camera Planning and Observation Anchoring
Thank you for raising this point. Camera planning and observation anchoring serve different purposes in the progressive reconstruction loop. Planning aims to expand scene coverage by directing the next trajectory toward regions that are uncovered in the current reconstruction. Rendering support is therefore the relevant planning signal: it identifies which parts of a candidate view are already represented and whether its uncovered regions are adjacent to sufficient rendered context for repair. Using observation anchoring as the trajectory-selection criterion would instead favor views with greater overlap with content directly verified by the input images, without directly measuring whether a trajectory exposes currently uncovered scene content. The planner therefore deliberately optimizes coverage expansion rather than the proportion of directly verified pixels.
By contrast, observation anchoring determines which content from a selected view can serve as reliable evidence during repair and scene optimization. Real-input correspondences define the Grounded regions, which receive observation-based protection; Provisional regions remain editable, while Uncovered regions are generated. Direct generated-view supervision is restricted to the latter two regions, and real inputs remain the only source of verification and depth anchoring. Thus, trajectories containing substantial Provisional content but limited Grounded content can still contribute to scene exploration without treating uncertain renderings as verified observations. In this way, coverage-aware planning and observation anchoring serve complementary purposes, enabling progressive scene expansion while preserving the distinction between real observations and uncertain content.
[Q3] Computational Cost and Baseline Comparisons
We appreciate the reviewer's comments on computational cost. We report the mean training time and peak GPU memory across the eight Replica scenes, measured on NVIDIA RTX 5090 GPUs. Training time includes input preparation, initialization, diffusion generation and repair, and optimization. Times are in minutes. The results are summarized below.
Table. Computational cost on Replica.
| Method | Mean training (min) | Mean peak GPU memory (GB) |
|---|---|---|
| Difix3D+ | 51.97 | 10.12 |
| ArtiFixer | 105.90 | 24.01 |
| WorldStereo | 28.89 | 11.12 |
| GuidedVD | 88.02 | 32.04 |
| G4Splat | 66.47 | 32.31 |
| Ours | 67.34 | 17.04 |
Compared with G4Splat, our method achieves higher mean PSNR with a comparable training time (67.34 vs. 66.47 minutes) while reducing mean peak GPU memory from 32.31 GB to 17.04 GB (47.3%). Our method also requires less training time and peak GPU memory than ArtiFixer and GuidedVD. Overall, these results demonstrate that our method achieves better reconstruction quality with comparable or shorter training time and substantially lower GPU memory usage than these baselines. These efficiency results have been added to Appendix A.10 of the revised version.
[Q4] Robustness to Inaccurate Generation
We thank the reviewer for raising this concern. Our framework incorporates several mechanisms to mitigate errors introduced by inaccurate generation. During RGB repair, Grounded regions are protected by real-input colors and excluded from direct generated-view supervision. During geometric recovery, generated depth and Gaussian seeds are screened for validity and confidence to reduce the influence of unreliable geometry. During scene optimization, generated RGB receives a lower supervision weight than real observations, while a fixed real-view sampling probability maintains the contribution of real-input supervision throughout progressive reconstruction. Together, these mechanisms help limit the propagation of errors from generated content.
To evaluate robustness under persistent generation errors, we apply smooth random spatial deformations to generated RGB at every reconstruction round. The perturbations affect only Provisional and Uncovered regions, while Grounded regions and real inputs remain protected. The subsequent depth recovery, scene optimization, and camera planning are performed using the perturbed results. We evaluate the complete pipeline on Replica scan3 under identical reconstruction settings, with Clean denoting no additional perturbation.
Table. Final reconstruction results on Replica scan3.
| Condition | PSNR ↑ | SSIM ↑ | LPIPS ↓ | AbsRel ↓ | RMSE ↓ | CD ↓ | F-Score ↑ | NC ↑ |
|---|---|---|---|---|---|---|---|---|
| Strong | 26.13 | 0.869 | 0.174 | 0.023 | 0.125 | 0.0075 | 0.796 | 0.856 |
| Moderate | 26.58 | 0.872 | 0.169 | 0.021 | 0.112 | 0.0071 | 0.824 | 0.870 |
| Clean | 26.59 | 0.870 | 0.167 | 0.021 | 0.110 | 0.0069 | 0.825 | 0.872 |
As shown in the table, our method maintains reconstruction quality close to the clean setting under moderate perturbations. Stronger perturbations lead to some degradation in rendering and geometric accuracy, but the reconstruction process remains functional. These results demonstrate that our framework can tolerate inaccurate generated supervision under the tested perturbation levels while maintaining effective reconstruction performance.
We further examine the progressive reconstruction process under strong perturbations to assess whether persistent generation errors disrupt subsequent refinement.
Table. Reconstruction progress under strong generation noise.
| Stage | PSNR ↑ | SSIM ↑ | LPIPS ↓ | AbsRel ↓ | RMSE ↓ |
|---|---|---|---|---|---|
| Initialization | 19.46 | 0.816 | 0.198 | 0.050 | 0.355 |
| Round 1 | 22.99 | 0.849 | 0.179 | 0.032 | 0.209 |
| Round 2 | 24.06 | 0.854 | 0.179 | 0.029 | 0.180 |
| Round 3 | 25.47 | 0.862 | 0.177 | 0.026 | 0.141 |
| Round 4 | 25.99 | 0.866 | 0.177 | 0.023 | 0.121 |
| Round 5 | 26.13 | 0.869 | 0.174 | 0.023 | 0.125 |
Despite persistent generation perturbations, rendering quality continues to improve across all five reconstruction rounds, accompanied by an overall reduction in depth error. These results show that progressive refinement remains effective under strong generation noise, without interruption of the reconstruction process.
Overall, the experiments demonstrate that GenAnchor maintains reconstruction stability and continues to improve scene quality under the tested generation errors, consistent with the benefits of its observation-based protection and supervision mechanisms.
[W1&W2/Q1&Q2] Generality of the Framework
We appreciate the reviewer’s comment. We use 2DGS because its oriented planar primitives can explicitly represent local surfaces and provide depth and normals for geometric supervision and surface reconstruction. These properties suit our joint appearance and geometry objectives, and the comparison results below show that 2DGS achieves the strongest overall performance among the tested representations. Nevertheless, the framework's observation verification, view repair, and supervision rules operate on rendered RGB-D and real-input correspondences, allowing them to transfer across Gaussian representations.
We have completed full progressive reconstruction using two alternative Gaussian-based reconstruction backbones: vanilla 3DGS and FSGS. Both substitutions cover all eight Replica scenes. The results are summarized below, together with the baselines reported in the paper.
Table. Representation substitutions and baseline comparisons on Replica.
Both substitutions outperform the compared baselines in PSNR, SSIM, CD, F-Score, and NC, demonstrating that the benefits of our framework extend beyond the original 2DGS implementation. These results support the applicability of our framework across different Gaussian-based reconstruction methods. The corresponding experiments have been added to Appendix A.11 of the revised version.
To evaluate generality across generative priors and assess the contribution of the pipeline itself, we test our method and G4Splat with ViewCrafter and WorldStereo on all eight Replica scenes, allowing the two methods to be compared under the same pretrained generative prior. Each variant retains its own reconstruction pipeline; Ours uses WorldStereo by default.
Table. Methods integrated with different diffusion models.
As shown in the table, our method achieves higher PSNR and F-Score and lower LPIPS and CD than G4Splat with both ViewCrafter and WorldStereo. The PSNR gains are 3.16 dB and 0.94 dB, respectively, with corresponding CD reductions of 25.1% and 7.6%.
These results are consistent with how our framework uses real observations to guide generation and reconstruction. Our tri-state masking preserves verified input appearance in Grounded regions, refines uncertain renderings in Provisional regions, and generates missing content in Uncovered regions. Our observation-anchored geometric completion preserves verified input depths and uses them to calibrate recovered geometry before it supervises Gaussian reconstruction. Our coverage-aware camera exploration targets uncovered regions adjacent to rendered context, enabling progressive scene expansion. Together, these designs direct generation toward content requiring completion or refinement and convert repaired views into reconstruction supervision constrained by verified input geometry. These consistent improvements demonstrate that the performance advantages stem from our framework design rather than from the choice of pretrained diffusion model. These results have been added to Appendix A.12 of the revised version.