acceptodds
Withdrawn from ICLR 2027

GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent 3D reconstruction methods achieve impressive results with dense multi-view imagery but struggle when only a few views are available. Various approaches, including regularization techniques, semantic priors, and geometric constraints, have been implemented to address this challenge. Recent diffusion-based approaches further improve performance by generating novel views to augment training data. Despite this progress, we identify three critical limitations in current state-of-the-art approaches: (i) inadequate coverage beyond known view peripheries, (ii) geometric inconsistencies across generated views, and (iii) computational inefficiency due to expensive pipelines. We introduce GaMO (Geometry-aware Multi-view Outpainter), a framework that reformulates sparse-view reconstruction through multi-view outpainting. Instead of generating new viewpoints, GaMO expands the field of view from existing camera poses, which inherently preserves geometric consistency while providing broader scene coverage. Our approach employs multi-view conditioning and geometry-aware denoising strategies in a zero-shot manner without training. Extensive experiments on Replica, ScanNet++, and Mip-NeRF 360 demonstrate strong reconstruction performance across sparse-view settings (3, 6, and 9 input views). Notably, our method is significantly more efficient than existing diffusion-based approaches, reducing the overall runtime to within 10 minutes.

Primary area:
generative models
Keywords:
Sparse-view 3D Reconstruction, Multi-view Outpainting, Diffusion Models, Geometry-aware Generation, Novel View Synthesis

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_EF9KOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

In the field of sparse-view 3D reconstruction, instead of generating novel views for more view coverage, this paper proposes to conduct multiview outpainting on the input views alone for larger FOV coverage. The authors mainly present a Geometry-aware Multi-view Outpainter along with a reconstruction pipeline. Specifically, they apply geometry-aware conditioning, latent blending and mask scheduling to the pre-trained MVGenMaster. The results show that GaMO can reconstruct more background area comparing with other sparse-view 3D reconstruction methods.

Critical strengths

The claim of multiview outpainting is more stable than generating novel views is definitely correct, and the authors provide sufficient experiments to verify that. While most methods are generating novel views, I consider this idea a novel one.

Critical weaknesses

Outpainting can reconstruct more background area is expected, and comparing with methods that are not meant for outpainting is not fair. Simply using outpainting will never be the solution to sparse-view 3DGS reconstruction because unseen areas apart from the background remains unseen. This can be proved from the provided results where DIFIX3D+'s details are obviously better. Outpainting is useful, but disgarding generating novel views is not.

Questions for authors

I suggest the authors train a model that can both generate novel views (or fix view artifacts like DIFIX) and conduct outpainting, so that the claim should be more applicable.

1 comment
AuthorsOpenReview ↗

Revised Positioning and Complementarity with NVS

1. Revised Paper Positioning: Outpainting as a Complement to NVS (W1/Q1)

We thank the reviewer for the constructive feedback and for recognizing the usefulness of outpainting. We agree that outpainting cannot replace novel-view synthesis, particularly because expanding the original FOV alone cannot recover occluded regions. Accordingly, we have reframed our core contribution: GaMO is a zero-shot multi-view outpainting diffusion model that generates geometrically consistent outpainted views to assist sparse-view 3D reconstruction. These views expand scene coverage and complement existing NVS methods rather than replacing them.

To examine whether outpainting and NVS can be effectively combined, as the reviewer suggests, we incorporated GaMO-generated outpainted views into Difix3D, GenFusion, and GuidedVD-3DGS, and evaluated their performance under the 3-view and 6-view settings on Replica and ScanNet++.

The averaged results (PSNR↑ / SSIM↑ / LPIPS↓) are summarized below:

MethodWithout GaMOWith GaMO Views
Difix3D19.65 / 0.762 / 0.23321.84 / 0.785 / 0.197
GenFusion21.88 / 0.813 / 0.20923.04 / 0.823 / 0.190
GuidedVD-3DGS23.94 / 0.830 / 0.18824.55 / 0.845 / 0.166

For reference, standalone GaMO-GS achieves 23.93 / 0.843 / 0.156.

The results demonstrate that incorporating GaMO-generated outpainted views improves the average PSNR, SSIM, and LPIPS across all three methods, with PSNR gains of +2.19, +1.16, and +0.61 dB, respectively. These findings confirm that outpainting can effectively complement existing NVS methods by expanding scene coverage.

We also agree that directly ranking our pipeline against methods not designed for outpainting should not be the primary comparison. Therefore, the revised Table 1 emphasizes comparisons of each existing method with and without GaMO-generated outpainted views, directly demonstrating the complementary benefits of our approach.

Terminology: In the revised manuscript, GaMO denotes the outpainting diffusion model, while GaMO-GS refers to the complete 3DGS reconstruction pipeline using GaMO-generated views, previously referred to as "Ours".

2. Revised Contributions and Integration with Existing NVS Methods (W1/Q1)

Following the reviewer's suggestion, we have substantially revised the paper's positioning and contribution statements. Rather than presenting outpainting as an alternative to NVS, we now emphasize GaMO as a zero-shot multi-view outpainting diffusion model that generates geometrically consistent views to complement existing NVS methods, expanding scene coverage and improving sparse-view 3D reconstruction.

Regarding the suggestion to train a single model that performs both novel-view generation (or artifact fixing, as in Difix3D) and outpainting, we agree that this is a valuable direction. Since GaMO operates in a zero-shot manner without requiring additional training, we instead investigate its integration with existing novel-view generation and artifact-refinement methods through complementary outpainted training views.

Our results demonstrate improvements across all three methods, validating the effectiveness of combining outpainting with NVS. While jointly training a unified model remains a promising direction for future work, our experiments demonstrate that the two capabilities can already work effectively together without additional model training.

These revisions are reflected throughout the updated manuscript, with the complete results presented in Table 1 of the main paper and implementation details provided in Appendix A.

Reviewer_oyYVOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GaMO, a multi-view diffusion outpainting method for sparse-view 3D reconstruction. The authors observe that synthesizing intermediate novel views can degrade 3D reconstruction performance due to geometric inconsistencies. Instead, they propose expanding the FoV of existing input images using a pretrained MVGenMaster model, incorporating coarse 3D geometry priors, mask latent blending, and iterative mask scheduling. The generated outpainted views are then used alongside the original input views to refine the 3DGS reconstruction. Experiments on various datasets demonstrate competitive reconstruction performance while substantially reducing computational costs compared to existing diffusion-based reconstruction methods.

Critical strengths
  • I think this framework is a simple and efficient approach for sparse-view reconstruction. Instead of generating additional novel views or refining each rendered image, GaMO expands the FoV of input images to increase the scene coverage.
  • The authors evaluate the effectiveness of GaMO through extensive experiments across multiple datasets and sparse-view settings. The ablation studies also demonstrate the contributions of the proposed components.
Critical weaknesses
  • The advantage of outpainting over novel-view synthesis could be better justified. The authors observe that generating intermediate novel views can degrade 3D reconstruction performance, while outpainting improves it. However, existing methods such as GenFusion and GuidedVD-3DGS have demonstrated that novel-view generation can improve sparse-view reconstruction. Moreover, Figure 11-(b) shows that SEVA-based outpainting performs worse than the original 3-view baseline in terms of SSIM and LPIPS. These observations suggest that the effectiveness of either approach may depend on specific implementation details, such as the generation backbone and supervision strategy. Therefore, the results in Figure 2 may not be sufficient to support a general conclusion that intermediate-view synthesis degrades reconstruction quality whereas outpainting improves it.

  • Unclear improvements in reconstruction accuracy versus scene completion. My main concern is that it is unclear whether the improvements mainly come from generating plausible content in previously unobserved regions or from more accurately reconstructing the underlying 3D scene. As shown in Figure 1, the qualitative improvements mainly appear to come from filling missing regions and reducing visible holes. However, generating visually plausible content does not necessarily guarantee accurate 3D geometry. The authors only report image-based metrics such as PSNR, SSIM, and LPIPS, without directly evaluating geometric reconstruction accuracy. Therefore, it remains unclear whether GaMO improves faithful 3D reconstruction or mainly enhances visual completeness.

Questions for authors
  • Could the authors clarify why intermediate-view synthesis degrades reconstruction performance in Figure 2, whereas existing methods successfully leverage generated novel views? In particular, how do the authors reconcile this observation with the SEVA-based outpainting results in Figure 11-(b)?
  • Could the authors evaluate reconstruction quality separately for regions already observed in the input views and regions newly covered by outpainting?
5 comments
AuthorsOpenReview ↗

Clarification of Outpainting and Novel-View Synthesis

1. Clarification of Outpainting vs. Novel-View Synthesis (W1/Q1)

We thank the reviewer for identifying the overly generalized conclusion in our original Figure 2, which we have removed from the revised manuscript (the original remains accessible in the previous version). Our intention was to show that directly adding generated novel views as supervision without sufficient cross-view consistency constraints can introduce inconsistencies and degrade reconstruction, rather than suggesting that NVS is inherently harmful. In contrast, GenFusion and GuidedVD-3DGS anchor generation to the current 3D reconstruction to promote geometric and cross-view consistency. We have clarified that the effectiveness of generated views depends on their guidance and consistency mechanisms.

We also acknowledge that SEVA-based outpainting improves PSNR but not SSIM or LPIPS over the 3-view baseline, showing that outpainting is not universally beneficial and depends on the generation backbone and supervision strategy. Accordingly, we removed the original Figure 2 and SEVA-related comparisons (including Figure 11 and the adapted-SEVA appendix comparison); these remain accessible in the previous version. We now position GaMO as a zero-shot multi-view outpainting diffusion model that generates geometrically consistent outpainted views to complement, rather than replace, NVS.

2. Complementarity with Existing NVS Methods (W1/Q1)

To test this revised positioning, we incorporated GaMO-generated outpainted views into Difix3D, GenFusion, and GuidedVD-3DGS and evaluated 3-view and 6-view reconstruction on Replica and ScanNet++. Average results (PSNR↑ / SSIM↑ / LPIPS↓) are:

MethodWithout GaMOWith GaMO views
Difix3D19.65 / 0.762 / 0.23321.84 / 0.785 / 0.197
GenFusion21.88 / 0.813 / 0.20923.04 / 0.823 / 0.190
GuidedVD-3DGS23.94 / 0.830 / 0.18824.55 / 0.845 / 0.166
GaMO-GS (standalone)23.93 / 0.843 / 0.156—

GaMO views improve average PSNR, SSIM, and LPIPS for all three methods, with PSNR gains of +2.19, +1.16, and +0.61 dB, respectively. Across 12 individual evaluation settings, LPIPS improves in all 12, while PSNR and SSIM improve in 11; the only decreases are marginal (−0.08 dB PSNR and −0.002 SSIM for GuidedVD-3DGS on ScanNet++ with 6 views). Full results are in Table 1 of the revised manuscript.

Terminology: GaMO denotes the outpainting diffusion model; GaMO-GS denotes the complete 3DGS pipeline using GaMO views (previously called “Ours”).

3. Geometric Accuracy and Region-wise Evaluation (W2/Q2)

We agree that visual completion alone does not establish accurate 3D reconstruction. We therefore added geometric evaluations on Replica and ScanNet++ (3-view and 6-view). We use Replica's ground-truth depth maps and render test-view ground-truth depth from ScanNet++'s laser-scanned meshes. We report Depth MAE, AbsRel, and F-score@5cm after per-scene median scale alignment. Test-view pixels are divided into Observed and Unobserved regions according to whether their corresponding ground-truth 3D points are visible from any input view. The averaged results are:

Method / RegionPSNR↑SSIM↑LPIPS↓MAE↓AbsRel↓F@5cm↑
FSGS / Full21.080.7960.2260.1600.1020.443
FSGS / Observed24.330.8330.1950.1530.0970.470
FSGS / Unobserved12.750.4370.4530.3420.2330.252
GuidedVD / Full23.940.8300.1880.1650.1050.418
GuidedVD / Observed25.560.8520.1660.1460.0920.451
GuidedVD / Unobserved18.140.6540.3330.3940.2480.243
GaMO-GS / Full23.930.8430.1560.1260.0820.520
GaMO-GS / Observed25.710.8620.1360.1130.0740.560
GaMO-GS / Unobserved16.990.6310.3060.3140.1960.266

GaMO-GS achieves better geometric accuracy in both observed and unobserved regions than the compared methods, showing benefits beyond plausible scene filling. We also acknowledge that unobserved-region image quality does not consistently improve across every metric. Evaluation details and additional results are in Appendix D (Table 8); key findings are discussed in Sec. 4.2.

Thank you for the clarifications and additional experiments, which address some of my concerns. With the revised positioning, I would appreciate further clarification of the methodological contribution. The finding that cross-view-consistent outpainted images improve sparse-view reconstruction supports the approach’s effectiveness, but does not by itself establish its novelty. The approach appears primarily to extend existing diffusion-based strategies to outpainting using established conditioning, blending, and resampling techniques.

Could the authors clarify what nontrivial methodological contribution GaMO offers beyond adapting these strategies to multi-view FOV expansion?

AuthorsOpenReview ↗

Methodological Novelty Beyond Existing Diffusion Techniques

We thank the reviewer for the follow-up question. We agree that conditioning, latent blending, and resampling are established diffusion techniques, and we do not claim these operations as individually novel. Our contribution lies in establishing geometry-aware multi-view FOV expansion as a complementary supervision strategy for sparse-view reconstruction, together with the specific designs that make geometrically consistent outpainting possible without training.

1. A complementary and plug-and-play source of supervision.

Existing diffusion-based sparse-view reconstruction methods primarily rely on synthesizing or refining views at new camera poses. In contrast, GaMO expands the FOV of the existing input views, providing additional scene coverage while retaining the original camera poses. This is a complementary source of supervision that can be incorporated into existing NVS-based pipelines without retraining their generative components.

Adding GaMO-generated views to Difix3D, GenFusion, and GuidedVD-3DGS improves LPIPS across all 12 evaluated settings (3 methods × 2 datasets × 2 view counts), with average PSNR gains of +2.19, +1.16, and +0.61 dB, respectively.

2. Why this is not a direct application of existing techniques.

Generating the outpainted views with the multi-view backbone alone, i.e., with its standard warped conditioning and none of our components while keeping every other stage of the pipeline unchanged, gives 24.39 dB novel-view PSNR and 0.152 LPIPS; the full GaMO reaches 25.20 dB and 0.135 (Tab. 3, rows 1 vs. 5). Three design choices close this gap:

  • Blending with a multi-view geometry prior, not the image’s own known pixels. RePaint-style blending re-injects the known region of the same image. In outpainting, the only known region is the input view at the center, which gives no constraint on the periphery to be generated. GaMO instead blends with a wide-FOV rendering of a coarse 3DGS built from all input views, so content observed by other views is carried over through 3D and generation is confined to regions the coarse reconstruction does not cover. Adding this blending raises novel-view PSNR by 0.54 dB (Tab. 3, rows 2 vs. 3), and replacing the coarse-3DGS prior with a direct DUSt3R point projection lowers it by 0.31 dB (Replica) and 0.62 dB (ScanNet++) (Tab. 7).

  • Handling an imperfect prior. The coarse 3DGS is itself a sparse-view reconstruction, and it is least reliable at the boundary of the region it covers, which is exactly where generated content has to connect to it. GaMO therefore starts from a dilated mask, letting the model overwrite this unreliable fringe, and tightens the mask over three blending steps to lock onto the reliable interior. This improves the outpainted views over a fixed mask (LPIPS 0.173 → 0.169) and is 1.8× faster than blending at every step (Tab. 4).

  • Conditioning that carries all known information into the enlarged view. In FOV expansion, the target retains the input camera pose while changing its FOV, so the input maps exactly onto the center of the target, independent of depth. Standard warped conditioning does not exploit this and leaves the generated content misaligned with the observed region (Fig. 6a). GaMO places the exact input observation at the center and keeps the cues warped from all input views around it, so everything that is already known constrains the generation.

In summary, our contribution is not a new diffusion primitive, but a geometry-aware, zero-shot multi-view outpainting formulation that addresses the specific challenges of fixed-pose FOV expansion. These designs let outpainted views serve as plug-and-play supervision that complements existing NVS-based sparse-view methods, without training or per-scene fine-tuning.

Reviewer_mSHdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies sparse-view 3D reconstruction. The authors argue that generating more novel views can introduce cross-view inconsistencies and may not provide reliable supervision. They instead expand the field of view at the existing camera poses. The method first builds a coarse 3D Gaussian scene, then uses geometry conditioning and mask-based guidance to jointly outpaint the input views with a multi-view diffusion model. The outpainted images are used to reinitialize and refine the final 3D Gaussian scene. Experiments on Replica, ScanNet++, and Mip-NeRF 360 show competitive reconstruction quality and substantially lower runtime than more expensive diffusion-based reconstruction pipelines.

Critical strengths
  1. Expanding the coverage of the existing views is an intuitive way to help sparse-view reconstruction. The observed regions serve as anchors, reducing the difficulty of changing the camera position while also generating new content. Figures 2 and 11 support this motivation with MVGenMaster and SEVA, respectively. Adding the outpainted views to GuidedVD also improves its results, suggesting that this additional supervision is useful.

  2. The method is well supported by experiments, and the efficiency gain has practical value. The paper studies geometry conditioning, latent blending, mask scheduling, resampling, and the final reconstruction stage. Results across three datasets and different numbers of input views support its usefulness. The appendix also compares end-to-end runtime on the same A6000 GPU, which makes the speed comparison convincing.

Critical weaknesses

The full-model results differ across the ablation tables without a clear explanation. Section 4.3 gives a common description of the six-view setup on Replica and ScanNet++, including the input cropping procedure. I would therefore expect Tables 3, 4, and 5 to use the same evaluation setup, but their full-model results differ. In particular, the novel-view PSNR in Table 5 is 1.28 dB higher than in Table 4. This does not necessarily invalidate the comparisons within each table, but it makes it difficult to understand how these ablations relate to the final method.

Questions for authors

1. Full-model results across ablation tables

From the setup described in Section 4.3, I understand Tables 3, 4, and 5 to use the same set of six-view inputs. However, the full-model novel-view PSNR is 23.53, 23.65, and 24.93 in these tables, respectively. Could you explain why these results differ? If the tables use different scenes or configurations, please clarify the differences.

2. Per-scene parameter selection

Appendix D mentions that the FOV expansion factor is adjusted between 0.5 and 0.7 depending on the scene, and that some 3DGS refinement parameters are also adjusted per scene. What information is used to make these choices? Is there a shared rule, or are the settings selected through trial and error for each scene? Listing these parameters and how they are chosen would help clarify how much manual tuning is needed in practice.

3. The role of coarse-GS optimization

After DUSt3R initialization, the method optimizes a coarse GS for 10,000 steps to guide outpainting. With sparse input views, this optimization may also introduce inaccurate geometry. What does this step mainly add compared with using projections of the DUSt3R point cloud directly for guidance? Could you include a comparison with the rest of the pipeline held fixed, to show its effect on outpainting and the final reconstruction?

4. Additional viewpoints and the scope of the claims

Appendix K suggests generating wide-FOV images from other camera positions to help with occluded regions. I am interested in this direction: have you made any preliminary attempts? Could adding a few such views alongside the current outpainted views further improve reconstruction? I would also suggest distinguishing regions outside the input FOV from occluded regions in the introduction, and softening the contribution statement about "eliminating" holes and other artifacts to reflect the occlusion limitations already acknowledged in the paper.

1 comment
AuthorsOpenReview ↗

Clarifications, Additional Experiments, and Manuscript Revisions

1. Clarification and Re-evaluation of Ablation Settings (W1/Q1)

We thank the reviewer for pointing out this inconsistency. Tables 3 and 4 originally used a 0.6× cropping setting to ensure ground-truth images were available for the outpainted regions, enabling quantitative evaluation of generation quality. This setting differed from that of the main comparison experiments, causing discrepancies in the Full Model results.

To resolve this, we re-evaluated the novel-view results of all ablations using the same evaluation settings as the main comparisons, averaging over the 6-view settings on Replica and ScanNet++. The updated results show that our full model achieves the best performance on nearly all metrics. We have updated Tables 3–5 in the revised manuscript. For clarity, the outpainted-view quality columns in Tables 3 and 4 still use 0.6× cropped inputs, since ground truth is required for these regions, whereas all novel-view columns now follow the main comparison protocol. This distinction is explicitly stated at the beginning of Sec. 4.3.

2. Clarification of Per-Scene Parameter Selection Criteria (Q2)

We thank the reviewer for this suggestion. Parameter selection is primarily guided by the geometric coverage and sparsity of the initial coarse 3D reconstruction. For each scene, we select an FOV expansion factor of 0.6 or 0.7 based on the initial coverage, and enable Point Re-initialization when needed to improve coverage in under-reconstructed regions. These choices are most relevant for sparse 3-view inputs, where scene coverage varies considerably. We have also corrected the originally stated 0.5–0.7 range: only 0.6 and 0.7 are used.

Separately, we use 3,000 optimization iterations for all 3-view settings to limit overfitting, and 7,000 for all 6-view and 9-view settings. We have added the complete per-scene settings in Appendix B.1 (Table 6) to improve reproducibility. Automating the coverage-based selection is an interesting direction for future work.

3. Ablation Study on Coarse-GS Optimization (Q3)

We thank the reviewer for suggesting this comparison. We conducted a controlled study on Replica and ScanNet++ under the 6-view setting, comparing RGB/alpha rendered from coarse 3DGS with RGB/mask obtained by directly projecting the DUSt3R point cloud, while keeping the diffusion, refinement, and training settings identical.

GuidanceReplica PSNR↑SSIM↑LPIPS↓ScanNet++ PSNR↑SSIM↑LPIPS↓
DUSt3R projection26.090.8730.11322.790.8230.210
Coarse 3DGS (Ours)26.400.8820.10423.410.8350.181

Coarse 3DGS consistently improves all three metrics on both datasets, providing more continuous RGB and geometric coverage guidance with fewer holes and discontinuities than direct point-cloud projection. We have added the experimental details and qualitative RGB/mask comparisons in Appendix C (Table 7 and Figure 9).

4. Additional Viewpoints and Clarification of Occlusion Limitations (Q4)

We conducted preliminary experiments on six Replica scenes, adding novel views at new camera poses (normal-FOV, wide-FOV, and bird's-eye). None improved upon GaMO-GS: full-image PSNR dropped by 1.2–1.9 dB. The degradation was concentrated in unobserved regions (2.5–5.2 dB), while observed-region PSNR decreased by less than 0.9 dB.

SettingPSNR↑SSIM↑LPIPS↓Observed PSNR↑Unobserved PSNR↑
GaMO-GS (Ours)26.400.8820.10427.0518.40
+ Novel views25.190.8650.12226.3015.87
+ Wide-FOV views24.650.8570.12726.2613.46
+ Bird's-eye view24.550.8580.12726.2213.22

We attribute this result to weaker observed-image anchoring at new camera poses: unlike input-pose outpainting, generated novel views lack fixed observed pixels and can introduce cross-view inconsistencies. Reprojecting input views to new poses and generating only disoccluded regions is a promising future direction. These preliminary results are discussed in Appendix J (Table 14).

Scope of claims. We agree that regions outside the input FOV should be distinguished from occluded regions. We have clarified this distinction in the Introduction and removed the claim of “eliminating holes and artifacts” from the contributions to accurately reflect the method's occlusion limitations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.