acceptodds
Withdrawn from ICLR 2027

DreamSurfels: Single-Image 3D Generation with Geometry-Conditioned Video Diffusion

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generating a detailed 3D mesh from a single image requires both plausible completion of unseen regions and consistent 3D geometry across the generated views. Existing methods that reconstruct 3D from synthesized views can propagate inconsistencies across viewpoints into the final mesh. Video diffusion provides a powerful prior for coherent view sequences, yet visual coherence does not guarantee that the views share a consistent 3D surface. To this end, we introduce DreamSurfels, a geometry-conditioned video diffusion framework for single-image 3D mesh generation. To provide explicit geometric guidance to video diffusion, we lift the visible surface from the input view into Gaussian surfels and render the partial geometry at target viewpoints. Residual conditioning branches inject these renderings into the diffusion backbone, anchoring the generated views to the observed surface as the model completes unseen regions. Beyond consistent view synthesis, accurate mesh reconstruction requires surface orientation cues that RGB appearance alone cannot reliably provide. We therefore develop a joint RGB-normal video diffusion model that uses intermediate RGB features to guide surface normal predictions, aligning geometry with the generated views. With the resulting geometrically consistent RGB views and surface normals, our framework then reconstructs detailed textured meshes. Experiments show that DreamSurfels produces consistent multi-view geometry and high-quality mesh reconstructions.

Primary area:
generative models
Keywords:
3D Generation

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_HCctOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DreamSurfels for single-image 3D generation. It uses Gaussian surfels to provide geometric guidance for video diffusion and adopts a dual-stream DiT to generate RGB images and surface normals. A coarse-to-fine reconstruction pipeline is then used to produce textured meshes. Experiments show improvements over several baselines.

Critical strengths
  1. The paper is clearly written and easy to follow.
  2. The paper addresses the important problem of geometric inconsistency in single-image 3D generation. The motivation is clear, and the overall framework is logically organized.
  3. The paper provides a reasonably comprehensive experimental evaluation, including comparisons with representative methods across RGB generation, normal prediction, and mesh reconstruction.
Critical weaknesses
  1. The technical novelty is limited. Geometry-guided view generation and RGB-normal generation have been studied in previous works. The dual-stream DiT and MLP-based feature injection are also relatively simple. The authors should better explain what is new compared with existing methods.
  2. The evaluation of Gaussian surfel conditioning is insufficient. Figure 2 only shows qualitative results, without quantitative comparisons. Also, cross-view geometric consistency is a key motivation, but the paper does not directly evaluate it. More ablation studies and geometric consistency metrics are needed.
  3. The comparison with related methods is incomplete. Since this paper uses video diffusion for single-image 3D generation, it would be useful to compare with video-based methods such as SV3D and Hi3D. The authors should provide comparisons under similar settings or explain why these methods are not included.
  4. The qualitative results mainly show relatively simple objects. The authors should provide more challenging cases, such as objects with complex structures, thin parts, or severe self-occlusions, to better demonstrate the robustness of the proposed method.
Questions for authors

Please see the weaknesses.

Reviewer_ve3vOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper targets on addressing single-image 3D mesh generation through geometry-conditioned video diffusion. The method first estimates the velsisible geometry and surface normals of the input image using a fine-tuned MoGe-2 model. These estimates are converted into oriented Gaussian surfels, whose renderings provide partial RGB and normal conditions at target viewpoints. The generation model is built on Wan2.1-VACE-14B and uses separate RGB and normal diffusion streams. The RGB stream is trained first and then frozen during inference, and its intermediate features are injected into the normal stream through learned adapters. The generated observations are then used for the reconstruction process by Instant-NSR, Gaussian-surfel refinement, and screened Poisson surface extraction.

The evaluation was conducted on 100 held-out Objaverse objects, and compared three selected baseline methods. The overall approache proposed makes sense to me but the technical novelty and distinctive contributions of this paper are very limited in my opinion.

Critical strengths
  • Although I think the idea is not new, the motivation distinguishes visual coherence across generated views from compatibility with a shared 3D surface is plausible. Thus, the combination of both is motivated. The paper appropriately acknowledges that its geometric conditions are soft and do not guarantee agreement on unobserved surfaces.

  • DreamSurfels achieves the best reported results in Tables 1–3.

  • Appendix D compares normal generation with and without intermediate RGB features while retaining the normal backbone and partial-normal conditioning.

Critical weaknesses
  • As I mentioned above, the idea of combining normal and rgb is well motivated but not new/novel to me. The motivation of normal generation doesn't support the choice of surfel representation in this paper. The comparisons in Tables 1–3 involve systems with different backbones, adaptation procedures, geometry estimators, and reconstruction pipelines. Consequently, they do not establish how much of the reported improvement comes specifically from surfels or from the proposed reconstruction sequence. The most informative controls would be 1) Geometry conditioning; 2) Visibility; 3) Reconstruction refinmention.

  • The related-work discussion omits NOVA3D [1] and SViM3D [2], both of which are directly relevant to the RGB–normal video-generation component.

  • The current measurements support improved fidelity and downstream reconstruction quality. However, they do not directly establish whether generated views agree on a shared surface. E.g., high foreground IoU is compatible with substantial disagreement in interior surface orientation, and therefore should not be interpreted as a direct measure of geometric alignment.

  • I am also concerned that several omissions affect both reproducibility and confidence in cross-method comparisons. First, the conversion from MoGe-2 outputs to the target-view coordinate system. Second, the actual surfel visibility mechanism is underspecified. Flattening a Gaussian covariance does not by itself define signed front/back visibility. Third, the evaluation protocol needs clarification. For normals, the authors should explain how outputs from different methods are converted into a common coordinate convention, which foreground mask is used, and how missing foreground predictions are treated. For meshes, normalizing distances by the ground-truth bounding-box diagonal does not specify how predictions are registered to ground truth.

  • The objectives, loss weights, optimization schedules, and extraction settings needed to reproduce the Instant-NSR and surfel stages are missing as well.

[1] NOVA3D: Normal Aligned Video Diffusion Model for Single Image to 3D Generation [2] SViM3D: Stable Video Material Diffusion for Single Image 3D Generation

Questions for authors

See weakness section for details.

Reviewer_mXqUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DreamSurfels, a single-image 3D generation framework that integrates geometry-conditioned video diffusion with surface reconstruction. The method first estimates the visible geometry from the input image and represents it using oriented Gaussian surfels. These surfels are rendered from target viewpoints to provide partial RGB and normal guidance for video diffusion. The authors further introduce RGB-guided normal denoising, where intermediate features from a pretrained RGB diffusion stream guide normal generation. Finally, the synthesized RGB views and normals are used for coarse-to-fine mesh reconstruction through Instant-NSR, Gaussian surfel refinement, and Poisson reconstruction.

Experiments on Objaverse demonstrate improvements in novel-view synthesis, normal estimation, and final mesh reconstruction compared with the evaluated baselines.

Critical strengths

Effective integration of geometric guidance into video diffusion. The use of oriented Gaussian surfels to provide view-dependent geometric guidance is well motivated. The method combines explicit input-view geometry with video diffusion priors to encourage structural consistency across generated views. The quantitative results demonstrate improvements in view synthesis and final mesh quality.

Promising RGB-normal generation with supporting ablations. The proposed RGB-guided normal denoising effectively couples appearance generation with surface orientation prediction. The ablation demonstrates substantial improvements in normal accuracy and RGB-normal alignment when RGB feature guidance is enabled. The resulting mesh reconstruction also achieves competitive geometric fidelity.

Critical weaknesses

Insufficient ablations to isolate the central contributions. While the paper evaluates the effect of RGB-guided normal denoising, it lacks controlled ablations for the proposed Gaussian surfel conditioning. In particular, it is unclear how much improvement comes from using oriented Gaussian surfels rather than simpler geometric representations such as point clouds, depth projections, or standard 3D Gaussians. A comparison against the same video diffusion backbone without geometric conditioning is also missing. Since the method builds on a powerful pretrained Wan2.1-VACE-14B backbone and a fine-tuned geometry estimator, the current experiments do not convincingly establish the individual contributions of the proposed components.

Limited evaluation and computational efficiency. The quantitative evaluation uses only 100 held-out Objaverse objects with predefined camera trajectories, which limits evidence of generalization to more diverse objects and real-world images. Moreover, the baseline methods use different model architectures and training procedures, making it difficult to attribute the reported improvements solely to the proposed methodology. The complete pipeline also requires approximately 15 minutes per object, considerably limiting its practicality compared with more efficient feed-forward approaches.

Questions for authors

Can the authors provide controlled ablations comparing Gaussian surfel conditioning against standard 3D Gaussians, point-cloud projections, and no geometric conditioning, using the same video diffusion backbone and training setup?

What is the individual contribution of the proposed coarse-to-fine reconstruction pipeline? For example, how does reconstruction using only Instant-NSR compare with the full pipeline including Gaussian surfel refinement? It would also be useful to quantify how much the generated normal maps improve final mesh geometry.

Can the authors evaluate the method on a larger and more diverse benchmark, including real-world images and unseen camera trajectories? Additionally, can they provide an inference-time breakdown and comparisons with more recent efficient 3D generation methods?

Reviewer_A5YUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents DreamSurfels, a framework for single-image 3D generation using geometry-conditioned video diffusion. The method introduces Gaussian surfel-based geometric conditioning and RGB-guided normal diffusion, followed by a coarse-to-fine mesh reconstruction pipeline. Experiments on Objaverse demonstrate improvements over existing methods in multi-view synthesis, normal prediction, and mesh reconstruction.

Critical strengths

Using oriented Gaussian surfels for geometric conditioning is an interesting design choice that helps preserve the observed surface geometry.

Critical weaknesses
  1. Limited novelty. The main technical components are closely related to existing works:

Geometry-conditioned video diffusion: ViewCrafter [1] and VidSplat [2] have explored video diffusion guided by rendered 3D geometry.

Joint RGB-normal generation: Wonder3D [3], and SViM3D [4] have investigated joint RGB-normal generation with video diffusion models.

Surface reconstruction: Wonder3D++[5] has explored coarse-to-fine mesh extraction.

[1] Yu et al. "ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis." IEEE TPAMI, 2025.

[2] Tang et al. "VidSplat: Gaussian Splatting Reconstruction with Geometry-Guided Video Diffusion Priors." SIGGRAPH, 2026.

[3] Long et al. "Wonder3D: Single Image to 3D using Cross-Domain Diffusion." CVPR, 2024.

[4] Engelhardt et al. "SViM3D: Stable Video Material Diffusion for Single Image 3D Generation." ICCV, 2025.

[5] Yang et al. "Wonder3D++: Cross-Domain Diffusion for High-Fidelity 3D Generation From a Single Image." IEEE TPAMI, 2026.

  1. Outdated baselines. The experimental comparisons are largely limited to old methods. In particular, all baselines for the final 3D mesh generation task (Table 3) are from 2024 or earlier, despite substantial progress in single-image 3D generation. Recent methods such as TRELLIS, Hunyuan3D 2.1, and TRELLIS.2 are missing. Furthermore, SViM3D and are highly relevant to the proposed RGB-normal video generation framework but are not discussed or evaluated.

  2. Limited evaluation and baseline fairness. The quantitative evaluation uses only 100 synthetic Objaverse objects. Generalization to real-world images remains unclear.

Questions for authors

see above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.