acceptodds
Withdrawn from ICLR 2027

Gate3DGen: A Pre-processing Model that Selectively Purifies Real-World Images for 3D Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Single-image 3D generation has achieved high fidelity on curated benchmarks, yet degrades sharply on real-world images with uneven illumination, blur, and occlusions. Purifying such inputs requires deciding when to intervene and preserving the visual cues needed for 3D generation. We introduce **Gate3DGen**, a selective pre-processing model that gates image inputs to existing 3D generators without retraining them. A degradation-aware router bypasses inputs classified as clean and extracts features to guide purification of the remaining inputs. Conditioned on these features, a 3D-oriented one-step purifier purifies degraded inputs with spatial losses that encourage preservation of object identity, contours, and textural cues. To provide paired supervision, we construct a synthetic data engine that renders clean views of 3D assets and synthesizes composite degradations. On the synthetic benchmark, the purifier improves image fidelity and downstream 3D geometry, reducing TRELLIS.2 mesh distance by 61.5% relative to degraded inputs. VLM evaluation and human preferences on real photographs further support identity preservation and degradation correction. Code and model will be released.

Primary area:
generative models
Keywords:
3D Generation, Image Preprocessing, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_HggaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Gate3DGen is a preprocessing framework for improving single-image 3D generation from degraded photographs. It combines a degradation-aware router with a one-step image purifier, while keeping downstream 3D generators frozen. The authors construct synthetic paired training data and evaluate image restoration and downstream 3D quality using two generators, supplemented by real-image evaluations. Although the application is relevant and the purifier shows promising results, the evidence does not sufficiently establish the effectiveness of the selective routing mechanism or the proposed 3D-oriented spatial constraints.

Critical strengths
  1. Relevant practical problem. Improving robustness to occlusions, blur, and uneven illumination addresses an important limitation of single-image 3D generation. Preprocessing inputs without retraining downstream generators is a practical approach.
  2. Encouraging synthetic-benchmark results. The purifier improves several geometry and appearance metrics over degraded inputs, including a substantial reduction in mesh distance on TRELLIS.2.
Critical weaknesses
  1. The central claim of selective purification is not validated end to end. Section 4.1 explicitly states that all degraded inputs are purified and routing decisions do not affect the main reconstruction results. These experiments therefore establish the utility of the purifier, but not the benefit of the complete gated system. Moreover, on real photographs, the router achieves 58.33% accuracy, matching the trivial strategy of always predicting PURIFY under the proxy-label protocol. This is a substantial gap for a method whose central motivation is deciding when to intervene.
  2. The ablations provide weak support for the proposed spatial constraints. In Table 5, adding spatial losses worsens MD from 0.755 to 0.764 and lowers 2D PSNR, while providing no improvement in the reported F1 metrics. The improvements in shaded and albedo LPIPS are only 0.0004 and 0.0002, respectively, without uncertainty estimates. These results do not convincingly support the claim that the spatial constraints better preserve the geometric cues needed for downstream reconstruction.
  3. The methodological contribution is insufficiently separated from task-specific fine-tuning. The purifier is initialized from an existing image-editing model and trained with standard latent, photometric, perceptual, region-weighted, and edge objectives. No direct 3D supervision or gradient through the downstream generator is used. This does not invalidate the approach, but makes controlled comparisons particularly important. The main baselines are general-purpose editors, and the paper lacks a comparably trained restoration baseline using the same paired data and training budget. Consequently, it remains unclear how much of the improvement comes from the proposed design rather than adaptation to the training distribution.
  4. Real-world evidence does not adequately establish the claimed robustness. The real-image evaluation contains only 96 photographs and relies on VLM judgments, proxy routing labels, and human preferences without ground-truth geometry. The human study reports 341 valid ratings, but the allocation across comparisons, participant details, and uncertainty estimates are insufficiently specified. Furthermore, the method does not lead the VLM asset-quality scores, and the reported FD-DINOv2 results do not improve over unedited inputs. These outcomes warrant a more limited conclusion about real-world benefits.
  5. The efficiency motivation lacks measured support. A single purifier evaluation does not by itself establish low end-to-end cost, since the pipeline also includes a VLM router and feature extraction. Without latency and memory measurements for the complete system, the practical benefits of one-step purification and selective bypass remain unsubstantiated.
Questions for authors
  1. On a mixed set of clean and degraded inputs, how does the complete gated pipeline compare with always purifying and never purifying in downstream 3D quality? Are such results available for real photographs?
  2. What evidence supports the necessity of the spatial losses given the results in Table 5? Are the small rendering improvements consistent across runs, and is the claimed improvement in contour preservation quantitatively measured?
  3. Is there a baseline trained on the same paired data with a comparable parameter budget and standard restoration objectives? Such a comparison is needed to distinguish the contribution of the proposed architecture and conditioning from task-specific training.
  4. How are identity, geometry, and viewpoint consistency verified when generating degraded training pairs with image editors? Please specify the synthesis models, filtering criteria, and any manual validation of the resulting pairs.
  5. What are the measured end-to-end latency and peak memory usage, including routing and purification? For the human study, how many independent participants and comparisons support each reported preference rate?
Reviewer_G3r6OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

Gate3DGen improves 3D generation from single images by preprocessing image inputs. A VLM-router (Qwen2.5-VL based) selects KEEP or PURIFY operations. If PURIFY, then VLM's hidden states go through a 1-layer DiT to preprocess the input, making it ready for 3D lifting (TRELLIS.2 based). They report synthetic purification results and 61.5% reduction in TRELLIS.2 mesh distance, alongside VLM and human evaluations.

Critical strengths
  1. Clear writing and easy to understand figures, which is appreciated.
  2. The preprocessing can be useful for 3D lifting.
Critical weaknesses
  1. I am not sure about the motivation of this work. It's evident that 3D lifting networks benefit from "purified" 2D images. However, the distinction between "3D-oriented purification" and image restoration is not clear. Because the objectives in Sec. 3.4 and Eq. 8–9 supervise image and latent reconstruction, without directly measuring downstream geometry. One specific aspect compared to conventional image restoration could be occlusion, but the paper does not discuss important baselines like Amodal3R and pix2gestalt either, even though Sec 3.2 mentions training regime includes occlusion. Besides that, edge constraints make sense but Tab 5 does not benefit from them [see W3]. What makes the purification specifically "3D-oriented" therefore remains unclear to me.

  2. Why do we need a router? The paper states purification can introduce unwanted changes (Sec 3.1-3.3) but they don't show what happens when already good samples are passed through the purifier. Do we have speedups from not purifying all samples? Every input still needs to pass through the VLM anyway, and the paper does not report runtime measurements, either. This raises a question about the necessity of a router.

  3. No unified evaluation. Tab 1 only measures gating, and Tab 3-7 only measure reconstruction with all samples purified already (lines 278-279). So the paper doesn't have an experiment what do we benefit from adding a routing.

  4. Additional losses remain unclear to me. The authors introduce these losses to preserve structural cues for 3D reconstruction (Sec 3.4 and Eq 8). However Tab 5 shows that Full produces similar or slightly worse geometry scores than B, with only small improvements in image similarity. Moreover, Sec 4.6 states that Full preserves contours better than B, but provides no direct comparison of contour accuracy.

Questions for authors
  1. [W1] What specifically distinguishes "3D-oriented purification" from faithful image restoration and completion, considering the image-based training objectives and related work like pix2gestalt and Amodal3R?

  2. [W2] Does purifying already good inputs harm downstream reconstruction? Does skipping purification provide a meaningful speedup?

  3. [W3] How does the complete gated pipeline compare with always-purify and always-keep cases, in terms of runtime and reconstruction quality?

  4. [W4] What evidence supports the claimed geometry benefits of the spatial losses, given that Full does not improve geometry scores over B in Tab. 5?

Reviewer_NnUGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents Gate3DGen, a preprocessing system for improving image-to-3D generation from degraded photographs. A Qwen-based router decides whether to keep an image unchanged or send it to a one-step purifier adapted from FireRed. The purifier is trained on synthetic image pairs using reconstruction, perceptual, region-weighted, and edge losses. The processed images are then passed to frozen TRELLIS.2 and Pixal3D models. The authors evaluate image restoration and downstream 3D quality, with separate routing experiments and additional VLM and human evaluations on real photographs.

Critical strengths

The paper addresses a practical problem. Occlusions, blur, and poor lighting can lead to unwanted geometry or appearance artifacts in image-to-3D generation. Improving the input without retraining the downstream generator makes the approach easy to apply to different models.

The purifier shows measurable benefits. The evaluation includes 1,082 synthetic cases with 3D ground truth and two downstream generators. The geometry improvements and real-image user preferences provide evidence that the trained purifier is useful in the tested settings.

Critical weaknesses
  1. The methodological contribution is limited. The approach mainly adapts existing models for input preprocessing. The claimed “3D-oriented” losses do not show a clear benefit: in Table 5, adding them worsens mesh distance from 0.755 to 0.764, with only a small improvement in rendering LPIPS. This leaves the value of a key proposed component unclear.

  2. The benefit of selective routing is not established. The main generation experiments purify every degraded input rather than test the complete gated pipeline. On real images, routing accuracy against GPT-5 proxy labels is 58.33%, matching always predicting PURIFY. A comparison on mixed clean and degraded inputs is needed to show that routing improves 3D quality while avoiding unnecessary edits.

Questions for authors

How does the complete gated pipeline compare with always purifying and always keeping the input on a mixture of clean and degraded real photographs? Please evaluate downstream 3D quality and how often purification makes a suitable input worse.

What clear benefit do the spatial losses provide over Variant B in Table 5? Can you provide evidence for the claimed improvement in contour preservation and explain why the full configuration is preferable despite its slightly worse geometry metrics?

How reliable are the GPT-5 routing labels on real photographs? Have they been checked against human judgments of whether purification actually improves the resulting 3D asset?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.