acceptodds
Withdrawn from ICLR 2027

UniComplete: Unified Multimodal High-Resolution 3D Shape Completion via Anchored Latent Sets

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing shape completion methods accept a single input modality and, when built on dense voxel grids, deliver watertight surfaces only at low resolution. Latent vector sets decouple the latent from the grid, but their latent tokens come from input-dependent queries, lack of consistent correspondence between partial and complete shapes, and surface extraction still needs a dense field query. We present UniComplete, a framework for multimodal shape completion built on an anchored latent set: each token is bound to a fixed canonical 3D anchor, so it keeps the same identity and spatial reference across shapes, modalities, and completeness levels. An SDF VAE defines the shared latent space, and modality-specific encoders are distilled into it, so signed and unsigned distance fields, occupancy grids, point clouds, and depth maps share one decoder and one completion model. Completion is a posterior-mean rectified flow: a partial encoder regresses the posterior-mean estimate of the complete latent, and a rectified flow started from that estimate yields multiple plausible completions in a few steps. Sparse surface encoding and a coarse-to-fine Surface Gate confine computation to the surface at the input and the output, so cost scales with the surface rather than the volume. We further introduce SfMComplete, a benchmark pairing partial shapes reconstructed by structure-from-motion from rendered images with exact CAD ground truth across modalities and resolutions. UniComplete is competitive on 3D-EPN and PatchComplete, improves over compared baselines on SfMComplete at higher resolutions, and completes all five input modalities at SDF resolutions up to .

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D shape completion, generative 3D, latent set representation, flow matching, multimodal 3D input, high-resolution 3D generation, Shape completion benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Ej4zOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

UniComplete reconstructs and completes 3D shapes from partial views in one framework. It takes five input modalities, SDF, UDF, occupancy grids, point clouds, and depth maps, and runs them through one shared decoder and one completion transport. The representation at the center is a set latent with fixed anchors: 40 tokens, each bound to a fixed canonical 3D position, where the positions come from a one-time k-means fit over training-set surface points. Because the anchors never move, a token keeps the same identity and spatial reference across shapes, modalities, and levels of completeness, and a partial view fills every anchor from the observed surface instead of leaving holes. This latent is defined by an SDF VAE built on the COD-VAE backbone. Every modality encoder, and later every partial encoder, is distilled into the frozen teacher latent with the alignment loss of Eq. 1, which combines per-token ℓ1 error, pairwise token distances, KL, and an SDF term; the partial encoder therefore becomes a posterior-mean regressor. Completion starts from its output: a rectified flow with source noise carries the estimate to complete latents in 2 Euler steps, and the noise scale α is the one knob between fidelity and diversity. A surface-only design keeps high resolution affordable: at the input, the sparse encoder reads only a 3-voxel band; at the output, a Surface Gate classifier cuts dense SDF queries down to surface-bearing cells, saving over 80% of queries with 100% recall; in the loss, supervision is split over surface, near-surface band, and outer regions. The authors also introduce SfMComplete. It has about 35K ShapeNet objects and four SfM trajectories each, so about 140K partials carry real pipeline noise while the ground truth stays the exact CAD model, in all five modalities at 64³ to 512³. On this benchmark their method leads every baseline at matched modality and resolution, though the baselines run out of memory beyond 128³ and 256³. On the standard 32³ benchmarks it is second to BridgeShape.

Critical strengths
  1. Encoder-only modality extension is shown to work. The SDF teacher stays frozen, and four encoders are distilled against the shared anchors. Their alignment scores span 62.81 to 64.14 at 128³, about 1.3 IoU points, and their completion scores sit within about one point of the SDF teacher, with UDF 64.79 and PC 64.72 slightly above the SDF teacher's 64.09, all through one shared transport and decoder (Tab. 4). In practice this means a new input modality can be added without touching the decoder or flow, and the staged schedule in Appx. B makes the whole process reproducible.

  2. SfMComplete fills a real benchmark gap, and it is built soundly. Existing 32³ benchmarks give clean synthetic scans with approximate or low-resolution ground truth, so real pipeline noise and exact ground truth did not meet in one benchmark before. Here they do: the partials carry the noise of a real pipeline, holes, viewpoint-dependent density, and floaters, while the ground truth stays the exact CAD model, across all five modalities and 64³ to 512³. The splits also prevent leakage, because they are 80/5/15 by object identity and the four trajectories of one object share a split. Appx. D and Tab. 9 justify the design against 3D-EPN and PatchComplete. The baseline protocol is unusually careful as well: official code, one model for all categories, shared sampling seeds, documented retraining details, and both best-of-five and mean-of-five reported per column.

  3. Surface-only computation is checked at both ends, and the cost is stated. The Surface Gate catches every sign crossing while saving 81.6 to 87.8% of queries (Tab. 6). Training memory up to 512³ is dominated by the O(R²) surface term (Fig. 7). The efficiency protocol in Tab. 13 is fixed and documented, including the analytic-sphere model used for the resolution sweep, so the numbers can be compared across methods. The paper also does not hide the fidelity cost: reconstruction peaks at 128³, 88.43 IoU, and falls to 82.52 at 512³.

Critical weaknesses
  1. The anchored latent is a compromise between existing designs, and the "none before us" claim fails against uncited prior art. Grid latents already bind every token to a fixed identity and position. XCube does so sparsely and is cited. LATTICE (arXiv:2512.03052, CVPR 2026, uncited) proposes exactly "a compact set of latent vectors anchored to a coarse voxel grid", which predates this submission. What UniComplete actually adds is one shared 40-point anchor set instead of an input-dependent active-voxel subset. That variant is meaningful for token correspondence, but it is one query-scheme choice inside a known design space of FPS queries, learnable queries, and fixed anchors, and Tab. 7 shows exactly that trade-off. So the representation is a variant, not a new problem framing. The completion half has the same status, because posterior-mean regression then flow is PMRF applied in the latent, latent-to-latent transport from the partial is BridgeShape's starting point, and the backbone is adopted COD-VAE with a TripoSG-style DiT. The paper's own ablation puts the gain over a Brownian bridge at 1.04 IoU and 0.19 CD, 65.58 against 64.54 (Tab. 8), which is real but small.

  2. The two headline components are technically separate; only the story joins them. The anchored latent serves modality alignment and the posterior-mean start, and the surface-only parts, sparse surface encoders, coarse-to-fine output gating, and stratified supervision, serve efficiency. If the paper were one connected idea, then removing one component should hurt the other. It does not: the flow consumes the same anchored latent at 64³ as it does at 512³, and nothing in the Surface Gate depends on anchors being fixed. So the claim of addressing both limits coherently reduces to running both parts in one system. The resolution story even works against itself, because the ceiling is set by the fixed-size latent, which the surface-only design cannot lift: reconstruction peaks at 128³ and falls at 256³ and 512³ (Tab. 4). Three competent adaptations on one backbone is a fair description; one insight it is not.

  3. "Multimodal" is not fusion, and the modalities never help each other. Each modality has its own encoder, each encoder is trained on its own against the frozen teacher, and nothing couples them at train or test time, so inference takes exactly one modality. The paper contrasts this with multi-input methods, RGB beside a point cloud, prompts, and condition branches, and the contrast is accurate, but it hurts the paper's own case: those works use whichever extra sense is available, and this use is exactly the benefit cross-modal completion offers under ambiguous partials, while UniComplete gives it up by design. Encoder-only extension is good engineering, but calling modality substitution the unified multimodal completion problem overstates it. One more caveat belongs here: the depth row of Tab. 4 uses clean rendered depth rather than the noisy SfM partial, so it is the only modality whose input is not pipeline-realistic, and the paper flags this correctly.

  4. The decisive comparisons run where the baselines cannot. On the two standard 32³ benchmarks UniComplete is second to BridgeShape everywhere: 0.043 against 0.039 ℓ1 on 3D-EPN, 67.7 against 70.1 IoU on PatchComplete synthetic, and 52.0 against 53.5 on the real split. The abstract claim of improvement at higher resolutions rests on SfMComplete operating points where DiffComplete runs out of memory at 128³ and BridgeShape at 256³. Matching by modality and protocol is fair, but a win obtained only where the other method cannot run does not prove superiority at high resolution. The reconstruction comparison cuts both ways, and neither direction settles the question. At 64³ BridgeShape's VAE is clearly ahead, 92.08 against 86.43 IoU, so part of the completion advantage there is a swap of transport for VAE rather than a representation win at equal capacity. At 128³ the ranking reverses sharply, 88.43 against 72.60, and I read that reversal as evidence about BridgeShape rather than about UniComplete: its dense 16³ VQ latent is not built to extend past the resolution it was trained at, so its VAE collapses there. The paper anticipates this and states that the grid-code edge "lasts only while the grid is compact" (§4.2). That reading does not rescue the high-resolution claim, however, because it converts the 128³ win into one more instance of the same problem this weakness describes: the decisive result is produced at an operating point where the comparison method degrades rather than one where both are at capacity.

Questions for authors
  1. Reconcile the "None gives token i both a fixed identity and a fixed position" claim with LATTICE/VoxSet (arXiv:2512.03052, CVPR 2026, uncited), with VQ and grid latents, and with the irregular-latent-grid line you already cite as 3DILG (Zhang et al., 2022a), which §2 lists among schemes that "fix positions but resample them per input". Beyond that, measure what the fixed 40-anchor set adds over the scheme comparison of Tab. 7; per-token posterior variance of partial versus complete latents of the same object, or tokenizing one object with two modalities and decoding the cross-swapped token sets, would do it directly.

  2. Can the framework take two partial modalities at once, for example concatenated tokens on the shared anchors? If yes, report it. If not, a depth-plus-SDF probe would show how much complementary information the substitution design gives up. In that case, the word "multimodal" in the abstract and introduction should become modality-swappable.

  3. The conclusion says one Euler step, while §4.1 and Tab. 5 use two. Which is it? Also report the accuracy-diversity frontier over α, since the paper shows a single point at α=1. And Tab. 3 marks TMD with ↑ while the text treats "our TMD is the lowest" as a cost, so please fix one of the two.

  4. To support "improves over compared baselines at higher resolutions" beyond the baselines' OOM points, add a capacity-matched bridge baseline, for example BridgeShape with reduced VQ channels, or restate the claim as "at resolutions where the compared baselines cannot run".

  5. Redraw Figure 1 so the caption matches the figure: one object per block, or an explanation of the two objects, the middle row, and the arrows. Enlarge the panel text of Figure 2. Fix the typos listed in the Presentation field, including the broken "Fig. ??" reference in the Ethics statement.

Reviewer_yjfHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents UniComplete, a unified framework for multimodal 3D shape completion. It introduces an anchored latent set to align different input modalities and partial/complete shapes. A posterior-mean rectified flow is used for shape completion, while sparse surface encoding and a SurfaceGate enable high resolution outputs. The authors also introduce SfMComplete, a benchmark for evaluating shape completion across different modalities and resolutions.

Critical strengths

The anchored latent representation provides a reasonable approach to unifying different input modalities within a shared latent space.

The framework supports high resolution shape completion, with encouraging results on the proposed SfMComplete benchmark.

Critical weaknesses

Limited validation on real-world noisy inputs. The main performance improvements are demonstrated on the proposed SfMComplete benchmark, which is constructed from synthetic renderings of ShapeNet objects. Although SfM introduces reconstruction artifacts, the evaluation does not fully reflect the complexity of real-world scans. In addition, baseline BridgeShape outperforms the proposed method on 3D-EPN and PatchComplete. Additional evaluation on real-world noisy inputs is needed to support the generalization claims.

Insufficient validation of multimodal generalization. While the method supports five input modalities, most are derived from the same underlying SfM reconstruction setting, rather than independently collected sensor observations. The depth inputs are clean renderings from CAD models, making them less challenging and not directly comparable to the other modalities. It remains unclear how well the unified latent representation handles real-world inputs, such as noisy depth measurements or LiDAR point clouds.

Questions for authors

Can the authors provide additional experiments on real-world scans with different noise levels, outliers, or incomplete observations to explain the performance gap with BridgeShape on PatchComplete Real?

Can the authors clarify how sensitive the anchored latent representation is to object alignment and unseen object categories?

How well does the proposed framework generalize to real world input modalities, such as real depth sensor measurements and LiDAR point clouds?

Reviewer_whZBOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a method for 3D shape completion from multimodal inputs. The method and network design is simple. Have different encoders for different modalities and map the information to a unified space and then decode the latents to a final SDF of the complete 3D shape.

Critical strengths
  • Unified latent space for shape reconstruction and completion. This method can actually reconstruct or complete 3D shapes from different modalities because of the unified latent space.

  • The paper is well written and clear to follow.

Critical weaknesses
  • Novelty: I could not find any novelty in this approach. The network design and the approach to have a unified space seems obvious way to solve this problem.

  • Results are not good enough. The method despite taking multi-modal inputs does not give best results compared to the baselines as shown in Table 1 and 2.

  • I feel this problem can be easily solved with 3D generative models like Trellis. So I was wondering if the authors have tried this approach. May be, a simple baseline could be to just train encoders as a conditioning signal to the diffusion model and then use the generative power of 3D generative models to reconstruct the final shape. I feel this baseline is more simple and could even perform better than this method.

  • The presentation of the paper could be significantly improved, especially the teaser is very small and also the results and pipeline figure i.e. Fig. 2 and Fig. 3 could be improved.

Questions for authors

I would like to see how 3D foundational generative models perform on this task.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.