- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes UMI3D, a training-free framework that restructures cross attention by explicitly routing each voxel to its most informative image, which prevents conflicting visual evidence from being indiscriminately mixed during generation, thereby unlocking strong performance on inconsistent inputs. To realize this routing, UMI3D introduces Simultaneous Focusing Cross-Attention (SFC-Attn), which activates all conditioning images at each denoising step while allowing each voxel to focus on the single image that best explains it. Extensive experiments show that UMI3D unlocks the multi-image potential of single-image 3D generation frameworks across diverse tasks.
Critical strengths
A training-free framework that adapts single-image 3D foundation models to unconstrained multi-image inputs without updating the backbone parameters. A new benchmark for unconstrained multi-image 3D generation with realistic cross-view inconsistencies.
Critical weaknesses
How to verify the consistency of multi-image 3D generation in the experiment? The corresponding quantitative results and qualitative results should be provided. Why the results of UMI3D and other methods in terms of other metrics (LPIPS, DINO, CLIP) present near performance while User Study results of UMI3D has significant gain compared other methods?
Questions for authors
There are many generation designs are proposed for 3D generation, such as [1] DyFusion: Cross-attention 3D object detection with dynamic fusion. [2] Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attention [3] Ground-to-Satellite Localization in Unconstrained Image Collections for 3D Scene Reconstruction [4] Wonder3d++: Cross-domain diffusion for high-fidelity 3D generation from a single image [5] Svim3d: Stable video material diffusion for single image 3d generation [6] MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models There methods should be compared and discussed. The similar designs should be ablated in the experiments. How to evaluate the performance of multi-view 3D generation enhancement? Some artifacts are generated in results of UMI3D in figure 8. Figure 2 should be modified and improved. The core designs should be highlighted. How to achieve the Simultaneous Focusing Cross-Attention (SFC-Attn), how to prove the Simultaneous design in this module. The corresponding motivation should be improved. Why use the Voxel Reference Score to route each voxel? Are there any other routing methods? What are the corresponding results compared with other methods?
5 comments
Response to Reviewer nV6m (2/2)
Q5: Implementation of SFC-Attn.
SFC-Attn performs VRS-guided voxel grouping and group-wise cross-attention, as detailed in Sec. 3.3 and Fig. 2(c). Within each denoising step and DiT block, VRS assigns each voxel to a reference image. Each voxel group then attends to its assigned reference, and the group-wise outputs are merged into a unified voxel feature tensor. Thus, all references contribute within the same denoising step, while each voxel selectively uses the reference assigned to it.
As illustrated in Supp. A.1, switching references across denoising steps can cause conditioning oscillation and detail loss, whereas indiscriminately combining reference-conditioned outputs can mix conflicting evidence. SFC-Attn avoids both by keeping all references available within each step while routing their evidence selectively. Here, “simultaneous” refers to using all references within the same step, and “focusing” refers to voxel-wise reference selection.
Q6: Why use the Voxel Reference Score (VRS)?
Different voxels may rely on different input images. VRS measures the attention mass allocated to local image tokens and provides a model-intrinsic proxy for voxel–image support, enabling reference selection without an additional matching model or trained router.
We compare VRS routing with random, opposite-VRS, and soft VRS-weighted routing in Table 4 and Fig. 8. Among these evaluated strategies, VRS routing achieves the best CLIP, DINO, and LPIPS scores.
| Routing strategy | CLIP ↑ | DINO ↑ | LPIPS ↓ |
|---|---|---|---|
| Baseline | 85.95 | 72.99 | 0.3413 |
| Random CA Routing | 85.61 | 72.36 | 0.3423 |
| Opposite VRS Routing | 84.42 | 69.63 | 0.3439 |
| Soft VRS-Weighted Fusion | 85.77 | 72.35 | 0.3435 |
| VRS Routing (Ours) | 86.42 | 74.62 | 0.3356 |
I have read the authors' response and authors generally addressed my main concerns. I decide to maintain my original score.
Response to Reviewer nV6m (1/2)
W1: Evaluation of multi-image consistency
We evaluate reference-view fidelity by rendering each generated 3D asset from the input viewpoints and comparing the renderings with their corresponding references using CLIP, DINO, and LPIPS.
Our benchmark is created by independently editing views rendered from source meshes. Because these edits may alter geometry and appearance, the original mesh is not a valid common ground truth for all edited references. We therefore do not separately quantify cross-view consistency. Instead, our user study includes renderings from additional viewpoints and provides complementary evidence of perceived structural and visual coherence. We have clarified the scope and this limitation in the revised appendix.
W2: Automatic metrics versus user preference
Two factors may contribute to this difference.
Although the input views are unconstrained, they depict the same object. As a result, most methods preserve its broad semantic identity and may receive similar semantic-similarity scores.
The evaluations cover different viewpoints. As described in Appendix A.4, our user study includes additional viewpoints, allowing participants to assess structural defects, missing details, and the overall coherence of the 3D asset—qualities that reference-view metrics may not fully capture.
To illustrate this limitation, Fig. 17 shows representative cases in which outputs with visibly different perceptual quality receive similar CLIP, DINO, and LPIPS scores. These examples suggest that semantic similarity and the restricted viewpoints evaluated by the metrics can obscure perceptual quality differences. Thus, metric scores may not fully reflect the quality gap between outputs, motivating our use of a user study as a complementary assessment.
Q1: Related work
Ultra3D [2], Wonder3D++ [4], and Svim3D [5] are single-image 3D generation methods and do not support multi-image input. In addition, the official code for Ultra3D and Svim3D is unavailable, precluding direct comparisons in our setting. DyFusion [1] focuses on 3D detection, [3] on 3D scene reconstruction, and MICON-Bench [6] on in-context image generation; these works address different tasks and are not directly comparable to our multi-image 3D generation method.
We have added [2], [4], and [5] to the Related Work section.
Q2: Performance of multi-view 3D generation enhancement
We evaluate SFC-Attn on 20 randomly selected Sketchfab assets. For each asset, we render three input images from the same fixed viewpoints, uniformly distributed around the asset. The complete GLB model serves as the geometric ground truth, and both generated and reference geometries use the same coordinate system. We report CD (×10³), F-score at a threshold of 0.1, and the CLIP, DINO, and LPIPS metrics used in our paper. The baseline and SFC-Attn-enhanced TRELLIS.2 use identical input views.
As shown below, SFC-Attn improves all five metrics over the baseline, indicating better geometric accuracy and reference-view fidelity.
Multi-view generation results
Q3: Artifacts in Fig. 8
We acknowledge that artifacts remain in Fig. 8, including inaccurate text on the watch screen. This is an auxiliary application of VRS-based cross-attention routing on TRELLIS.2, rather than our primary unconstrained multi-image generation task. Compared with the TRELLIS.2 baseline, SFC-Attn improves specific regions, including the knight’s armor texture and SpongeBob’s mouth, but does not eliminate all artifacts. We will explore further improvements to multi-view 3D generation with VRS and SFC-Attn in future work.
Q4: Advice on Fig. 2.
The core components are shown in Fig. 2(b) and (c): panel (b) illustrates how VRS derives voxel–image assignments from attention, while panel (c) shows how SFC-Attn uses these assignments for grouped cross-attention and feature merging. Panel (a) presents the overall backbone pipeline. We will emphasize the labels of panels (b) and (c) and clarify this distinction in the caption.