acceptodds
Under review as a conference paper at ICLR 2027

OwnerFusion3D: Mask-Guided Local Image-to-3D Fusion

Abstract

Recent image-conditioned 3D generators can synthesize high-quality assets from single visual conditions. However, they cannot directly use a user-drawn 2D mask to assign an independent reference image to a chosen region of a 3D asset: treating multiple images as global conditions causes reference appearance to leak beyond the target region and weakens preservation elsewhere. We present OwnerFusion3D, a training-free framework for mask-guided local image-to-3D fusion. Given a structure image, an image-plane mask, and a content reference, OwnerFusion3D transfers reference appearance to the selected region while preserving the complementary structure. First, we introduce a Mask-Conditioned Ownership Estimator (MOE) that lifts the 2D mask to a sparse 3D owner field by contrasting frozen cross-attention responses to selected and competing foreground patches. Second, we propose Owner-Exclusive Hierarchical Routing (OEHR), which uses this field to restrict every sparse query to its assigned condition source throughout the generator hierarchy. For multi-part fusion, Categorical Ownership Resolution (COR) resolves competing part assignments before the same routing process. OwnerFusion3D requires neither training nor 3D supervision and uses no post-hoc mesh or texture editing. Experiments across diverse object categories and visual styles demonstrate accurate local reference transfer together with strong preservation outside the selected mask. An anonymous implementation is available at https://anonymous.4open.science/r/anonymous-3d-release-7k2p.

Primary area:
generative models
Keywords:
3D generation, controllable 3D generation, mask-guided 3D fusion

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_E3QLOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Owner-Fusion3D, a training-free framework for mask-guided local image-to-3D fusion. Given a structure image, an image-plane mask, and a content reference, OwnerFusion3D transfers reference appearance to the selected region while preserving the complementary structure. A Mask-Conditioned Ownership Estimator (MOE) is introduced that lifts the 2D mask to a sparse 3D owner field by contrasting frozen cross-attention responses to selected and competing foreground patches. An Owner-Exclusive Hierarchical Routing (OEHR) is proposed that uses this field to restrict every sparse query to its assigned condition source throughout the generator hierarchy. For multi-part fusion, Categorical Ownership Resolution (COR) resolves competing part assignments before the same routing process. OwnerFusion3D requires neither training nor 3D supervision and uses no post-hoc mesh or texture editing.

Critical strengths

Formulating mask-specified local image-to-3D fusion as sparse ownership discovery followed by source-specific execution in a frozen image-to-3D generator. OEHR enforces source-exclusive pre-softmax access across the shape-to-texture hierarchy; COR resolves ownership conflicts in multi-part fusion.

Critical weaknesses

What is the “COR” in figure 1?The contributions of Introduction section should be more detailed presented. The paper organization should be revised, such as figure 2 should be located at the top of Page 3. What is the “e_i” in figure3? How to obtain the O_LR or what process is conducted for O_LR in figure 3? Please clarify. The font size of figure 2 and figure 4 is too small. In the main paper, these are no description of Table 1. The layout of Table 1 and Table 2 should be adjusted.

Questions for authors

There are many cross-attentions and generation designs are proposed for 3D generation or 3D fusion, such as [1] DyFusion: Cross-attention 3D object detection with dynamic fusion. [2] Ultra3D: Efficient and High-Fidelity 3D Generation with Part Attention [3] Pixal3D: Pixel-aligned 3D generation from images [4] Wonder3d++: Cross-domain diffusion for high-fidelity 3D generation from a single image [5] Direct3d: Scalable image-to-3d generation via 3d latent diffusion transformer [6] Instant3D: Instant Text-to-3D Generation There methods should be compared and discussed. The similar designs should be ablated in the experiments.

How to identify which qualitative result is best in figure 5 and figure 6? Do these datasets have the ground-truth? The better fair and accurate qualitative comparison should be provided. The corresponding quantitative results are provided, which will be more convincing. The color appearance has significant changes after fusion for the proposed method, how to address this problem under this model.

In ablation study, existing Mask-Conditioned Evidence and Owner-Exclusive Routing designs from other methods should be compared and discussed.

4 comments

Weaknesses

Thank you for pointing out that the contribution description, COR, and the presentation of the figures and tables needed to be clearer. We have revised the Introduction and Method sections so that MOE estimates sparse ownership from the user mask, OEHR enforces source-exclusive access during generation, and COR resolves competing assignments in multi-part fusion. COR is now introduced explicitly as the multi-part extension, and its role is explained before routing.

We have also clarified the transition from the attention evidence to the ownership field. For each active sparse query, e_i is the normalized preference for selected foreground evidence over competing unselected foreground evidence, aggregated across the fixed block-head readout. O_LR is not the score itself; it is the discrete ownership field obtained by thresholding the scores and regularizing assignments on the structure-derived sparse support. The regularization removes isolated assignments and performs score-qualified neighborhood expansion only within that support.

We adjusted the figure placement and layout for readability, including positioning Figure 2 at the top of page 3 and improving the presentation of Figures 2 and 4. The text now explains the symbols alongside the method equations. We also added an interpretation after Table 1 and adjusted the layouts of Tables 1 and 2. In particular, the comparison shows that OwnerFusion3D achieves the best single-mask LPIPS, CLIP-I, and DINO-I, as well as the best multi-part CLIP-I and DINO-I. Fuse3D obtains a slightly lower multi-part LPIPS. Table 2 provides complementary blinded multi-view VLM ratings.

Q1. Related 3D generation and attention methods

We appreciate these pointers. The Related Work section now discusses DyFusion, Ultra3D, Pixal3D, Wonder3D++, Direct3D, and Instant3D, describing the generation, attention, or fusion setting addressed by each. We position them as complementary directions rather than treating general-purpose generation methods as directly equivalent regional-fusion systems.

Q2. Interpreting the qualitative comparisons

Thank you for raising this point. We evaluate the qualitative results according to the task’s stated objectives, rather than ranking them by general visual appeal alone. The structure image defines the asset to preserve, the mask identifies the region to edit, and the content reference specifies the appearance to transfer within that region. A successful result should therefore both reflect the reference inside the selected region and preserve the complementary structure outside it.

The same criteria apply to multi-part fusion: each mask is paired with its own reference, and each reference should influence its assigned region without leaking into other parts. Thus, when comparing Figures 5 and 6, we assess whether the requested appearance is transferred to the specified region, whether distinct references remain associated with their intended regions, and whether unselected regions retain the structure image’s appearance. We do not treat these figures as ground-truth rankings; the quantitative metrics and multi-view VLM ratings provide complementary evidence for these objectives.

Appearance changes inside a selected region are part of the requested transfer. Changes outside the selected region, or mixing between references in the multi-part setting, instead indicate a failure of locality or source separation..

Q3. Ablations of ownership evidence and exclusive routing

The paper already documents these variants in Section 4.4 and Table 3, while Appendix D.1 provides their qualitative comparison under identical inputs. Appendix B further explains the fixed calibrated block-head readout and the All-block-head aggregation. Table 3 reports the quantitative results, whereas Appendix D.1 illustrates the corresponding behavioral differences.

In w/o MOE, mask-conditioned ownership estimation is removed, so no owner field is available for source-exclusive routing. In w/o OEHR, MOE still estimates the owner field, but it is not applied as an exclusive attention gate and both sources remain accessible. All-block-head retains MOE and OEHR and changes only the fixed calibrated readout to an unweighted average of available block-head evidence.

These ablations isolate the roles of ownership estimation, ownership enforcement, and readout selection within the same generator. We discuss the cited methods in Related Work, but do not present their internal modules as drop-in ablations: they use different tasks and generator interfaces, so transplanting their components into TRELLIS would not be a controlled reproduction of those methods.

I have read the authors' response and authors generally addressed my main concerns. I decide to maintain my original score.

Thank you for taking the time to read our responses and for letting us know. We appreciate that our clarifications have addressed your main concerns. We understand and respect your decision to retain your original rating. Thank you again for your thoughtful feedback and time.

Reviewer_TW4COpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OwnerFusion3D, a training-free framework for mask-guided local image-to-3D fusion. Given a structure image, a mask, and reference images, the method transfers reference appearance to selected 3D regions while preserving the remaining structure. The pipeline introduces three components: Mask-Conditioned Ownership Estimation to lift 2D masks into sparse 3D ownership, Owner-Exclusive Hierarchical Routing to enforce source-specific conditioning throughout generation, and Categorical Ownership Resolution to handle conflicting assignments in multi-part fusion. The evaluation results are detailed and sufficient overall.

Critical strengths
  • This paper focuses on mask-based 3D assets fusion, an area that lacks strong training-free methods. The formulation based on sparse 3D ownership is intuitive and interesting.
  • This paper is easy to follow.
  • The qualitative results demonstrate promising local appearance transfer and preservation of unselected regions.
Critical weaknesses
  • The image-based metrics used in the quantitative evaluations seem to be weak. The paper can include VLM grading and user-study to further evaluate the methods.
  • The comparison with image editing methods use Qwen-Image-2.0-Pro and FLUX. More stronger recent models like GPT Image 2.5 and Nano Banana 2.1 can be included.
  • The cases shown in Fig. 19 seem to be easy cases and the seperated structures lack sophisticated details. More visualizations on the hard cases and failure cases can better illustrate the capabilities of the sparse ownership designs.
  • The illustration of the failure case shows only one sample.
Questions for authors
  • In the comparison, does Fuse3D use TRELLIS or TRELLIS.2?
  • As the fusion involves reference images and the goal of the reference images can often be ambiguous, how to incorporate text prompts to cope with this in the proposed method?
  • How sensitive is the method to mask quality and attention-head selection?
6 comments
AuthorsOpenReview ↗

W1 & W2 & Q1. Evaluation, Stronger Baselines, and Backbone Fairness

Q1. Fuse3D backbone

In the original native-pipeline qualitative comparison, Fuse3D uses TRELLIS, whereas OwnerFusion3D and the corresponding original baselines use TRELLIS.2. Table 1 is a separate matched-backbone comparison: OwnerFusion3D is transferred to TRELLIS, the 2D composites are lifted with TRELLIS, and Fuse3D uses its TRELLIS pipeline. We clarify this distinction so the native-pipeline results are not mistaken for a controlled backbone comparison.

W1. Perceptual evaluation

Thank you for pointing out that image metrics alone may not fully capture constrained 3D fusion. We added a blinded, rubric-based multi-view VLM evaluation using GPT-6.1-SOL on the same 50 single-mask assets and 20 multi-part assets (74 requested regions). Method identities are hidden, and each result is evaluated from four corresponding views.

MethodS-PFS-LRS-OPS-VCS-OQS-MeanM-PFM-CSM-OPM-VCM-OQM-Mean
FLUX.23.6003.3003.7803.5403.4003.5243.2423.3003.6503.7003.2003.418
FLUX.33.5803.4203.7403.4603.4603.5323.3143.4503.8004.1003.3503.603
Fuse3D3.4003.3403.8203.3403.2803.4363.1233.2503.5503.6003.2003.345
NanoBanana 2.13.6003.3803.7803.5603.4403.5563.3483.5003.9003.9503.5003.640
Qwen-Image 2.1 Pro3.5803.3603.7603.5003.5003.5403.3263.5503.7503.8003.5003.585
GPT Image 2.53.7003.4803.7003.5203.5003.5803.4993.8003.9504.2003.8003.850
OwnerFusion3D3.6203.5803.8003.7003.6203.6643.6303.8504.0004.0503.8503.876

S denotes single-mask and M denotes multi-part. PF measures reference fidelity within the selected region; LR measures locality and resistance to reference leakage; CS measures separation between different references; OP measures preservation outside the target region; VC measures consistency across views; and OQ measures overall constrained-fusion quality. All scores are on a 1--5 scale, with higher being better. Mean is the unweighted average of the five criteria. We report these VLM ratings as complementary perceptual evidence, not as a human user study or ground-truth 3D metric.

W2. Stronger baselines and matched-backbone results

We appreciate the suggestion to add stronger image-editing baselines. Table 1 includes FLUX.3, NanoBanana 2.1, Qwen-Image 2.1 Pro, and GPT Image 2.5 in addition to FLUX.2 and Fuse3D. In this comparison, all methods use the original TRELLIS reconstruction backbone, the same task inputs, shared structure-only references, and the same evaluation protocol.

MethodSingle LPIPS ↓Single CLIP-I ↑Single DINO-I ↑Multi LPIPS ↓Multi CLIP-I ↑Multi DINO-I ↑
FLUX.20.36730.79330.61650.35550.77880.5446
FLUX.30.35840.79890.62030.35030.78410.5539
Fuse3D0.35870.78610.59290.34470.78710.5754
NanoBanana 2.10.36120.80100.63820.35230.78300.5768
Qwen-Image 2.1 Pro0.36990.79970.62330.35030.77980.5399
GPT Image 2.50.36530.80500.63510.34840.79340.5904
OwnerFusion3D0.34020.82450.68320.34660.80770.6245

OwnerFusion3D leads all three single-mask metrics and the multi-part CLIP-I and DINO-I metrics. Fuse3D has a slightly lower multi-part LPIPS (0.3447 vs. 0.3466).

The gains are consistent with the method’s design: MOE contrasts evidence from selected and competing foreground patches to estimate which sparse queries belong to the requested region; OEHR then restricts each query to its assigned source before attention normalization. This helps avoid the region spillover and cross-source mixing that can occur when image-space edits are lifted to 3D. For multiple parts, COR resolves competing assignments before routing. The component ablation provides supporting evidence:

VariantJoint LPIPS ↓Joint CLIP-I ↑Joint DINO-I ↑
w/o MOE0.39190.75420.5173
w/o OEHR0.36310.78880.5957
All-block-head0.36160.81820.6804
Full (Ours)0.35790.83890.7061

These results support the roles of mask-conditioned ownership, exclusive routing, and a calibrated readout. They also show why richer image edits alone do not guarantee better regional control: broader edits may transfer more appearance, but can affect unselected areas or mix attributes across requested parts.

W3. More challenging visual examples

We agree that the earlier examples did not sufficiently illustrate complex assignments. Appendix D.2 now contains four multi-part ownership visualizations with neighboring regions and more structured shapes, shown from four viewpoints; Appendix D.5 adds final fusion assets. The examples illustrate how assignments behave in more complex layouts, while the tables above provide the quantitative evidence.

W4. Failure and limitation analysis

Thank you for noting that one failure example was insufficient. Appendix D.4 now analyzes three distinct cases. First, a whole-truck mask selects the truck but cannot express that track-like appearance should apply only to its wheels; the limitation is mask granularity, not localization of the selected truck. Second, a single-view mask on a robot head also assigns a rear component hidden behind the head; the 2D input does not provide the depth evidence needed to separate them, and OEHR faithfully routes the same reference to both assigned regions. Third, a thin eagle-talon region is localized, but the generated shape remains anatomically unconvincing because sparse ownership does not guarantee fine geometric correspondence or topology. These cases distinguish input/visibility limitations from the backbone’s difficulty with fine details.

Q2. Text prompts and ambiguous references

This is an interesting question that has prompted us to think further. If text prompts could be incorporated into our method to provide more precise information, they could potentially improve the fusion results. However, this would require the backbone itself to have the corresponding text-understanding capability. The TRELLIS.2 backbone we currently use does not provide an available text-related model or text-conditioned pipeline. Although this capability might be introduced by adding text-related data and training the model, doing so may conflict with our current training-free setting.

Nevertheless, this is an interesting direction worth exploring. TRELLIS itself has both an image-conditioned generation pipeline and a text-conditioned generation pipeline, meaning that it supports text conditions. In such a setting, MOE could still determine the ownership of regions in 3D space, the image could provide the appearance reference, and the text could specify which attributes should be borrowed from the reference image and which attributes should not be transferred. These are only preliminary ideas, and we may investigate them in future work. Thank you for this interesting question.

Q3. Sensitivity to readout and mask quality

Thank you for raising these important questions. We evaluated both block-head sensitivity and mask-boundary sensitivity on the same 50 matched single-mask assets.

Block-head selection

The member-level diagnostics show that ownership quality depends on which attention heads are read by MOE.

Readout profileHead-member recordsMean reliabilityMean support focusMean seed-IoU diagnosticMean threshold-selected ratio
Worst-five heads2500.09610.00000.10000.0000
Fixed Stable Mid-Late-42000.93861.80820.53590.3669

The worst-five members provide no threshold-selected ownership evidence or focused support for the masked region. Consequently, MOE produces no valid ownership seeds in these cases. Neighborhood completion can regularize or extend evidence-backed assignments, but cannot infer the intended region when the selected heads provide no useful mask-conditioned signal. The failure therefore occurs during ownership evidence acquisition, before OEHR applies the source assignment.

The fixed four-member readout provides substantially stronger and more focused evidence. The All-block-head ablation offers a complementary comparison: averaging all heads reduces reliance on any one head, but also mixes diffuse or weak responses. On the 50 single-mask assets, All-block-head obtains 0.3616/0.8182/0.6804 for Joint LPIPS/CLIP-I/DINO-I, compared with 0.3579/0.8389/0.7061 for the calibrated full method. These results explain why both arbitrary poor-head selection and indiscriminate averaging can underperform a fixed readout calibrated for reliable ownership evidence.

Mask-quality sensitivity

We also evaluated mask-boundary sensitivity on the same 50 single-mask cases. For each case, we applied a 4-pixel dilation or erosion to the binary mask while keeping the structure image, reference image, backbone, four-step LR probe, stable mid/late four-head readout, and threshold fixed.

Mask conditionMean mask-covered patchesMean MOE-selected LR queriesMean selected ratioPaired change vs. original
Original mask202.96 (37.88%)1,097.136.32%—
Dilated by 4 px213.78 (39.97%)1,138.837.86%+1.54 pp [1.16, 1.98]
Eroded by 4 px186.96 (34.82%)1,053.734.89%−1.43 pp [−1.79, −1.09]

The intervals are 95% paired case-level bootstrap confidence intervals. Dilation increased the selected ratio in all 50 cases, whereas erosion decreased it in all 50 cases. None of the cases lost all selected LR queries under either perturbation. The median absolute change was 0.93 percentage points for dilation and 0.90 percentage points for erosion; the corresponding 90th-percentile changes were 4.03 and 3.27 percentage points.

These changes are consistent with the behavior of our ownership estimator. Dilation adds boundary evidence to the selected region, while erosion removes part of that evidence. Because the mask is mapped onto a discrete patch/token grid, the resulting changes are not perfectly symmetric. Nevertheless, the selected ownership support remains nonempty in every case under these moderate perturbations, indicating that the method tolerates small local boundary errors while still responding to the user-provided mask.

These experiments characterize the sensitivity of MOE ownership to attention-head choice and local mask-boundary changes. They do not claim robustness to arbitrary mask errors, severely inaccurate masks, or visibility-ambiguous regions, where insufficient or misleading mask evidence can affect ownership assignment and the resulting controlled fusion.

Reviewer_YmVbOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

OwnerFusion3D lets a user specify a region on a structure image and supply an independent reference image for that region. It estimates ownership over the generator's sparse 3D representation, then restricts each element to its assigned image source during shape and texture generation. The task and interface are compelling.

Critical strengths

This work is training free. The qualitative results show useful local control, and Table 1 reports the best joint CLIP-I and DINO-I scores on both the single-mask and multi-part benchmarks. Table 3 also shows gains from the ownership and routing components.

Critical weaknesses
  1. The novelty is narrower than the task formulation suggests. Fuse3D already studies multi-image regional 3D fusion, maps specified 2D image regions to sparse 3D elements through attention, and tune the attention to localize their influence. This work differs in where the user places the mask and in its contrastive ownership estimate and hard routing across the generator hierarchy. Those differences are interesting, but the paper should show which of them accounts for the improvement.

  2. The Fuse3D comparison changes the generator as well as the method. Fuse3D runs on TRELLIS; this work runs on TRELLIS.2. Pairing each method with a reference from its own backbone helps, but does not isolate the effect of ownership and routing. A Fuse3D-style soft-routing baseline on TRELLIS.2, or this work on the original TRELLIS, would be the most useful comparison.

  3. The multi-parrt contribution is not fully tested. COR is compared with score-only assignment visually, but the paper gives no downstream quantitative result without COR. please report how much COR changes the final multi-part metrics or preferences on the 20 multi-part assets.

  4. Evidence for control across 3D views limited. The image metrics use an aligned view, and the automated multi-view preference test covers only single-mask examples. Since the contribution is control over a 3D asset, measurements of reference leakage and structure preservation from unseen views would make the claim more convincing.

  5. The ablation needs clarification. Table 3 says "without MOE" also remoes routing, whereas “without OEHR” retains the owner field but allows access to both sources. Please explain precisely how each variant uses the owner field and why their results differ.

Questions for authors

see the weakness

7 comments
AuthorsOpenReview ↗

W1–W2. Novelty and matched-backbone comparison

W1. Novelty

Thank you for this thoughtful comparison with Fuse3D. We agree that the distinction between the general task of regional multi-image 3D fusion and the specific contribution of OwnerFusion3D should be made more explicit.

Fuse3D demonstrates regional multi-image 3D fusion and uses attention responses to identify where a local reference can influence the sparse 3D representation. Our distinction lies in how the user's regional request is represented and enforced during generation. Fuse3D primarily relies on response-based localization to identify influential regions, whereas OwnerFusion3D starts from an explicit 2D mask provided by the user and converts this request into generator-native sparse ownership.

The mask specifies the structural region that the user intends to modify. It is therefore an input-level control signal rather than an independently learned module. MOE then contrasts the evidence from the masked region with complementary structural evidence to estimate which sparse 3D queries are supported by the selected region. When MOE is removed, the mask-derived ownership field is no longer available, and the explicit regional source assignment is consequently lost.

OEHR operates after ownership estimation. During conditional generation, it converts the estimated ownership into an ownership-exclusive source-access constraint applied before softmax normalization. Removing OEHR preserves the MOE ownership estimate, but allows the structure and content conditions to remain jointly accessible to each query. For multiple references, COR further resolves overlapping candidate assignments before routing.

The central distinction is therefore not the use of attention itself, but the conversion of a user-specified 2D region into an explicit ownership field and the enforcement of this ownership throughout low-resolution shape, high-resolution shape, and texture generation. The existing ablations distinguish the roles of ownership estimation and ownership enforcement, while the matched-backbone comparison below shows that the observed gains cannot be explained solely by using TRELLIS.2. We will revise the Introduction and Method sections to make this ownership-based distinction more explicit.

W2. Matched-backbone comparison

Thank you for suggesting the matched-backbone comparison. We reran the comparison on all 50 single-mask assets and 20 multi-part assets using a unified TRELLIS image-to-3D reconstruction backbone, the same task inputs, the same structure-only references, and the same evaluation protocol. The comparison therefore does not mix different backbone versions. The tables include all evaluated methods, including the newly added FLUX.3, NanoBanana 2.1, and GPT Image 2.5 baselines.

Table 1: Single-mask comparison on 50 assets.

MethodJoint LPIPS ↓Joint CLIP-I ↑Joint DINO-I ↑
FLUX.20.36730.79330.6165
FLUX.30.35840.79890.6203
Fuse3D0.35870.78610.5929
NanoBanana 2.10.36120.80100.6382
OwnerFusion3D0.34020.82450.6832
Qwen-Image 2.1 Pro0.36990.79970.6233
GPT Image 2.50.36530.80500.6351

Table 2: Multi-part comparison on 20 assets.

MethodJoint LPIPS ↓Joint CLIP-I ↑Joint DINO-I ↑
FLUX.20.35550.77880.5446
FLUX.30.35030.78410.5539
Fuse3D0.34470.78710.5754
NanoBanana 2.10.35230.78300.5768
OwnerFusion3D0.34660.80770.6245
Qwen-Image 2.1 Pro0.35030.77980.5399
GPT Image 2.50.34840.79340.5904

Bold and italic indicate the best and second-best means, respectively. Each joint score equally weights reference similarity inside the selected region and preservation of the complementary region. For multi-part inputs, content scores are averaged within each asset before averaging across assets.

OwnerFusion3D obtains the best result on all three single-mask metrics. On multi-part inputs, it obtains the best Joint CLIP-I and Joint DINO-I, while Fuse3D obtains the lowest Joint LPIPS. The LPIPS difference between Fuse3D and OwnerFusion3D is 0.0019. A paired case-level bootstrap gives a 95% interval of [-0.0015, +0.0055] for OwnerFusion3D minus Fuse3D, so we do not interpret this small difference as a decisive advantage for either method.

The corresponding multi-part gains in Joint CLIP-I and Joint DINO-I are 0.0207 and 0.0492. Thus, the advantage of OwnerFusion3D is not dependent on using a stronger TRELLIS.2 backbone. It persists when the methods are evaluated under a common TRELLIS reconstruction backbone and a common task protocol.

AuthorsOpenReview ↗

W3. Quantitative evidence for multi-part ownership resolution

Thanks for the valuable comments. We evaluated all 20 multi-part cases using ownership metrics rather than relying only on visual comparisons or conflict counts.

COR operates on per-query candidate ownership, so a no-resolution comparison needs an explicit rule for queries with competing or missing candidates. We define Score-only as follows: each eligible query is assigned to its highest-scoring candidate, while a query with no eligible candidate remains structure-owned. The full pipeline combines the ownership readout, spatial candidate processing, categorical resolution, and completion used by OwnerFusion3D. Both runs use the same backbone, task inputs, and sampling seeds. The results therefore quantify the downstream effect of enabling the complete ownership-resolution pipeline, with COR providing the categorical conflict-resolution and completion stage used before exclusive routing.

A generated 3D asset is not necessarily a pixel-aligned reconstruction of its conditioning image. This mismatch is inherited from the image-to-3D backbone, which generates a 3D asset from a 2D condition rather than reproducing its pixels one by one; it is not introduced by COR. We use the same per-case alignment parameters, camera settings, normalization, and registration for both variants to minimize these additional factors and ensure matched viewing conditions.

Table 3: Projected ownership results on 20 multi-part assets.

MetricScore-onlyFullΔCIHolm p
Part Macro-IoU ↑0.63310.7492+0.1160[+0.0628, +0.1789]1.20e-4
Correct-owner rate ↑71.09%83.71%+12.62 pp[+6.28, +20.27]7.63e-6
Assignment coverage ↑80.16%89.25%+9.09 pp[+2.62, +17.62]N/A
Unassigned rate ↓19.84%10.75%-9.09 pp[-17.58, -2.59]8.05e-4

Here, Δ denotes Full minus Score-only, and CI denotes the paired 95% bootstrap confidence interval. Rate differences are reported in percentage points. We decode color-coded ownership renders against the registered input-view part labels while excluding a two-pixel boundary band. Part Macro-IoU averages part classes within each asset and then across assets. Correct-owner rate measures label agreement at visible evaluable locations. Assignment coverage is the fraction decoded to a part color, and the unassigned rate is its visible complement. These are projected ownership metrics for the final categorical assignment, rather than claims of complete hidden-surface 3D segmentation accuracy.

Holm correction covers Part Macro-IoU, correct-owner rate, unassigned rate, and cross-part error among assigned locations. Cross-part error decreases from 11.05% to 6.26%, although this secondary change is not statistically significant after correction. Across the 20 cases, Part Macro-IoU improves in 19/20 assets, correct-owner rate improves in all 20 assets, and assignment coverage improves in 18/20 assets. Overall, Macro-IoU increases by 0.1160, correct-owner rate increases by 12.62 percentage points, and the visible unassigned rate decreases by 9.09 percentage points.

Table 4: Ownership changes by initial assignment difficulty.

LoadnConflictUnassignedChangedΔCLIP-IΔDINO-I
Low71.5%9.7%13.0%+0.0012+0.0010
Medium42.9%13.2%19.5%+0.0054+0.0281
High927.3%19.1%46.9%+0.0285+0.0464

Low-load cases have conflict below 5% and unassigned rate below 15%; high-load cases have conflict at least 15% or unassigned rate at least 50%; the remaining cases are medium-load. The Conflict and Unassigned columns are macro-averaged over cases, while Table 3 reports pooled visible projection rates. Changed denotes the fraction of ownership locations reassigned by the full pipeline.

The largest gains occur in high-load cases, where the initial conflict rate is 27.3% and 46.9% of ownership locations are changed. Selected-region CLIP-I improves by 0.0285 and DINO-I by 0.0464, with both metrics improving in 8/9 assets. Across all 20 assets, selected-region CLIP-I and DINO-I increase by 0.0143 and 0.0261, respectively.

These results show where the ownership-resolution pipeline is most useful: when independent score-based assignment produces competing claims or leaves part regions unresolved. COR converts these query-level candidates into categorical, spatially completed assignments before OEHR applies source-exclusive routing, leading to improved ownership accuracy and assignment completeness.

AuthorsOpenReview ↗

W4–W5. Multi-view evidence and ablation interpretation

W4. Multi-view evidence for multi-part control

Thank you for pointing out that the original evaluation provided limited multi-view evidence for multi-part control. We extended the four-view evaluation to all 20 multi-part assets, covering 74 requested fusion regions. Each evaluation includes the structure image, all mask-reference pairs, and a shared structure-only 3D reference. A GPT-6.1-SOL judge evaluates corresponding front, back, front-oblique, and back-oblique renders with method identities hidden.

The five 1–5 criteria directly target the requested behaviors. PF measures reference fidelity within each requested part; CS measures separation between requested parts and resistance to reference leakage; OP measures preservation of visible regions outside the requested union; VC measures consistency across views; and OQ measures overall constrained-fusion quality. PF is averaged over parts within each asset, and each criterion is then averaged over the 20 assets. Mean is the unweighted average of the five criteria.

Table 5: Multi-view multi-part VLM ratings.

MethodPF ↑CS ↑OP ↑VC ↑OQ ↑Mean ↑
OwnerFusion3D3.6303.8504.0004.0503.8503.876
FLUX.23.2423.3003.6503.7003.2003.418
FLUX.33.3143.4503.8004.1003.3503.603
Fuse3D3.1233.2503.5503.6003.2003.345
GPT Image 2.53.4993.8003.9504.2003.8003.850
NanoBanana 2.13.3483.5003.9003.9503.5003.640
Qwen-Image 2.1 Pro3.3263.5503.7503.8003.5003.585

OwnerFusion3D obtains the highest PF, CS, OP, and OQ ratings. In particular, its CS and OP scores are 3.850 and 4.000, compared with 3.250 and 3.550 for Fuse3D. This directly supports improved separation of multiple references and preservation of visible non-target regions across views, rather than only improved similarity from a single aligned view.

GPT Image 2.5 obtains the highest VC score, indicating stronger cross-view consistency on this criterion. Its mean score is 3.850, close to OwnerFusion3D's 3.876, so we do not claim universal superiority over this strong baseline. The main advantage of OwnerFusion3D is concentrated in the criteria most directly related to the paper's contribution: part-specific reference fidelity, source separation, and preservation outside the requested regions.

W5. Exact behavior and interpretation of the ablations

We appreciate your suggestion. We will make the exact implementation and interpretation of the ablations clearer.

The full method builds sparse support from the structure image, estimates mask-conditioned ownership with MOE, and uses OEHR to restrict each query's direct cross-attention access to the assigned source throughout the TRELLIS.2 shape and texture stages.

In w/o MOE, the structure-derived initial support is retained, but no ownership field is estimated. Content-only conditioning is used globally for subsequent shape and texture sampling, without the owner-exclusive gate. Thus, this variant removes the ownership-based local-control chain, including the routing that depends on MOE.

In w/o OEHR, MOE is computed as in the full method, but the resulting owner field is not used by the attention gate. The sampler receives concatenated structure and content tokens, and every query can attend to both sources. The ownership field is available for diagnostics but has no source-restriction effect on generation.

This explains why the two variants produce different outputs. w/o MOE uses global content conditioning on structure-derived support, whereas w/o OEHR uses unrestricted joint structure-content conditioning. Their difference should therefore not be interpreted as the effect of an estimated but unused owner field. The full-versus-w/o-OEHR comparison is the direct test of whether ownership assignments are enforced during generation.

The All-block-head row separately replaces the calibrated readout while retaining routing. Its reasonable performance shows that broad attention aggregation already contains useful correspondence evidence. The stronger calibrated readout is consistent with suppressing diffuse or conflicting contributions from some head-block combinations, rather than implying that all head-block combinations are ineffective.

We will revise the ablation description to make these conditioning paths and their scope explicit.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

Dear Reviewers,

We sincerely thank all reviewers for their careful evaluations and constructive feedback. Your comments helped us strengthen the empirical evidence, and better explain the scope and limitations of OwnerFusion3D.

[Our Contributions]

OwnerFusion3D is a training-free framework for mask-guided local image-to-3D fusion. MOE converts an explicit 2D mask into generator-native sparse 3D ownership. OEHR restricts each sparse query to its assigned source throughout shape and texture generation, reducing reference leakage outside the selected region. For multi-part fusion, COR resolves competing ownership candidates before exclusive routing. The method requires no additional 3D supervision, fine-tuning, mesh editing, or texture post-processing.

[Additional Evidence and Clarifications]

• Novelty relative to Fuse3D: We clarified that Fuse3D primarily uses attention responses to identify influential regions, whereas OwnerFusion3D starts from an explicit user-provided mask and converts it into sparse ownership. MOE estimates ownership, OEHR enforces source-exclusive access throughout hierarchical generation, and COR resolves competing assignments for multiple regions.

• Matched-backbone comparison: We added FLUX.3, NanoBanana 2.1, and GPT Image 2.5. All methods use the same TRELLIS reconstruction backbone, task inputs, structure-only references, and evaluation protocol. OwnerFusion3D achieves the best results on all three single-mask metrics and the best multi-part Joint CLIP-I and Joint DINO-I. Fuse3D obtains a slightly lower multi-part LPIPS, with a difference of only 0.0019. We also distinguish the native qualitative comparison from the matched-backbone quantitative comparison.

• Multi-view evaluation: We extended the blinded four-view evaluation using GPT-6.1-SOL to all 50 single-mask assets and 20 multi-part assets covering 74 requested regions. OwnerFusion3D obtains mean scores of 3.664 and 3.876 in the single-mask and multi-part settings. In the multi-part setting, it leads part fidelity, cross-source separation, outside-region preservation, and overall constrained-fusion quality. GPT Image 2.5 obtains the highest view-consistency score.

• Quantitative evidence for COR: On the 20 multi-part assets, the complete ownership-resolution pipeline improves Part Macro-IoU from 0.6331 to 0.7492, correct-owner rate from 71.09% to 83.71%, and assignment coverage from 80.16% to 89.25%. The visible unassigned rate decreases from 19.84% to 10.75%. The largest improvements occur in high-conflict cases, where the initial conflict rate is 27.3% and selected-region CLIP-I and DINO-I improve by 0.0285 and 0.0464.

• Sensitivity analysis: On the same 50 single-mask assets, the calibrated Stable Mid-Late-4 readout produces substantially stronger ownership evidence than the worst-five-head readout. Under 4-pixel mask dilation and erosion, the selected-query ratio changes by only +1.54 and -1.43 percentage points, and all cases retain nonempty ownership support.

• Hard cases: limitations, and ablations. We added more complex multi-part visualizations, additional fusion assets, and three failure analyses covering mask granularity, single-view occlusion ambiguity, and fine geometric correspondence on thin articulated structures. We also clarified the ablations: w/o MOE removes ownership estimation and uses global content conditioning; w/o OEHR retains MOE but removes source-exclusive routing; All-block-head retains routing but replaces the calibrated readout with unweighted aggregation.

• Related work and reproducibility: We expanded the discussion of related 3D generation, attention, and image-fusion methods, added citations, and clarified the mask, alignment, crop, metric, baseline, and VLM protocols in the appendix.

We have incorporated these experiments and clarifications into the revised manuscript. We sincerely appreciate the reviewers’ time and suggestions, which helped us substantially improve the paper. We respectfully invite the reviewers to reconsider their initial assessments and welcome further discussion.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.