acceptodds
Under review as a conference paper at ICLR 2027

INLAY3D: PRECISE PART-LEVEL TEXTURE AND SHAPE MANIPULATION OF 3D ASSETS

Abstract

real-world 3D asset production pipelines, artists need to edit and refine the details of the local regions, yet this problem remains insufficiently explored. Such local edits should provide the following capabilities: 1) Part-level spatial control: identifying the complete 3D target part intended by the user while preserving non-target regions and boundaries; and 2) Attribute-specific editing: supporting texture editing, shape manipulation, or joint shape-and-texture manipulation within the target part. However, existing methods often define the editing region imprecisely, either with coarse 3D boxes or by implicit region selection. They lack an editing framework that enables decoupled texture and geometry editing. To address these issues, we propose Inlay3D, a novel framework for precise part localization and decoupled texture and shape editing. Specifically, Inlay3D first performs Target Part Extraction, selecting an explicit 3D target part from 3D part candidates guided by a lightweight 2D mask. It then applies Two-Stage Editing to the selected part. In Isolated Mesh Editing, we edit a part-isolated representation that contains only the selected region. As a result, the edit can focus on the target part itself, leading to high-quality results while reducing contextual coupling from non-edited regions. We further introduce Boundary Refinement, which refines a narrow boundary region to remove boundary artifacts while preserving the non-target region. Extensive experiments demonstrate that Inlay3D achieves state-of-the-art results in editing quality and non-target region preservation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Local 3D Editing, 3D Part Segmentation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7TSjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents Inlay3D, a novel framework for precise part localization and decoupled texture and shape editing. Guided by a 2D mask, the framework first identifies an explicit target part from a set of 3D part candidates through Target Part Extraction. It then applies a two-stage editing process. It edits the isolated target part, followed by boundary refinement.

Critical strengths
  1. Effective boundary refinement. Mend combines boundary-localized refinement with O-Voxel replacement to improve integration while preserving non-target regions. This design could also benefit other 3D editing frameworks.
  2. Comprehensive evaluation. Experiments compare against state-of-the-art methods on both texture and geometry editing, using stage-specific metrics to demonstrate strong performance in localization, editing quality, and non-target preservation.
Critical weaknesses
  1. Limited component-wise ablations. The Full Mesh Inpainting comparison does not isolate the contributions of individual editing stages.
  2. Insufficient boundary refinement validation. The ablation provides only qualitative texture-editing results, lacking quantitative comparisons and dedicated evaluation of geometric boundary blending.
Questions for authors
  1. Limited component-wise ablations. The Full Mesh Inpainting comparison does not isolate the contributions of individual editing stages.
  2. Insufficient boundary refinement validation. The ablation provides only qualitative texture-editing results, lacking quantitative comparisons and dedicated evaluation of geometric boundary blending.
  3. Failure case. How does the precomputed boundary band handle large changes in the edited part’s attachment location? Please provide a failure case illustrating its limitations.
  4. Replacement strategy. Compare O-Voxel replacement with latent replacement in terms of boundary refinement quality.
2 comments

Thank you for your valuable suggestion to include more ablations. In response, we conducted additional quantitative ablations of latent replacement and boundary refinement. We also show geometric boundary blending results and present a failure case in which the region produced by the first stage extends beyond the precomputed boundary band. 1. Additional Ablation Studies (W1/W2/Q1/Q2/Q4) The existing ablation study on full-mesh inpainting demonstrates the effectiveness of the two-stage editing strategy. We conducted additional ablation studies on specific components of Inlay3D, including Iterative Part Selection and O-Voxel replacement. To evaluate Iterative Part Selection, we removed it and used the best-matching candidate from a single pass of P3-SAM as the target part. To assess O-Voxel replacement, we replaced it with latent replacement. We also included quantitative results for a configuration without boundary refinement ("w/o Boundary Refinement"). FID and KID show larger changes than the other metrics. We hypothesize that these strategies mainly affect boundary regions, which occupy only a small fraction of each rendered image. FID and KID appear more sensitive to these changes. The experimental results are presented in the table below.

MetricOursw/o Boundary Refinementw/o Iterative Part SelectionLatent Replacement
FID ↓54.1559.0859.3457.79
KID ↓0.0500.2150.0890.091

The results show that the complete method achieves the lowest FID and KID values, demonstrating the effectiveness of these strategies. We have added these quantitative ablations to Table 4 of the revised manuscript.

2. Geometric Boundary Blending Results (Q2) Following your advice, we have added the results before and after boundary blending to Figure 7 of the revised manuscript. The results show that Boundary Refinement smooths the boundary band and reduces boundary artifacts.

3. Failure Case (Q3) We acknowledge that boundary refinement is sensitive to the precomputed boundary band. If the result of Isolated Mesh Editing changes in size or position, boundary refinement can be less effective. We applied scaling and translation to the first-stage results to simulate deviations that may occur in practice. We have added this failure case to Appendix E (Figure 8) of the revised manuscript.

Reviewer_7TSjOpenReview ↗

Concerns Addressed

The authors have addressed my concerns, so I will maintain my score.

Reviewer_yKoeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Inlay3D is a training-free Cut-Edit-Mend framework for local editing of textured meshes. It selects and recursively subdivides 3D part candidates using a single-view mask, edits the isolated target with TRELLIS.2, and refines an interface band using O-Voxel replacement. Evaluation covers target extraction on Part-Objaverse-Tiny and texture/shape editing on a collected 100-asset dataset.

Critical strengths
  1. The target-extraction claim has meaningful support. Table 1 reports 74.30% average mIoU, with gains across all eight categories. This supports iterative selection and merging over the evaluated fixed-candidate approaches, although not yet robustness to imperfect user masks.
  2. The texture-editing workflow has a relevant internal comparison. Against full-mesh inpainting, Table 2 reports improved CLIP-Direction (5.27 to 6.55), non-target PSNR (24.75 to 28.30), and LPIPS (0.068 to 0.040). Together with Figure 4, these results support a useful texture-editing/preservation trade-off from the two-stage design.
Critical weaknesses
  1. Table 1 omits SegViGen, a closely related baseline. SegViGen already supports single-view 2D-map-conditioned 3D segmentation. A fair comparison and discussion of methodological differences are needed to establish the additional value of Inlay3D's recursive target extraction.
  2. The pipeline's complexity is insufficiently justified. Recursive segmentation, image-prompt generation, isolated editing, and boundary refinement introduce multiple dependencies, thresholds, and manual tuning. The full-mesh inpainting comparison supports two-stage editing but does not establish the necessity of the complete hierarchy. The paper should clarify the performance gains relative to added overhead and sensitivity to intermediate errors.
  3. Cut lacks discussion of correspondence-based alternatives. CUPID/Pixal3D exploit explicit 2D-3D correspondence; Inlay3D itself already has pixel-to-face correspondence (Algorithm 1). Why is recursive candidate matching preferable to direct visible-mask lifting followed by region completion? Correspondence alone cannot resolve occluded semantic regions, and these generators are not drop-in segmenters, but the paper should explain what recursion adds beyond simpler correspondence-based localization.
  4. Reliable shape editing and attribute decoupling remain unproven. The geometry-editing score is 4.80/10 (Table 3), near the rubric's "partial or inconsistent" level. Appendix D does not describe showing the judge the source asset or target mask, limiting verification of actual target changes. No geometry-only evaluation measures target-texture preservation, so non-target geometry scores cannot establish attribute decoupling within the edited part.
Questions for authors
  1. How does Inlay3D differ from SegViGen's image-guided segmentation? Please include a fair comparison.
  2. What justifies the full pipeline's complexity, and how much overhead or manual tuning does it require?
  3. Could correspondence-based localization simplify Cut? Please discuss what recursive matching adds beyond direct mask lifting and region completion, in relation to CUPID/Pixal3D.
  4. What evidence supports reliable shape editing and preservation of the target texture during geometry-only edits?
2 comments

Thank you for the constructive feedback. We address the comparison with SegviGen, the contribution and cost of the pipeline, the role of recursive matching, and the evidence for shape editing.

1. Comparison with SegviGen (W1/Q1) We appreciate your suggestion to compare with SegviGen. SegviGen encodes a 2D mask as a conditioning signal and predicts colors that indicate part labels across the 3D asset. Inlay3D uses a binary target mask to evaluate the visible projections of pretrained 3D part proposals. We evaluated both methods for target part extraction using the same Part-Objaverse-Tiny benchmark, evaluation protocol, and 2D masks. The results are shown below.

CategoryOursSegviGen (2D map)P3-SAM Free Seg.P3-SAM Point PromptSAMPart3D
Human-Shape80.8170.2445.0521.5346.65
Animals74.6862.8849.0414.1532.98
Daily-Used74.4944.2343.7017.2933.47
Buildings & Outdoor69.9238.5943.3917.3039.26
Transportations70.0756.8043.768.4834.59
Plants66.3364.5942.7321.0234.90
Food76.9562.0443.7929.0343.38
Electronics81.1356.6750.6420.5641.40
Category mean74.3057.0145.2618.6738.33

SegviGen achieves a category mean IoU of 57.01%. Inlay3D achieves 74.30% and outperforms SegviGen across all eight categories. We have added this comparison to Section 4.3 and Table 1 of the revised manuscript.

2. Contribution and Cost of the Pipeline (W2/Q2) Previous end-to-end solutions often lack the ability to modify specific local details. Inlay3D follows the process an artist uses to edit a 3D asset. The Cut–Edit–Mend workflow provides interactive and interpretable steps for refining local details. Each step contributes to precise, granular editing. Inlay3D includes several components. We conducted ablation studies to assess their contributions. The results are shown below.

MetricOursw/o Boundary Refinementw/o Iterative Part SelectionLatent Replacement
FID ↓54.1559.0859.3457.79
KID ↓0.0500.2150.0890.091

We report FID and KID because they show the largest changes in these ablations. Removing Boundary Refinement or Iterative Part Selection increases both FID and KID. Changes in CLIP-T, CLIP-Dir, PSNR, SSIM, and LPIPS are small. We have added the FID and KID results to Table 4 of the revised manuscript.

We also tested sensitivity to several thresholds. In the Cut stage, we varied τ\tau, η\eta, and DD on 50 of the 200 samples in the Part-Objaverse-Tiny dataset. In Two-Stage Editing, we varied the boundary expansion threshold τb\tau_{\mathrm{b}} on 50 of the 100 samples in our evaluation dataset. We have added this analysis to Appendix A of the revised manuscript. Figure 5 shows that small changes to these parameters produce only minor differences in the metrics. The default settings therefore work well in practice. A higher τ\tau favors more precise matching, while a lower τ\tau makes candidate acceptance more permissive. Users can adjust τ\tau for a particular asset if needed.

Finally, we measured the the texture-editing inference time of each component and compared it with that of the corresponding baseline. We ran the experiments on an RTX 3090 GPU and averaged runtimes over five runs. The results are shown below.

StageBaselineExecution TimeProposed MethodExecution TimeAdded Time
Part ExtractionP3-SAM63.34Target Part Extraction129.49+66.15
Image Prompt GenerationQwen-Image-Edit230.46Image Prompt Generation236.36+5.90
3D EditingTRELLIS.2 Texture Generation84.47Two-Stage Editing102.70+18.23
Whole PipelineBaselines378.27Inlay3D468.55+90.28

Our method adds Iterative Part Selection, local rendering, forward and inverse affine transformations, and an additional editing stage. These steps add 90.28 seconds (23.9%) to the baseline runtime of 378.27 seconds. We have added this runtime comparison to Section 4.5 and Table 5 of the revised manuscript.

3. Comparison with CUPID and Pixal3D (W3/Q3) CUPID and Pixal3D are designed to reconstruct an entire object from a single image. Because they rely on learned priors to estimate object geometry, their reconstructions can introduce errors. Our iterative approach selects parts directly from the input mesh, so it does not introduce reconstruction errors. When applied to local regions, methods designed for complete objects treat these regions as complete, which can lead to errors around the boundaries. This makes our approach better suited to target part selection for subsequent editing.

4. Shape Manipulation and Target Texture Preservation (W4/Q4) The qualitative results in Figure 3 of the revised manuscript show that our method can substantially deform objects. We agree that the original image and the 2D mask are useful inputs to the GPT-based evaluation of shape manipulation. Following your suggestion, we reran the evaluation with these inputs. The results are shown below.

MetricOursVoxHammerVecSet-Edit3DEditFormerNano3DInstant3dit
GPT Editing Score ↑5.383.723.594.233.801.54

Our method achieves a score of 5.38, the highest among the evaluated methods, indicating good alignment with the editing instructions. We have updated the scores in Table 3 and described the evaluation inputs in Appendix D of the revised manuscript. Appendix E discusses the remaining limitations of geometric editing. In future work, we will explore how to improve shape manipulation.

For target texture preservation, we acknowledge that our method does not provide a theoretical guarantee of complete decoupling. Shape manipulation can alter the topology and UV map of the edited region. In our experiments, Inlay3D approximately preserves the target texture using texture projection. For each texel in the edited part's texture map, we find the nearest point on the original part's surface to the corresponding point on the edited surface. We sample the original texture at those locations and bake the sampled colors into the edited part's texture map. We have added the texture projection procedure to Appendix C of the revised manuscript. Figure 6 shows that the target texture is largely preserved after shape manipulation.

Reviewer_912fOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper propose a 3-stages framework inspired by artistic workflow Cut-Edit-Mend: first localize the target part based on 2D mask, then performs edit only on the target part region, and finally further refine the boundary region of target edit to resolve artifacts. The results demonstrate that the method can produce high-quality edited results while preserving non-edited region.


Authors have generally addressed my concerns so I will keep my score.

Critical strengths
  • The separate 3-stages editing process is technically sound.
  • Paper exposition is generally easy-to-follow but would need some revision. (see below)
  • Figure 2 is very polish and easy to grasp the overall idea of the method.
  • Results are clearly main advantages of the paper, a clear win across the benchmarks.
Critical weaknesses
  • In the editing stage, the authors say that the isolation of target region for edit is to avoid contextual effects of unedited region. However, this raises a concern that for some types of edit like adding an object like adding armrest to a chair, if model is not seen the context of unedited region like the shape of whole chair, the newly added armrest is likely to have inconsistent shape structure with respect to the whole object.
  • It seems that the method only focuses on som certain types of 3D editing like modifying existing shape (both geometry and texture) and replacing target shape. No adding and removal.
  • 2D mask is cheap to obtain but whether it is reliable to accurately localize target part considering common factor like occlusion and depth ambiguity of projection.

Exposition

  • the intro needs to reorganize. There is no discussion of what other have done and what is the existing gap the paper try to solve. They just jump directly to their solution. And then talk about what others have done, this part should be right after the first paragraph. This kind of structure makes it hard for the audience to get the story since the paper method and the past work are not clearly separated.
  • citation error: L363.
  • Notation of Isolated Mesh Editing is quite dense to follow.
Questions for authors
  • Basically, the refinement stage still use whole-view prompt to edit based on whole input shape and using the mask to replace some voxels at the boundary. I wonder if whole-prompt is already good, why not start it why we need local editing first. Or we can do in reserse flow: global edit first, give reference to local edit.
  • Question for the Target Part Extract, how the method handle failure of 3D segmentation model if the edit part is not detected.
1 comment

Thank you for your valuable questions and suggestions. We first discuss why our approach adopts an editing strategy that proceeds from local to global, then examine the limitations of shape manipulation techniques when applied to tasks involving the addition or removal of target parts, and finally explain how our method handles cases where the 3D segmentation model fails to detect the specific part intended for editing.

1. Local-to-Global Editing Strategy (W1/Q1) We conducted experiments using a whole-view prompt for full-mesh inpainting. Section 4.4 of the revised manuscript discusses these results, which are shown in Figure 4 and Table 2. These results show that this approach produces poor edits in local regions, such as the back of a dragon's wings. These experiments demonstrate the need for Isolated Mesh Editing.

We then conducted a small experiment to evaluate your suggested "reverse flow" approach. Specifically, we used full-mesh inpainting as the initial stage and then applied Target Part Extraction and Isolated Mesh Editing to its output. We have included the results in Figure 1 of the supplementary material. Full-mesh inpainting produces high-quality results in the boundary regions. However, Target Part Extraction relies solely on object geometry for segmentation and can still produce imprecise boundaries. Isolated Mesh Editing introduces artifacts in these boundary regions. Based on these results, we consider the reverse workflow unsuitable.

2. Shape Manipulation in Inlay3D (W1/W2) We acknowledge that Inlay3D currently does not support adding or removing objects. However, our precise segmentation results provide a foundation for object removal.

3. Reliability of 2D Mask-Guided Part Localization under Occlusion and Depth Ambiguity (W3) In Target Part Extraction, we use a rendering of the complete mesh to resolve depth ambiguity based on visibility. However, the target part can be partially occluded in the user view. In such cases, a single-view 2D mask provides limited information for target part extraction. The ability to segment parts depends heavily on the capabilities of the pretrained 3D segmentation model. If the extracted target part is incomplete, the user can adjust the expansion parameters during boundary extraction until the entire target part is covered. Boundary Refinement can then refine the incomplete part. In the revised manuscript, Section 3.2 describes the visible-projection procedure, Appendix A explains how to adjust the boundary band, and Appendix E discusses the limitations of single-view masks.

4. Handling Failures of the 3D Segmentation Model (Q2) Thank you for your valuable question. We checked whether Inlay3D produced empty target regions during the segmentation evaluation. Among the 200 objects evaluated, six yielded empty target regions despite having non-empty input masks. This outcome depends in part on the capabilities of the 3D segmentation model and on the random seed used for segmentation. Changing the seed yielded non-empty target regions in five of these six cases. Lowering the acceptance threshold τ\tau yielded a non-empty target region in the remaining case. When the default settings yield an empty target region, users can try changing the seed or lowering the acceptance threshold. For the case raised in W3, users can expand the boundary region during Boundary Refinement.

5. Writing and Expression Following your advice, we have clarified the relevant passages in the revised manuscript.

Comments on the paper

1 comment

General Response: Contributions and Manuscript Revisions

We sincerely thank the reviewers and the Area Chair for their time and thoughtful consideration. We appreciate the reviewers' constructive feedback and recognition of the following contributions:

  • Precise part-level editing. Inlay3D introduces a Cut–Edit–Mend workflow for localized texture and shape manipulation of 3D assets, enabling fine-grained control over target regions while preserving unedited areas.
  • Reliable target part extraction. Our approach combines pretrained 3D part proposals with mask-guided iterative selection and boundary refinement to accurately identify target regions.
  • Local-to-global editing strategy. The proposed two-stage editing process integrates isolated mesh editing with full-mesh inpainting to balance local editing accuracy and global consistency.

Following the reviewers' suggestions, we have expanded the experiments and revised the manuscript. The main updates are:

  1. Comparisons with stronger baselines (yKoe). Section 4.3 and Table 1 now include a comparison with SegviGen under the same benchmark, evaluation protocol, and 2D masks. Inlay3D achieves a category mean IoU of 74.30%, outperforming SegviGen (57.01%) across all eight categories. We also clarify the differences from CUPID and Pixal3D, particularly regarding reconstruction errors and local editing suitability.

  2. Additional ablation studies (yKoe, 7TSj). Table 4 now includes quantitative ablations of Iterative Part Selection, Boundary Refinement, and O-Voxel replacement. The complete method achieves the best FID (54.15) and KID (0.050), demonstrating the contributions of these components. Figure 7 further provides geometric boundary blending results, showing reduced boundary artifacts after refinement.

  3. Validation of the editing strategy (912f). Section 4.4, Figure 4, and Table 2 provide additional evidence for the necessity of Isolated Mesh Editing through comparisons with full-mesh inpainting. We also investigate the suggested reverse workflow, which performs full-mesh inpainting before isolated mesh editing. The supplementary results show that this alternative introduces boundary artifacts, supporting our original local-to-global design.

  4. Shape manipulation and texture preservation (yKoe, 912f). We revise the GPT-based shape editing evaluation to include the original image and 2D mask. Inlay3D achieves the highest editing score of 5.38 among the evaluated methods (Table 3). Appendix C further clarifies the texture projection procedure for approximately preserving target textures after shape manipulation. We also acknowledge the current limitations of geometric editing, including unsupported object addition and removal.

  5. Efficiency, robustness, and failure analysis (yKoe, 912f, 7TSj). Section 4.5 and Table 5 report component-wise runtime measurements, showing that Inlay3D requires 468.55 seconds compared with 378.27 seconds for the corresponding baseline pipeline, an additional 23.9% cost. Appendix A and Figure 5 analyze parameter sensitivity, demonstrating relatively stable performance under small parameter variations. We further investigate segmentation failures and provide practical adjustments to improve target extraction. Appendix E discusses limitations related to occlusion, incomplete segmentation, and boundary refinement, including a new failure case illustrated in Figure 8.

Our individual responses provide the corresponding experimental results and detailed explanations. We thank the reviewers again for their valuable suggestions, which have strengthened the empirical validation, clarified the design choices, and improved the presentation of Inlay3D. We sincerely hope that our revisions and responses have addressed the reviewers' concerns, and we would greatly appreciate any further feedback or comments from the reviewers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.