acceptodds
Withdrawn from ICLR 2027

SAP3D: Spatially Adaptive Personalization for Consistent 3D Generative Editing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Image-to-3D foundation models produce clean, detailed 3D assets, but offer no direct way to edit an existing one, be it generated or manually crafted. A natural route is to edit a 2D render of the asset and regenerate from it. Existing editors either hold part of the source's latent representation fixed, as frozen latent tokens or cached voxel features, which produces geometric artifacts and preserves nothing under a global edit, or couple the generation to the source's trajectory, which holds the edit back. We instead give the generator a prior for the specific asset: a low-rank adapter is fitted to the source object and applied at a variable strength per latent token, so that tokens in the edit region run without the adapter and follow the edited image, while every other token runs with it and reproduces the source. To decide which tokens belong to the edit, we use two probes: a 2D probe on the transformer's image cross-attention, which selects tokens by image content, and a 3D probe on the shape decoder's cross-attention, which grounds the region in 3D and reaches geometry the edited view does not show. Since VecSet tokens are not bound to fixed locations and drift during denoising, we re-estimate the assignment at every step. Because the adapter carries the asset's identity rather than any one region of it, a uniform strength also turns it into a control for global edits. On Edit3D-Bench, our method achieves the highest edit fidelity of all compared methods at numerically competitive preservation, and participants in a user study preferred its results in 88% of the comparisons that were not ties.

Primary area:
generative models
Keywords:
Deep Learning, Generative AI, image-to-3D, 3D editing, 3D Personalization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_5QuvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SAP3D, a spatially adaptive personalization framework for consistent 3D asset editing. Given a source 3D asset, an edited 2D image, and a specified 3D editing region, the method aims to achieve the desired modifications while preserving the unedited parts of the original object. To address the limitations of existing latent preservation methods, particularly for VecSet-based representations where tokens lack fixed spatial locations, the authors propose to personalize the 3D generative model using an asset-specific LoRA. During generation, the LoRA strength is adaptively controlled at the token level using complementary 2D and 3D attention probes, with token assignments dynamically updated throughout the denoising process.

The method is evaluated on Edit3D-Bench using Hunyuan3D 2.1 as the backbone, with comparisons against five existing 3D editing methods. The results demonstrate competitive preservation performance and improved edit fidelity, supported by quantitative evaluations, qualitative comparisons, user studies, and ablation experiments.

Critical strengths
  1. Clear problem motivation and analysis of existing methods. The paper provides a well-motivated analysis of the limitations of existing 3D editing methods, particularly the challenges of applying latent preservation strategies to VecSet-based representations. I find the problem formulation convincing, and the authors demonstrate a good understanding of the differences between structured and unstructured latent representations.

  2. Thoughtful method design with promising results. The proposed combination of asset-specific personalization and spatially adaptive LoRA control is an interesting approach to balancing edit fidelity and preservation. The overall design is reasonable and reflects the authors' own insights into the problem. The competitive experimental results, particularly in edit fidelity, further support the effectiveness of the framework.

Critical weaknesses
  1. High computational cost of per-asset LoRA training. The method requires training a separate LoRA for each 3D asset, which could significantly limit its practical applicability, especially for interactive 3D editing. Although the authors mention up to 5,000 training steps, the actual training time and computational overhead are not reported. I believe this is an important concern, particularly compared with existing training-free methods.

  2. Insufficient evidence for the necessity of dynamic localization. While SAP3D achieves the best edit fidelity among the compared methods, its preservation performance is less competitive than that of some baselines. More importantly, the ablation results in Table 3 raise questions about the contribution of the proposed localization mechanism. In particular, fixing the editing region at τm\tau_m achieves slightly better edit fidelity on both DINOv2 and CLIP metrics, with nearly identical preservation performance. The authors acknowledge in Appendix C that the benefits of dynamic localization are concentrated on a subset of cases and report qualitative improvements in several examples. However, the current evidence does not sufficiently establish when and why dynamically updating token assignments is necessary, especially given the additional complexity and potential computational overhead. A more systematic per-case analysis would strengthen this central technical contribution.

Questions for authors
  1. Computational cost and practical efficiency. Could the authors provide the actual training time required for a per-asset LoRA? More specifically, how long does it take to edit a new 3D asset, including LoRA training and the complete editing pipeline? I think this is important for assessing the practical applicability of the proposed method, especially compared with training-free approaches.

  2. Effectiveness of the proposed components. While Table 1 shows improvements in edit fidelity, the preservation results are less competitive. More importantly, in Table 3, the benefits of the localization components appear relatively limited, and the fixed-region variant even achieves slightly better edit fidelity than the full method. Could the authors further explain these results and provide stronger evidence for the effectiveness and necessity of dynamic localization?

  3. Core novelty and technical contribution. Given the relatively limited gains from the localization components in Table 3, I am somewhat concerned about the significance of the technical novelty. Could the authors clarify what they consider the most important technical contribution of this work, beyond combining per-asset LoRA personalization with attention-based localization? At present, I am not fully convinced that the demonstrated contributions are sufficiently substantial for ICLR, and a clearer justification would help inform my assessment.

Reviewer_9UwMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SAP3D, a method for editing 3D assets using pretrained image-to-3D diffusion models. Specifically, the author propose training a LoRA model for the source asset and selectively disable personalization for tokens corresponding to the edited region. The author first train a lora model for each source asset, then they localize the editing tokens based on cross-attentions and apply lora on the unedited region while using pretrained model for generating new part.

Critical strengths
  • The proposed method is technically sound and provides a reasonable way to balance edit fidelity and source preservation
  • The method achieves competitive performance on Edit3D-Bench. The authors also conduct a user study to support their findings and provide a useful analysis of the limitations of existing evaluation metrics.
Critical weaknesses
  • Since the model requires personalized LoRA for source assets, the editing time for each assets would be impractical when users only need to edit an asset once.
  • The novelty of the paper mostly come from the dynamic token assignment. However, the proposed approach for token assignment is closely tied to the architecture of Hunyuan3D 2.1. In particular, the 3D probe assumes a decoder with cross-attention between 3D query points and latent tokens. This structure is not available in many other image-to-3D models. It is therefore unclear how easily the method can extend to different architectures.
  • The effectiveness of dynamic token assignment also questionable at each perform closely with fixed region as in Table 3
  • The proposed approach only tested on Hunyuan3d 2.1, it would be more convincing if the author also report performance/ablation on different models. It would also be useful to analyze the sensitivity of important hyper-parameters, including LoRA strength and the editing schedule, across different models.
Questions for authors

it would be helpful to add the training time for lora model of each asset.

Reviewer_M2uROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper first uses LoRA to personalize the model to a source object, then determines which 3D tokens can change and which should preserve the source. It identifies edit-related tokens using 2D and 3D attention and updates this selection during generation. Experiments show that the method can faithfully perform the requested edits.

Critical strengths
  1. Using an object-specific LoRA to preserve the source while allowing local changes is a valuable idea, and the experiments support the benefit of personalization.
  2. The evaluation checks both whether the result preserves the source and whether the requested edit is actually completed, providing a more comprehensive assessment.
Critical weaknesses

1.The need to reselect tokens at every step is not sufficiently demonstrated. In Table 3, selecting the tokens once and keeping them fixed does not produce worse edit fidelity, while source preservation remains nearly identical. The evidence therefore supports the effectiveness of LoRA more clearly than the additional benefit of dynamic reassignment. 2. The actual runtime is unclear. Each object requires separate LoRA training, and editing involves additional Decoder evaluations. However, the paper does not report training or editing times, making its practical computational cost difficult to assess.

Questions for authors

1.In Table 3, fixing the token selection achieves comparable preservation and slightly better edit fidelity than dynamic reassignment. Could the authors clarify when dynamic reassignment is beneficial? 2.Could the authors report the per-object LoRA training time, per-edit latency, and additional runtime introduced by the 3D probe?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.