acceptodds
Under review as a conference paper at ICLR 2027

Scaling Versatile 3D Assets Editing with a Million-Scale Dataset

Abstract

Although recent 3D generative models produce increasingly realistic assets, controllable 3D asset editing remains challenging. Existing methods are limited by scarce training data, insufficient source-aware modeling, and a lack of practical evaluation protocols. To address these limitations, we present Alchemy3D, a unified framework for training and evaluating versatile 3D asset editors that covers data construction, model architecture, and benchmark evaluation. Specifically, we curate Alchemy3D-1M, a large-scale 3D editing dataset containing 1.25M assets and 1.38M editing pairs across seven editing types. On this data, we train a family of generative flow models for general-purpose 3D asset editing. The model family supports image- and text-conditioned editing, few-step inference, and transfer to multi-view 3D part segmentation. We further introduce GEdit3D-Bench, a large-scale, open-world benchmark with a multi-dimensional evaluation protocol. Across existing and newly introduced benchmarks, our method outperforms prior methods on most metrics of editing fidelity, source preservation, and visual quality. Our dataset, models, and benchmark will be made publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Generation, 3D Editing, 3D Dataset

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_A93uOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This work focuses on 3D asset editing and argues that progress is limited by the lack of large paired 3D data. The authors build Alchemy3D-1M, which is a dataset consisting of 1.38M editing pairs across 7 edit types. They also train a TRELLIS.2-based editor on it along with a faster distilled version. Furthermore they introduce GEdit3D-Bench, a new benchmark that uses held-out assets and MLLM judge. On several existing benchmarks and a user study, their models are stated to beat prior methods.

Critical strengths
  1. The paper provides large-scale resources. Alchemy3D-1M is much bigger than previous 3D editing datasets and covers more edit types (Tab 1). GEdit3D-Bench uses assets that are not in the training data, which makes the evaluation fairer.

  2. The results are strong in some sections. On add, remove, and replace, Alchemy3D gets around 90% success while the baselines get ~50-60% and users picked it ~70% of the time (Tab 2). It also does better on the other benchmarks (Tab 3-4).

Critical weaknesses
  1. A central premise of the paper is that 3D editing is limited by the lack of large paired 3D data (Abstract, Sec. 1). I believe this premise is never tested.
  • Most importantly, there is no scaling experiment. The model is never trained on less data (for ex. a percentage of the 825K pairs), so we cannot tell whether more data actually helps. This matters in my opinion because scale is the focus of the title and abstract.

  • The gains could also come from other factors. First, Alchemy3D starts from the strong TRELLIS.2 backbone, while all image-conditioned baselines (Nano3D, 3DEditFormer, PartFlow) build on TRELLIS. For example, Alchemy3D beats 3DEditFormer on 3DEditVerse (Tab 4), but two models differ in both training data and backbone, so this does not show that the data is the reason. Another example is that Tab 7 suggests the backbone gives most of the benefit (28.54 mIoU from scratch, 58.68 from TRELLIS.2, and only 60.08 from Alchemy3D, a small gain).

  • Second, the 2D edit seems to do much of the work. In Tab 14, editing one view in 2D and then lifting it with Alchemy3D beats direct text editing on most edit types (ex. 50.0% vs 6.7% on global appearance). If the 3D model mainly lifts a 2D edit, it is unclear how much paired 3D data it really needs. The paper also does not explain why it creates new 2D edit pairs with FLUX.2 instead of reusing existing 2D editing datasets such as AnyEdit.

  1. The benefit of one unified model for all seven edit types is not clear.
  • The authors stated appearance edits are learned poorly when trained together with geometry edits (Appendix A.6). In Tab 9, Alchemy3D reaches 28.3% success on global appearance, compared with ~90% on add, remove, and replace. Since there is no comparison with separate models trained on the same data, we don't know whether combining the tasks helps or hurts.
  1. The evaluation does not fully support the claims.
  • Training assets for structural and appearance edits are regenerated with TRELLIS.2 (Sec. 3). The benchmark includes real Sketchfab assets, but the paper does not say how many, and does not report their results separately. So it is unclear how well the model works on real assets.

  • The paper argues that one edit can have many correct results (Sec. 5), but it appears to test only one result per input (Sec. 6.1). We do not know how much results vary across random seeds.

Questions for authors
  1. [W1] Can the authors train the model on less data (for ex. 10% and 50% of the 825K pairs) and report the results? This would show whether a large dataset is really needed.

  2. [W1] The paper evaluates on 3DEditVerse, but can authors also train Alchemy3D with 3DEditVerse train set and compare it with their current model? This would separate the effect of their data & TRELLIS.2 backbone.

  3. [W2] Can the authors compare the unified model with separate models trained for each edit type on the same data? This would show whether training all edit types together helps or hurts.

  4. [W3] Can the authors report results on the real Sketchfab assets alone? This would show how well the model works on real assets, not only generated ones.

Reviewer_Vu2aOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces Alchemy3D, a 1.38M-pair 3D editing dataset covering seven edit types, a family of TRELLIS.2-based editors using source-target token concatenation, and GEdit3D-Bench, an open-world benchmark with MLLM-based evaluation.

Critical strengths
  1. The dataset is roughly 10× larger than existing 3D editing datasets and covers more edit types, including animation and appearance editing. Its release would benefit the community.

  2. GEdit3D-Bench uses assets independent of the training data and avoids relying on a single reconstructed target as ground truth.

  3. The results are consistently strong across five benchmarks, with substantial gains in success rate and human preference. The Turbo variant runs in approximately 1.2 s, and the transfer to part segmentation is a useful additional result.

Critical weaknesses
  1. The scaling claim is not experimentally supported. Despite the title, the paper provides no data-scaling experiments, and the baselines are not retrained on Alchemy3D-1M. It is therefore difficult to separate the contributions of data scale, architecture, and the TRELLIS.2 backbone, especially since most baselines use TRELLIS v1.

  2. The architectural contribution is not isolated. Source-target token concatenation is proposed as an alternative to ControlNet-style injection, but the two are not compared under the same training conditions. Similar concatenation strategies are already common in image editing, while the distillation stage mainly combines existing techniques (MeanFlow, DMD, and AnyFlow).

  3. The evidence for source preservation is limited. Identity preservation is evaluated only by an MLLM. On 3DEditVerse, Alchemy3D underperforms in target alignment (CD: 24.48 vs. 13.36; F1: 44.15 vs. 72.18), with similar concerns on Nano3D-100K. The authors attribute this to baseline training data, but PartFlow also performs better on Nano3D-100K. Geometric preservation metrics outside edited regions would provide stronger evidence.

Questions for authors
  1. How does performance scale with dataset size (e.g., 10%, 30%, and 100%)?

  2. How does token concatenation compare with ControlNet-style injection when trained on the same data?

  3. How well do the MLLM evaluation scores correlate with human judgments?

  4. What are the licensing terms and release plans for the Sketchfab assets and the 1.25M generated assets?

Reviewer_Jf8bOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents Alchemy3D, an integrated framework for large-scale 3D editing data, model development, and evaluation. Alchemy3D-1M contains approximately 1.25 million assets and 1.38 million editing pairs, with about 825,000 pairs used for unified training. Building on TRELLIS.2, the model jointly attends to source-asset and noisy target tokens across sparse-structure, geometry, and material stages. It supports image and text conditioning, MeanFlow/DMD-based three-step distillation, and transfer to part segmentation. The authors introduce the 2,100-case GEdit3D-Bench and report broad improvements, particularly on structural edits.

Critical strengths
  1. Substantial dataset contribution!! The resource expands both scale and task coverage, including appearance and pose changes beyond common structural modifications.
  2. Edit masks and stored generation trajectories constrain unchanged regions more carefully than independently reconstructing source and target assets.
  3. Multiple benchmarks, multi-view metrics, model-based judging, and interactive 3D user evaluation provide complementary evidence, especially for structural edits.
  4. Turbo reduces reported sampling time from 5.202 to 1.231 seconds, while segmentation experiments suggest that editing pretraining yields transferable features.
Critical weaknesses
  1. Data, backbone, and architectural improvements are confounded. Main comparisons simultaneously change the underlying generator, training-set scale, and source-conditioning mechanism. They therefore do not establish whether joint token attention is responsible for the gains over auxiliary conditioning branches. Matched-data, matched-budget architectural comparisons and data-scaling experiments are needed. The separate contributions of MeanFlow and DMD distillation are also insufficiently isolated.

  2. Appearance editing is considerably weaker than structural editing. The direct text-conditioned model achieves only 6.7% success on global appearance changes, below Steer3D’s 18.3% in Table 14. The agentic 2D-editing pipeline improves performance but adds external models and cost. The explanation based on joint-training task conflict needs single-task or rebalanced-training controls.

  3. Timing uses ten examples on one H200.

Questions for authors

Please refer to the weakness 1 2 and 3. The authors are recommended to supplement more details about efficiency, appearance editing discussion and detailed ablation.

Comments on the paper

1 comment

Dear reviewers,

Thanks for your continued efforts. The authors have provided a rebuttal to the initial reviews. Please review it carefully, along with all other reviews and the corresponding author responses, and consider whether you have any further questions for the authors or whether the rebuttal has addressed your concerns. There are only a few days remaining before the discussion period closes, so please use this time to seek any further clarifications or raise any additional questions about this paper.

Best,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.