- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work focuses on 3D asset editing and argues that progress is limited by the lack of large paired 3D data. The authors build Alchemy3D-1M, which is a dataset consisting of 1.38M editing pairs across 7 edit types. They also train a TRELLIS.2-based editor on it along with a faster distilled version. Furthermore they introduce GEdit3D-Bench, a new benchmark that uses held-out assets and MLLM judge. On several existing benchmarks and a user study, their models are stated to beat prior methods.
Critical strengths
-
The paper provides large-scale resources. Alchemy3D-1M is much bigger than previous 3D editing datasets and covers more edit types (Tab 1). GEdit3D-Bench uses assets that are not in the training data, which makes the evaluation fairer.
-
The results are strong in some sections. On add, remove, and replace, Alchemy3D gets around 90% success while the baselines get ~50-60% and users picked it ~70% of the time (Tab 2). It also does better on the other benchmarks (Tab 3-4).
Critical weaknesses
- A central premise of the paper is that 3D editing is limited by the lack of large paired 3D data (Abstract, Sec. 1). I believe this premise is never tested.
-
Most importantly, there is no scaling experiment. The model is never trained on less data (for ex. a percentage of the 825K pairs), so we cannot tell whether more data actually helps. This matters in my opinion because scale is the focus of the title and abstract.
-
The gains could also come from other factors. First, Alchemy3D starts from the strong TRELLIS.2 backbone, while all image-conditioned baselines (Nano3D, 3DEditFormer, PartFlow) build on TRELLIS. For example, Alchemy3D beats 3DEditFormer on 3DEditVerse (Tab 4), but two models differ in both training data and backbone, so this does not show that the data is the reason. Another example is that Tab 7 suggests the backbone gives most of the benefit (28.54 mIoU from scratch, 58.68 from TRELLIS.2, and only 60.08 from Alchemy3D, a small gain).
-
Second, the 2D edit seems to do much of the work. In Tab 14, editing one view in 2D and then lifting it with Alchemy3D beats direct text editing on most edit types (ex. 50.0% vs 6.7% on global appearance). If the 3D model mainly lifts a 2D edit, it is unclear how much paired 3D data it really needs. The paper also does not explain why it creates new 2D edit pairs with FLUX.2 instead of reusing existing 2D editing datasets such as AnyEdit.
- The benefit of one unified model for all seven edit types is not clear.
- The authors stated appearance edits are learned poorly when trained together with geometry edits (Appendix A.6). In Tab 9, Alchemy3D reaches 28.3% success on global appearance, compared with ~90% on add, remove, and replace. Since there is no comparison with separate models trained on the same data, we don't know whether combining the tasks helps or hurts.
- The evaluation does not fully support the claims.
-
Training assets for structural and appearance edits are regenerated with TRELLIS.2 (Sec. 3). The benchmark includes real Sketchfab assets, but the paper does not say how many, and does not report their results separately. So it is unclear how well the model works on real assets.
-
The paper argues that one edit can have many correct results (Sec. 5), but it appears to test only one result per input (Sec. 6.1). We do not know how much results vary across random seeds.
Questions for authors
-
[W1] Can the authors train the model on less data (for ex. 10% and 50% of the 825K pairs) and report the results? This would show whether a large dataset is really needed.
-
[W1] The paper evaluates on 3DEditVerse, but can authors also train Alchemy3D with 3DEditVerse train set and compare it with their current model? This would separate the effect of their data & TRELLIS.2 backbone.
-
[W2] Can the authors compare the unified model with separate models trained for each edit type on the same data? This would show whether training all edit types together helps or hurts.
-
[W3] Can the authors report results on the real Sketchfab assets alone? This would show how well the model works on real assets, not only generated ones.
Dear reviewers,
Thanks for your continued efforts. The authors have provided a rebuttal to the initial reviews. Please review it carefully, along with all other reviews and the corresponding author responses, and consider whether you have any further questions for the authors or whether the rebuttal has addressed your concerns. There are only a few days remaining before the discussion period closes, so please use this time to seek any further clarifications or raise any additional questions about this paper.
Best,
AC