acceptodds
Withdrawn from ICLR 2027

HiFi3D: High-Fidelity 3D Asset Editing for Sparse Voxel Representations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing 3D editing methods suffer from two critical limitations: optimization-based approaches require lengthy per-instance tuning, while recent flow-based methods struggle to preserve fine geometric structures and material details during editing. We introduce HiFi3D, a training-free 3D editing framework built upon TRELLIS2's factored O-Voxel representation, enabling precise instruction-guided modifications without model retraining. Our method overcomes two fundamental challenges: fragility of sparse voxel topology under stochastic perturbations, and the entanglement between geometric structure and surface appearance in joint editing pipelines. For geometry manipulation, we propose Dual-Space Geometry-Preserving Flow Editing, which combines mask-free regional dynamics with dual-level preservation, maintaining trajectory stability while voxel-supervised alignment ensures decoded geometric consistency through a differentiable decoder. For appearance control, we introduce Geometry-Conditioned Material Editing with PBR consistency constraints, reformulating texture editing as a geometry-aware process where multi-sample velocity estimation stabilizes material flow and decoded-space regularization enforces physically plausible PBR attributes. Extensive experiments demonstrate that HiFi3D achieves accurate instruction-following modifications while outperforming state-of-the-art baselines in both visual quality and geometric fidelity.

Primary area:
generative models
Keywords:
3D Asset Editing, Material Editing, 3D Generative Models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_h74BOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a training-free 3D asset editing framework built on TRELLIS2. Geometry editing combines source–target flow differences with latent anchoring and decoded voxel-space preservation. Material editing uses multi-sample velocity estimation and PBR regularization. Experiments on a proposed 200-task benchmark report improvements in geometry preservation, appearance quality, inference time, and user preference.

Critical strengths

This paper addresses a practically relevant problem: performing localized edits while preserving source geometry and materials. Combining latent-space anchoring with decoded-space preservation is a reasonable direction. The evaluation covers multiple editing categories, includes separate geometry-component ablations, and supplements automatic metrics with a 40-participant user study. The qualitative examples suggest potentially useful editing capabilities.

Critical weaknesses

The central physical assumptions and theoretical guarantees are not sound as stated. Appendix B.2 treats highly metallic, highly rough materials as physically implausible, although rough conductors are valid in standard metallic–roughness PBR models. Consequently, the proposed penalty can suppress legitimate materials rather than enforce physical validity. Similarly, surface roughness does not generally require spatially smooth base-color textures. These terms could potentially serve as empirical priors, but their applicability and effects require validation. Furthermore, Theorem 2 does not follow from the stated Lipschitz assumptions. Eq. (43) establishes descent only when the gradient norm is sufficiently large; no condition relates gradient magnitude to loss value or excludes positive-loss stationary points. Thus, the argument cannot establish the claimed convergence to a physically valid manifold. These concerns affect a principal methodological contribution rather than presentation alone.

The evaluation leaves key conclusions unverifiable and does not sufficiently isolate the incremental contribution. The benchmark description lists source geometry and source/target images, without explaining the target 3D references required for edited-region Chamfer Distance and overall F-Score. Comparing edited regions against source geometry would measure deviation rather than editing correctness. The material consistency metric averages LPIPS across viewpoints without describing correspondence compensation, confounding viewpoint changes with inconsistency. Comparisons also mix generative backbones, making it difficult to attribute improvements to the proposed editing mechanism. Closely related approaches, including AnchorFlow and the TRELLIS2-based VS3D, are omitted. Material ablations compare the complete module against TRELLIS2 without separating sample averaging and individual regularizers. Hardware, resolution, runtime scope, and uncertainty are insufficiently reported.

Questions for authors

1, How are target 3D references and evaluation regions constructed? Please clarify the references, registration, and sampling protocols for Chamfer Distance and F-Score. 2, What justifies excluding rough metallic materials? Can corrected regularizers and component-wise ablations establish their benefit without suppressing valid appearances? 3, What additional assumptions make Theorem 2 valid, and can matched-backbone comparisons establish gains beyond existing flow-based preservation strategies?

Reviewer_uPUMOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes HiFi3D, a training-free 3D editing framework based on TRELLIS2. It introduces dual-space geometry preservation and geometry-conditioned material editing to improve structural consistency and appearance quality.

Critical strengths
  1. The proposed method addresses important challenges in 3D editing by introducing dual-space geometry preservation and geometry-conditioned material editing, with a clear and intuitive overall design.
  2. The method achieves strong quantitative and qualitative performance across various 3D editing tasks, demonstrating improvements in geometric fidelity, appearance quality, and editing efficiency.
Critical weaknesses
  1. The experimental evaluation omits several highly relevant and recent 3D editing methods, including AnchorFlow (which is discussed in the Related Work section), Towards Scalable and Consistent 3D Editing (ICML 2026), and VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Space (3DV 2026). Without comparisons against these competitive approaches, it is difficult to establish the effectiveness and superiority of the proposed method over the current state of the art.
  2. The paper defines the editable region by comparing the source and target images but does not provide implementation details on how the semantic difference regions are identified. Although the method is described as "mask-free," it still explicitly relies on the identified editable and invariant regions, which makes this claim potentially misleading. Furthermore, the proposed framework heavily depends on the accuracy of these regions, yet no ablation or robustness analysis is provided to investigate how region identification affects editing performance.
  3. In Table 2, the base model without either latent or voxel supervision already achieves a Chamfer Distance of 0.0161, outperforming all competing methods reported in Table 1. This suggests that the strong performance may largely originate from the underlying baseline rather than the proposed components.
Questions for authors
  1. Could the authors provide comparisons with more recent and relevant methods, such as AnchorFlow, Towards Scalable and Consistent 3D Editing, and VoxHammer, to better establish the proposed method's competitiveness?
  2. How exactly are the editable regions identified from the source and target images? Given the method's dependence on these regions, how is the "mask-free" claim justified, and how sensitive are the results to region identification accuracy?
  3. Why does the base model in Table 2 already outperform all competing methods in Table 1? Could the authors clarify the baseline configuration and ensure a fair comparison to better isolate the contributions of the proposed modules?
Reviewer_fgnxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes HiFi3D, a training-free method for editing 3D assets in TRELLIS.2’s O-Voxel representation. First, they edit a rendered view of an asset using a 2D editor (Nano Banana Pro) to obtain the edited image. Then, they encode the source asset into TRELLIS.2’s latents, which store the location, material, and shape of each voxel. Next, they run two separate flow edits, one for geometry and the other for materials, conditioned on the source and edited images. For geometry editing, they iteratively apply preservation terms to keep unedited regions close to the source in both latent and voxel space. The region mask is also applied only to these preservation terms instead of the edit velocity. For material editing, they add PBR penalties to keep the materials physically plausible. They evaluate the method on a new benchmark, HiFi200, consisting of 200 samples, reporting the best results across all geometry, appearance, and runtime metrics. They show a 60.31% preference rate in the user study. The ablation also examines the effects of both preservation terms.

Critical strengths

It is well motivated to include geometric consistency in voxel space instead of just in latent space. Since the decoder is nonlinear, a small latent deviation can cause a large geometric error. The material edit follows the same idea by constraining decoded PBR values, and it gives more view-consistent materials than TRELLIS.2’s texture optimization. Together, they adapt flow editing to sparse voxel representations.

Critical weaknesses

It is unclear whether the improvement is from the TRELLIS.2 backbone or from the proposed design. Table 1 does not explain it since Nano3D, which applies FlowEdit to the original TRELLIS and copies the source back into the unedited regions, is different from HiFi3D. Table 2 partially separates them, since the Base is FlowEdit on TRELLIS.2 without the preservation terms. However, Table 2 reports only CD(R_i) (L403-411), which measures the unedited regions, so it does not show how much the Base already gains on the other metrics in Table 1. Similarly, the material edit is compared only with TRELLIS.2’s texture optimization (L470-477), which is a different method, so the effect of the added PBR penalties remains unknown. Overall, apart from CD(R_i), the paper does not show how the two contributions improve the results.

Questions for authors
  1. Can you show how much of the improvement is from the TRELLIS.2 backbone and how much is from the proposed design?
  2. Do the preservation terms affect the quality of the edited regions? Could you report Table 2 on all the metrics in Table 1? This would also answer Q1.
  3. Can you show what the PBR penalties contribute?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.