acceptodds
Under review as a conference paper at ICLR 2027

HGA3D: High-Fidelity Geometry-Aware 3D Generation Acceleration

Abstract

Recent VecSet-based 3D diffusion models, which represent 3D geometry as a compact set of latent vectors, achieve high generation quality but suffer from prohibitive inference latency due to multi-step denoising and expensive decoding. The central challenge is identifying reducible computational redundancy without corrupting the zero-level surface that determines the final mesh. To address this issue, we leverage geometry preservation as a guiding principle to accelerate 3D generation, implementing acceleration strategies across both the diffusion and decoding phases. In the diffusion stage, we introduce geometry-aware token pruning, which exploits decoder cross-attention between sparse dynamic surface anchors and latent tokens to retain surface-critical tokens while eliminating redundancy. In the decoding stage, we propose curvature-adaptive volume decoding, which facilitates efficient mesh decoding by adaptively concentrating high-resolution queries on geometrically complex regions while sparsifying redundant queries in smooth near-surface areas. Experiments on TripoSG and Hunyuan3D-2.0 demonstrate total inference speedups of and , respectively, while preserving high-fidelity and structurally consistent 3D generation. Our code will be released.

Primary area:
generative models
Keywords:
3D generation, 3D generative models, diffusion models, inference acceleration, token pruning, efficient inference

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3UsfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HGA3D speeds up 3D generation by selectively updating latent tokens during denoising and reducing decoding queries based on surface curvature. It requires no additional training and reports roughly 1.5× speedups on TripoSG and Hunyuan3D-2.0, with some loss in geometric quality.

Critical strengths
  • The method requires no additional training and works on two different 3D generation backbones.
  • It achieves about 1.5× overall speedups while keeping most geometric metrics close to the original models.
Critical weaknesses
  • The pruning ratios and refresh schedules used for RAS and SADA are not reported, making the comparisons difficult to assess.
  • Some key settings are missing, including the pruning start and end steps and the SDF thresholds. The text and Algorithm 1 also describe different anchor removal rules.
  • Pruning occurs within the first 10 denoising steps, but the reliability of surface anchors at this early stage is not evaluated.
  • Experiments are limited to TripoSG and Hunyuan3D-2.0. Performance on newer models with more tokens and finer geometry such as ultrashape remains unclear.
Questions for authors
  • Does the hierarchical decoding baseline in Table 2 include FlashVDM’s Adaptive KV Selection?
  • During sparse denoising steps, are skipped tokens removed from self-attention entirely, or are their cached keys and values retained?
  • When does surface-based token pruning start, and how do you avoid unreliable anchor selection early in denoising?
  • The text describes "anchors that consistently deviate from the surface over consecutive timesteps are identified as redundant and subsequently removed", but Algorithm 1 removes invalid anchors immediately. Which is used in the experiments?
  • Can you evaluate HGA3D on a newer model with more tokens, such as UltraShape, to test whether the speedup holds while preserving finer geometric details?
2 comments

Thank you for the clarifications and the additional experiments on UltraShape-1.0. These results help address my concern about applicability to models with larger token counts.

However, the qualitative results currently presented in the paper are too limited to adequately assess the claimed preservation of geometric fidelity. More visual evidence is needed to complement the aggregate metrics.

Could you provide additional qualitative comparisons on Hunyuan3D-2.0 and UltraShape-1.0, showing the input image, vanilla output, and HGA3D output under matched settings and viewpoints? Comparisons with the tuned RAS and SADA baselines would also be helpful.

Please include a broader range of examples with close-up views of thin structures, sharp edges, and fine geometric details, so that readers can better assess the quality–speed trade-off.

Thank you for the detailed rebuttal and clarifications. Your responses have addressed my main concerns, and I have increased my score accordingly.

Reviewer_udH2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a method to accelerate the 3D generation procedure. The proposed method has two main section: 1. the geometry-aware token pruning, which aims at eliminating the redundancy of diffusion procedure by diffuse the tokens that plays more important role in the 3D generation; 2. curvature-adaptive volume decoding, which selects the voxels with high estimated curvature to reduce the computation of hierarchical decoding. In the experiments, the authors claim that the proposed method can be around 1.5 times faster than the SOTA baselines while preserving high-fidelity.

Critical strengths
  1. the design of geometry-aware token pruning and curvature-adaptive volume decoding are novel and interesting.
  2. the proposed method optimized both the diffusion and decoding, which displays the comprehensive thinkness of the 3D generation problem.
  3. the proposed method can be utilized directly to DiT-based model without fine-tuning.
Critical weaknesses
  1. The experiments could be more convincing: in the experiments, the authors compare the perfromance with some SOTA 3D generation methods. I wonder what would happen if the number of time-steps used in original TripoSG is decreased? It would be interesting to compare the time-cost and performance between the TripoSG + proposed-method and the TripoSG + reduced timesteps.
  2. In geometry-aware token pruning, the tokens are selected based on the cross-attention between latent tokens and anchor points. Such design sounds reasonable, but I think more theoretical explanation and studies are needed.
Questions for authors

I am little bit confused by the formula 3 and related descriptions. Formula 3 is used to estimate the generated latent vector in the diffusion procedure, but the formula 3 is clearly used for flow matching, I assume the "diffusion phase" mentioned in this paper is actually means the inference phase of flow matching, but such description is a little bit confusing.

Reviewer_HYcbOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a training-free, vecset-based 3D diffusion acceleration framework, including geometry-aware token pruning during denoising, and curvature-based surface decoding. The core idea is to utilize the cross-attention probing / curvature to identify the importance of latent tokens / sampled decoding queries, and retain those near the surface. Two representative vecset-based 3D diffusion models TripoSG and Hunyuan3D-2.0 are adopted to demonstrate the acceleration. Experiments on Toys4K show that the proposed method achieves ~1.5x speedup with less quality degradation than baselines.

Critical strengths
  1. Clear motivation. The paper addresses a practical problem in 3D generation. Using surface information to guide computation is a reasonable design choice.
  2. Useful experimental analysis. The paper includes evaluations on two backbones, additional tests on thin-structure categories, and ablations on several design choices. The runtime breakdown also helps explain the cost of the proposed modules.
  3. Strong qualitative results. In Figure 4, the proposed method can preserve geometric quality both in both global shape and local details, even better than vanilla backbone in some cases.
Critical weaknesses
  1. The technical novelty appears limited. FlashVDM already uses sparse spatial queries and decoder attention to select relevant latent tokens, together with hierarchical near-surface decoding. The main new idea here is to use evolving surface information to guide token updates during denoising. This is more like a meaningful yet incremental extension. The curvature-based refinement also appears to be a relatively small modification of existing hierarchical decoding.
  2. The application impact is not yet strongly established. A 1.5× speedup is useful but not significant, especially considering the tradeoff between quality and speed. The complete framework also depends on the VecSet representation and cannot be directly applied to different representations, such as SDF or O-Voxel models, which limits its broader application value as a general 3D diffusion acceleration method. Also, the paper should demonstrate whether the gains remain obvious when combined with existing inference optimizations or fewer sampling steps.
Questions for authors

The authors may discuss more about the technical contributions beyond FlashVDM.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.