acceptodds
Withdrawn from ICLR 2027

MeshCrafter: Structurally Robust Autoregressive Mesh Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Autoregressive mesh generation becomes increasingly challenging as mesh complexity and sequence length grow. Although recent methods improve scalability through compact representations or localized generation, geometric structure is typically imposed through discrete partitioning and fixed serialization, which discards ambiguous transitions between neighboring regions. Moreover, standard next-token training optimizes local prediction likelihood rather than the geometric and structural quality of the complete generated mesh. We present MeshCrafter, a structure-aware autoregressive framework that incorporates geometric organization throughout region modeling, serialization, and post-training. MeshCrafter first constructs latent geometric regions from pretrained geometry features and refines their organization with a triplet objective, while explicitly estimating a continuous interface-proximity field to preserve structural transitions discarded by hard assignments. Based on this representation, we introduce soft-interface-guided hierarchical mesh tokenization, where discrete region ownership constrains the traversal space and continuous interface proximity determines the traversal priority. A hierarchical coarse-to-fine spatial encoding further provides a compact representation of 3D vertices. Finally, we introduce structure-aware reinforcement post-training, which compares groups of complete mesh trajectories using geometric and latent structural feedback and directly optimizes the generation policy at the sequence level. Experiments on a 1k-mesh benchmark demonstrate consistent improvements over representative autoregressive mesh generators. MeshCrafter achieves state-of-art results compared among strongest evaluated baselines, while presenting its robustness and effectiveness.

Primary area:
generative models
Keywords:
3D Generation, Autoregressive Modeling, Mesh Generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LFoVOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

MeshCrafter is a structure-aware autoregressive mesh generation framework for complex 3D assets.

Critical strengths

None.

Critical weaknesses
  1. Poor figure quality:

Through this paper, many renderings contain noticeable gray or black speckles. In some cases, these artifacts appear on the mesh surface, while in others they extend into the surrounding background. Examples can be found in the bottom-left part of Figure 2, as well as several cases in Figure 4 (e.g., the fourth image in the first row, the first and third images in the second row, and the fourth image in the last row). It is unclear whether these artifacts come from the generated geometry or from the rendering pipeline, but either way, they significantly affect the visual quality of the comparisons.

There are also apparent visibility issues in the wireframe renderings. For example, in the illustration below "Tokenization Method" in Figure 2, wireframe edges from the back of the object seem to be rendered together with the visible edges on the front surface (In my experinence, it is a phonomenon when you rendered in the 'double side' mode), making the topology extremely hard to see. Even the teaser figure uses inconsistent rendering styles: the shoe and the small prince character are shown with background boxes, while the other examples are not. These inconsistencies make the figures look truly unpolished and unprofessional.

More importantly, these problems are particularly concerning given the topic of the paper. For artist-mesh generation, readers need to inspect the generated topology and mesh structure. Noisy or poorly configured wireframe rendering makes it much harder to assess these properties.

Figure 2 also needs substantial improvement. The overall pipeline is difficult to follow and several important components are either missing or poorly illustrated. For example, the figure should clearly show how the point cloud latent features are passed to the hourglass Transformer through cross attention. There are also basic textual errors, such as "hiearchical" instead of "hierarchical". More importantly, the figure does not clearly explain what the proposed tokenization method actually does. The difference between the output of the initial mesh generation pipeline and the final output after GSPO is also difficult to identify, and the GSPO section itself does not communicate the training procedure clearly. (I truly recommend authors to learn how to draw the method picture from Mesh-Pro, which shares the same topic but draws much better and clearer than this paper.)

The number and diversity of qualitative results are insufficient. Several examples are reused across different figures. For instance, the scarecrow appears repeatedly in the teaser, method overview and qualitative comparison for 3 times, while the shoe example is also shown multiple times. Considering that the authors report training on approximately 1 million meshes, I would expect a much broader and more diverse set of qualitative results to demonstrate the model's capabilities and generalization. Repeatedly showing the same few examples makes it difficult to judge how well the method performs across different categories and mesh complexities. (BTW, the cases in this paper are not complicated, at least it would be better to annotate the face count of cases.)

  1. Poor writing quality:

The paper suffers from serious writing and presentation issues. There are many unconventional expressions through the whole paper, some of which read like AI-generated academic language rather than terminology commonly used in this field. I understand that the authors have acknowledged the use of AI tools for writing assistance in AI Use Statement. However, it is still the authors' responsibility to ensure that the terminology is appropriate and familiar to the intended research community.

For example, the term "mesh trajectories" is not a common expression in the 3D generation. Similarly, "mesh generators" has a well-established meaning in traditional geometry processing, where it often refers to meshing algorithms that discretize continuous surface into meshes or finite elements. In modern learning-based 3D generation, terms such as "mesh generation models" or "3D generative models" would be more natural. These are just two examples of a much broader problem with the terminology used throughout the paper.

The abstract and introduction also introduce too many self-defined terms and symbols before properly explaining the problems they are intended to solve (e.g., Lines 70–78). Some statements are repeated almost verbatim, including the claim that "Standard next-token training optimizes local prediction likelihood rather than the geometric and structural quality of the complete generated mesh." Overall, the introduction feels repetitive and poorly organized, making it difficult to identify the actual research motivation and the specific limitations of previous methods.

I believe the authors should spend much more effort explaining the concrete challenges faced by existing autoregressive methods when generating long mesh sequences. For example, the paper could discuss how a fixed sliding-window size limits the available context as the sequence grows, potentially preventing later predictions from accessing information about earlier parts of the mesh. This can lead to accumulated errors and inconsistencies (e.g. topological asymmetry is also one of this problems) between distant but geometrically connected regions. Such explanations, ideally supported by illustrative figures, would be much more informative than repeatedly introducing abstract terms such as "structural transitions" or "local prediction likelihood" without explaining the underlying mechanisms.

  1. Poor mathematical presentation and missing references:

The mathematical presentation also needs considerable improvement. Most equations are introduced without clear denotations and the connections between successive equations are often difficult to follow. This makes the proposed method extremely hard to understand and reproduce.

For example, the Equation (16), where the authors introduce the structure-aware reward function R3DR_{3D} without providing its explicit mathematical definition and this reward is central to the proposed reinforcement post-training strategy. I am also concerned about the novelty and empirical justification of the proposed GSPO-based post-training strategy. The combination of supervised pretraining and reinforcement post-training has already been explored in several recent works on autoregressive mesh generation. DeepMesh adopts DPO, Mesh-Pro introduces ARPO, Mesh-RFT proposes Masked DPO, and QuadGPT explores topology-aware tDPO. However, the related work discussion does not adequately cover these developments, particularly Mesh-Pro, Mesh-RFT, and QuadGPT.

More importantly, the paper does not clearly explain what makes the proposed GSPO strategy better suited to mesh generation than these existing alternatives. Simply applying GSPO with and without a structure-aware reward (GSPO) does not establish a feasible reinforcement learning method and meanwhile it can not establish any advantage over DPO, ARPO or other existing post-training strategies. The authors should explain the specific limitations of existing approaches and demonstrate how their method addresses them. More broadly, this paper contains many heuristic design choices with too few ablation studies to justify them.

Questions for authors

The authors claimed that "Our test set contains 1k meshes, including 400 meshes generated by TRELLIS from held-out prompts and the remaining 600 meshes sampled from the aforementioned datasets. The test meshes contain an average of approximately 5.1K faces."

However, TRELLIS is a diffusion-based 3D generation method that typically produces meshes with hundreds of thousands faces even after some operations like mesh simplification and decimation. In my experience with TRELLIS, it is impossible to receive the reported average of only 5.1K faces. Therefore, I further examined the face-count distribution in Figure 7 and estimated the minimum possible average face count by taking the lower bound of each interval:

0.07(1000)+0.17(2000)+0.25(4000)+0.18(6000)+0.13(8000) +0.08(12000)+0.06(16000)+0.04(24000)+0.02(32000) =7050>5100.$$ Even under this conservative estimate, the minimum average face count is more than 7K, substantially higher than the reported 5.1K. It therefore appears mathematically inconsistent, assuming Figure 7 represents the actual test-set distribution. I hope authors can explain the above phenomenon which are really infeasible for me.
Reviewer_88fbOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MeshCrafter, a structure-aware autoregressive framework for generating high-quality 3D meshes from point-cloud conditions.

  1. MeshCrafter combines discrete geometric regions with a continuous interface-proximity representation and uses this information to guide mesh serialization. It introduces a hierarchical coarse-to-fine tokenization scheme to shorten autoregressive sequences while preserving local face adjacency.

  2. After pre-training, the model is refined through GSPO-based reinforcement post-training. Experiments show improvements in geometric fidelity, surface quality, watertightness, and user preference over several autoregressive mesh-generation baselines.

Critical strengths
  1. The paper is clearly written, well organized, and easy to follow.

  2. By combining supervised pre-training with reinforcement post-training, the paper presents a relatively comprehensive framework for autoregressive mesh generation.

Critical weaknesses
  1. Autoregressive modeling constitutes an established line of research for mesh generation, and diffusion-based methods represent another dominant paradigm. The paper would be strengthened by a more explicit discussion of the unique advantages of autoregressive formulations relative to diffusion-based alternatives in the mesh generation setting.

  2. The intuition that hard region partitioning discards transitional information along region boundaries and thereby induces topological defects is plausible. Even so, the motivation as currently presented lacks sufficient persuasiveness and would benefit from supporting empirical evidence and more in-depth analysis.

  3. The adoption of GSPO as a refinement to GRPO appears incremental, with limited methodological novelty. Additionally, online reinforcement post-training for autoregressive mesh generation incurs considerable rollout and inference overhead. The authors should clarify how these computational expenses are managed.

  4. Figures 5 and 7 seem to be rasterized instead of vector graphics, leading to visible blurring. The authors should replace these plots with vector-format figures to enhance visual quality and readability.

Questions for authors

See weakness

Reviewer_Do9TOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MeshCrafter is a point-cloud-conditioned autoregressive framework for generating structurally coherent triangle meshes. It combines latent geometric region structuring and a continuous interface-proximity signal with region-constrained traversal and coarse-to-fine coordinate tokenization. The model is subsequently refined through group-sequence reinforcement post-training using geometric and latent structural feedback. Experiments on a mesh benchmark report improvements in geometric fidelity, watertightness, and sequence compactness over representative baselines, supported by a user study and interface-robustness analyses.

Critical strengths

The paper connects region representation, interface-guided serialization, and sequence-level post-training around a concrete problem: preserving mesh structure during long autoregressive generation. This is a useful design direction, and the qualitative examples illustrate its intended benefit for fine structures and region transitions.

Critical weaknesses
  • Region-constrained traversal can fail on disconnected clusters. The traversal only visits adjacent faces within the same region, while k-means clustering and triplet refinement do not guarantee spatially connected regions. Consequently, the traversal frontier can become empty before all faces are visited, yet Algorithm 1 provides no restart mechanism. The soft-interface signal only prioritizes accessible faces and therefore cannot resolve this failure.
  • The interface-proximity estimator does not explicitly measure geometric proximity to region boundaries. Equation (5) assigns high scores whenever intra-region and inter-region feature distances are similar, which can also occur in poorly separated or noisy feature regions. Consequently, the signal may prioritize geometrically irrelevant faces during traversal, since feature ambiguity alone does not establish proximity to a structural interface.
  • The experiments do not isolate or fully specify the claimed structure-aware improvements. The reward remains an abstract function in Equation (16), and the promised scoring terms, weights, and normalizations are not provided in the supplied appendix, making the post-training result difficult to reproduce.
Questions for authors

-Please give the complete inverse tokenizer that reconstructs faces and stitches regions from the deduplicated vertex stream, together with an encode–decode round-trip example. How are empty traversal frontiers handled when feature-based clusters contain disconnected components? -What is the exact reward, including its geometric and latent terms, weights, normalization, and invalid-mesh treatment?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.