acceptodds
Under review as a conference paper at ICLR 2027

SeedFACE: Rethinking Autoregressive Mesh Initialization through First-Face Modeling

Abstract

Autoregressive models have shown strong potential for native 3D mesh generation, but their performance degrades on broad and diverse shape distributions. We show that this difficulty is highly non-uniform across the generation process. In particular, autoregressive mesh generation suffers from an initialization bottleneck: the first face must establish geometric context from an empty history, while later faces are strongly constrained by previously generated geometry. Prefix experiments show that providing only the first ground-truth face substantially improves generation, and further analysis reveals that the first face has the highest prediction uncertainty and becomes increasingly difficult to model as shape diversity grows. Motivated by this asymmetry, we propose SeedFACE, which separates geometric initialization from autoregressive continuation. We factor mesh generation into a seed distribution and a conditional continuation model. Rather than generating the seed face token by token from an empty context, SeedFACE represents it in a regularized latent space and learns its distribution with a lightweight latent generative model. The sampled seed is then decoded by the shared mesh decoder and serves as the geometric context for standard autoregressive generation. This preserves autoregression where geometric context is informative while avoiding token-by-token prediction in the least constrained regime. Experimental results on Objaverse demonstrate that SeedFACE substantially improves generation quality and diversity over the state-of-the-art autoregressive baselines.

Primary area:
generative models
Keywords:
3d mesh generation, autoregressive modeling, diffusion model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_k4h7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the initialization bottleneck in autoregressive 3D mesh generation. The authors observe that providing a single ground-truth face significantly improves subsequent mesh generation, suggesting that predicting the first face from an empty context is considerably more difficult than generating later faces. To address this issue, the authors propose SeedFACE, which separates first-face generation from autoregressive mesh continuation. Specifically, a VAE learns a compact latent representation of the first face, while a latent diffusion model learns its distribution. The generated first face then serves as the geometric context for the original autoregressive model. Experiments on Objaverse and ShapeNet demonstrate improvements in distribution-level metrics, including Coverage (COV), Minimum Matching Distance (MMD), and 1-Nearest-Neighbor Accuracy (1-NNA). The authors also provide analyses of first-face prediction uncertainty, distribution diversity, and modeling gaps, together with ablations on different seed-generation strategies.

Critical strengths

S1. An interesting observation about autoregressive mesh initialization.

The paper identifies an interesting phenomenon: providing only the first ground-truth face substantially improves subsequent autoregressive generation. The authors support this observation through prefix experiments, cross-entropy analysis, and investigations of first-face distribution complexity. This provides a useful perspective on where modeling difficulties arise during sequential mesh generation.

S2. A simple and effective approach with improvements on Objaverse.

The proposed framework separates geometric initialization from autoregressive continuation while largely preserving the existing mesh generation architecture. The improvements over FACE on Objaverse are substantial, particularly in COV, MMD, and 1-NNA. The experiments also demonstrate that explicit seed modeling can be implemented using different generative approaches.

Critical weaknesses

W1. The first-face bottleneck may depend on mesh serialization and the absence of global conditioning.

The paper identifies first-face prediction as a major bottleneck in autoregressive mesh generation. However, the first face is not an intrinsic property of a mesh; it is determined by the adopted face ordering strategy.

The experiments primarily follow the face-based serialization framework of FACE, which uses deterministic spatial ordering. Different traversal or ordering strategies may result in substantially different first-face distributions and prediction difficulties. Without evaluating alternative ordering strategies, it remains unclear whether the observed bottleneck reflects a general limitation of autoregressive mesh generation or a property of the particular serialization method.

Furthermore, the experiments focus on category-agnostic and category-conditioned generation without instance-specific geometric guidance. In practical mesh generation tasks, autoregressive models are often conditioned on images, point clouds, or other global geometric representations. Such conditioning could significantly reduce uncertainty during first-face prediction. Therefore, the generality of the proposed first-face bottleneck remains insufficiently established.

W2. The evaluation does not fully establish the necessity of the proposed design or its advantages in mesh quality.

The paper primarily evaluates generation performance using COV, MMD, and 1-NNA computed from sampled point clouds. While these metrics are appropriate for measuring distributional similarity, they do not directly evaluate geometric validity, mesh connectivity, topology, or surface quality.

A generated mesh can be geometrically plausible while differing from the empirical dataset distribution. Conversely, improved distribution matching does not necessarily imply better mesh structure. Additional evaluations of geometric validity and topology would strengthen the claim that SeedFACE improves mesh generation quality.

Moreover, Table 4 shows that an isolated autoregressive first-face generator achieves performance comparable to SeedFACE, including a slightly better MMD (10.43 versus 10.46). This suggests that explicitly separating first-face modeling from subsequent generation may account for much of the improvement, while the additional benefits of the proposed VAE and latent diffusion architecture remain unclear. Given the additional architectural and training complexity, a more systematic comparison of model capacity, computational cost, and generation quality would be valuable.

Questions for authors

Q1. Sensitivity to Face Ordering

How sensitive is the proposed first-face bottleneck to different mesh serialization strategies? Since the first face depends on the adopted ordering, it would be valuable to evaluate whether the reported uncertainty gap and the benefits of SeedFACE persist under alternative ordering strategies, such as different spatial sorting orders or graph-based traversal methods, with models retrained under matched settings.

Q2. Generalization to Conditional Mesh Generation

Does the first-face bottleneck remain significant when the autoregressive mesh model is conditioned on global geometric information, such as point clouds or image features? Such conditioning may already provide sufficient information for first-face prediction. Evaluating SeedFACE in these settings would help establish whether the proposed method addresses a general challenge in autoregressive mesh generation or a difficulty primarily associated with unconditional generation.

Q3. Necessity of Latent Diffusion

Table 4 shows that the isolated autoregressive seed generator achieves results comparable to SeedFACE. Could the authors provide a more detailed comparison of these approaches, including parameter counts, training costs, inference latency, and generation quality? This would help clarify the specific advantages of introducing a VAE and latent diffusion model rather than using a simpler dedicated first-face generator.

Q4. Evaluation of Geometric Validity

Could the authors provide additional evaluations that directly assess the geometric quality of generated meshes, beyond distribution-level metrics computed from point clouds? For example, measurements of disconnected components, degenerate faces, non-manifold geometry, or other mesh validity properties would help determine whether the improvements in COV, MMD, and 1-NNA translate into meaningful improvements in mesh quality.

3 comments

We thank the reviewer for the positive assessment of our technical insights, method effectiveness, and experimental analysis. We address the concerns below.

Q1: Sensitivity to Face Ordering.

Following your suggestion, we retrained FACE and SeedFACE using two additional graph-based traversal strategies, breadth-first search (BFS) and depth-first search (DFS), under matched experimental settings. The results are shown below.

MethodCOV ↑MMD ↓1-NNAV/F
FACE44.3418.0069.3460.76 / 99.27
SeedFACE46.6815.2158.1176.19 / 116.70
FACE (BFS)42.1917.2170.0273.26 / 92.33
SeedFACE (BFS)50.7815.5256.3584.95 / 106.72
FACE (DFS)43.7517.1969.5389.03 / 132.87
SeedFACE (DFS)47.6615.4358.40102.07 / 149.93

SeedFACE consistently improves all three distribution-level metrics over FACE under both traversal strategies. In particular, BFS improves COV from 42.19 to 50.78 and 1-NNA from 70.02 to 56.35, achieving even larger gains than the default zyx ordering. DFS also maintains consistent improvements across all metrics. These results demonstrate that the effectiveness of SeedFACE is not tied to a particular serialization strategy. Regardless of the traversal order, the first face must still be predicted without preceding geometric context, supporting the generality of the identified initialization bottleneck.

Q2: Generalization to Conditional Mesh Generation.

We further conducted image-conditioned generation experiments on Objaverse using renderings of untextured gray meshes as conditions, together with a first-face perturbation study.

MethodCOV ↑MMD ↓1-NNAV/F
FACE-cond (w/o seed generator)50.9814.4153.8167.67 / 108.22
SeedFACE-cond (w/ seed generator)54.4320.4351.9075.44 / 120.42

As the results shown, explicit seed-face modeling further improves COV from 50.98 to 54.43 and brings 1-NNA closer to the ideal 50% (53.81->51.90). Although MMD increases, this may reflect a trade-off between broader distribution coverage and nearest-neighbor matching fidelity. These results suggest that image conditioning provides useful geometric context but does not eliminate the benefits of explicit first-face modeling.

More importantly, we also present a visualization comparison in Appendix Figure 10, showing that slightly perturbing the first face while keeping the image condition unchanged can substantially alter subsequent generation, resulting in incomplete geometry or deviations from the intended object structure. This further demonstrates that first-face initialization remains influential even with global visual guidance, rather than being a challenge exclusive to unconditional generation.

Q3: Necessity of Latent Diffusion.

We would like to clarify that the isolated AR experiment is intended to evaluate alternative first-face modeling strategies, rather than establish the superiority of diffusion over autoregression. The comparable performance of different seed generators supports our central insight that explicitly separating initialization from autoregressive continuation effectively addresses the first-face bottleneck.

Our contribution extends beyond introducing an additional seed generator. We identify and systematically analyze the initialization bottleneck, and propose a latent-space formulation that models the seed distribution while retaining the shared autoregressive mesh-decoding pathway. Compared with directly modeling discrete first-face tokens, our VAE provides a compact, regularized latent representation for diffusion-based generation.

Regarding computational cost, the isolated AR and latent-diffusion seed generators are matched in model size, providing a comparison with comparable model capacity. The latent model operates only on a compact seed representation, while subsequent mesh generation retains the original autoregressive process. We will clarify these architectural and computational considerations in the revision.

Q4: Evaluation of Geometric Validity.

We appreciate the suggestion to evaluate mesh structure beyond point-cloud-based distribution metrics. Accordingly, we additionally report the average numbers of vertices and faces (V/F) over 2,048 generated samples per method on Objaverse, providing complementary information about mesh complexity and vertex sharing. The results are shown below.

MetricMeshFlowPolyGenMeshGPTPivotMeshFACESeedFACE
COV ↑51.8640.6626.4138.6740.4352.88
MMD ↓12.3311.3815.8012.2814.4010.46
1-NNA54.5461.1182.1662.7976.1053.56
V/F327.82 / 109.3970.89 / 152.9625.86 / 47.9732.44 / 59.2645.04 / 78.2858.45 / 109.16

MeshFlow generates substantially more vertices than faces (327.82/109.39), suggesting limited vertex sharing and fragmented connectivity. In contrast, SeedFACE achieves a more balanced V/F ratio (58.45/109.16), comparable to PolyGen, while generating more faces than MeshGPT, PivotMesh, and a comparable number to MeshFlow. These results indicate that SeedFACE maintains reasonable vertex sharing while generating richer mesh structures, consistent with the qualitative comparisons.

We also note that the training meshes are not necessarily watertight or manifold, making strict topological validity criteria less representative of the target data distribution. We therefore use V/F as a complementary structural statistic rather than treating manifoldness or watertightness as universal requirements for generation quality.

Thank you for the detailed rebuttal and additional experiments.

The BFS/DFS experiments adequately address my concern regarding sensitivity to face ordering. I also appreciate the new image-conditioned generation experiments.

However, some concerns remain. In the conditional setting, MMD deteriorates substantially despite improvements in COV and 1-NNA, leaving the benefits for conditional generation unclear. Additionally, the reported V/F statistics do not directly establish mesh validity or connectivity quality.

I acknowledge the additional evidence supporting the importance of first-face initialization, but I do not believe the remaining concerns regarding evaluation and practical effectiveness have been fully resolved. Therefore, I will maintain my current rating.

Reviewer_gbkJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the importance of the first face in autoregressive mesh generation. It shows that providing a ground-truth first face substantially improves subsequent generation. Based on this observation, SeedFACE generates the first face using a VAE and latent diffusion model, then continues with standard autoregressive decoding. The method achieves clear improvements on Objaverse.

Critical strengths
  • The prefix experiments show that first-face initialization has a large impact on generation quality, providing a strong motivation for the work.
  • Compared with baselines, SeedFACE substantially improves coverage, MMD, and 1-NNA on public datasets.
Critical weaknesses
  • The paper shows that first-face initialization matters, but the additional benefit of diffusion over autoregressive first-face modeling remains unclear. In Table 4, an isolated AR seed model performs almost as well as SeedFACE. This suggests that the gains may come mainly from dedicated first-face training. Comparisons with first-face loss reweighting and a compute-matched AR seed model would help clarify the reason.
  • The experiments are restricted to meshes with fewer than 500 faces on Objaverse and 800 faces on ShapeNet. It remains unclear whether the improvements hold for more detailed meshes with much longer generation sequences. Evaluation on meshes with higher face counts would strengthen the paper’s scalability claims.
Questions for authors

/

1 comment

We thank the reviewer for the positive assessment of our motivation, technical insights, and experimental results. We address the concerns below.

W1: Necessity of Separate First-Face Modeling

We introduce FACE+aux, which increases the loss weight of the nine first-face tokens during autoregressive training. Under the same 5k-mesh Objaverse setting, FACE+aux yields only marginal improvements over FACE, whereas SeedFACE achieves substantially better results. These results demonstrate that simply increasing first-face supervision is insufficient to address the initialization bottleneck.

MethodCOV↑MMD↓1-NNAV/F
FACE44.3418.0069.3460.76 / 99.27
FACE+aux45.5117.8566.7061.14 / 98.53
SeedFACE46.6815.2158.1176.19 / 116.70

Comparison with Isolated AR. We would like to clarify that the purpose of Table~4 is to investigate different first-face modeling strategies, rather than establish the superiority of diffusion over autoregression. The comparable performance of isolated AR further supports the effectiveness of explicitly separating initialization from autoregressive continuation, which is central to our contribution. Our technical contribution extends beyond introducing a seed generator: we identify and systematically analyze the initialization bottleneck and propose a dedicated latent-space modeling framework to address it. We adopt latent diffusion for its compact, regularized representation and its compatibility with the shared mesh decoder. For computational fairness, the isolated AR and latent-diffusion seed generators are already matched in model size, ensuring comparable model capacity.

Together, these comparisons distinguish the benefit of explicit seed-distribution modeling from simply increasing first-face supervision, while demonstrating that our initialization-centered formulation is effective across different generative mechanisms.

W2: Generalization to Higher-Face-Count Meshes.

Our evaluation follows the established settings of existing mesh generators, including MeshGPT, PivotMesh, and MeshFlow, whose publicly available implementations primarily target relatively low-poly meshes. Our focus is to investigate and address the initialization bottleneck within this established setting, rather than to extend the maximum mesh resolution.

We further believe that the initialization bottleneck is largely independent of the face count. Since an autoregressive model generates each face conditioned on the preceding ones, a higher-face-count mesh still goes through the same process: the first face is generated without any geometric context, and the subsequent faces are generated by continuation. We therefore expect the behavior to be similar to the low-poly setting we study. Moreover, our experiments show that this challenge becomes more pronounced as shape diversity increases, which suggests it is unlikely to diminish for more complex meshes. We will clarify the scope of our experimental evaluation and note the face-count limitation in the revised manuscript.

Reviewer_kVncOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates performance bottlenecks in autoregressive 3D mesh generation, identifying the prediction of the initial face as both critical and challenging. It demonstrates that providing the ground-truth (GT) initial face or modeling it separately significantly enhances the quality of the subsequent mesh generation. Building on this insight, the paper proposes a framework that models the initial face independently using a VAE+Diffusion approach, followed by standard autoregressive generation methods (such as FACE or PivotMesh) to complete the mesh. Experiments conducted on ShapeNet and Objaverse confirm a significant improvement in performance.

Critical strengths
  1. The identification of an initialization bottleneck in autoregressive 3D mesh generation is informative and valuable. This finding shares a similar spirit with observations in LLM, such as The Pitfalls of Next-Token Prediction (2024) and Beyond the 80/20 Rule (2025), which suggest that a small subset of tokens can play a more important role in autoregressive generation. Through detailed empirical analyses, the authors establish the importance of the first face in mesh generation, providing useful new insight.
  2. The authors propose a practical solution to this problem. Modeling the first face separately substantially improves overall autoregressive generation quality, with particularly clear gains over FACE.
  3. The paper includes extensive ablation studies. The fact that multiple first-face modeling plugins improve performance strengthens the evidence for the underlying observation. The investigation into the number of seed faces is also valuable.
Critical weaknesses
  1. Lack of naive baselines for the first-face issue. This is one of my major concerns. The difficulty of learning the first-face distribution may arise partly from insufficient supervision weight, optimization resources, or model capacity allocated to this step during full-sequence training. A straightforward baseline would be to increase the loss weight of the first face in the original autoregressive model or increase its frequency in training, encouraging the model to focus more on initialization. If such modifications substantially alleviate the difficulty of modeling the first-face distribution, the need for a separate generative model would be less compelling.
  2. In Table 4, the standalone AR plugin performs very similarly to the proposed VAE-plus-diffusion initialization model. This somewhat weakens the contribution of the proposed latent-space formulation for first-face modeling.
  3. Further discussion with diffusion-based mesh generation methods would be helpful. If a DiT can better model the first-face distribution, a natural question is why the remaining faces should be generated autoregressively, rather than modeling all faces jointly with a DiT, as explored in recent diffusion-based approaches such as NEXUS. Addressing this question would help clarify the practical value of the proposed hybrid design.
  4. 1The meshes shown on ShapeNet and Objaverse are relatively simple, and the experiments are restricted to shapes with fewer than 800 / 500 faces, respectively. This limits the scope of the conclusions about the initialization bottleneck. It remains unclear whether improving the first face alone would yield substantial gains for more complex meshes with more parts and challenging surface structures, or whether other tokens would become similarly difficult and consequential as in LLM. Moreover, the qualitative comparisons in Figures 4 and 5 do not show a clear visual advantage.
Questions for authors

As discussed above, it would be appreciated if the authors could provides more evidence of the necessity of separate initialization rather than naively increasing the focus of AR model on first faces. Moreover, discussions about AR vs Diffusion for mesh generation are also helpful.

2 comments

We thank the reviewer for the positive assessment of our motivation, technical insights, and experimental findings. Below, we respond to the main concerns.

W1: Necessity of Separate First-Face Modeling. Following your suggestion, we introduce FACE+aux, which increases the loss weight of the nine first-face tokens during autoregressive training. The results are summarized below.

MethodCOV↑MMD↓1-NNAV/F
FACE44.3418.0069.3460.76 / 99.27
FACE+aux45.5117.8566.7061.14 / 98.53
SeedFACE46.6815.2158.1176.19 / 116.70

FACE+aux yields only marginal improvements over FACE, whereas SeedFACE achieves substantially better results under the same experimental setting. Notably, although the stronger first-face supervision broadens the distribution of starting faces (higher COV), MMD and 1-NNA remain close to the baseline. Since the first face and the subsequent faces still share a single objective, over-fitting the first face degrades the subsequent autoregressive generation, and we observe more fragmented and discontinuous meshes. These findings further support our motivation that the initialization bottleneck cannot be adequately addressed by simply increasing first-face supervision. Instead, explicitly modeling the seed-face distribution provides a more effective solution than reweighting the original autoregressive objective.

W2: Standalone AR vs. Latent Diffusion in first-face modeling. We thank the reviewer for this observation, and we believe it is in fact consistent with our analysis. As shown in W1, as long as the first face is trained jointly with the autoregressive continuation, the shared objective prevents the model from fitting the first-face distribution well, so the improvement is limited. A standalone AR removes this constraint: a model dedicated to only nine tokens can fit the first faces almost by brute-force memorization, which is in effect close to supplying ground-truth first faces from the training set. Its comparable performance therefore confirms that the bottleneck lies in first-face modeling capacity, rather than weakening our contribution.

Importantly, our contribution goes beyond introducing an additional initialization model. We identify and systematically analyze the first-face modeling bottleneck, and propose an initialization-centered framework that addresses this challenge while preserving autoregressive continuation. Within this framework, we adopt latent diffusion for its compact, regularized representation and efficient modeling of the seed distribution, rather than claiming that diffusion is the only effective solution.

W3: Autoregressive vs. Diffusion-Based Mesh Generation. We would like to clarify that our work focuses on addressing the initialization bottleneck in autoregressive mesh generation, rather than establishing the superiority of autoregression over diffusion-based approaches. Our analysis reveals that the first face presents a distinct modeling challenge due to the absence of preceding geometric context, whereas subsequent faces benefit from increasingly constrained autoregressive continuation. Accordingly, we adopt diffusion specifically to model the first-face distribution, rather than suggesting that diffusion is inherently more suitable for generating the entire mesh.

Full-mesh diffusion methods such as NEXUS represent an alternative generation paradigm and are complementary to our investigation. The effectiveness of diffusion for first-face initialization does not imply that replacing the entire autoregressive generation process with diffusion would be preferable. Instead, our contribution is to identify and address a specific limitation of existing autoregressive mesh generators while preserving their effective continuation mechanism. We have added a discussion of the relationship between first-face modeling and diffusion-based mesh generation methods to the supplementary material.

W4: Mesh Complexity and Qualitative Improvements. We appreciate the reviewer's suggestion. The current evaluation follows the established settings of existing mesh generators, including MeshGPT, PivotMesh, and MeshFlow, whose publicly available implementations and evaluation protocols primarily target relatively low-poly meshes. More importantly, we believe the initialization bottleneck we study is largely independent of mesh complexity. Since an autoregressive model generates each face conditioned on the preceding ones, a mesh with more faces or more complex geometry still goes through the same process: the first face is generated without any geometric context, and the subsequent faces are generated by continuation. We therefore expect the behavior to be similar to the low-poly setting we study and our findings to carry over to more complex meshes, although verifying this empirically at larger scale is left to future work.

Regarding qualitative results, the main advantage of SeedFACE lies in generating a broader diversity of geometric structures rather than improving fine-grained geometric details. As illustrated in Figs.4 and5, SeedFACE produces more varied object configurations, while existing autoregressive baselines tend to favor simpler shapes. This observation is further supported by the improvements in COV and 1-NNA, particularly on Objaverse. We will clarify this distinction in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.