- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces MeshCarve, an artistic mesh generation framework from input point cloud. MeshCarve separately generates vertex and edge both in compact latent spaces, supported by its proposed VertexVAE and EdgeVAE. Spatial-aware compression and vertex-link encoding improve the latent compactness and thus diffusability / scalability. Experiments demonstrate competitive surface fidelity and strong connectivity generation when reference vertices are provided, compared with recent SOTA methods including meshflow, LATO.2.
Critical strengths
- A compact representation design for diffusion-based mesh generation. Although previous methods have already explored separate vertex-edge latent diffusion system or edge connectivity encoding & decoding algorithm, MeshCarve pushes it into a more compact latent space for both vertex and edge, especially for edge latents. Also, vertex anchoring, permutation-invariant neighborhood encoding, and spatially aware compression form a well-motivated approach to reducing generative sequence length.
- Both qualitative and quantitative results demonstrate clear improvements over strong baselines such as LATO.2. Detailed ablations further show the effects of different design choices.
- Improved adherence to the requested vertex budget, as shown in supplementary Table 5. This also reflects the flexibility of compact latent generation to some extent. This is a useful practical capability for applications where mesh complexity must be controlled.
Critical weaknesses
- MeshCarve requires point cloud as input, and does not demonstrate the image-to-mesh generation ability. Although point-to-mesh is already useful, I am curious whether the whole pipeline could effectively handle image input, especially the single-forward coarse anchor generation model. This coarse occupancy accuracy is generally critical to the downstream models.
- It would be better to show different generated meshes under various vertex budget for the same input, to demonstrate the effectiveness of budget control.
- The contribution of compression to generation quality is not sufficiently isolated. Appendix I reports that the released LATO.2 vertex and topology flows contain approximately 162M and 221M parameters, whereas MeshCarve uses 1.27B and 2.37B parameters. Consequently, its generation gains may reflect both the proposed representation and substantially greater model capacity. The experiments do not establish how much improvement compression provides at a fixed model size or compute budget.
Questions for authors
- What fraction of the topological defects is attributable to face recovery? Could the authors reconstruct faces from ground-truth vertices and ground-truth adjacency using the current three-cycle enumeration procedure, and report face precision/recall and non-manifoldness? How do the authors explain the substantial non-manifold edge percentages that remain with reference vertices?
- In Table 3a, the reconstruction performance of LATO.2 V-VAE is very poor. Yet the performance of end-to-end generation of LATO.2 in Table 2 is very close to MeshCarve (even better in HD). How should this be interpreted?
2 comments
Thank you for the detailed rebuttal. My major concerns are largely resolved. I will keep my positive rating.
Thank you for the positive assessments and the constructive questions. With all weights unchanged, we now sample the vertex flow from a start-noise temperature of 0.7 instead of 1.0 with 25 steps for faster and better generation results. The limitations that were pointed out are discussed in the revised abstract and main text, and we also answer each point below for your convenience.
1. Image input (W1)
Thank you for raising this. We agree that image to 3D generation is a valuable direction. At this moment MeshCarve cannot take an image as input, and we state this limitation in the revised main text. MeshCarve is conditioned on a point cloud by design and focuses more on designing the VAE that achieves better compression to save training/inference cost. Though interesting, image-to-mesh generation is beyond the scope of this submission, so we leave it to future work.
2. Several vertex budgets for one input (W2)
Thank you for the suggestion. A new figure in the revised PDF shows four Objaverse inputs, each generated at vertex budget scales of 0.5, 0.75, 1.0, 1.25 and 1.5. Over the 185 Objaverse meshes, the number of generated vertices rises monotonically with the scale, from 0.61 to 1.34 times the count at scale 1.0.
3. Compression and model capacity (W3)
Thank you for raising this point, and we ran a new experiment for it. On speed, our inference latency is directly comparable to LATO.2, 5.76 s against 4.99 s per mesh on one H200, despite the larger flow models.
We designed the VAE to have strong compression ability so that the cost of training larger flow matching models can be significantly reduced. We have studied the scaling of the edge flow model during this rebuttal period and the results are shown below. At 226M size, directly comparable to LATO.2's, we train our model with the same recipe as our reported 2B model, it mostly surpasses LATO.2's T-FLOW model on both evaluation sets and lags behind the 2B model. This suggests that 1) our VAE design does contribute to the performance uplift, and 2) scaling the flow matching model is useful and also improves the generation quality.
Both columns are measured with the edge flow given the reference vertices. The full metrics of the 226M model are in Appendix L of the revised PDF.
4. Topology error from face recovery (Q1)
We ran the 3-cycle enumeration on the reference vertex voxels at 512³ with their reference edges:
The enumeration recovers every reference face and adds 2.6 and 1.9 points of non-manifold edges. It remains an interesting direction to avoid these losses introduced by 3-cycle and we leave it for future work. With reference vertices, our non-manifold edge ratio is already lower than that of concurrent work. Flow matching models lead in surface fidelity and in handling complex shapes, but their topology, including that of current commercial models, is still not perfect. Our future work will focus on improving the topology quality.
5. LATO.2 end to end despite its V-VAE (Q2)
The vertex F1 of Table 3a requires an exact match of each vertex voxel, which LATO.2 also uses in their work. We found that their released V-VAE reconstruction performance decreases beyond the resolution of 256. Regarding their end-to-end performance, metrics such as CD and HD are more sensitive to surface fidelity and less sensitive to the exact location of individual vertices. In general, for VAEs, higher reconstruction performance is preferred.