acceptodds
Under review as a conference paper at ICLR 2027

Topology Matters: Evaluating Topological Fidelity in CAD Generation

Abstract

Geometric similarity and execution validity do not guarantee structural correctness in Computer-Aided Design (CAD): small geometric perturbations can change a solid's connected components, tunnels, or cavities. We find that such changes can arise during representation preprocessing alone: among 7,590 DeepCAD references reconstructed from both original CAD programs and quantized vectors, 762 (10.0%) have different global topology. Motivated by this gap, we study topology fidelity across five neural CAD generators, using the Betti signature of each final solid as a diagnostic. Under standard generation, exact signature fidelity ranges from 78.2% to 88.9%, and error directions differ across generation paradigms: text-conditioned models tend to omit existing structures, while the evaluated reconstruction-based models show more balanced or over-realized errors. Fidelity is also substantially lower on construction-novel patterns, with a gap persisting after standardization over measured topological complexity and construction length. Finally, even when a model reproduces the baseline topology, it rarely realizes explicit topology-changing instructions, with strict success rates of 0–1.7% across the evaluated text-conditioned models. These findings show that aggregate topology fidelity alone is insufficient to characterize CAD generators: models can differ substantially in their failure modes, sensitivity to construction patterns, and ability to modify existing topology.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer-Aided Design, CAD Generation, Topological Fidelity, Model Evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_GeKrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Small changes in CAD construction sequence parameters can cause topological differences in the final model. When these changes cause holes not to pierce the intended faces, shells to become disjoint or the formation of internal voids, these errors are “deal breaker” for the usability of the geometry for downstream processes.

Topology Matters studies these deal breaking problems, comparing results from 5 well known CAD generation models. They discover that 10% of standard dataset used for academic CAD generation experiments (DeepCAD) cannot be rebuilt with the original topology, when parameters are quantized to 8 bits of precision (two or four times the resolution used in many academic papers).

Results are additionally reported stratified by a coarse complexity measure and construction sequence novelty. Finally prompts the effectiveness of prompts which request a change to the genus of the solid are investigated.

Critical strengths

It’s very interesting to see a paper which focuses on better metrics for existing works rather than just promoting a new architecture. The ML CAD community doesn’t do enough of this kind of thing.

Changes to the betti numbers of a solid indicate serious problems in the generation. Other topological comparisons like face and edge counts and Weisfeiler Lehman graph hash are too strict.

I found the section which compared the fidelity metric between novel and covered sequences interesting.

Critical weaknesses

The biggest weakness of the paper is the treatment of non-watertight generation results. The DeepCAD network is known to be dreadful for generating non-watertight geometry. . Admittedly auto-encoding is better than generation, but the authors still find more than 10% of generations are invalid. The way these invalid generations are combined with the other metrics completely changes the rankings, with DeepCAD scoring best on the Sig Fidelity metric and third best when invalidity is taken into account. I believe what is happening here is that the invalidity effectively filters the generated data distribution and reduces the number of complex bodies. This modified distribution is then much easier to achieve a high Sig Fidelity score on.

One way to compare across all methods is to consider only the set of examples for which all methods create watertight solids. This might restrict the number of complex solids too much, but it has the advantage that the remaining distribution is within the generation capability of all models.

184> We assign a topological signature only to watertight welded meshes; invalid or non-watertight samples are excluded from topology-based comparisons

I don’t feel this stressed how dependent the result are on how the non-watertight bodies are taken into account.

Many papers report a “valid ratio” which uses the Open Cascade function BRepCheck_Analyzer.IsValid(). This is rather stricter than watertightness of the output mesh. From the CAD perspective, if your solid doesn’t meet this bar then the data cannot be used. Looks like the authors didn’t use this function. The betti number results would be more convincing to me if used on generations which already meet the BRepCheck_Analyzer.IsValid() bar.

DeepCAD and SkexGen use latent vectors or codebook codes at the auto-encoder bottleneck. The other methods use text. Text to cad is much harder task. Appendix G makes this clear. It would be much better if this was in the main paper. It’s a detail which is critical to understanding the results in Section 4.1. If there isn’t space then make this clearer in Evaluated Models (line 162).

The authors use a complexity score based on betti numbers rather than face count. I think face count is a better measure of the perceived complexity of a B-Rep body.

The main goals of this paper are to convince the reader that metrics based on betti number are valuable. In my experience, the original DeepCAD network was considered to be a rather easy benchmark to beat, but these metrics imply it’s actually quite competitive. Given this disconnect between my personal experience and my “vibe checks” based on examining images of 1000s of DeepCAD generations, I am a little concerned that the new metric isn’t working as expected. I remain convinced that disjoint shells and incorrect genus are measuring real problems which real designers care about. Perhaps the problem is that all these generative algorithms are not product ready. It would be interesting to see if Astra hits this out of the park.

Questions for authors

Why evaluate the betti numbers from the mesh? You can extract the number of shells and lumps from the OCC B-Rep. Then use the Euler formular to find the genus. See

Partial entity structure: a compact non-manifold boundary representation based on partial topological entities, Sang-Hun Lee and Kunwoo Lee

for details of the extended Euler formula

3 comments
AuthorsOpenReview ↗

Official Comment by Authors

We sincerely thank you for recognizing the importance of metric-focused research in the CAD generation community. We would like to address your questions individually as follows.


Q1: Why evaluate the betti numbers from the mesh?

A1: We emphasize that our goal is to evaluate global topological fidelity, rather than to advocate for a particular representation or extraction algorithm. Betti numbers characterize intrinsic topological properties and can be computed using either mesh-based or B-Rep-based representations. Thus, our mesh-based pipeline is an implementation choice.

Following your suggestion, we implemented an independent B-Rep-based extractor using OCC solid/lump information, shell classification, and the extended Euler–Poincaré relation. Across all 50,751 watertight-valid samples, the two approaches agree on 50,574 (99.65%).

This result demonstrates strong agreement between the two extraction approaches. We therefore retain the mesh-based pipeline, and use the B-Rep-based results as an independent cross-validation of our topological measurements.


W2: The generation paradigms of the evaluated models are not sufficiently clarified

A2: We sincerely appreciate the insightful suggestion and will add the corresponding explanation to the manuscript.


W3: Could you evaluate Betti fidelity using only generations that pass OpenCascade's BRepCheck_Analyzer.IsValid() check?

A3: Thanks for your suggestion, and we applied BRepCheck_Analyzer.IsValid() as a stricter validity criterion and reevaluated the results. We report the individual coverage rates(valid rate) under the combined generation-success, watertightness, and OCC-validity criteria in the table below. To ensure a fair comparison, we evaluated all five models on their common valid subset, consisting of 5,106 out of 7,925 samples (64.43%). The results are as follows:

ModelIndividual Coverage(valid rate)Signature Fidelity on common valid subset
DeepCAD89.7%93.09%
SkexGen81.2%88.27%
Text2CAD84.5%86.47%
CAD-Coder93.9%91.03%
Cadrille-SFT89.5%85.47%

These results show that DeepCAD retains the highest Signature Fidelity even under the stricter OCC validity criterion and on an identical evaluation subset. We will include these additional results in the revised manuscript.

Reviewer_GeKrOpenReview ↗

Thank you for the extra results

Thank you for the extra results. These are very interesting.

Evaluating the betti numbers from the mesh Thank you. Yes. This makes sense. Good to see the lump count is in agreement with your mesh based values.

Signature Fidelity on common valid subset

Thank you for the extra numbers. These clearly show that DeepCAD still does very well with the Signature Fidelity metric. Your experiment proves that the gains don't come from filtering more complex examples from the distribution.

I'm still really confused by the results though. My experience is that DeepCAD is a rather easy baseline to beat. Probably the mismatch is because most papers show DeepCAD generation, while your results are for auto-encoding. That would make the comparison with SkexGen sensible.

To compare correctly with the text based intermediate representations one would need to do something like converting to point cloud, creating the latent from the point cloud and then decoding. That's probably too much to ask for, although it might be an interesting experiment for you to try. I would expect to see the Signature Fidelity fall off a cliff when the DeepCAD latent isn't passed through directly.

I'm happy with the paper as long as it's made very clear that the DeepCAD and SkexGen results are auto-encoding and so can't be directly compared with the other methods.

I'm even surprised that DeepCAD beats SkexGen, but maybe it's because quantization of the codebooks introduce topological changes.

AuthorsOpenReview ↗

Further clarification of evaluation settings and an additional text-conditioned experiment

We sincerely thank you for this insightful comment. We agree that DeepCAD and SkexGen are evaluated under reconstruction-based settings, whereas Text2CAD, CAD-Coder, and Cadrille-SFT use text-conditioned generation. Specifically, the former receive ground-truth latent representations or shape codes, while the latter generate CAD from expert-level textual descriptions. Therefore, their absolute Signature Fidelity scores should not be directly compared as measures of generation capability.

Additionally, we would also like to mention that our goal is to provide a unified topological evaluation protocol across different CAD modeling paradigms, rather than comparing performance across different paradigms. We will clarify these distinctions in the revised manuscript.

Following your suggestion, we conducted an additional experiment using an alternative pipeline to further investigate this issue. We used Text2CAD to generate CAD command sequences from expert-level descriptions, converted these sequences into DeepCAD's input format, and then processed them using DeepCAD's official construction pipeline.

Model / PipelineSignature Fidelity
DeepCAD (reconstruction-based)88.85%
Text2CAD (text-conditioned)80.37%
Text2CAD sequence + DeepCAD construction79.56%

The alternative pipeline achieves 79.56%, substantially lower than DeepCAD's reconstruction result and close to Text2CAD's 80.37%. This further illustrates why reconstruction-based and text-conditioned results should be interpreted separately. We will make this distinction explicit throughout the revised manuscript.

We thank you again for this valuable suggestion. We hope our response addresses your concerns and clarifies the distinction between the two evaluation settings. We would be glad to provide further discussion or clarification if any questions remain.

Reviewer_ugYfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper evaluates global topological fidelity in CAD generation using the Betti signature of the final solid. It studies five models under standard generation, construction-pattern novelty, and topology-directed interventions. The paper finds that quantization alone can change topology and that high aggregate fidelity does not necessarily imply reliable performance on unfamiliar constructions or topology-changing instructions. These findings are informative, but the descriptor's structural coverage and the interpretation of some experiments remain limited.

Critical strengths
  1. The paper addresses an important evaluation problem: valid and geometrically similar CAD models may still have incorrect connectivity, tunnels, or cavities. The finding that quantization changes the topology of approximately 10% of paired references is particularly consequential.
  2. The authors validate topology extraction through synthetic solids, tolerance checks, and an independent implementation. They also investigate how quantization affects error directions, strengthening confidence in the principal measurements.
Critical weaknesses
  1. The Betti signature provides a limited characterization of CAD structural correctness. Betti numbers describe global topological invariants but do not capture local structural details, specific adjacency relationships, or construction semantics. Consequently, models with substantially different local structures may receive identical signatures. In comparison, ComplexGen explicitly models B-Rep vertices, edges, faces, and their relationships, retaining finer structural information. The tasks differ, so a direct performance comparison would be inappropriate. Nevertheless, the paper should better explain what its compact, decomposition-independent descriptor preserves, which errors it misses, and how these omissions affect the interpretation of fidelity. Section 7 acknowledges this limitation, but its implications are not sufficiently developed in the discussion of the evaluation results.
  2. The construction-novelty and topology-intervention results admit important alternative explanations. For construction novelty, the partition is based on command-sequence fingerprints, while complexity and sequence length are adjusted only through coarse bins. Novel cases may remain more difficult within these bins or be more susceptible to quantization. The residual gap therefore cannot yet be confidently attributed to construction novelty itself. For topology-directed intervention, retaining the original description and appending a modification may introduce conflicting instructions or cause the added content to be ignored. Preserving topology under neutral prompts does not establish that the new instruction was executed. The very low success rates may therefore reflect general instruction-processing failures rather than a specifically topological limitation.
Questions for authors
  1. Are there generated results whose Betti signatures match the references but whose local structures differ substantially or contain errors? If so, how frequent are such cases among signature-matched samples, and how do they affect the interpretation of “structural fidelity”? Additionally, how does the proposed metric complement the B-Rep structural information used in ComplexGen?
  2. Does the covered–novel gap persist on representation-consistent references and under finer matching of complexity or sequence length?
  3. Could you verify that the complete appended instruction reaches the model and test a small subset using coherent, fully rewritten target descriptions and an active geometry-edit control whose execution is verified?
1 comment
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for recognizing the importance of evaluating global topological fidelity and the validation of our topology-extraction pipeline. We would like to address your questions individually as follows.


Q1: How frequent are local structural errors despite matching Betti signatures, and how does your metric complement ComplexGen’s B-Rep representation?

A1: We would like to clarify that our work aims to isolate errors in the global topology of the final solid that geometric similarity and validity metrics can miss. The Betti signature provides a compact, decomposition-independent comparison of connected components, tunnels, and cavities across different CAD representations.

However, matching Betti signatures do not imply that local structures match, and quantifying the frequency of such local discrepancies is challenging. This is because they encompass heterogeneous types of errors, such as geometric deviations, feature misplacements, and local connectivity errors. These discrepancies require different evaluation criteria, and there is currently no universally adopted metric that comprehensively measures them. Therefore, we do not report a unified frequency statistic for such cases.

Besides, ComplexGen evaluates finer relationships among B-Rep vertices, edges, and faces after matching corresponding entities. Applying that comparison directly across our models is difficult because they construct solids through different representations: equivalent solids can have different face and edge decompositions, making entity correspondence ambiguous. Our Betti metric is insensitive to those decomposition differences and provides a complementary check of final-solid connectivity, tunnels, and cavities.


Q2: Does the covered–novel gap persist on representation-consistent references and under finer matching of complexity or sequence length?

A2: We divided sequence length into quintiles within each CtopoC_topo stratum and recomputed the standardized difference in signature fidelity:

ModelCovered–novel gap (percentage points)95% bootstrap CI
DeepCAD16.1%[11.7, 21.1]
SkexGen11.2%[6.4, 16.5]
Text2CAD12.8%[8.9, 17.1]
CAD-Coder9.5%[5.8, 13.2]
Cadrille-SFT6.7%[3.0, 10.2]

The gap remains positive under finer sequence-length matching. Construction fingerprints depend on command types, so parameter quantization does not change covered–novel assignments.


Q3: Could you verify complete instruction delivery, test a small subset with coherent, fully rewritten target descriptions, and evaluate an active geometry-edit control with verified execution?

A3: We verified that CAD-Coder and Cadrille-SFT received the complete appended instructions. Text2CAD truncates 41/600 prompts (6.8%; 22 add-hole and 19 remove-hole), but operations still had zero successful edits excluding them. Thus, truncation does not explain the observed failure of topology modification.

Besides, we would like to clarify that our work aims to assess whether models can incorporate new topological requirements through incremental instructions while preserving the original design context, but fully rewritten descriptions are a separate setting and bypass the incremental instruction-following capability we aim to evaluate.

To further examine the effect of instruction formulation, we tested more detailed, expert-level incremental instructions on 120(60+60) Text2CAD samples for each operation, while retaining the original descriptions. The success rates remained low:

Topology-editing instructionSuccessful casesStrict-hit rate
Add a through-hole1/601.7%
Remove a through-hole0/600.0%

This further indicates that the observed difficulty is not simply due to concise instructions. As for active geometry-edit control, we would like to clarify that our study specifically investigates the realization of requested global topological changes, rather than general-purpose editing ability. These represent different evaluation objectives.

Reviewer_kjQVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies global topological fidelity (matching the numbers of connected components, tunnels, and enclosed cavities) in procedural CAD generation using the Betti signature of the final solid. It tests whether models generate the correct Betti topology, preserve it on unseen construction patterns, and change it as instructed. Across five models, the authors report different tendencies to miss or add topological structures and lower fidelity on unseen construction patterns, even after accounting for measured complexity. The three text-conditioned models also rarely follow the tested instructions to add or remove holes.

Critical strengths
  1. The evaluation reveals distinct structural failure modes that are not captured by conventional geometric similarity or validity metrics. The component-wise and directional analyses in Section 4.1 provide useful diagnostic information.
Critical weaknesses
  1. Limited empirical contributions. The paper identifies several interesting failure patterns in CAD generation, but the main findings in Section 4 are largely descriptive, with limited insight into why these failures occur or what they imply for improving CAD generation models. The quantization analysis in Appendix F provides a more concrete explanation, but is limited to DeepCAD. Overall, the broader significance of the empirical contributions remains unclear.
  2. Potential issue with the topology-editing experiment. The models are trained on detailed expert-level descriptions, but the editing experiment (§4.3) only appends short, vague instructions to the original descriptions. This mismatch may explain the low success rates, rather than a lack of topology-editing ability.
Questions for authors
  1. Insight: What deeper insights can be drawn from the observed failure patterns, and how could these insights guide future improvements in CAD generation methods?
  2. Topology editing: Have the authors considered using revised expert-level descriptions instead of short instruction suffixes (§4.3)? Would this change the reported success rates?
1 comment
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for recognizing the value of our component-wise and directional analyses in revealing structural failure modes. We would like to address your questions individually as follows.


Q1: What deeper insights can be drawn from the observed failure patterns?

A1: Our findings provide three key insights into the observed failure patterns and their implications for improving CAD generation.

First, at the representation level, our DeepCAD analysis shows that quantization can disrupt topology-critical geometric relations, altering the topology of 10.0% of the examined references. This highlights the importance of topology-preserving CAD representations.

Second, at the generation level, topology errors exhibit distinct directional patterns: text-conditioned generators predominantly under-realize existing topological structures, whereas reconstruction-based models show more balanced or over-realization errors. These differences suggest the potential value of topology-aware generation and targeted error correction.

Third, at the generalization and transformation levels, topological fidelity drops substantially on construction-novel patterns. Furthermore, models rarely execute explicit topology-changing instructions, even when their baseline topology is correct. These findings suggest the need for greater construction-pattern diversity and more explicit mechanisms for topology transformation.

Taken together, these insights suggest that improving CAD generation requires addressing topology beyond geometric similarity and execution validity. Promising directions include topology-preserving representations, topology-aware generation and correction mechanisms, more diverse construction patterns, and explicit topology-aware planning.


Q2: Have the authors considered using revised expert-level descriptions instead of short instruction suffixes (§4.3)? Would this change the reported success rates?

A2: Our experimental setting is motivated by practical CAD scenarios, where users commonly express incremental design modifications through short and natural instructions, rather than rewriting the entire design specification.

To further investigate the potential effect of instruction detail, we conducted an additional experiment on 120(60+60) Text2CAD samples using more detailed, expert-level topology-editing instructions, while retaining the original design descriptions. The results are shown below.

Topology-editing instructionSuccessful casesStrict-hit rate
Add a through-hole1/601.7%
Remove a through-hole0/600.0%

These additional results further support our original findings, indicating that the observed difficulty in topology editing is not merely a consequence of using concise editing instructions.

Reviewer_yYBHOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper evaluates topological fidelity in five CAD generators using Betti signatures. It examines standard generation, unseen construction patterns, and topology-changing instructions, revealing structural errors overlooked by conventional evaluation.

Critical strengths
  1. The problem is practically relevant. The finding that quantization alone changes topology in 10% of comparable references is particularly useful.
  2. The extraction pipeline is well validated, and the complementary diagnostic experiments provide informative failure analyses.
Critical weaknesses
  1. The contribution remains primarily diagnostic. The paper provides no topology-aware post-training or improvement baseline demonstrating that its evaluation signals can guide better generation. A focused fine-tuning or training-data-selection experiment would strengthen its practical value.
  2. The evaluation mainly uses DeepCAD-derived data, and topology interventions are limited to through-holes. Appending instructions may also create conflicting requirements, complicating the interpretation of low success rates.
Questions for authors
  1. Can a topology-aware training baseline improve signature fidelity while preserving geometric quality and validity?
  2. Do the intervention results persist with self-consistent rewritten prompts?
1 comment
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for recognizing the practical relevance of the quantization analysis and the validation of our topology-extraction pipeline. We would like to address your questions individually as follows.


Q1: Can a topology-aware training baseline improve signature fidelity while preserving geometric quality and validity?

A1: We would like to clarify that our work is positioned as an empirical study of global topological fidelity rather than a proposal for a new training method. As you noted, our findings show that quantization can alter reference topology and that signature fidelity drops substantially on construction-novel patterns. Moreover, although we include SkexGen as a relevant topology-aware representation baseline, it does not explicitly optimize the global topology of the final solid. Since existing topology-aware CAD methods primarily focus on representation-level structure, such as B-Rep connectivity, our findings further highlight the importance of global topological fidelity in model training. We therefore agree that topology-aware training is a promising direction for future work.


Q2: Do the intervention results persist with self-consistent rewritten prompts?

A2: We would like to clarify that our intervention experiment is designed to assess whether models can execute a requested topology change while preserving the original design context. Accordingly, the appended instruction is intended to update the relevant part of the original description rather than replace it entirely. While self-consistent rewritten prompts may yield different results, our current findings specifically characterize the incremental modification setting. In contrast, fully rewritten descriptions constitute a different evaluation setting, as they directly specify the modified target rather than testing the ability to perform an incremental modification. We therefore consider this a valuable complementary direction for future evaluation.

Comments on the paper

2 comments

Dear Reviewers,

The authors have posted their rebuttal. Please read it and reply in the discussion, covering:

Whether each of your questions and concerns has been addressed. Whether your rating and recommendation change, and why. Any remaining concern that must be settled before the meta-review. Best.

AuthorsOpenReview ↗

The revised PDF has been updated based on the reviewers’ suggestions

We sincerely thank all reviewers for their insightful comments and constructive feedback. Following the review process, we have revised the manuscript to address the reviewers' concerns and improve its clarity and rigor. All modifications are highlighted in blue color in the revised PDF. The revisions are summarized below:

1. Clarifying the distinction between reconstruction-based and text-conditioned paradigms. Following the suggestions of Reviewer GeKr, we revised Evaluated Models (Sec. 3) to explicitly distinguish reconstruction-based and text-conditioned models according to their input modalities and task objectives. We also reorganized Table 1 in Sec. 4.1 into two groups and clarified that results across these paradigms are not directly comparable.

2. Addressing the covered–novel gap under finer-grained sequence-length matching. Following suggestions of Reviewer ugYf, we added a new subsection, "Finer-Grained Sequence-Length Matching," in Appendix B.2 to further examine the covered–novel gap under more fine-grained sequence-length matching.

3. Examining the effect of instruction detail in topology-directed intervention. Following suggestions of Reviewer kjQV and ugYf, we added a new subsection, "Additional Experiment: Expert-Level Editing Instructions," in Appendix C.1. It reports an additional experiment using detailed expert-level instructions for adding and removing through-holes on 120 Text2CAD samples (60 for each operation), while retaining the original design descriptions.

We would greatly appreciate it if you could review our revised PDF and let us know whether they address your concerns. We would be glad to clarify any remaining questions during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.