acceptodds
Under review as a conference paper at ICLR 2027

GeomCAD: A Benchmark for Precision Reconstruction of Generative CAD Operations from Point Clouds

Abstract

Point cloud to CAD models are judged by whether their programs execute and resemble the target shape, yet a CAD program is only useful if its parameters are right. We show that these criteria diverge sharply: models whose programs execute on over 90% of inputs recover the exact parameter set for at most 2.7% of them. To measure this gap, we introduce GeomCAD, a benchmark of 4 million executable CadQuery programs, four times larger than any prior program-paired corpus and balanced across extrude, revolve, sweep, and loft, with every parameter an integer on a 1,000-level grid so that exact parameter match is well defined; 36% of its faces are neither planar nor cylindrical, versus at most 12% in prior program-paired corpora. Its protocol GeomCAD-Eval scores exact parameter recovery alongside shape and validity. Our reference model CAD-Exact, trained on GeomCAD, outperforms all zero-shot and fine-tuned baselines on every operation, recovering 27–49% of ground-truth parameter values (best baseline: 13–40%) and matching the target volume within 5% on 49–95% of inputs (best baseline: 36–84%). The corpus, evaluation code, and all checkpoints are released.

Primary area:
datasets and benchmarks
Keywords:
CAD reconstruction, point-cloud-to-program, benchmark, exact parameter recovery, program synthesis, LLM code generation, CadQuery, precision-aware evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_KAf1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper argues that point-cloud-to-CAD reconstruction is judged by the wrong criteria: programs are scored on whether they execute and produce a similar shape, yet a CAD program is only useful if its parameters are right, and the authors show these criteria diverge sharply — models whose programs almost always execute recover the exact ground-truth parameter set only rarely. To make parameter precision measurable, they introduce GeomCAD, a procedurally generated corpus of executable CadQuery programs balanced across four generative operations (extrude, revolve, sweep, loft), with every literal stored on an integer grid so that exact parameter match is well defined, and with a surface mix far richer than prior program-paired corpora. The accompanying GeomCAD-Eval protocol pairs exact and graded parameter recovery with standard shape-similarity metrics and a failure-mode decomposition of program validity. A reference model trained on the corpus (a CAD-Recode-style point-conditioned LLM) outperforms zero-shot and fine-tuned baselines on every operation, yet still recovers only a minority of ground-truth parameter values. The corpus, evaluation code, and checkpoints are released.

Critical strengths
  1. Making exact parameter recovery measurable, and showing that it diverges from the metrics the field actually reports. The integer-grid design is what makes PEM well defined at all — with float literals, any comparison would need a tolerance, as the Zero-to-CAD discussion notes. The headline observation, that models executing on over 90% of inputs match the full parameter set on almost none of them, is useful to have on record: it suggests that recent shape-metric progress says less about dimensional correctness than is commonly assumed, and it gives follow-up work a concrete target to optimize against.
  2. The corpus is large and more diverse than prior program-paired data, and the release engineering is careful. Four operations at roughly a million samples each, with about a third of faces neither planar nor cylindrical, should make it usable as training data and not only as a test set. On the release side — pinned OCCT container, mesh round-trip audit, split-versus-pool checks as a release gate, versioned scoring — the main claims can be recomputed by a reader rather than taken on trust, which is not something every benchmark paper manages.
Critical weaknesses
  1. Robustness to realistic input noise is never tested. All point clouds are sampled from ideal CAD surfaces. If the end goal is scanning a real object and recovering how it was made, the input comes from a scanner, and scanner point clouds always carry noise, outliers, and partial coverage. The evaluation never enters this regime: the encoder sees clean 8,192-point clouds, and no noisy variant of the splits exists. This matters more here than for shape-similarity benchmarks, because the paper's central claim is about precision at 0.1% grid resolution — a metric suite built around exactness is only convincing if the exactness survives realistic sensing conditions. Even a small noise-level sweep on a subset would substantially strengthen the paper.
  2. Every sample is a single-operation program, and it is unclear whether the evaluation survives composition. Real CAD programs are sequences of operations, and once operations compose, construction ambiguity grows quickly: materially different operation sequences can produce the same solid, so strict literal-sequence matching (PEM) — and to a lesser degree multiset recall (rPEM) — becomes increasingly ill-defined exactly where the task becomes realistic. The only real-data evidence in the paper is zero-shot transfer to DeepCAD, which is extrude-only and shares sketch provenance with the training pool ("only partly external", as the authors themselves put it). Whether GeomCAD-Eval remains meaningful for multi-operation reconstruction is an open question, and it bounds how far the benchmark's framing as a measure of progress toward useful CAD reconstruction can currently reach.
  3. PEM is meaningful only where the ground-truth program is identifiable from shape, and the paper's own results show this fails on a quarter of the benchmark: loft PEM is 0.0 even for CAD-Exact joint at TVF@5% 94.8, because free-form sections admit multiple valid parametrizations. The flagship metric therefore cannot distinguish "wrong parameters" from "a different but equally valid parametrization" exactly where the geometry is richest. The graded rPEM variants mitigate this, but the headline framing ("exact parameter recovery remains largely unsolved") still leans on strict exact match, and the abstract's 2.7% figure inherits the caveat.
Questions for authors
  1. Have you considered an equivalence-aware variant of PEM — e.g., crediting predictions whose executed solid matches GT within tolerance even when the literal sequence differs, or canonicalizing section parametrizations for loft? How would the headline ≤2.7% change under such a metric? This seems important for interpreting PEM as a measure of reconstruction quality rather than of parametrization luck.
  2. Do the Tier-1/Tier-3 picture and the executability–exactness gap survive realistic sensing conditions (noise, partial views, varying density)? Is a noisy variant of the splits planned?
  3. Real CAD programs are sequences of operations. Does the integer-grid PEM remain well defined under operation-order permutations once programs compose extrude/revolve/sweep/loft, and do you see GeomCAD extending to that setting?
1 comment
AuthorsOpenReview ↗

Follow-up before discussion deadline

Dear Reviewer KAf1,

Thank you again for the careful review. Since our earlier replies, the revised PDF has added:

  1. Noise-robustness sweep (§F.5, W1): a 500-sample Gaussian-noise sweep. Validity drops from 94.2% to 51.0%, aggregate TVF@5% from 73.6% to 7.6%, and strict PEM from 4/500 to 0/500; the rPEM margin over a shuffled-reference baseline remains positive across the sweep (+26.5 pp on clean inputs, +4.2 pp at σ = 0.02).

  2. Multi-operation pilot (§F.6, W2/Q3): a 1,497-sample zero-shot pilot on two-type Boolean-union compositions. 76.6% of CAD-Exact (joint) predictions execute, aggregate TVF@5% is 22.4%, and 0.0% match the reference literal sequence; the executability–exactness gap persists under composition.

  3. PEM-eq10 metric (§3.5, §E.1, Table 3, W3/Q1): a TVF-gated, order-invariant, tolerance-based literal recovery variant (±10 grid units) addressing your parametrization equivalence question. For CAD-Exact (joint), the TVF-gated recovery rate reaches 27.4% on loft and 21.5% on revolve.

If these additions address your concerns, we would be grateful if you would consider updating your assessment before the discussion deadline.

Respectfully,

The authors

Reviewer_99VQOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces GeomCAD, a dataset of 4M CADQuery programs, all single-operation, using extrude, revolve, loft, or sweep. The numerical parameters are represented on a 1000-level integer grid. It also introduces GeomCAD-Eval, a set of metrics to measure exact parameter recovery, shape similarity, and validity, and CAD-Exact, a CAD-Recode v1.5 architecture (Qwen2-1.5B) trained on the GeomCAD dataset to generate CADQuery code from an input point cloud. The validation consists of evaluating CAD-Exact against cadrille (zero-shot and fine-tuned), CAD-Recode, and PC2CAD (trained on DeepCAD) on GeomCAD's test set using the GeomCAD-Eval protocol.

Critical strengths

The paper is well-written and easy to follow.

Critical weaknesses

The main weakness of this paper is that the benchmark is limited to only single-operation CAD programs. This problem is trivial and won't measure the ability of these AI models to generate CAD programs from point cloud of real-world geometries. In Table 2, GeomCAD has 9 median faces per solid, compared to ABC, which has 187 or Zero-to-CAD, which has 30. This shows the gap between this benchmark and real-world CAD geometries, where multi-operation CAD programs lead to a higher number of faces per solid. The real problem in point-cloud to CAD program is how can we recover accurate programs with accurate parameters in a multi-operation CAD geometry setting which is the problem tackled in papers like CAD-Recode and cadrille. Other weaknesses:

  • Very limited number of baseline models for a benchmark
  • CAD-Exact joint trained on GeomCAD achieves extremely lower performance than baselines like CAD-Recode and cadrille on DeepCAD (Table 13)
Questions for authors

Why do you limit the scope of the paper to single-operation CAD programs?

Reviewer_tDVnOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents GeomCAD and CAD-Exact, a benchmark and a reference model for point-cloud-to-CAD reconstruction that emphasizes exact parameter recovery and multi-operation coverage. Its contributions are three-fold: (1) an integer-canonical program representation that makes Parameter Exact Match (PEM) strictly definable, together with GeomCAD-Eval, a three-tier evaluation protocol covering parameter precision, shape similarity, and program validity with failure-mode decomposition; (2) GeomCAD, a 4M executable CadQuery programs balanced across four generative operations (extrude, revolve, sweep, and loft); and (3) CAD-Exact, a reference model trained on GeomCAD. The authors evaluate nine systems (four baseline models and five variants of the proposed CAD-Exact) on GeomCAD’s per-operation test splits. The results show that CAD-Exact recovers more parameter values than all baselines. However, the results also indicate that exact recovery remains an open problem: although models can generate executable programs for more than 90% of inputs, they fully reproduce the parameter set for at most 2.7% of them.

Critical strengths

This paper focuses on the issue of reconstruction precision in point-cloud-to-CAD. Existing evaluation metrics used in prior work, such as CD and IoU, primarily measure appearance similarity, but there are no metrics to evaluate reconstruction precision at the parameter level. To address this issue, this work proposes a comprehensive solution that includes a dataset, evaluation metrics, and a model. Specifically, when processing the data, the authors propose a 1,000-level integer grid representation that makes Parameter Exact Match (PEM) strictly definable; this approach is inspiring. Furthermore, both the dataset and the evaluation metrics are valuable to the community.

Critical weaknesses
  1. As shown in Table 3, none of the proposed models, regardless of the training strategy, achieves the best performance across all metrics for all operations. Therefore, how to leverage this new dataset more effectively to develop a stronger model is an area worth further exploration.
  2. Regarding the proposed GeomCAD-Eval, is the PEM metric reasonable? The PEM metric requires identical literal-sequence length and position-wise exact matching, making it highly sensitive to literal ordering and parametrization. Since the paper acknowledges multiple valid parametrizations for the same solid (e.g., the PEM for loft in Table 3 is 0.0), it is unclear whether PEM unfairly penalizes semantically or geometrically equivalent correct programs. Furthermore, is an order-invariant or normalized exact-match variant needed for such operations?
Questions for authors
  1. In Table 3, the proposed model variants exhibit an interesting pattern: under the staged training strategy, each variant appears to perform better on the operation focused on in the previous stage. For instance, on the Extrude test split, CAD-Exact (stage 2) outperforms CAD-Exact (stage 1) on more metrics, even though Stage 1 was specifically fine-tuned for the Extrude operation. Could the authors provide an explanation for this?
  2. There are multiple formatting issues (e.g., indexing failures), such as Line413 and L483.
1 comment
AuthorsOpenReview ↗

Follow-up before discussion deadline

Dear Reviewer tDVn,

Thank you again for the careful review. Since our 2026-11-07 reply, the revised PDF has added:

  1. PEM-eq10 metric (§3.5, §E.1, Table 3): a TVF-gated, order-invariant, tolerance-based literal recovery variant (±10 grid units), directly addressing your W2 question on whether an order-invariant or normalised exact-match variant is needed. For CAD-Exact (joint), the TVF-gated recovery rate reaches 27.4% on loft (where strict PEM is 0.0%) and 21.5% on revolve.

  2. Clarified staged vs. joint trade-off in §6 (W1, Q1): we now state the regime-specific trade-offs explicitly (e.g., staged leads on sweep TVF@5% 59.2 vs. joint 49.0; joint leads on loft 94.8 vs. staged 58.4), and explain why stage 2 can improve stage 1 on Extrude through continued training from stage-1 weights plus Extrude replay.

  3. Cross-reference fixes at L413 and L483 (Q2).

If these additions address your concerns, we would be grateful if you would consider updating your assessment before the discussion deadline.

Respectfully,

The authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Revised PDF - change log

Revised PDF - change log (summary)

We thank the three reviewers and the Area Chair. This revision (27 pages; main text spans pages 1-10 and ends on page 10, within the original 10-page limit) implements the commitments in our individual replies. We summarise the three substantive additions here; the full per-item change log, with reviewer attribution for each edit, is at https://anonymous.4open.science/r/CAD-Exact-5707/CHANGELOG.md.

(i) Order-invariant, tolerance-based literal recovery. A new Tier-1 metric PEM-eq10 (TVF-gated, order-invariant full-literal recovery within ±10 grid units) is defined in §3.5, formalised in §E.1, and reported for all eight CadQuery configurations × four operations in Table 3. This responds to the order-invariant exact-match variant asked for by tDVn (W2) and to KAf1 (W3, Q1)'s equivalence question. A τ-sweep for τ ∈ {1, 5, 10} is in §F.4; for CAD-Exact (joint), the TVF-gated recovery rate rises from 0.0% at τ=1 to 27.4% at τ=10 on loft and from 3.5% to 21.5% on revolve.

(ii) Noise and multi-operation diagnostics. A 500-sample Gaussian-noise sweep (§F.5, addressing KAf1 W1) and a 1,497-sample zero-shot pilot on two-type Boolean-union compositions (§F.6, addressing KAf1 W2/Q3 and 99VQ's scope concern) show that the executability-exactness gap GeomCAD-Eval exposes persists under input perturbations and under two-type compositions. Under noise, validity drops from 94.2% to 51.0%, aggregate TVF@5% from 73.6% to 7.6%, strict PEM from 4/500 to 0/500; the rPEM margin over a shuffled-reference baseline remains positive across the sweep (+26.5 pp on clean inputs, +4.2 pp at σ = 0.02). Under composition, 76.6% of predictions execute and 0.0% match the reference literal sequence.

(iii) Shape-metric vs parameter-recovery analysis. A new appendix §F.7 (addressing 99VQ follow-up Q2) reports Spearman correlations (Table 19: strict PEM has |ρ| ≤ 0.19 against IoU/CD where defined; CD vs. rPEM shows moderate correlation on Revolve/Sweep/Loft, |ρ| = 0.41 to 0.62), a conditional pass-rate analysis (Table 20: restricting to IoU ≥ 0.95 and CD% ≤ 0.5, strict-PEM pass rate is still 0.0% to 6.0% per operation), and four per-sample exemplars (Table 21).

Reproducibility artefacts (anonymous preview, available during the discussion period)

  • Code: https://anonymous.4open.science/r/CAD-Exact-5707 (updated 2026-11-12)
  • Weights and test splits: https://doi.org/10.5281/zenodo.24158668 (CC-BY-NC-4.0; concept DOI https://doi.org/10.5281/zenodo.24158667)
    • cad-exact-joint.tar: reproduces the joint row of Table 3 (all four operation columns, including PEM-eq10, and the Appendix F.5 / F.6 / F.7 analyses).
    • cad-exact-loft.tar: reproduces the stage-4 (loft) row of Table 3.
    • GeomCAD_test_splits_v1.tar.gz (502 MB): the 6,244 test samples scored in Table 3 (1,500 extrude + 1,500 revolve + 1,500 sweep + 1,744 loft). The layout matches the --datainfo and --gt_dir conventions of infer_batched.py and eval.py. With the joint and stage-4 checkpoints above, this is sufficient to reproduce the joint and stage-4 (loft) rows of Table 3 end to end.
    • The anonymous preview does not include the stage-3 checkpoint associated with the Revolve results of 2.7% strict PEM and 28.6% PEM-eq10, nor the stage-1 and stage-2 checkpoints; those, together with the full 4M-program corpus and the generator and training code, will be released upon acceptance.

Internal corrections. No Table 3 result changes as a consequence of these corrections except a stage-3 Sweep PEM-eq10 rounding fix (22.3% to 22.4%); the added PEM-eq10 column is a new metric, not a correction. The original strict-PEM headline of 2.7% in the abstract is unchanged; the revised abstract adds the 28.6% PEM-eq10 result as a new reported quantity. Corpus count updated from 4,006,182 to 4,006,426 (the 244-sample difference is the 244 additional loft test programs used to score the loft column of Table 3). The pseudocode in §B (Algorithms 1–3) and the LoRA parameterisation in §D are also aligned to the released code. Full list of corrections at the CHANGELOG link above.

Page count. 27 pages overall. Main text (Sections 1-6) ends on page 10, within the original 10-page limit; we did not use the +1-page discussion-period allowance. The AI Use, Ethics, and Reproducibility statements follow on page 11 onward.

We hope these additions address the concerns raised and would be grateful if the reviewers could reconsider their ratings in light of the revised PDF, the per-reviewer discussion threads, and the reproducibility artefacts above.

Dear Reviewers, the authors have posted their comments in response to the reviews. Please make an effort to discuss your review and the authors' response. - regards AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.