acceptodds
Under review as a conference paper at ICLR 2027

Measure Before You Code: Tool-Augmented CAD Program Synthesis from Point Clouds

12 views

Abstract

Synthesizing executable CAD programs from point clouds requires recovering both discrete modeling sequences and continuous geometric parameters. Recent point-to-CAD methods rely on multimodal large language models (MLLMs) to compress point clouds into latent tokens and infer parametric programs end-to-end. While these methods show impressive progress in recovering overall shape structure, this paradigm overlooks the primary advantage of point clouds over 2D images: direct access to explicit 3D coordinates. Dimensions are therefore predicted rather than measured, which makes them prone to error. We propose QM-CAD (Query-and-Measure CAD), an agentic framework guided by a simple principle: measuring before coding. An MLLM drafts a modeling plan that directs geometric queries to the raw point cloud. A query-driven segmenter identifies the surface points relevant to each planned operation. A geometry fitter returns measured parameters and point projections. The MLLM uses this numerical and visual feedback to generate the final CAD program. To support this paradigm, we construct an automated pipeline for synthesizing plan-measure-generate trajectories with real tool executions. Supervised fine-tuning on these trajectories teaches the model to formulate queries and use their feedback. Reinforcement learning then refines the model, using the solid Intersection-over-Union (IoU) of executed CAD models as the reward. Experiments on DeepCAD and Fusion360 show that QM-CAD achieves higher geometric reconstruction accuracy than recent point-to-CAD methods while keeping a low invalidity ratio.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
CAD reconstruction, CAD program synthesis, point clouds, multimodal large language models, tool use, reinforcement learning, reverse engineering

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_oP7gOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose QM-CAD, a framework for generating executable CAD programs from point clouds. The main motivation is that existing LLM/MLLM-based methods generally don't have strong priors for point-cloud tokens and end up estimating geometric parameters from latent representations, even though these parameters could be measured directly from the point cloud.

To address this, the authors propose a plan-measure-generate approach. An MLLM first generates a construction plan and queries for the geometric features it wants to measure. A query-driven point segmenter then identifies the relevant points, and a geometry fitter extracts measurements and projection images. These are fed back to the MLLM to generate the final CAD program.

The authors also introduce an automated pipeline for generating training trajectories, which are used for supervised fine-tuning, followed by RL using the IoU of the generated CAD model as the reward. They evaluate QM-CAD on DeepCAD and Fusion360, reporting improvements in reconstruction accuracy over the included baselines. Ablation studies further show that introducing measurement tools improves performance, while RL provides additional gains.

Critical strengths

The paper is well-motivated and addresses a limitation of existing LLM/MLLM-based CAD reconstruction methods. Since point clouds already contain explicit 3D coordinates, estimating geometric parameters from latent representations introduces unnecessary complexity when these parameters can be measured directly. The proposed separation of planning, measurement, and code generation allows the MLLM to focus on construction planning while relying on geometric tools for numerical measurements.

QM-CAD achieves higher reconstruction accuracy than the included baselines on DeepCAD and Fusion360. The ablation study in Table 2 shows that planning alone provides relatively small improvements, whereas introducing measurement tools leads to more substantial gains. This provides evidence that explicit geometric feedback improves CAD reconstruction. The automated pipeline for generating approximately 1.19 million training trajectories is another contribution, providing supervision for learning tool interactions and CAD program generation.

Critical weaknesses

The main contribution of the paper seems to be the learned query-and-measure framework, where an MLLM decides what geometric features should be segmented and measured. However, the experiments do not directly isolate the contribution of this learned query-selection mechanism. While Table 2 shows that introducing measurement tools improves reconstruction accuracy, it does not establish how much of this improvement comes from the MLLM learning what to measure, as opposed to simply having access to additional geometric information. Furthermore, Table 3 evaluates segmentation and measurement accuracy using reference queries derived from ground-truth CAD programs rather than queries generated by the MLLM. As a result, these experiments establish that the measurement tools work when provided with appropriate queries, but do not directly evaluate how well the MLLM learns to generate those queries. A controlled comparison against a fixed or heuristic query-selection approach, while keeping the measurement tools unchanged, would help isolate this contribution.

Another concern is that it is unclear how much of the final CAD program is actually grounded in explicit measurements. For instance, in sketch-extrude operations, the geometry fitter provides measurements such as extents and extrusion depth, along with projection images, but the MLLM still needs to infer parts of the sketch geometry, including vertices and arcs. The paper does not quantify how many of the final CAD parameters are directly measured versus inferred by the model, or how accurate these parameters are individually. Given that the main motivation is to avoid estimating geometric parameters from latent representations, a parameter-level evaluation would help establish this claim. Similarly, while the RL ablation shows improvements in final reconstruction accuracy, it remains unclear whether RL improves query selection or primarily improves code generation. This makes it difficult to determine how much of the reported improvement can be attributed to the paper's main methodological contributions.

Questions for authors
  1. Can the authors provide an evaluation of the queries generated by the MLLM itself, rather than using reference queries from ground-truth CAD programs as in Table 3? How accurately does the model identify what needs to be measured, and how does this affect downstream segmentation and measurement accuracy?

  2. Can the authors compare the learned query-selection mechanism against a fixed or heuristic approach while keeping the measurement tools unchanged? This would help isolate how much of the improvement comes specifically from learning what to measure.

  3. How much of the final CAD program is actually grounded in explicit measurements versus parameters inferred by the MLLM? For instance, in sketch-extrude operations, the model still needs to recover vertices, arcs, and other sketch parameters from projection images. Can the authors provide a breakdown of directly measured versus inferred parameters and their respective reconstruction errors?

  4. The authors claim that RL jointly improves planning, querying, and code generation. However, the current ablation only shows improvements in final reconstruction accuracy. Is there evidence that RL improves the quality of generated queries? Would optimizing only the code-generation stage, while keeping the query-generation policy fixed, achieve similar improvements?

4 comments
AuthorsOpenReview ↗

Response to Reviewer oP7g (Part 1/2: Q1, Q2 & W1)

We thank Reviewer oP7g for the positive evaluation and for highlighting the motivation of QM-CAD and our 1.19M trajectory synthesis pipeline. We address all four questions below with controlled experiments on the DeepCAD test set and will include these results in the revised paper.


Response to Q1 & W1: Accuracy of Model-Predicted Queries and Downstream Tool Performance

A single 3D CAD solid can be constructed in multiple ways, such as by extruding an annular sketch or subtracting one cylinder from another. We therefore matched model-predicted queries against reference queries on the full DeepCAD test set. The model-predicted queries of QM-CAD match about 65% of the reference queries and achieve 97.7% operation accuracy and 91.4% plane accuracy. We compare the results for these matched model-predicted queries with those reported for reference queries in Table 3.

Query SourceSeg. IoU ↑Precision ↑Recall ↑Measurement Coverage ↑[email protected]% ↑[email protected]% ↑Acc@1% ↑
Reference queries (Table 3)94.6%96.2%98.1%96.5%90.2%94.4%95.0%
Model-predicted queries (ours)93.4%95.5%96.5%95.8%89.8%93.5%94.1%

QDPS segmentation accuracy on model-predicted queries is very close to that on reference queries, and SGF measurements remain accurate across parameter types. These results show that the model reliably identifies what to measure at inference time.


Response to Q2 & W1: Learned Query Selection vs. Fixed or Heuristic Queries

Following your suggestion, we kept the measurement tools (QDPS and SGF) and the second-turn code generator fixed, and replaced only the first-turn queries using the same tool-call format on a 1,000-case DeepCAD test subset. We compared fixed three-plane queries using global boxes on the XY, YZ, and ZX planes, and heuristic RANSAC queries extracted via multi-plane RANSAC on the point cloud.

Query Selection Strategy (1k subset)Med. CD (×10³) ↓Mean IoU (%) ↑Med. IoU (%) ↑IR (%) ↓
Fixed three-plane3.20367.271.31.8
Heuristic RANSAC0.27673.893.31.1
Learned own (ours)0.15795.499.10.5

With the same measurement tools and code generator, learned queries yield substantially higher reconstruction accuracy than either fixed three-plane queries or heuristic RANSAC queries.

AuthorsOpenReview ↗

Response to Reviewer oP7g (Part 2/2: Q3, Q4 & W2)

Response to Q3 & W2: Directly Measured vs. Inferred Parameters and Their Reconstruction Errors

1. Proportions of Directly Measured and Inferred Parameters

Direct measurements provide information about both non-sketch parameters and 2D sketch parameters. We report statistics for both categories across the full DeepCAD test set.

  1. Non-sketch parameters. Among matched steps, SGF directly measures 99.9% of workplanes, 90.4% of cylinder and hole radii, 69.8% of box centers, and 46.8% of sketch extrusion depths. In the remaining extrusion steps, multi-step Boolean operations often remove or occlude one end face, leaving insufficient information to measure the depth directly.
  2. 2D sketch parameters. Some sketch vertex coordinates can be obtained directly from the 2D bounding box measured by SGF (uv_bbox). The bounding box determines one coordinate of a vertex along an edge and both coordinates of a vertex at a corner. Among predicted sketch vertices in steps with a measured uv_bbox, 57.1% lie on the bounding box boundary, including 23.2% at its corners. The model infers interior vertex coordinates and curves from projection images.

2. Reconstruction Errors of Directly Measured vs. Inferred Parameters

On matched steps across the full DeepCAD test set, we compare the reconstruction errors of parameters supported by SGF measurements with those of parameters inferred by the model. We express errors as percentages of the part's longest-axis length LL and report the median (Med.) and 90th percentile (P90) for each parameter type and group.

Parameter TypeMeasured Med. / P90 Error (% LL) ↓Inferred Med. / P90 Error (% LL) ↓
Extrusion depth0.0024 / 0.02340.0312 / 1.8427
Arc endpoint0.0002 / 0.44120.1542 / 0.8317
Sketch vertex uu0.0000 / 0.00090.0001 / 0.0346
Sketch vertex vv0.0000 / 0.00390.0001 / 0.0463

Across all four parameter types, parameters supported by SGF measurements have lower median and P90 reconstruction errors. For extrusion depth, the P90 error is 0.0234% of LL for parameters with measurement support and 1.8427% of LL for inferred parameters.


Response to Q4 & W2: Disentangling RL Gains on Query Generation vs. Code Generation

We evaluated query quality before and after joint RL on the full DeepCAD test set. To assess the contribution of each turn, we paired the SFT and joint-RL checkpoints at inference time while keeping QDPS and SGF fixed.

1. Query and Downstream Segmentation Quality

MethodPredicted StepsExact Step-Count Cases (%) ↑Step-Count MAE ↓Query Prec. / Rec. / F1 ↑Geo. [email protected] ↑Matched QDPS Mask IoU (%) ↑
SFT13,16572.40.4640.640 / 0.657 / 0.6490.40091.8
SFT + RL (ours)11,72576.80.3520.708 / 0.647 / 0.6760.42893.4

The table shows that after joint RL, query F1 rises from 0.649 to 0.676 and mask IoU rises from 91.8% to 93.4%.

2. Turn-Wise Policy Swap

Turn 1 (Plan & Query)Turn 2 (Code Generation)Med. CD (×10³) ↓Mean IoU (%) ↑IR (%) ↓
SFTSFT0.16193.70.44
SFTSFT + RL0.16094.30.15
SFT + RLSFT0.15894.90.37
SFT + RLSFT + RL0.15695.30.10

Comparing rows 1 and 2 shows that switching to the RL checkpoint for code generation reduces the IR from 0.44% to 0.15% and increases mean IoU from 93.7% to 94.3%. Comparing rows 1 and 3 shows that switching to the RL checkpoint for planning and querying yields a larger IoU gain, from 93.7% to 94.9%. Using the RL checkpoint for both turns (row 4) gives the highest mean IoU (95.3%) and the lowest IR (0.10%).

Reviewer_oP7gOpenReview ↗

Acknowledging Author Response to my Concerns

Thanks to the authors for the additional experiments. The new comparisons against fixed and heuristic query selection, along with the turn-wise RL ablation, help addresses the questions I raised about whether the learned querying mechanism contributes to the overall improvement. The parameter-level analysis also helps clarify how much of the reconstruction is grounded in measurements.

One remaining question is about the comparison against heuristic RANSAC queries. Since the second-turn code generator was trained using learned queries, could the performance gap partly come from the heuristic queries producing feedback that differs from the training distribution? Also, the authors mention that model-predicted queries match approximately 65% of the reference queries. It would be useful to understand how much of the remaining portion comes from valid alternative CAD decompositions versus incorrect or missing operations.

Overall, the rebuttal addresses most of my original concerns, and I am maintaining my weak acceptance rating.

AuthorsOpenReview ↗

Response to Reviewer oP7g's Follow-up Questions

We thank Reviewer oP7g for the thoughtful follow-up and for confirming that our additional experiments address your main concerns. We answer both remaining questions below on the same 1,000-case DeepCAD test subset.


1. Heuristic RANSAC Queries: Turn-2 Distribution Shift vs. First-Turn Measurement Loss

We agree that feedback distribution mismatch at the second-turn code generator can account for part of the end-to-end performance gap. To remove the second-turn code generator and its training distribution from the comparison, we directly evaluated the first-turn SGF tool measurements returned under heuristic RANSAC queries and under QM-CAD's learned queries against ground-truth step parameters.

Following the tool-evaluation protocol of Table 3, we report the measured rate and the measured-and-correct rate (error at most 0.5% of LL in every coordinate) for each parameter type.

GT Step ParameterHeuristic RANSAC (Meas. / [email protected]% ↑)Learned Queries (Meas. / [email protected]% ↑)
Extrusion depth42.1% / 42.1%70.1% / 69.7%
Extrusion start / end position42.1% / 42.1%70.1% / 69.8%
2D profile bounds34.7% / 33.3%49.9% / 45.8%

Even before reaching the second-turn code generator, heuristic RANSAC queries recover 27.6 percentage points fewer accurate extrusion depths and 12.5 percentage points fewer accurate 2D profile bounds than learned queries. Because RANSAC issues coarse plane-level queries without decomposing multi-step features or selecting suitable primitive fitters, many step-level dimensions are never measured in Turn 1. Thus, while Turn-2 distribution shift may play a secondary role, the primary bottleneck comes from insufficient step-level measurements in the first turn.


2. Breakdown of the Remaining Unmatched Queries

On the 1,000-case subset, our step-matching rule matches 64.9% of reference steps, leaving 564 unmatched reference steps and 424 unmatched model-predicted queries. We audited these unmatched items by ablating steps and comparing the resulting signed 3D volumetric changes, both one-to-one and across multi-step combinations or reorderings.

Audit CategoryUnmatched Reference Steps (N=564N = 564)Unmatched Predicted Queries (N=424N = 424)
Valid alternative decomposition245 (43.4%)203 (47.9%)
Consistent local 3D geometry (>95%), no 1-to-1 step match84 (14.9%)61 (14.4%)
Reference no-op or empty-profile step41 (7.3%)7 (1.7%)
Partially matched local geometry (50%–95%)80 (14.2%)73 (17.2%)
Incorrect or missing local feature (≤50%)66 (11.7%)33 (7.8%)
Unmatchable for step-level comparison48 (8.5%)47 (11.1%)

Valid alternative decompositions (such as step merging, splitting, reordering, or using alternative primitives and axes), steps that still produce consistent local 3D geometry without an isolated one-to-one step match, and empty or zero-change reference steps together account for 65.6% of the unmatched reference steps. By contrast, clear local geometric errors or missing features account for only 11.7% of the unmatched reference steps, and 7.8% of the unmatched predicted queries. These results explain why QM-CAD achieves a mean 3D solid IoU of 95.4% on this subset despite a strict step-by-step query match rate of about 65%.


Manuscript Update. We have also uploaded a revised manuscript PDF incorporating the new experiments and analyses from our earlier response (Section 4.3, Tables 3, 4, and 5b, and Appendix I, Tables 15–17), where newly added content is highlighted in red and deleted text is marked in blue with strikethrough. We hope these clarifications resolve your remaining questions, and we thank you again for the constructive discussion.

Reviewer_pgpLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

QM-CAD reconstructs CadQuery programs from point clouds through a plan-measure-code workflow. A multimodal model first produces a construction plan and geometric queries. A query-driven point segmenter identifies the surface points associated with each step, and a geometry fitter returns measured parameters, projection images, and measurement statuses. The model then generates the complete program using this feedback. The authors construct approximately 1.19 million training trajectories with actual tool execution, apply supervised fine-tuning, and subsequently optimize both generation turns using a solid-IoU reward. Experiments on DeepCAD and Fusion360 show improved reconstruction accuracy.

Critical strengths

1. A well-motivated integration of planning and geometric measurement.

The method gives the model access to explicit geometric measurements while retaining its ability to plan construction and generate code. Constructing training trajectories through actual tool execution provides a practical way to teach the model to use imperfect geometric feedback. This is a meaningful contribution to tool-assisted CAD reconstruction.

2. Promising results.

QM-CAD improves CD and IoU on both evaluated benchmarks while maintaining a low invalidity ratio. Table 2 includes a direct point-to-code baseline sharing the backbone and training data, as well as a planning-only variant. These comparisons provide evidence that the complete tool-assisted workflow contributes beyond simply using the chosen backbone and training corpus.

Critical weaknesses

See the details below.

Questions for authors

1. The contribution of numerical measurement is not isolated from the other tool feedback.

The central motivation emphasizes grounding dimensions in measurements rather than inferring them from latent features. However, the tools jointly provide measured numbers, and also coordinate-scaled projection images. The w/o tools ablation in Table 2 removes these together. It therefore establishes the value of the entire feedback package, but does not determine how much improvement comes from numerical measurement versus easier visual interpretation. Numerical-only and image-only variants would clarify their respective contributions.

2. Fine-grained geometric attributes of the final dimensions is not evaluated.

The paper does not clearly quantify which parameters required for reconstruction can be directly measured, which must be inferred, or their respective proportions. Moreover, the final reconstruction is assessed using overall CD and IoU. This leaves a gap in the evidence showing whether tool measurements translate into more accurate final dimensions. A useful evaluation would compare fine-grained geometric attributes, such as hole radii and feature centers, against ground truth and the no-tool baseline.

3. Tool accuracy under model-predicted queries remains unclear.

The results in Table 3 are obtained using reference queries derived from the reference programs. At inference, however, the queries are predicted by the model. It remains unclear how much accuracy degrades under these predicted queries.

My primary concerns are 1 and 2. I would raise my score if the authors provide convincing additional evidence addressing them.

2 comments
AuthorsOpenReview ↗

Response to Reviewer pgpL

We thank Reviewer pgpL for the positive assessment of our plan-measure-code workflow and trajectory synthesis pipeline and for the clear, actionable questions. We address each point below and will include the new ablations and parameter-level evaluations in the revised paper.


Response to Q1: Contributions of Numerical Measurements and Projection Images

Following your suggestion, we used the same initialization and training sample order to compare a direct generator (w/o planning and tools) with three SFT variants that differ only in the tool feedback returned after querying. Given the time available, we trained all variants with the SFT stage only and evaluated them on the full DeepCAD test set. Full plan + tools corresponds to w/o RL in Table 2.

SFT Variant (DeepCAD Test)Med. CD (×10³) ↓Mean IoU (%) ↑IR (%) ↓
w/o planning and tools0.18588.20.37
Numerical-only feedback0.17391.30.81
Image-only feedback0.16991.80.76
Full plan + tools (ours, w/o RL)0.16193.70.44

Compared with the variant without planning or tools, the numerical-only and image-only variants improve mean IoU by 3.1 and 3.6 percentage points, respectively. The numerical-only result shows that explicit measurements can support a substantial gain even without projection images. Numerical feedback provides measurements such as extrusion depths and sketch bounds, while projection images show the sketch profile in a coordinate frame with numeric ticks. Combining these complementary sources yields the best reconstruction accuracy, with a mean IoU of 93.7% and a median CD of 0.161.


Response to Q2: Measured vs. Inferred Parameters and Fine-Grained Dimensional Accuracy

1. Proportions of Directly Measured and Inferred Parameters

Direct measurements provide information about both non-sketch parameters and 2D sketch parameters. We report statistics for both categories across the full DeepCAD test set.

  1. Non-sketch parameters. Among matched steps, SGF directly measures 99.9% of workplanes, 90.4% of cylinder and hole radii, 69.8% of box centers, and 46.8% of sketch extrusion depths. In the remaining extrusion steps, multi-step Boolean operations often remove or occlude one end face, leaving insufficient information to measure the depth directly.
  2. 2D sketch parameters. Some sketch vertex coordinates can be obtained directly from the 2D bounding box measured by SGF (uv_bbox). The bounding box determines one coordinate of a vertex along an edge and both coordinates of a vertex at a corner. Among predicted sketch vertices in steps with a measured uv_bbox, 57.1% lie on the bounding box boundary, including 23.2% at its corners. The model infers interior vertex coordinates and curves from projection images.

2. Fine-Grained Geometric Accuracy Relative to Ground Truth

To assess whether tool measurements improve final dimensional accuracy, we extracted cylinder and hole features directly from the final reconstructed 3D B-Reps across the full DeepCAD test set. We evaluated QM-CAD and the trained no-tool baseline against the ground truth.

Method (Full DeepCAD Test)Feature Center & Axis Prec. / Rec. / F1 (≤ 5% LL) ↑Matched Hole / Cylinder Radius [email protected]% ↑Overall GT Hole / Cylinder Radius [email protected]% ↑
No-tool baseline60.3 / 51.3 / 55.5%55.3%28.4%
With tools (ours)66.0 / 61.7 / 63.8%99.0%61.0%

Without tools, the model misses more cylinders and holes and often misestimates their radii. Using measurement tools substantially improves the accuracy of the final reconstructed dimensions.


Response to Q3: Tool Accuracy Under Model-Predicted Queries

Following your suggestion, we evaluated tool accuracy using model-predicted queries. The same CAD solid can be constructed in multiple ways, such as by extruding an annular sketch or subtracting one cylinder from another. On the full DeepCAD test set, we matched model-predicted queries to about 65% of the reference queries. We compare the results for these matched model-predicted queries with those reported for reference queries in Table 3.

Query SourceSeg. IoU ↑Precision ↑Recall ↑Measurement Coverage ↑[email protected]% ↑[email protected]% ↑Acc@1% ↑
Reference queries (Table 3)94.6%96.2%98.1%96.5%90.2%94.4%95.0%
Model-predicted queries (ours)93.4%95.5%96.5%95.8%89.8%93.5%94.1%

These results show that the tools performed reliably on the matched model-predicted queries.

AuthorsOpenReview ↗

Summary of Manuscript Revisions for Reviewer pgpL

Dear Reviewer pgpL,

Following our earlier author response, we have uploaded a revised manuscript PDF incorporating the additional experiments and results addressing your questions (newly added content is highlighted in red, and deleted text is marked in blue with strikethrough):

  • Q1 (Contributions of numerical measurements vs. projection images): Added SFT ablations isolating numerical-only and image-only tool feedback in Section 4.3 and Table 2b.
  • Q2 (Measured vs. inferred parameter proportions and final B-Rep feature accuracy): Added parameter proportion statistics and fine-grained cylinder/hole accuracy against the no-tool baseline in Section 4.3, Table 5a, and Appendix I.3 (Table 17).
  • Q3 (Tool accuracy under model-predicted queries): Added evaluation under matched model-predicted queries in Section 4.3 and Table 3.

We hope these additional experiments and manuscript revisions have addressed your concerns, and we thank you again for your time and constructive feedback!

Reviewer_ZmFCOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The main motivation of this paper is that existing MLLM-based point-to-CAD methods typically encode point clouds into latent features and then use language models to predict both discrete modeling operations and continuous geometric parameters. However, predicting numerical values from latent features inevitably introduces estimation errors, while point clouds already contain explicit 3D coordinates. Therefore, this paper explores a way to directly measure geometric parameters from raw point clouds instead of relying entirely on latent representations. The authors propose query-and-measure CAD (QM-CAD), a tool-augmented framework for reconstructing executable and editable CAD programs from 3D point clouds. The framework consists of three stages: (1) planning: an MLLM breaks down the target object into a sequence of CAD modeling steps and generates geometric queries for each step. (2) measurement: a query-driven point segmenter (QDPS) identifies the relevant surface points, and a statistical geometry fitter (SGF) estimates the geometric parameters and generates 2D projection images. (3) code generation: The MLLM uses the modeling plan, measured parameters, and projection images to generate the final CadQuery program. Experiments on the DeepCAD and Fusion360 benchmarks show that QM-CAD outperforms existing Point-to-CAD methods in terms of geometric reconstruction accuracy.

Critical strengths
  1. The method is well-motivated with practical value. Existing reconstruction methods based on end-to-end transformer predictions often have noticeable errors in numerical fitting. However, these numerical values are actually already embedded in the input model itself. The approach explored in this paper is reasonable and practical.
  2. Query-conditioned segmentation is an interesting task. Although this task is quite common in the 3D field, it has other potential applications in the CAD domain, such as open-vocabulary feature recognition.
Critical weaknesses
  1. The proposed method does not fully support its core motivation when dealing with complex models. The main motivation of this paper is that point clouds already contain explicit 3D coordinates, so MLLMs should not need to regress continuous geometric parameters from latent point tokens, thereby avoiding numerical errors. However, for complex sketch-extrude operations, SGF mainly measures parameters such as bounding extents, extrusion depth, and start/end positions. The detailed structure and coordinates of the sketches are not obtained through explicit geometric fitting. Instead, these parameters still need to be inferred by the MLLM based on the 2D projection images, point tokens, and other context. In particular, the 2D projection provided by the method is an image rather than structured geometric primitives.
  2. Insufficient robustness analysis against point cloud segmentation errors. The accuracy of SGF depends on the quality of the step-wise point masks generated by QDPS. In practice, point cloud segmentation inevitably introduces errors, such as incorrectly including points from unrelated surfaces or missing some points from the target geometric region. These segmentation errors may significantly affect the accuracy of subsequent geometric parameter estimation. Although the paper evaluates geometric measurement accuracy using predicted segmentation masks, it lacks a systematic analysis of how different types and levels of segmentation errors affect the geometric fitting results and the final CAD reconstruction quality.
  3. Limited support for CAD operations. The paper only supports sketch-extrude operations and combines the generated CAD models through Boolean operations. However, the framework does not yet support other common CAD modeling operations such as revolve, fillet, and chamfer.
Questions for authors
  1. Could the authors provide a more detailed analysis of reconstruction performance across different sketch complexities (e.g., grouped by the number of lines/arcs), including errors in vertex coordinates, arc parameters, and sketch profiles?

  2. Could the authors evaluate the robustness of SGF and the overall QM-CAD framework to segmentation errors by artificially perturbing the predicted segmentation masks?

3 comments
AuthorsOpenReview ↗

Response to Reviewer ZmFC (Part 1/2: W1 & Q1)

We thank Reviewer ZmFC for the constructive feedback and for recognizing the practical value of grounding numerical CAD parameters in raw 3D point clouds and the potential of query-conditioned segmentation in CAD. We address each point below and will include the new experiments and analyses in the revised paper.


Response to W1: Numerical vs. Visual Grounding in Sketch-Extrude Operations

We agree that the detailed structure and interior vertex coordinates of sketches are inferred by the model. However, some sketch vertex coordinates can be obtained directly from measurements. SGF measures a 2D bounding box (uv_bbox) for each sketch, which determines one coordinate of a vertex along an edge and both coordinates of a vertex at a corner. Therefore, direct measurements provide information about both non-sketch parameters and 2D sketch parameters. We report statistics for both categories across the full DeepCAD test set.

  1. Non-sketch parameters. Among matched steps, SGF directly measures 99.9% of workplanes, 90.4% of cylinder and hole radii, 69.8% of box centers, and 46.8% of sketch extrusion depths. In the remaining extrusion steps, multi-step Boolean operations often remove or occlude one end face, leaving insufficient information to measure the depth directly.
  2. 2D sketch parameters. Among predicted sketch vertices in steps with a measured uv_bbox, 57.1% lie on the bounding box boundary, including 23.2% at its corners.

Extending SGF to return structured 2D fitted primitives alongside projection images is a promising direction, and we will discuss it in the revised paper.


Response to Q1: Reconstruction Performance and Fine-Grained Errors Across Sketch Complexities

Following your suggestion, we provide a detailed analysis of reconstruction performance across different sketch complexities (grouped by the number of sketch curves, i.e., lines, arcs, and circles), including the mean and median of 3D IoU and the F1 scores of vertex coordinates, circles, and arcs. The matching tolerance is τ=max⁡(0.001L,0.05)\tau = \max(0.001L, 0.05), where LL is the longest axis of the part.

Sketch CurvesCases NNMean 3D IoU (%) ↑Med. 3D IoU (%) ↑Vertex F1 (≤ τ\tau) ↑Circle F1 (≤ τ\tau) ↑Arc F1 (≤ τ\tau) ↑
1–44,18597.199.70.7220.4570.013
5–82,01694.798.90.7800.2630.057
9–161,35892.097.60.6760.1550.037
>1648790.196.20.5770.1220.014

As sketch complexity increases, 3D IoU and F1 scores of vertex/circle/arc decrease gradually, although 3D IoU remains high even for sketches with more than 16 curves (90.1% mean and 96.2% median). Arc F1 is lower across bins because the model often approximates short arcs with line segments, which preserves the overall 3D shape.

AuthorsOpenReview ↗

Response to Reviewer ZmFC (Part 2/2: W2, Q2 & W3)

Response to W2 & Q2: Robustness to Segmentation Mask Perturbations

Following your suggestion, we evaluated the robustness of SGF and the overall QM-CAD framework to segmentation errors by perturbing the predicted segmentation masks at rates of 10%, 30%, and 50% across four modes: random point removal (drop), boundary point removal (erode), contiguous patch removal (block_drop), and adding neighboring non-target points (dilate). On a 1,000-case stratified subset, we evaluated both SGF fitting accuracy ([email protected]%, as in Tables 3 and 14) on matched steps and end-to-end reconstruction quality (Mean IoU and Med. CD, as in Table 1).

Perturbation (10% / 30% / 50%)SGF depth [email protected]% (%) ↑SGF uv_bbox [email protected]% (%) ↑Mean IoU (%) ↑Med. CD (×10³) ↓
Unperturbed77.778.994.00.173
drop77.8 / 77.6 / 77.379.4 / 79.7 / 79.993.8 / 93.9 / 93.90.175 / 0.175 / 0.177
erode77.6 / 77.6 / 77.379.4 / 79.4 / 79.793.7 / 93.7 / 93.80.175 / 0.176 / 0.175
block_drop77.8 / 77.9 / 77.678.6 / 60.7 / 42.893.6 / 92.3 / 89.20.178 / 0.200 / 0.230
dilate78.0 / 78.0 / 78.075.9 / 75.9 / 75.992.5 / 92.0 / 91.90.182 / 0.185 / 0.185

The results show the following.

  1. Under drop and erode, SGF fitting accuracy and reconstruction quality remain nearly unchanged even when 50% of the points are removed.
  2. The dilate perturbation reduces performance by slightly expanding uv_bbox.
  3. Under block_drop, performance changes slightly at 10% removal and drops noticeably at 30%–50% removal. At these higher rates, removing all points from a face or corner can shrink uv_bbox or eliminate the end-face points needed to measure extrusion depth.

Actual QDPS errors are typically small and concentrated near boundaries, so SGF and QM-CAD remain robust in practice.


Response to W3: Limited Support for CAD Modeling Operations (Revolve, Fillet, Chamfer)

We agree that the current method has limited operation coverage, as noted in the Conclusion of the manuscript. Consistent with existing point-to-CAD datasets (DeepCAD, CAD-Recode, and Omni-CAD), our current training data cover only sketch-extrude operations (boxes, cylinders, and general profiles) and Boolean operations.

Nevertheless, the plan-measure-code workflow can extend to additional operations once training data are available. First, step surface masks in Eq. (8) depend only on point-to-B-Rep surface distance, which applies directly to revolved, filleted, and chamfered faces. Second, SGF is modular, allowing dedicated deterministic fitters such as axis and (r,z)(r, z) profile projection for revolve or blend-radius fitting for fillet to be added directly. Third, CadQuery natively supports .revolve(), .fillet(), and .chamfer().

AuthorsOpenReview ↗

Summary of Manuscript Revisions for Reviewer ZmFC

Dear Reviewer ZmFC,

Following our earlier author response, we have uploaded a revised manuscript PDF incorporating the additional experiments, fine-grained analyses, and future-work discussions addressing your concerns (newly added content is highlighted in red, and deleted text is marked in blue with strikethrough):

  • W1 & W3 (Sketch grounding and future extensions): Added statistics on directly measured parameters and uv_bbox-constrained sketch vertices in Section 4.3 and Appendix I.3 (Table 17), and updated Section 5 to discuss structured 2D primitive fitting in SGF and extensions to additional operations including revolves.
  • Q1 (Performance across sketch complexities): Added reconstruction and fine-grained vertex/circle/arc F1 analyses across sketch curve-count groups in Section 4.3 and Appendix I.4 (Table 18).
  • W2 & Q2 (Robustness to segmentation mask perturbations): Added SGF fitting and end-to-end robustness evaluations under four mask perturbation modes in Section 4.3 and Appendix I.5 (Table 19).

We hope these additional results and manuscript revisions have addressed your concerns, and we thank you again for your time and valuable feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.