- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
In this paper, the authors propose QM-CAD, a framework for generating executable CAD programs from point clouds. The main motivation is that existing LLM/MLLM-based methods generally don't have strong priors for point-cloud tokens and end up estimating geometric parameters from latent representations, even though these parameters could be measured directly from the point cloud.
To address this, the authors propose a plan-measure-generate approach. An MLLM first generates a construction plan and queries for the geometric features it wants to measure. A query-driven point segmenter then identifies the relevant points, and a geometry fitter extracts measurements and projection images. These are fed back to the MLLM to generate the final CAD program.
The authors also introduce an automated pipeline for generating training trajectories, which are used for supervised fine-tuning, followed by RL using the IoU of the generated CAD model as the reward. They evaluate QM-CAD on DeepCAD and Fusion360, reporting improvements in reconstruction accuracy over the included baselines. Ablation studies further show that introducing measurement tools improves performance, while RL provides additional gains.
Critical strengths
The paper is well-motivated and addresses a limitation of existing LLM/MLLM-based CAD reconstruction methods. Since point clouds already contain explicit 3D coordinates, estimating geometric parameters from latent representations introduces unnecessary complexity when these parameters can be measured directly. The proposed separation of planning, measurement, and code generation allows the MLLM to focus on construction planning while relying on geometric tools for numerical measurements.
QM-CAD achieves higher reconstruction accuracy than the included baselines on DeepCAD and Fusion360. The ablation study in Table 2 shows that planning alone provides relatively small improvements, whereas introducing measurement tools leads to more substantial gains. This provides evidence that explicit geometric feedback improves CAD reconstruction. The automated pipeline for generating approximately 1.19 million training trajectories is another contribution, providing supervision for learning tool interactions and CAD program generation.
Critical weaknesses
The main contribution of the paper seems to be the learned query-and-measure framework, where an MLLM decides what geometric features should be segmented and measured. However, the experiments do not directly isolate the contribution of this learned query-selection mechanism. While Table 2 shows that introducing measurement tools improves reconstruction accuracy, it does not establish how much of this improvement comes from the MLLM learning what to measure, as opposed to simply having access to additional geometric information. Furthermore, Table 3 evaluates segmentation and measurement accuracy using reference queries derived from ground-truth CAD programs rather than queries generated by the MLLM. As a result, these experiments establish that the measurement tools work when provided with appropriate queries, but do not directly evaluate how well the MLLM learns to generate those queries. A controlled comparison against a fixed or heuristic query-selection approach, while keeping the measurement tools unchanged, would help isolate this contribution.
Another concern is that it is unclear how much of the final CAD program is actually grounded in explicit measurements. For instance, in sketch-extrude operations, the geometry fitter provides measurements such as extents and extrusion depth, along with projection images, but the MLLM still needs to infer parts of the sketch geometry, including vertices and arcs. The paper does not quantify how many of the final CAD parameters are directly measured versus inferred by the model, or how accurate these parameters are individually. Given that the main motivation is to avoid estimating geometric parameters from latent representations, a parameter-level evaluation would help establish this claim. Similarly, while the RL ablation shows improvements in final reconstruction accuracy, it remains unclear whether RL improves query selection or primarily improves code generation. This makes it difficult to determine how much of the reported improvement can be attributed to the paper's main methodological contributions.
Questions for authors
-
Can the authors provide an evaluation of the queries generated by the MLLM itself, rather than using reference queries from ground-truth CAD programs as in Table 3? How accurately does the model identify what needs to be measured, and how does this affect downstream segmentation and measurement accuracy?
-
Can the authors compare the learned query-selection mechanism against a fixed or heuristic approach while keeping the measurement tools unchanged? This would help isolate how much of the improvement comes specifically from learning what to measure.
-
How much of the final CAD program is actually grounded in explicit measurements versus parameters inferred by the MLLM? For instance, in sketch-extrude operations, the model still needs to recover vertices, arcs, and other sketch parameters from projection images. Can the authors provide a breakdown of directly measured versus inferred parameters and their respective reconstruction errors?
-
The authors claim that RL jointly improves planning, querying, and code generation. However, the current ablation only shows improvements in final reconstruction accuracy. Is there evidence that RL improves the quality of generated queries? Would optimizing only the code-generation stage, while keeping the query-generation policy fixed, achieve similar improvements?
4 comments
Response to Reviewer oP7g (Part 2/2: Q3, Q4 & W2)
Response to Q3 & W2: Directly Measured vs. Inferred Parameters and Their Reconstruction Errors
1. Proportions of Directly Measured and Inferred Parameters
Direct measurements provide information about both non-sketch parameters and 2D sketch parameters. We report statistics for both categories across the full DeepCAD test set.
- Non-sketch parameters. Among matched steps, SGF directly measures 99.9% of workplanes, 90.4% of cylinder and hole radii, 69.8% of box centers, and 46.8% of sketch extrusion depths. In the remaining extrusion steps, multi-step Boolean operations often remove or occlude one end face, leaving insufficient information to measure the depth directly.
- 2D sketch parameters. Some sketch vertex coordinates can be obtained directly from the 2D bounding box measured by SGF (
uv_bbox). The bounding box determines one coordinate of a vertex along an edge and both coordinates of a vertex at a corner. Among predicted sketch vertices in steps with a measureduv_bbox, 57.1% lie on the bounding box boundary, including 23.2% at its corners. The model infers interior vertex coordinates and curves from projection images.
2. Reconstruction Errors of Directly Measured vs. Inferred Parameters
On matched steps across the full DeepCAD test set, we compare the reconstruction errors of parameters supported by SGF measurements with those of parameters inferred by the model. We express errors as percentages of the part's longest-axis length and report the median (Med.) and 90th percentile (P90) for each parameter type and group.
| Parameter Type | Measured Med. / P90 Error (% ) ↓ | Inferred Med. / P90 Error (% ) ↓ |
|---|---|---|
| Extrusion depth | 0.0024 / 0.0234 | 0.0312 / 1.8427 |
| Arc endpoint | 0.0002 / 0.4412 | 0.1542 / 0.8317 |
| Sketch vertex | 0.0000 / 0.0009 | 0.0001 / 0.0346 |
| Sketch vertex | 0.0000 / 0.0039 | 0.0001 / 0.0463 |
Across all four parameter types, parameters supported by SGF measurements have lower median and P90 reconstruction errors. For extrusion depth, the P90 error is 0.0234% of for parameters with measurement support and 1.8427% of for inferred parameters.
Response to Q4 & W2: Disentangling RL Gains on Query Generation vs. Code Generation
We evaluated query quality before and after joint RL on the full DeepCAD test set. To assess the contribution of each turn, we paired the SFT and joint-RL checkpoints at inference time while keeping QDPS and SGF fixed.
1. Query and Downstream Segmentation Quality
| Method | Predicted Steps | Exact Step-Count Cases (%) ↑ | Step-Count MAE ↓ | Query Prec. / Rec. / F1 ↑ | Geo. [email protected] ↑ | Matched QDPS Mask IoU (%) ↑ |
|---|---|---|---|---|---|---|
| SFT | 13,165 | 72.4 | 0.464 | 0.640 / 0.657 / 0.649 | 0.400 | 91.8 |
| SFT + RL (ours) | 11,725 | 76.8 | 0.352 | 0.708 / 0.647 / 0.676 | 0.428 | 93.4 |
The table shows that after joint RL, query F1 rises from 0.649 to 0.676 and mask IoU rises from 91.8% to 93.4%.
2. Turn-Wise Policy Swap
| Turn 1 (Plan & Query) | Turn 2 (Code Generation) | Med. CD (×10³) ↓ | Mean IoU (%) ↑ | IR (%) ↓ |
|---|---|---|---|---|
| SFT | SFT | 0.161 | 93.7 | 0.44 |
| SFT | SFT + RL | 0.160 | 94.3 | 0.15 |
| SFT + RL | SFT | 0.158 | 94.9 | 0.37 |
| SFT + RL | SFT + RL | 0.156 | 95.3 | 0.10 |
Comparing rows 1 and 2 shows that switching to the RL checkpoint for code generation reduces the IR from 0.44% to 0.15% and increases mean IoU from 93.7% to 94.3%. Comparing rows 1 and 3 shows that switching to the RL checkpoint for planning and querying yields a larger IoU gain, from 93.7% to 94.9%. Using the RL checkpoint for both turns (row 4) gives the highest mean IoU (95.3%) and the lowest IR (0.10%).
Acknowledging Author Response to my Concerns
Thanks to the authors for the additional experiments. The new comparisons against fixed and heuristic query selection, along with the turn-wise RL ablation, help addresses the questions I raised about whether the learned querying mechanism contributes to the overall improvement. The parameter-level analysis also helps clarify how much of the reconstruction is grounded in measurements.
One remaining question is about the comparison against heuristic RANSAC queries. Since the second-turn code generator was trained using learned queries, could the performance gap partly come from the heuristic queries producing feedback that differs from the training distribution? Also, the authors mention that model-predicted queries match approximately 65% of the reference queries. It would be useful to understand how much of the remaining portion comes from valid alternative CAD decompositions versus incorrect or missing operations.
Overall, the rebuttal addresses most of my original concerns, and I am maintaining my weak acceptance rating.
Response to Reviewer oP7g's Follow-up Questions
We thank Reviewer oP7g for the thoughtful follow-up and for confirming that our additional experiments address your main concerns. We answer both remaining questions below on the same 1,000-case DeepCAD test subset.
1. Heuristic RANSAC Queries: Turn-2 Distribution Shift vs. First-Turn Measurement Loss
We agree that feedback distribution mismatch at the second-turn code generator can account for part of the end-to-end performance gap. To remove the second-turn code generator and its training distribution from the comparison, we directly evaluated the first-turn SGF tool measurements returned under heuristic RANSAC queries and under QM-CAD's learned queries against ground-truth step parameters.
Following the tool-evaluation protocol of Table 3, we report the measured rate and the measured-and-correct rate (error at most 0.5% of in every coordinate) for each parameter type.
| GT Step Parameter | Heuristic RANSAC (Meas. / [email protected]% ↑) | Learned Queries (Meas. / [email protected]% ↑) |
|---|---|---|
| Extrusion depth | 42.1% / 42.1% | 70.1% / 69.7% |
| Extrusion start / end position | 42.1% / 42.1% | 70.1% / 69.8% |
| 2D profile bounds | 34.7% / 33.3% | 49.9% / 45.8% |
Even before reaching the second-turn code generator, heuristic RANSAC queries recover 27.6 percentage points fewer accurate extrusion depths and 12.5 percentage points fewer accurate 2D profile bounds than learned queries. Because RANSAC issues coarse plane-level queries without decomposing multi-step features or selecting suitable primitive fitters, many step-level dimensions are never measured in Turn 1. Thus, while Turn-2 distribution shift may play a secondary role, the primary bottleneck comes from insufficient step-level measurements in the first turn.
2. Breakdown of the Remaining Unmatched Queries
On the 1,000-case subset, our step-matching rule matches 64.9% of reference steps, leaving 564 unmatched reference steps and 424 unmatched model-predicted queries. We audited these unmatched items by ablating steps and comparing the resulting signed 3D volumetric changes, both one-to-one and across multi-step combinations or reorderings.
| Audit Category | Unmatched Reference Steps () | Unmatched Predicted Queries () |
|---|---|---|
| Valid alternative decomposition | 245 (43.4%) | 203 (47.9%) |
| Consistent local 3D geometry (>95%), no 1-to-1 step match | 84 (14.9%) | 61 (14.4%) |
| Reference no-op or empty-profile step | 41 (7.3%) | 7 (1.7%) |
| Partially matched local geometry (50%–95%) | 80 (14.2%) | 73 (17.2%) |
| Incorrect or missing local feature (≤50%) | 66 (11.7%) | 33 (7.8%) |
| Unmatchable for step-level comparison | 48 (8.5%) | 47 (11.1%) |
Valid alternative decompositions (such as step merging, splitting, reordering, or using alternative primitives and axes), steps that still produce consistent local 3D geometry without an isolated one-to-one step match, and empty or zero-change reference steps together account for 65.6% of the unmatched reference steps. By contrast, clear local geometric errors or missing features account for only 11.7% of the unmatched reference steps, and 7.8% of the unmatched predicted queries. These results explain why QM-CAD achieves a mean 3D solid IoU of 95.4% on this subset despite a strict step-by-step query match rate of about 65%.
Manuscript Update. We have also uploaded a revised manuscript PDF incorporating the new experiments and analyses from our earlier response (Section 4.3, Tables 3, 4, and 5b, and Appendix I, Tables 15–17), where newly added content is highlighted in red and deleted text is marked in blue with strikethrough. We hope these clarifications resolve your remaining questions, and we thank you again for the constructive discussion.
Response to Reviewer oP7g (Part 1/2: Q1, Q2 & W1)
We thank Reviewer oP7g for the positive evaluation and for highlighting the motivation of QM-CAD and our 1.19M trajectory synthesis pipeline. We address all four questions below with controlled experiments on the DeepCAD test set and will include these results in the revised paper.
Response to Q1 & W1: Accuracy of Model-Predicted Queries and Downstream Tool Performance
A single 3D CAD solid can be constructed in multiple ways, such as by extruding an annular sketch or subtracting one cylinder from another. We therefore matched model-predicted queries against reference queries on the full DeepCAD test set. The model-predicted queries of QM-CAD match about 65% of the reference queries and achieve 97.7% operation accuracy and 91.4% plane accuracy. We compare the results for these matched model-predicted queries with those reported for reference queries in Table 3.
QDPS segmentation accuracy on model-predicted queries is very close to that on reference queries, and SGF measurements remain accurate across parameter types. These results show that the model reliably identifies what to measure at inference time.
Response to Q2 & W1: Learned Query Selection vs. Fixed or Heuristic Queries
Following your suggestion, we kept the measurement tools (QDPS and SGF) and the second-turn code generator fixed, and replaced only the first-turn queries using the same tool-call format on a 1,000-case DeepCAD test subset. We compared fixed
three-planequeries using global boxes on the XY, YZ, and ZX planes, and heuristicRANSACqueries extracted via multi-plane RANSAC on the point cloud.three-planeRANSACown(ours)With the same measurement tools and code generator, learned queries yield substantially higher reconstruction accuracy than either fixed three-plane queries or heuristic RANSAC queries.