- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies generating an edited CAD program from three dimensioned orthographic drawings and a natural-language editing instruction, without a source CAD program at inference. CAD-D2E covers 15 edit subtypes with executable build123d source/target programs. Its evaluation set contains 2,692 editing examples from 940 source parts and is source-disjoint from training. JORE uses a single autoregressive policy to generate a source program followed by an edited program. CARPO combines GDPO-style reward-component normalization with stage-dependent routing: source geometry credit applies only to Source, while final geometry and format credit cover both stages. With a LoRA-adapted Qwen3.5-9B, JORE reports 87.63% final-program executability and 78.50% mean IoU among successful executions, improving on Joint GRPO by 1.96 percentage points. Validation includes direct-generation and cascade comparisons, component/routing ablations, and source-reward weight sensitivity.
Critical strengths
-
A practically relevant end-to-end task with verifiable supervision. CAD-D2E connects drawing reconstruction and instruction-guided editing without requiring source CAD at inference, allowing reconstruction errors to affect downstream editing. Paired executable source/target programs enable verification of intermediate and final geometry. Source-disjoint evaluation and human inspection of all evaluation examples further strengthen its research value.
-
Targeted controlled ablations. Joint GRPO, RCN-only, and JORE share initialization, data, verifier, and update budget. Table 2 tests normalization and stage assignment, while Table 4 tests all four cross-stage routing configurations. JORE improves final IoU by 1.56 percentage points over RCN-only and 2.58 points over Stage-local, providing useful empirical support for the routing design under this configuration.
Critical weaknesses
1. Evaluation does not yet establish correctness of the requested edit, limiting interpretation of the benchmark and main result. The task requires a local modification while preserving unrelated geometry, but reports only whole-shape volumetric IoU, global surface Chamfer distance, and executability. Small holes, shallow counterbores, or fillets can leave global similarity high when editing is omitted; Figure 6(e)'s 0.25 mm fillet illustrates this risk. Reference-source and predicted-source no-op comparisons and feature/dimension-level editing-success measures are missing. Appendix A.2 (lines 671–672) further restricts geometric metrics to successful executions, potentially comparing different subsets. Using Table 1's rounded aggregates and assigning failures zero IoU gives approximately 68.79% for JORE, 65.56% for Joint GRPO, and 66.11% for Direct-C1 GRPO; JORE remains ahead, so conditional reporting alone does not overturn its ranking. The more consequential issue is unverified edit sensitivity, compounded by unspecified IoU coordinates, alignment, and scale handling. Independently rescaling predictions and references could conceal absolute-size errors. The protocol should explicitly preserve physical dimensions and verify performance by edit type; 78.50% IoU cannot be interpreted as an editing-completion rate.
2. The value and scope of the additional stage routing require stronger empirical support. RCN explicitly follows GDPO, and Appendix A.1 uses a standard causality/score-function principle. The useful contribution is their application and empirical validation in joint reconstruction/editing, rather than a new fundamental gradient identity. In Table 2, CSA-only scores 76.48% versus GRPO's 76.54%; improvement is observed only in combination with RCN, with JORE gaining 1.56 IoU points over RCN-only. This supports the combination in the evaluated configuration, but independent training repetitions, uncertainty estimates, and gradient/training-dynamics evidence are missing, leaving robustness and the proposed reward-dispersion explanation unverified. The seed count is not reported, so it would be inappropriate to assert that all results are single-seed; Appendix B.1's paired weight-comparison intervals do not quantify training randomness. The starred baselines also lack sufficient details on backbones, training data, build123d/command-sequence or CadQuery adaptations, iterative feedback, and budgets. The weak cascade does not control the editor's GT-source/predicted-source distribution mismatch. These omissions limit attribution of the routing gain and broader superiority claims. Source-disjoint testing within a procedural generator also does not yet establish generalization to real engineering drawings.
Questions for authors
-
Editing validation: Please report reference-source no-op IoU/CD distributions, scores from retaining each method's predicted source, and feature/key-dimension correctness by edit subtype. Does JORE's advantage remain in completing the requested change while preserving unchanged geometry, particularly for small fillets, holes/counterbores, and bolt-circle edits, which account for approximately 27% of evaluation examples?
-
Geometry-protocol clarification: Is IoU computed through exact B-rep Boolean volume or approximation? What coordinates, alignment, and scaling are used? Are predictions and references normalized independently, and how are absolute dimensions and dimensional edits preserved? Please report full-test-set IoU with failures assigned zero and geometric tolerance success rates that include failures. A common-success subset is useful as an additional diagnostic, but cannot replace full-set evaluation.
-
Robustness validation: Are the comparisons among JORE, Joint GRPO, RCN-only, and Stage-local stable across independent training repetitions? Please provide means/dispersion and paired intervals clustered by source part, and specify Appendix B.1's resampling unit and failure handling. Group size G=4 and 741 updates alone do not establish that the differences are noise.
-
Baseline settings and error attribution: Please specify the starred methods' versions, backbones, training data/budgets, representation adaptations, prompts/API examples, and feedback/turn budgets, particularly the actual configurations of CAD-Editor, IterCAD, and Ortho2CAD. This asks for clarification of reported comparisons, not upgrades to versions released shortly before submission. Are a predicted-source-trained cascade, a GT-source editing upper bound, or comparisons using a fixed reconstruction available to test the explanation for the weak cascade?
-
Mechanism and theoretical scope: CSA-only shows no observed gain while RCN+CSA does; do training curves or gradient-noise measurements support the proposed explanation? Is Appendix A.1's raw-reward identity intended only as motivation for Equations (5)–(7)? It does not itself establish unbiasedness or guaranteed overall variance reduction for the normalized, length-averaged, clipped update. Please distinguish proved properties from empirical observations.
6 comments
3. Statistical Reliability and Independent Runs (Q3)
Completed paired comparisons. JORE achieves 78.50% Mean IoU. We report each method's mean over its successful executions in the original test set and the corresponding differences. For paired 95% confidence intervals, we jointly resample source parts across methods with replacement, retain all associated test instances, and recompute each method's mean using its own successful-execution mask within each resample.
| Comparator | Mean IoU (%) ↑ | JORE − comparator (pp) | Paired 95% CI (pp) |
|---|---|---|---|
| Joint GRPO | 76.54 | +1.96 | [0.78, 3.15] |
| RCN-only | 76.94 | +1.56 | [0.46, 2.65] |
| Stage-local | 75.92 | +2.58 | [1.47, 4.43] |
All three intervals exclude zero. In particular, the Stage-local comparison supports retaining final-edit feedback for Source tokens beyond conditioning editing on an intermediate reconstruction. These intervals quantify evaluation-sample uncertainty. We will add explicit resampling and failure-handling details to Appendix B.1.
Independent training repetitions. Additional independent runs of JORE, Joint GRPO, RCN-only, and Stage-local with different training seeds are underway under matched training settings and budgets. We will supplement this response with individual-run results and means ± standard deviations.
4. Baseline Implementations and Cascade Controls (Q4)
Implementation settings. Ortho2CAD∗, CAD-Editor∗, and IterCAD∗ are adapted from their official implementations to build123d, using Qwen3.5-9B and fixed task-specific prompts without in-context examples.
Ortho2CAD∗. SFT and Dr. GRPO use 1,028 and 3,161 unique source reconstruction pairs, respectively, with one epoch per stage. Evaluation uses one greedy generation without refinement, allowing 3,072 output tokens within a 6,144-token context.
CAD-Editor∗. The locating and infilling models are trained separately for one epoch on 3,000 editing examples, without RL or selective fine-tuning. Training uses GT sources; evaluation uses predicted sources and predicted masks. Inference consists of two greedy calls without execution feedback or repair, each allowing 3,072 output tokens within a 6,144-token context.
IterCAD∗. Training uses 3,000 multi-turn feedback SFT examples, followed by GSPO with GVPM on 5,932 RL tasks. Evaluation uses one greedy trajectory of up to five turns with execution and rendered-view feedback, allowing 8,192 output tokens per turn within a 32,768-token context.
Source-input controls and joint optimization. To address the source-distribution concern, we train an additional cascade whose editor uses predicted source programs from the SFT stage onward. We also evaluate the original editor with GT sources. The GT-source condition receives additional reference CAD information and serves only as a diagnostic reference. The table reports IoU on successful outputs and full-test-set IoU, with execution failures assigned zero in the latter.
| Setting | IoU on successful outputs (%) ↑ | Full-test-set IoU (%) ↑ | Exec. (%) ↑ |
|---|---|---|---|
| GT-source reference (GT CAD provided) | 87.67 | 81.80 | 93.31 |
| Original Source-GRPO + Edit-GRPO | 57.62 | 39.73 | 68.95 |
| Predicted-source-trained cascade | 54.21 | 39.21 | 72.33 |
| JORE | 78.50 | 68.79 | 87.63 |
JORE's advantage persists after training the cascade editor on predicted sources: it exceeds the adapted cascade by 24.29 percentage points in IoU on successful outputs, 29.58 percentage points in full-test-set IoU, and 15.30 percentage points in executability. Its advantage therefore holds both among successful outputs and when execution failures are included.
The GT-source reference shows that the original editor achieves high geometric accuracy when supplied with accurate source CAD. In the end-to-end setting, adapting the independently trained editor to predicted sources does not close the gap to JORE. These results support the benefit of jointly optimizing reconstruction and editing, complementing the controlled routing ablations in Table 4, which support directing final-edit feedback to reconstruction.
5. Training Dynamics and Theoretical Motivation (Q5)
Training dynamics. We summarize mean raw final-edit reward () over identical consecutive update ranges for all four configurations. Updates 5–741 are used, where each method has one contributing run.
| Method | Updates 5–250 | Updates 251–500 | Updates 501–741 |
|---|---|---|---|
| Joint GRPO | 69.63 | 69.81 | 70.38 |
| RCN-only | 69.10 | 69.61 | 69.66 |
| CSA-only | 68.76 | 69.93 | 69.43 |
| JORE | 70.00 | 70.77 | 71.34 |
JORE obtains the highest mean reward in all three intervals, consistent with the gains from combining RCN and CSA in the final evaluation.
Contribution. Section 4.2 explicitly attributes RCN to GDPO. Our contribution is its integration with dependency-aware credit assignment for reconstruction and editing, evaluated through controlled ablations.
Theoretical motivation and scope. The original manuscript already presents the theoretical argument as motivation for stage assignment. Section 4.2 uses “motivates,” and Appendix A.1 explicitly states that the reward-to-stage dependencies in Equation (11) “motivate the assignment of RCN-normalized component advantages in Equation 5.” The proof establishes that a source reward fixed by the completed Source prefix has zero expected score-function contribution to Edit continuations. The manuscript does not claim unbiasedness or overall variance reduction for the complete normalized, length-averaged, and clipped update. The effectiveness of the resulting training design is assessed empirically through the controlled ablations and training dynamics.
We thank the reviewer for recognizing the practical relevance of the end-to-end task, the verifiable source–target supervision, and the controlled evidence for stage-dependent credit assignment.
1. Validation of Requested CAD Edits (Q1)
No-op comparisons. We compare the reference source, JORE's predicted source, and its final output against the same edited target. The table reports Mean IoU (%) over all test instances, with execution failures assigned zero:
Final generation improves target agreement in all four categories, with an overall gain of 10.09 percentage points over retaining JORE's predicted source. For bolt-circle edits, which account for 26.97% of test examples, the gain is 18.85 percentage points.
The high reference-source no-op IoU for feature addition and removal reflects the large proportion of unchanged geometry. Larger source–target differences in geometry modification and pattern creation provide more room for improvement in global IoU. Since final generation can also correct reconstruction errors, we further examine the requested changes directly.
Local editing evidence. Our General Response to Reviewers defines IoVGT and Error volume. The targeted results below address holes, counterbores, fillets, and bolt-circle edits, with chamfers included as another small-feature operation. Correct reconstruction repairs are not counted as errors. Subtype results are per-instance averages over evaluable predictions within each subtype; the overall row pools evaluable predictions across all 15 subtypes. Unevaluable predictions are excluded from these local-metric averages.
Each entry reports IoVGT (%) ↑ / Error volume (% of GT source) ↓.
JORE improves both metrics over Joint GRPO in all six targeted subtypes. Across all 15 subtypes, it increases coverage by 6.25 percentage points and reduces Error volume by 44.85%, achieving the best aggregate results among the four configurations.
The ablations show why both metrics matter. For chamfers and fillets, CSA-only's low error volume accompanies substantially lower edit coverage, while RCN-only increases coverage over Joint GRPO but introduces more errors. JORE achieves the best aggregate coverage and error volume, with subtype-specific trade-offs against the individual components. These results support the complementary roles of component normalization and stage assignment.
2. Geometry Evaluation and Failure Handling (Q2)
Geometric evaluation. IoU uses B-Rep Boolean intersection and union volumes, rather than voxelization or point-sampling approximations. Both IoU and CD compare geometries in the original global CAD coordinate system, without geometric alignment or rescaling.
Full-test-set evaluation. With execution failures assigned zero, Mean IoU is 68.79% for JORE, 65.56% for Joint GRPO, and 66.11% for Direct-C1 GRPO.
Accuracy on successful executions. [email protected] and [email protected] report the percentage of each method's successfully executed outputs that meet the corresponding IoU threshold. Execution failures are excluded from these denominators.
JORE achieves the highest accuracy at both thresholds among successful executions.