- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies using GRPO post-training to improve the manufacturability of text-to-CAD generation. Instead of using an LLM as judge, the authors implement manufacturing DFM rules for three processes (CNC machining, FDM printing, and injection molding) as deterministic geometric checkers, and use them as part of the GRPO reward. CAD programs in the CADFS dataset are translated into CadQuery code, which serves as the SFT ground truth. The 3D solids obtained by executing these CadQuery code then serve as the reference geometry for the GRPO reward. Experiments are conducted on the original CADFS test set. Curriculum GRPO substantially reduces geometric violations over the SFT model, while keeping fidelity metrics largely unchanged. In comparison with open baselines, the proposed method clearly outperforms Text-to-CadQuery, CAD-Coder, and CADmium in both validity and DFM.
Critical strengths
The design of the reward mechanism is interesting and reasonable: gating the DFM term on faithfulness so that only the more faithful samples earns more manufacturability credit is an elegant safeguard against trading geometric fidelity for DFM score.
Critical weaknesses
The paper claims that rule-based DFM scoring is preferable to "using LLM as a judge" for training CAD generators, but does not support it by experiment. There is no experiment comparing GRPO with the proposed DFM reward against GRPO with an LLM (or VLM) judge reward. Without this comparison, the main contribution reduces to one untested design choice.
Questions for authors
How is the difficulty proxy trained? What are the input features and the regression target?
Are there ablations isolating the gain of key GRPO design choices (e.g., token-level loss, cyclic curriculum)?
2 comments
Response (2/2)
Q1. How is the difficulty proxy trained?
"How is the difficulty proxy trained? What are the input features and the regression target?"
The difficulty proxy is fitted offline using L2-regularized logistic regression with continuous targets. Its inputs are and 13 binary indicators of operation types in the reference-program metadata.
The target measures the SFT model's reconstruction miss under greedy decoding. It is 1 for an invalid reconstruction and, for a valid reconstruction, is
The floor accounts for same-part sampling differences. The fitted proxy remains fixed during GRPO and is used to order and sample training prompts; it is not an additional reward term. We do not claim that replacing it with construction-step count would produce equivalent training results.
Q2. Ablations isolating key GRPO design choices
"Are there ablations isolating the gain of key GRPO design choices (e.g., token-level loss, cyclic curriculum)?"
Token-level loss. Token-level loss aggregation follows DAPO (Yu et al., 2025). We have not performed a dedicated token-level versus sequence-level aggregation ablation, so we do not attribute a separately measured gain to this choice.
Cyclic curriculum. For sampling, the tested regular-sampling run deteriorates around step 240, whereas the cyclic configuration completes 3,000 steps. This documents a stability difference between the tested configurations.
Ablation results on the full CADFS test set (Table 3):
| Run | Validity ↑ | DFM ↑ | DFM (faithful) ↑ | Faithful ↑ |
|---|---|---|---|---|
| SFT | .786 | .678 | .589 | .716 |
| GRPO, regular sampling | collapsed from step $\sim$240, generating garbage completions | – | – | – |
| Curriculum GRPO, full reward | .850 | .729 | .634 | .776 |
| Curriculum GRPO, no DFM term () | .834 | .716 | .655 | .761 |
| Curriculum GRPO, no discount | .833 | .717 | .620 | .756 |
Reward terms. The table also reports conditions without the DFM term and without the fidelity discount. Relative to the no-discount condition, the full method has higher DFM (0.729 versus 0.717) and DFM on faithful outputs (0.634 versus 0.620). The no-DFM condition has lower overall DFM (0.716), but higher DFM on faithful outputs (0.655). We therefore do not claim that the full method dominates every ablation on every metric. These results characterize the tested configurations, while a fully controlled isolation of every GRPO design choice remains outside the present experiments.
References
- Yu et al. (2025). DAPO: An Open-Source LLM Reinforcement Learning System at Scale. arXiv:2503.14476.
Response (1/2)
We thank the reviewer for the thoughtful feedback. Table, figure, and section numbers refer to the revised manuscript.
W1. No comparison of GRPO with the DFM reward against GRPO with an LLM/VLM-judge reward
We agree that our experiments do not establish that geometric-reward GRPO outperforms judge-reward GRPO. We have revised the manuscript to distinguish this untested training comparison from the practical choice of evaluator for our explicit geometric rules. The new appendix, "Static Evaluation of LLM Judges for DFM" (Appendix D), reports an offline comparison of coverage, analytical accuracy, repeatability, and evaluation latency; the main results are below.
Controlled solids (Table 14). GPT-6.1 Terra performs well when given CAD code and explicit rule definitions. On all 96 geometry–process cases with independent analytical severity references, the code conditions provide every target estimate, with per-rule MAE below 5×10−5. Views alone yield only 12/144 complete process scores, while repeated code-only judgments vary by more than 0.01 in 5/71 cases.
Saved GRPO outputs (Table 15). Across 96 saved GRPO completions from six groups at one training step, all judge responses are structurally valid, but only 76 provide complete scores; one molding group has wall-uniformity abstentions on all 16 completions. Using this judge as the sole source of the DFM term would therefore require a validated procedure for handling missing estimates.
Evaluation latency (Table 16). On the same 96 completions, geometric scoring takes 40.9 ms on average, compared with 22.73 s for archived code-and-views judge responses. These are observed evaluation latencies for local computation and a remote service, not training-speedup measurements.
Model dependence. Reliability depends on model choice under the tested rubric: with identical code-only requests, Qwen3-Coder-480B-A35B-Instruct provides 77/144 complete scores, versus 133/144 for Terra (Appendix D).
These static results motivate our practical choice of evaluator; they do not establish superior policy performance over judge-reward GRPO or demonstrate reward hacking.