- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies program-based geometric construction from natural-language descriptions. It introduces PyGeoX, a geometric constraint-solving and verification framework, together with training data and evaluation benchmarks. It proposes Saturating Additive Rewards (SAR), combined with a success bonus and degeneracy penalty, for reward-weighted supervised fine-tuning and reinforcement learning.
Critical strengths
- The paper provides theoretical analysis of reward design for geometric constraint satisfaction.
- The authors conduct extensive experiments comparing reward functions across different training methods and geometric construction benchmarks.
Critical weaknesses
- Limited novelty. GeoLoom and MagicGeo address similar geometric construction tasks. The manuscript does not establish substantive novelty beyond differences in input/output format and solver access.
- Insufficient experimental settings and comparisons. The paper lacks comparisons with GeoLoom, MagicGeo, and other related methods. And the experiments do not isolate the effects of reward aggregation, residual transformation, temperature, or the success and degeneracy terms.
- Unclear training failure mechanism. The paper assumes that near-zero global rewards eliminate learning signals, but GRPO can still learn from differences between small rewards through group normalization. The link between near-zero rewards and training failure remains unclear.
- Incomplete reproducibility details. The total RL update count for Qwen3-8B, the adjusted SFT epochs for each reward variant, and the reward normalization used to obtain SFT sample weights require clarification. 5.Organization and presentation need improvement. The manuscript needs clearer organization, more polished writing, and improved figures.
Questions for authors
- Verifier correctness. Please explain why the isosceles trapezoid example in Appendix C.2 receives a reward of 10 despite violating the stated geometric constraints. Clarify whether the error lies in the example or the verifier and whether it affects the reported solving rates.
- Reward design. Please isolate the effects of aggregation, temperature, and the use of residuals versus squared residuals in the exponential kernel. Evaluate the sparse success bonus and degeneracy penalty separately.
- Training signal. Please clarify how near-zero global-norm rewards weaken the learning signal under the group_norm advantage estimator. Extend the analysis of non-zero GRPO advantages in Appendix C.8 to the main Qwen3-8B experiments and include an analysis of policy gradients.
3 comments
Q3 (continued): how near-zero global-norm rewards weaken the signal under group_norm
The reviewer asked us to extend the C.8 analysis of non-zero GRPO advantages to the main Qwen3-8B runs. That statistic needs the reward of every rollout in every group, which was not logged for these runs. The logs hold only the batch-average within-group reward std. It is always positive, so the signal does not vanish under group_norm. The global-norm reward weakens it in a different way: as the example at the end of W3 / Q3 shows, it can fail to give the better answer the higher reward, and group_norm keeps that ordering when it computes the advantages.
We tested the ranking directly on the stored Qwen3-8B answers to the 300 original PyGeoX-Bench problems. Among partially correct answers, a higher global-norm reward does not reliably mean more constraints satisfied:
Correlation of reward with the share of constraints satisfied: 0.38 for the global norm , 0.75 for SAR. Between two answers to the same problem, the one satisfying more constraints gets the higher global-norm reward in only 60% of cases (88% for SAR).
Simply counting satisfied constraints is not enough either, because it does not distinguish a constraint missed narrowly from one missed badly. 31.7% of the answer pairs satisfy the same number of constraints and would get identical reward, while SAR still separates 43% of these by how close the violated constraints are.
Follow-up before discussion closes
Dear Reviewer 1Ke1, as the discussion period closes today, we would be grateful to know whether our response and the revision summary above address your concerns, or whether anything remains we could clarify.
Response to Reviewer 1Ke1
Thank you for the detailed review.
Q1 You are right, and the bug was in the verifier. The Appendix C.2 construction is a parallelogram (diagonals 2 and 22). PyGeoX defined an isosceles trapezoid by AB∥CD and ∣BC∣=∣DA∣, which every parallelogram satisfies. We now also require ∣AC∣=∣BD∣. The C.2 output now scores 5.4 instead of 10. The constraint appears in 33 of 450 PyGeoX-Bench problems and in no Wild, CAD or Mech problem, and 0 to 4 outputs per model had been falsely accepted. No solving rate changes by more than 0.9 points, the model ranking is unchanged, and the OOD results are unaffected.
W1 We do not claim the input/output signature as the contribution. Geometric constraint solving has existed for decades, and pairing an LLM with a solver is established practice (Section 2). GeoLoom (ICML 2026, now cited) takes the same route. The novelty lies in the method, and in showing that it can be done. In both, the LLM writes a formal specification and a solver computes the coordinates. MagicGeo is training-free, and GeoLoom trains only the translator. In our work the LLM computes the coordinates itself, trained by RL on per-constraint residuals. MagicGeo itself notes that "directly using LLMs to solve for point coordinates leads to errors and hallucinations." That is the setting we study, and making it work is the contribution. We built the engine, the training data, the benchmarks and the agent harness, and provide theory for the reward design.
The result is geometric synthesis without the limits of a solver. A translate-then-solve system can only produce what its formal language can express, and it fails whenever translation errs. Our model writes a Python program, so it can construct anything that can be written as code. PyGeoX also goes beyond GeoLingua's shapes, lengths and angles, with equalities and inequalities over arbitrary algebraic expressions across 35 object types and 38 relationships.
W2 / Q2 We did not run GeoLoom or MagicGeo because (1) our agent harness (Table 2) already implements translate-then-solve on all four suites, (2) their languages do not cover the algebraic, transformation, congruence and arc constraints our suites test, and (3) they output rendered diagrams, not coordinates checked against every constraint.
The paper makes one claim: bounded per-constraint aggregation gives a better training signal than a global norm. The theory establishes it, and the experiments (22 training runs over four base models and two regimes) show that masking occurs in practice and that removing it gives real gains on real problems. Existing controls or theory cover the other factors:
The requested factorial needs ≥48 more RL runs per seed ($\sim$4 days each on 8×H200, $>$6 months).
W3 / Q3 You are right that group_norm rescales small rewards, and the sentence in Section 3.1 saying a near-zero reward gives a near-zero gradient has been removed. Masking survives for another reason (new Proposition A.5, derivation in our response to Reviewer Q4aB, W1 / Q1). Within a group, normalization multiplies every gap Rk−Rj by the same 1/σ, so it changes the scale of the gradient but not the relative weights of the rollouts. Under exp(−SSE/T), an improvement on constraint i by a rollout with error S on the other constraints counts exp(−S/T) times less than the same improvement by a rollout with S=0. Normalization does not remove this factor, so an outlier still masks progress elsewhere. SAR has no such factor. Normalization is also monotone, so GRPO keeps the ranking the reward assigns. With 16 constraints, rollout A satisfies 15 and misses one by 2.0, while rollout B misses all 16 by 0.4. The global norm scores A below B (0.670 vs. 0.774), and SAR scores A above B (0.938 vs. 0.018).
W4 RL: GRPO, 8 rollouts/prompt, lr 5×10−6 with cosine decay, prompt batch 144, max 8,192 tokens, trained for 69 steps (9,936 problems / 79,488 rollouts). SFT: lr 10−5, effective batch 512, LoRA rank 8, max 8,196 tokens, 2 epochs for dense variants after dropping traces with R≤2 from the 15,666-trace pool, and 4 epochs for Sparse (which keeps only fully correct traces) to match total token volume and optimizer steps, selected by validation loss. SFT weights: R∈[−4,10], drop R≤2, remaining traces weighted by R/10.
W5. Framework now precedes rewards. We will redraw the figures and polish the writing in the revision.