acceptodds
Under review as a conference paper at ICLR 2027

Internalizing Geometric Law: Learning from Solver Residuals for Precision-Critical Generation

Abstract

Large Language Models frequently hallucinate in precision-critical domains such as technical diagramming and mechanical design, where outputs must satisfy strict geometric constraints. We study this failure where it can be checked exactly: open-ended geometric synthesis from natural language, in which an LLM must turn a free-form description into a precise geometric construction whose objects simultaneously satisfy dozens of interacting constraints. To make it trainable we release PyGeoX, a programmable geometric DSL that compiles declarative constraints into a differentiable loss, an agent harness that exposes PyGeoX to any LLM, and a 900-problem evaluation suite with per-constraint verifiable rewards spanning procedurally generated, examination and engineering problems. Using PyGeoX as a verifier, we identify a failure mode we call Outlier Gradient Masking: when residuals are aggregated through a single norm, for example , one outlier constraint can nullify the learning signal from all the others. To address this we propose Saturating Additive Rewards (SAR), which decompose the reward into bounded per-constraint terms so that progress on satisfied constraints survives. Against MSE-based rewards, the natural baseline for geometry solvers, SAR improves the hard-tier solving rate by , and the resulting 8B model is competitive with much larger frontier systems.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Geometric Reasoning, Neuro-Symbolic Learning, Reinforcement Learning, Saturating Additive Rewards, Constraint Satisfaction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1Ke1OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies program-based geometric construction from natural-language descriptions. It introduces PyGeoX, a geometric constraint-solving and verification framework, together with training data and evaluation benchmarks. It proposes Saturating Additive Rewards (SAR), combined with a success bonus and degeneracy penalty, for reward-weighted supervised fine-tuning and reinforcement learning.

Critical strengths
  1. The paper provides theoretical analysis of reward design for geometric constraint satisfaction.
  2. The authors conduct extensive experiments comparing reward functions across different training methods and geometric construction benchmarks.
Critical weaknesses
  1. Limited novelty. GeoLoom and MagicGeo address similar geometric construction tasks. The manuscript does not establish substantive novelty beyond differences in input/output format and solver access.
  2. Insufficient experimental settings and comparisons. The paper lacks comparisons with GeoLoom, MagicGeo, and other related methods. And the experiments do not isolate the effects of reward aggregation, residual transformation, temperature, or the success and degeneracy terms.
  3. Unclear training failure mechanism. The paper assumes that near-zero global rewards eliminate learning signals, but GRPO can still learn from differences between small rewards through group normalization. The link between near-zero rewards and training failure remains unclear.
  4. Incomplete reproducibility details. The total RL update count for Qwen3-8B, the adjusted SFT epochs for each reward variant, and the reward normalization used to obtain SFT sample weights require clarification. 5.Organization and presentation need improvement. The manuscript needs clearer organization, more polished writing, and improved figures.
Questions for authors
  1. Verifier correctness. Please explain why the isosceles trapezoid example in Appendix C.2 receives a reward of 10 despite violating the stated geometric constraints. Clarify whether the error lies in the example or the verifier and whether it affects the reported solving rates.
  2. Reward design. Please isolate the effects of aggregation, temperature, and the use of residuals versus squared residuals in the exponential kernel. Evaluate the sparse success bonus and degeneracy penalty separately.
  3. Training signal. Please clarify how near-zero global-norm rewards weaken the learning signal under the group_norm advantage estimator. Extend the analysis of non-zero GRPO advantages in Appendix C.8 to the main Qwen3-8B experiments and include an analysis of policy gradients.
3 comments
AuthorsOpenReview ↗

Response to Reviewer 1Ke1

Thank you for the detailed review.

Q1 You are right, and the bug was in the verifier. The Appendix C.2 construction is a parallelogram (diagonals 22 and 222\sqrt{2}). PyGeoX defined an isosceles trapezoid by AB∥CDAB \parallel CD and ∣BC∣=∣DA∣|BC| = |DA|, which every parallelogram satisfies. We now also require ∣AC∣=∣BD∣|AC| = |BD|. The C.2 output now scores 5.4 instead of 10. The constraint appears in 33 of 450 PyGeoX-Bench problems and in no Wild, CAD or Mech problem, and 0 to 4 outputs per model had been falsely accepted. No solving rate changes by more than 0.9 points, the model ranking is unchanged, and the OOD results are unaffected.

W1 We do not claim the input/output signature as the contribution. Geometric constraint solving has existed for decades, and pairing an LLM with a solver is established practice (Section 2). GeoLoom (ICML 2026, now cited) takes the same route. The novelty lies in the method, and in showing that it can be done. In both, the LLM writes a formal specification and a solver computes the coordinates. MagicGeo is training-free, and GeoLoom trains only the translator. In our work the LLM computes the coordinates itself, trained by RL on per-constraint residuals. MagicGeo itself notes that "directly using LLMs to solve for point coordinates leads to errors and hallucinations." That is the setting we study, and making it work is the contribution. We built the engine, the training data, the benchmarks and the agent harness, and provide theory for the reward design.

The result is geometric synthesis without the limits of a solver. A translate-then-solve system can only produce what its formal language can express, and it fails whenever translation errs. Our model writes a Python program, so it can construct anything that can be written as code. PyGeoX also goes beyond GeoLingua's shapes, lengths and angles, with equalities and inequalities over arbitrary algebraic expressions across 35 object types and 38 relationships.

W2 / Q2 We did not run GeoLoom or MagicGeo because (1) our agent harness (Table 2) already implements translate-then-solve on all four suites, (2) their languages do not cover the algebraic, transformation, congruence and arc constraints our suites test, and (3) they output rendered diagrams, not coordinates checked against every constraint.

The paper makes one claim: bounded per-constraint aggregation gives a better training signal than a global norm. The theory establishes it, and the experiments (22 training runs over four base models and two regimes) show that masking occurs in practice and that removing it gives real gains on real problems. Existing controls or theory cover the other factors:

  • Aggregation: per-constraint wins both SAR vs. MSE and SAR+S+D vs. MSE+S+D (RL Hard: 0.200 vs. 0.080, 0.360 vs. 0.153), with MSE's TT tuned in its favor.
  • Kernel and temperature: Theorems A.1, A.3 and Prop. A.5 hold for any bounded decreasing kernel and ℓp\ell_p norm. C.8 trains the T→0T\to0 limit (PCT+S+D).
  • SS and DD: existing practice (Tao et al., 2025). They lift RL Hard from 0.200 to 0.360 (SAR) and 0.080 to 0.153 (MSE). Sparse (SS alone): 0.260.

The requested factorial needs ≥48\ge 48 more RL runs per seed ($\sim$4 days each on 8×H200, $>$6 months).

W3 / Q3 You are right that group_norm rescales small rewards, and the sentence in Section 3.1 saying a near-zero reward gives a near-zero gradient has been removed. Masking survives for another reason (new Proposition A.5, derivation in our response to Reviewer Q4aB, W1 / Q1). Within a group, normalization multiplies every gap Rk−RjR_k-R_j by the same 1/σ1/\sigma, so it changes the scale of the gradient but not the relative weights of the rollouts. Under exp⁡(−SSE/T)\exp(-\mathrm{SSE}/T), an improvement on constraint ii by a rollout with error SS on the other constraints counts exp⁡(−S/T)\exp(-S/T) times less than the same improvement by a rollout with S=0S=0. Normalization does not remove this factor, so an outlier still masks progress elsewhere. SAR has no such factor. Normalization is also monotone, so GRPO keeps the ranking the reward assigns. With 16 constraints, rollout A satisfies 15 and misses one by 2.0, while rollout B misses all 16 by 0.4. The global norm scores A below B (0.670 vs. 0.774), and SAR scores A above B (0.938 vs. 0.018).

W4 RL: GRPO, 8 rollouts/prompt, lr 5×10−65\times 10^{-6} with cosine decay, prompt batch 144, max 8,192 tokens, trained for 69 steps (9,936 problems / 79,488 rollouts). SFT: lr 10−510^{-5}, effective batch 512, LoRA rank 8, max 8,196 tokens, 2 epochs for dense variants after dropping traces with R≤2R \le 2 from the 15,666-trace pool, and 4 epochs for Sparse (which keeps only fully correct traces) to match total token volume and optimizer steps, selected by validation loss. SFT weights: R∈[−4,10]R \in [-4, 10], drop R≤2R \le 2, remaining traces weighted by R/10R/10.

W5. Framework now precedes rewards. We will redraw the figures and polish the writing in the revision.

AuthorsOpenReview ↗

Q3 (continued): how near-zero global-norm rewards weaken the signal under group_norm

The reviewer asked us to extend the C.8 analysis of non-zero GRPO advantages to the main Qwen3-8B runs. That statistic needs the reward of every rollout in every group, which was not logged for these runs. The logs hold only the batch-average within-group reward std. It is always positive, so the signal does not vanish under group_norm. The global-norm reward weakens it in a different way: as the example at the end of W3 / Q3 shows, it can fail to give the better answer the higher reward, and group_norm keeps that ordering when it computes the advantages.

We tested the ranking directly on the stored Qwen3-8B answers to the 300 original PyGeoX-Bench problems. Among partially correct answers, a higher global-norm reward does not reliably mean more constraints satisfied:

Correlation of reward with the share of constraints satisfied: 0.38 for the global norm exp⁡(−SSE/10)\exp(-\mathrm{SSE}/10), 0.75 for SAR. Between two answers to the same problem, the one satisfying more constraints gets the higher global-norm reward in only 60% of cases (88% for SAR).

Simply counting satisfied constraints is not enough either, because it does not distinguish a constraint missed narrowly from one missed badly. 31.7% of the answer pairs satisfy the same number of constraints and would get identical reward, while SAR still separates 43% of these by how close the violated constraints are.

AuthorsOpenReview ↗

Follow-up before discussion closes

Dear Reviewer 1Ke1, as the discussion period closes today, we would be grateful to know whether our response and the revision summary above address your concerns, or whether anything remains we could clarify.

Reviewer_yNx1OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies LLM hallucinations in precision-critical domains and frames GCS as an LLM alignment task. It introduces PyGeoX, a programmable geometric DSL that compiles declarative constraints into a differentiable loss, along with an agent harness that exposes PyGeoX to LLMs and a 900-problem evaluation suite with per-constraint verifiable rewards. Using PyGeoX as a verifier, the paper identifies a failure mode called outliner gradient masking and proposes saturating additive rewards (SAR) to address it.

Critical strengths
  • The paper frames GCS as an LLM alignment task to address precision-related hallucinations in CAD and mechanical synthesis, which is an under-explored problem.
  • The PyGeoX engine, a programmable geometric environment, and corresponding procedural data pipeline are released.
  • The paper identifies a failure mode called outliner gradient masking and proposes saturating additive rewards (SAR) to address it.
Critical weaknesses
  • The central claim that geometric constraint solving constitutes a new alignment problem is insufficiently justified. The setup is essentially an RLVR (reinforcement learning with verifiable reward) task on continuous geometric constraints. The paper does not clearly distinguish this formulation from existing verifiable reward work in mathematics and code. It conflates a new RL task instance with a novel alignment paradigm.
  • The paper focuses on deploying generative models in engineering domains, including technical diagramming, CAD, and kinematic mechanism design. However, PyGeoX appears to support only 2D geometry; its applicability to 3D tasks such as CAD generation is unclear.
  • The writing could be improved. For example, the ordering of Sections 3 and 4 is counterintuitive: the reward design is presented before the overall PyGeoX-RL framework. Weak logical transitions also make it difficult to follow the connections between claims.
Questions for authors
  • How does the notion of “constraint alignment” in this paper differ fundamentally from standard RLVR applied to continuous objectives?
  • Have you designed probing experiments to verify that the model truly internalizes geometric constraints, rather than learning reward hacking patterns that produce locally low residuals without consistent geometric understanding?
2 comments
AuthorsOpenReview ↗

Response to Reviewer yNx1

We thank the reviewer for the clear assessment. We have revised the paper accordingly.

W1 / Q1: "A new alignment problem" vs. RLVR

We agree with the first half of this point and disagree with the second.

Yes, this is RLVR, and the paper says so. The introduction places the work among "Standard Reinforcement Learning with Verifiable Rewards (RLVR) aggregators", Section 2 compares it with RLVR approaches in mathematics and code, and the conclusion states that the reward applies "to any RLVR setting where a solver returns a multi-dimensional residual". We used the word "alignment" for aligning a model to a verifier. We see that it reads as something larger, and we have replaced it with "RLVR with a continuous, multi-constraint verifier" throughout.

The paper does not claim a new alignment paradigm. We claim a new task and a better way to train on it. Most RL research is "just RL" on a particular problem in this sense, and its contribution lies in what had to be built and understood to make RL work there. That is the case here.

The distinction from RLVR in mathematics and code is stated in the paper, and it is specific. Section 2 notes that those approaches rely on binary outcome supervision (r ∈ {0, 1}): an answer matches or a test passes. Our verifier returns a vector of real-valued residuals, one per constraint. A scalar reward then has to aggregate that vector, and the paper is about that step: a single global norm lets one badly violated constraint dominate, and a bounded per-constraint sum does not. This question does not arise when the verifier returns one bit.

What is new is that RL was done here at all, and what it took.

  • The environment. To our knowledge, no prior package exposes per-constraint residuals at this precision, across 35 object types and 38 relationships, as a reward for training a language model. PyGeoX was built for that purpose.
  • The reward design, with theory explaining why per-constraint aggregation preserves the training signal.
  • The data, benchmarks and tools: about 100k validated training problems, a 900-problem evaluation suite, and an agent harness.

To answer Q1 directly: there is no difference in the learning formulation, and we do not claim one. The difference is a verifier with structured, continuous output, the reward-design problem this creates, and the infrastructure that makes training possible.

W2: 2D only

Yes. This is a stated limitation of PyGeoX (Section 6). It is a limitation of the current implementation and not of the approach. The reward design and the theory work on a vector of constraint residuals and do not depend on dimension, and the engine extends to 3D by adding 3D primitives. That extension was not done in time for this submission.

W3: Organisation

We have reordered the paper as suggested: the framework (task, PyGeoX, data) now comes before reward design, so the reward section can refer to a verifier the reader has already seen. We also added a short roadmap at the end of the introduction and bridging paragraphs between the framework, reward and experiment sections.

Q2: Internalisation vs. reward hacking

We address this in four ways.

1. The model constructs its answers. It does not hand them to a solver. Section 6 reports that about 90% of successful Hard-tier traces follow ruler-and-compass style derivations. We have now measured this on every evaluation problem. Of the 900 programs written by the RL SAR+S+D model, 16 (1.8%) call a numerical optimiser or a symbolic solver. The other 98.2% compute the coordinates step by step, using libraries only for basic arithmetic and trigonometry. On PyGeoX-Bench, none of its solved answers uses an optimiser or solver.

2. Trivial solutions earn nothing. The usual shortcut to low residuals is to collapse points or shapes onto each other. The degeneracy penalty removes the reward for this, and an answer counts as solved only if it is non-degenerate.

3. The problem space is too large for a shortcut to pay off. A Hard-tier problem combines 3 polygons from about 15 subtypes, about 23 objects, and 16 constraints drawn from 38 relationship types. That gives more than 10^17 distinct configurations, against about 10k training problems (Section 6). Every problem poses a different constraint system and each constraint is checked on its own, so no fixed output pattern can score well across problems. A model can only raise its reward by satisfying the constraints of the problem in front of it.

4. The gains hold on problems the model never saw. RL trained on medium problems only. Constraint satisfaction still rises from 63.4% to 76.8% on Bench-Hard (p < 0.0001) and from 82.1% to 88.7% on the human-written PyGeoX-Wild (p = 0.022).

AuthorsOpenReview ↗

Follow-up before discussion closes

Dear Reviewer yNx1, as the discussion period closes today, we would be grateful to know whether our response and the revision summary above address your concerns, or whether anything remains we could clarify.

Reviewer_Q4aBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents PyGeoX-RL, a framework for training LLMs to generate precise geometric constructions from natural-language descriptions. The authors introduce PyGeoX, a programmable geometry engine that converts geometric constraints into verifiable residuals, together with a 900-problem benchmark and a procedural training pipeline. The central methodological contribution is Saturating Additive Rewards, which aggregates bounded per-constraint rewards to preserve partial progress during reinforcement learning. Experiments using Qwen3-8B demonstrate improved geometric solving rates over sparse and global-norm rewards, with additional evaluations on out-of-distribution geometry and engineering tasks.

Critical strengths
  • Meaningful problem formulation and benchmark construction. The paper addresses precision-critical geometric generation, where correctness can be objectively verified through geometric constraints. PyGeoX provides a useful environment for studying this problem, and the combination of procedurally generated and independently constructed evaluation sets supports systematic analysis.

P- ractical reward design with substantial empirical gains. SAR exploits per-constraint residual information to provide fine-grained feedback during training. The controlled RL ablations show meaningful improvements on difficult geometric problems, particularly when combined with sparse success and degeneracy terms. The comparison between reward informativeness and downstream solving performance provides useful evidence for the proposed mechanism.

  • Comprehensive evaluation of training and inference strategies. The paper evaluates both SFT and RL variants, reports statistical comparisons, and contrasts direct generation with solver-assisted agent pipelines. The additional analysis of formalization errors and solver dependence provides valuable insight into the limitations of geometric reasoning systems.
Critical weaknesses
  • The theoretical analysis does not fully characterize the implemented learning mechanism. Theorem A.1 establishes reward-volume concentration for an unnormalized sum of per-constraint rewards, whereas the implemented SAR objective averages over constraints. The stated asymptotic conclusion does not generally hold under this normalization. Furthermore, Theorem A.3 analyzes reward derivatives with respect to constraint residuals, which do not directly characterize policy gradients under GRPO. The connection between the theoretical results and the claimed prevention of policy-gradient masking therefore remains incomplete.

  • The methodological novelty and reward comparisons need stronger justification. Combining dense partial-credit feedback with sparse correctness rewards is closely related to existing reward-shaping approaches. While SAR is effective, the experiments primarily contrast it with sparse rewards and exponentially transformed global MSE. Comparisons with normalized per-constraint satisfaction rates, clipped residual penalties, and matched-kernel additive rewards are needed to establish whether the proposed formulation offers benefits beyond standard constraint-wise reward decomposition.

  • The evidence for internalizing geometric reasoning remains limited. The generated programs can invoke external numerical and symbolic libraries, making it difficult to distinguish learned geometric reasoning from improved tool-assisted computation. Moreover, the primary training results use a single seed and one 8B model, while the reward advantage is less consistent on the CAD and mechanism benchmarks. The experiments demonstrate improved task performance but do not fully establish generalizable internalization of geometric principles.

Questions for authors
  • How does Theorem A.1 extend to the normalized SAR reward used in training? How do residual-level derivatives translate into effective policy-gradient signals under GRPO?

  • Were simpler alternatives such as the fraction of satisfied constraints, clipped residual rewards, or additive squared-error kernels evaluated under the same training budget?

  • Can the authors distinguish improvements in internal geometric reasoning from increased reliance on numerical optimization or symbolic computation in generated programs?

  • Can the authors provide multi-seed training results and explain why SAR's advantage over other reward designs is less consistent on the CAD and mechanism benchmarks?

2 comments

Thank you for the constructive review.

W1 / Q1: Theory

We restated and reproved both theorems. Their main conclusions hold in the revised version.

Theorem A.1. The proof was written for the sum, but Eq. 2 uses the average. Let μ=E[ϕ(r)]\mu=\mathbb{E}[\phi(r)] for rr uniform on [0,M][0,M]. By the law of large numbers the averaged SAR reward converges to μ\mu, so its reward volume tends to 1 for every ϵ<μ\epsilon<\mu, while the global-norm volume tends to 0 for every ϵ>0\epsilon>0. The theorem is restated with this condition and a new proof.

Theorem A.3. The conclusion holds under GRPO: an outlier still masks progress on the other constraints, not through a near-zero reward but through the relative weight each rollout gets. New Proposition A.5 shows this. The policy gradient (before clipping, KL) is

1G2σ∑k<j(Rk−Rj)(gk−gj),\frac{1}{G^2\sigma}\sum_{k<j}(R_k-R_j)(g_k-g_j),

with gk=∇θlog⁡πθ(yk∣x)g_k=\nabla_\theta\log\pi_\theta(y_k|x) and σ\sigma the group reward std. The reward enters only through gaps Rk−RjR_k-R_j, all divided by the same σ\sigma. If two rollouts differ by Δri\Delta r_i in constraint ii, their gap is ∂R∂riΔri\frac{\partial R}{\partial r_i}\Delta r_i to first order, so the derivative in Theorem A.3 is the weight on progress on ii. The 1/C1/C factor cancels in σ\sigma.

In exact form, with SS their squared error on the other constraints:

  • SAR: the gap is wC[ϕ(ri)−ϕ(ri′)]\frac{w}{C}[\phi(r_i)-\phi(r_i')], independent of SS.
  • exp⁡(−SSE/T)\exp(-\mathrm{SSE}/T): the same gap is multiplied by exp⁡(−S/T)\exp(-S/T).

Dividing by σ\sigma does not remove this factor unless all rollouts in the group share the same SS. Normalization also keeps the ranking, which the global norm can invert. For example, with 16 constraints, consider that solution A satisfies 15 and misses one by 2.0, while solution B misses all 16 by 0.4. In this case, the global norm ranks B above A (0.774 vs. 0.670), SAR ranks A above B (0.938 vs. 0.018).

W2 / Q2: Novelty and simpler alternatives

The dense-plus-sparse structure is existing practice, which we cite (Tao et al., 2025). The contribution is the bounded per-constraint aggregation of solver residuals, the theory behind it, and the engine, data and benchmarks that make training possible. To our knowledge this is the first use of per-constraint solver residuals as a reward for a language model. We give a fuller account of the novelty in our response to Reviewer 1Ke1 (W1).

The three alternatives listed are all instances of Definition 1, a sum of bounded decreasing per-constraint kernels:

  • Fraction of satisfied constraints is the zero-temperature limit of SAR.
  • A clipped residual penalty is the kernel ϕ(r)=1−min⁡(r,c)/c\phi(r)=1-\min(r,c)/c.
  • An additive squared-error kernel is ϕ(r)=exp⁡(−r2/T)\phi(r)=\exp(-r^2/T).

Theorems A.1 and A.3 and Proposition A.5 hold for any bounded decreasing kernel, so they apply to all three alternatives unchanged. Choosing among them is a question of kernel shape. The paper's claim is about scoring each constraint separately.

The experiments support this claim against a global norm. In SFT, all five reward variants use the same teacher traces and training settings, and the per-constraint reward beats the global-norm reward on every PyGeoX-Bench tier and on PyGeoX-Wild (p<0.0001p<0.0001 vs. MSE+S+D, p<0.05p<0.05 vs. MSE). In RL, the SAR+S+D model reaches the highest PyGeoX-Bench solving rate of all eleven configurations. Our response to Reviewer 1Ke1 (W2 / Q2) discusses these comparisons further.

The study already has 22 independent training runs over four base models and two training regimes. Training the three alternatives on Qwen3-8B with the seeds requested in Q4 would add at least 19 RL runs. At about 4 days each on an 8×H200 node, that is about 2.5 months of compute, which we cannot fit in the rebuttal period.

W3 / Q3: Internal reasoning vs. tool use

We measured this on all 900 evaluation problems. Of the programs written by the RL SAR+S+D model, 16 (1.8%) call a numerical optimiser or symbolic solver. The rest compute coordinates step by step with basic arithmetic and trigonometry, and none of its solved PyGeoX-Bench answers uses an optimiser or solver. The gains also hold on unseen problems. RL trained on medium problems only, and the fraction of constraints satisfied still rises from 63.4% to 76.8% on Bench-Hard (p<0.0001p<0.0001) and from 82.1% to 88.7% on the human-written PyGeoX-Wild (p=0.022p=0.022).

Q4: Seeds, CAD and Mech

Each RL run takes about 4 days on an 8×H200 node, so we prioritised breadth over seeds. The study has 22 training runs over four base models and two regimes, and every comparison is a paired test on identical problems.

CAD and Mech test two skills: turning an engineering requirement into a constraint system, and solving it. Our training targets the second. Error inspection shows that failures on these suites come mostly from the first, for example building the inner instead of the outer dead-centre of a slider, so the reward design has less room to matter there.

AuthorsOpenReview ↗

Follow-up before discussion closes

Dear Reviewer Q4aB, as the discussion period closes today, we would be grateful to know whether our response and the revision summary above address your concerns, or whether anything remains we could clarify.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions

We thank all reviewers for their time. To summarize up front, the central claim and results of the paper do not change. Per-constraint aggregation of solver residuals gives a better training signal (RL Hard-tier solving rate 0.360 vs. 0.153 for SAR+S+D vs. MSE+S+D, and 0.200 vs. 0.080 without S and D), and the gains carry over to problems not seen in training (constraint satisfaction 63.4% -> 76.8% on Bench-Hard, 82.1% -> 88.7% on PyGeoX-Wild). What the revision adds is stronger theory, new measurements, a verifier fix, and the missing reproducibility details.

On novelty and the scope of the contributions

Most of the criticism concerns SAR, namely its theory, its experiments and how novel it is, and we address all of this below. However, SAR is only one of the paper's contributions, and the others went largely unchallenged in the reviews, with Q4aB and yNx1 in fact listing the environment and benchmarks among the strengths of the paper.

These contributions are also independent of SAR. Before this work there was no way to train a model on this task with any residual-based reward, whether a global norm, the fraction of satisfied constraints or SAR, and it is the environment, data and training setup that make all of these possible. We do believe SAR is the better choice, and the experiments support it, but it is one choice among several, and the rest of the work stands without it and is valuable on its own.

  • The task itself. To our knowledge this is the first work where a language model is trained by RL to compute precise geometric constructions on its own. In GeoLoom (now cited) and MagicGeo the LLM writes a specification and a solver does the computing, whereas here the model computes the coordinates itself in Python, so it is not bounded by what a formal language can express or by the limitations of that language.
  • PyGeoX. No available tool exposes per-constraint residuals at this precision as a training reward, so we had to build one, covering 35 object types and 38 relationships, with equalities and inequalities over arbitrary algebraic expressions, all compiled into a differentiable loss.
  • Data and benchmarks. A procedural pipeline with about 100k validated training problems, together with a 900-problem evaluation suite that spans procedural, examination-style, human-written and engineering problems.
  • Getting RL to work at all. Beyond the reward itself, this required a degeneracy penalty (without it, collapsed constructions get rewarded) and a sparse success term, as well as 22 training runs across four base models and two training regimes.
  • The agent harness, released together with the engine, data and benchmarks.

On the novelty of SAR itself, what sets it apart from standard RLVR in math and code is that our verifier returns a residual per constraint rather than a single bit, and the claim of the paper is about how to aggregate that vector, which also covers the simpler alternatives the reviewers suggested, as detailed in our individual responses.

Theory (Q4aB, 1Ke1)

  • Theorem A.1 is now proved for the averaged reward that is actually used in training.
  • The new Proposition A.5 shows that masking survives GRPO group normalization, because with the global norm a rollout's reward gap is multiplied by exp(−S/T), where S is its error on the other constraints, and SAR has no such factor.
  • The sentence equating near-zero reward with near-zero gradient has been removed.

New measurements

  • Internalization (Q4aB, yNx1). Both reviewers asked whether the model really learns geometry or simply ends up relying on a solver in the loop. Since programs are allowed to import numerical libraries this was a fair concern, so we checked all 900 evaluation programs of the RL model and found that only 16 (1.8%) call an optimizer or symbolic solver, while the rest build the coordinates step by step with basic arithmetic and trigonometry, and not a single solved PyGeoX-Bench answer uses a solver.
  • Reward ranking (1Ke1). On the stored Qwen3-8B answers, SAR tracks the number of satisfied constraints much more closely than the global norm, so that given two answers to the same problem, the better one is ranked higher 88% of the time under SAR and only 60% of the time under the global norm.

Presentation and reproducibility (Q4aB, yNx1, 1Ke1) To make the paper easier to read, the framework now comes before the reward design, with a roadmap in the introduction and bridging paragraphs between sections, "alignment" is replaced by RLVR terminology, and the theorems now match the reward used in training. We also added the full RL and SFT hyperparameters and the SFT weighting, and will redraw the figures for the final version.

What we could not do One RL run takes about 4 days on an 8×H200 node, and the requested full factorial takes 48 runs per seed, which is not possible within the rebuttal window. As noted, the theory applies for the requested cases.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.