- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CPARL learns to translate natural-language problems into constraint programs through two rounds of supervised fine-tuning followed by solver-guided reinforcement learning. Its SE-GRPO objective assigns credit to blocks of edits and solver calls, while a shared outcome reward trains direct generation and route selection. The paper also introduces NL4CP, with solutions checked against formal XCSP3 specifications. Across five benchmarks, CPARL improves Qwen3.5-27B to 35.0% micro and 44.6% macro pass@1, with ablations examining supervision, routing, and credit assignment.
Critical strengths
- NL4CP checks returned assignments against formal specifications, providing stronger validation than objective matching alone offers.
- The staged training substantially improves the shared backbone across five benchmarks, supporting its practical utility.
- The same-reward trajectory control helps separate block credit assignment from changes in the reward design.
Critical weaknesses
- The small block-credit advantage remains uncertain because repeated evaluations vary decoding settings rather than training seeds.
- Unequal feedback access and unclear interface compatibility limit the conclusions drawn from external baseline comparisons.
- Unclear test-time validation and untested overconstraint risks limit confidence in the method's practical reliability.
Questions for authors
-
Table 2 shows gains of only 0.8 micro and 0.9 macro percentage points over the same-reward trajectory control. Do these gains persist across independent training runs under a common checkpoint-selection rule?
-
Section 5.1 compares CPARL with baselines lacking intermediate solver feedback. How much advantage remains over a fixed repair loop using the same SFT model, feedback, and generation budget?
-
Section 4.2 includes validation feedback during interactive generation. Is validate callable at inference, and does its feedback depend on benchmark references?
-
Successful output validation need not exclude overconstrained models rewarded by Equation (2). Have accepted, heavily pruned programs been audited for unintended restrictions, or evaluated with propagation credit removed?
-
For the near-zero baseline scores in Table 1, how often do failures reflect output-format incompatibility, program execution, or incorrect modeling?
-
The captions for Tables 1 and 2 appear below the tables; move them above to follow the required table format. Enlarge Table 2's text so that its means and standard deviations are legible.
7 comments
Response to Reviewer LS9H — Part 2/4
Response to W2.1/Q2. A fixed repair workflow with the same SFT model
(Q2) “Section 5.1 compares CPARL with baselines lacking intermediate solver feedback. How much advantage remains over a fixed repair loop using the same SFT model, feedback, and generation budget?”
To further demonstrate the importance of teaching the model to decide how to use tools, we added an evaluation of the same two-round SFT checkpoint with the predefined workflow construct → compile → repair if needed → propagate → repair if needed → solve → submit. This allows the model to revise its program based on solver feedback, while the operation order is fixed in advance. Key pass@1 results on all 900 test instances are below, reported as mean ± sample SD:
| Method | Micro (%) | Macro (%) |
|---|---|---|
| Two-round SFT + fixed tool workflow | 31.44 ± 0.16 | 39.59 ± 0.52 |
| Two-round SFT + autonomous interaction | 32.5 ± 0.4 | 41.1 ± 0.3 |
| Full CPARL (SFT + RL) | 35.0 ± 0.9 | 44.6 ± 1.5 |
The same SFT checkpoint with autonomous interaction exceeds the fixed workflow by 1.06 Micro and 1.51 Macro percentage points; full CPARL exceeds it by 3.56 and 5.01 points, respectively. This supports learning tool timing and interaction control.
We also evaluated strong general-purpose models with solver feedback. For example, Claude-Sonnet-4.6 achieves 58.8 ± 2.2% Micro and 62.2 ± 2.8% Macro on all 900 instances across repeated full-test-set evaluations with different evaluation seeds. These results further illustrate the value of tool feedback for general-purpose models' CP modeling. See Appendix K.2 for the settings and results.
Response to Reviewer LS9H — Part 3/4
Response to W2.2/Q5. Baseline failure attribution and execution-error subtypes
(Q5) “For the near-zero baseline scores in Table 1, how often do failures reflect output-format incompatibility, program execution, or incorrect modeling?”
We analyzed execution logs and checker reports for the five fine-tuned baselines across all 900 test instances, distinguishing program execution errors, output-format errors, timeouts, validation failures or objective mismatches, and other errors. Under CPMpy + OR-Tools, program execution errors are the main failure cause for all five models:
| Model | Program execution errors as a share of failed cases |
|---|---|
| ORLM-8B | 93.5% |
| OptMATH-7B | 75.9% |
| OptMATH-32B | 89.7% |
| SIRL-7B | 82.1% |
| SIRL-32B | 85.8% |
These percentages use failed cases as the denominator. Output-format errors account for a smaller share: for example, 5.5%, 7.6%, and 7.5% for OptMATH-7B, OptMATH-32B, and SIRL-7B, respectively. We further categorize execution errors into syntax/name errors, API errors, type/expression errors, data-binding errors, and other runtime errors.
A further breakdown by model and dataset reveals different sources of execution errors. Key results are below; each percentage uses program execution errors for the corresponding model and dataset as its denominator:
| Model | Main execution-error patterns | Key results |
|---|---|---|
| ORLM-8B | Syntax/name errors dominate on several datasets; other runtime errors are prominent on CP-Bench | Syntax/name errors account for 93.6% on TEXT2ZINC and 68.8% on CPEVAL; other runtime errors account for 69.7% on CP-Bench |
| OptMATH-7B | API errors dominate on most datasets, alongside syntax and data-binding problems | API errors account for 78.0% on CPEVAL; syntax/name errors account for 44.1% on IndusCP; data-binding errors account for 16.3% on NL4CP |
| OptMATH-32B | API errors are the largest category on every dataset | API errors account for 96.8% on CPEVAL and 73.1% on NL4CP |
| SIRL-7B | API errors are the largest category on every dataset, with substantial type/expression errors on some datasets | API errors account for 80.6% on CPEVAL and 70.0% on TEXT2ZINC; type/expression errors account for 28.2% on IndusCP |
| SIRL-32B | Both API and type/expression errors are prominent | API and type/expression errors account for 44.2% and 47.1% on NL4CP, respectively; type/expression errors account for 34.0% on IndusCP |
This breakdown shows that failures concentrate in code syntax, solver API usage, and type/expression construction, with clear differences across models and datasets. The low scores therefore cannot be explained solely by output-format incompatibility: reliably generating executable CP solving code remains a major bottleneck for these baselines under the current evaluation settings. See Appendix K.4 for the complete error analysis.
Response to Reviewer LS9H — Part 1/4
We thank the reviewer for recognizing NL4CP's solution checks against formal specifications, the practical gains from staged training across five benchmarks, and the value of the same-reward trajectory control for isolating credit assignment. We also appreciate the concerns about independent training stability, fair baseline comparisons, validation access, and the reliability of propagation rewards. We address each point with additional experiments and analysis below.
Response to W1/Q1. Independent training stability and common checkpoint selection
We conducted 15 independent RL runs: SE-GRPO, outcome-only GRPO, and the same-reward trajectory-level-credit control (E3), each with training seeds 17, 137, 2027, 3407, and 7919. All runs start from the same two-round SFT checkpoint. For each method and training seed, we select its own best checkpoint using the same Dev-1000 pass@1 criterion:
Each selected checkpoint is evaluated with five evaluation seeds on all 900 test instances. The table reports mean ± sample SD of Micro accuracy in percent. SD describes evaluation variability at each fixed checkpoint; separate rows correspond to independent training runs.
We acknowledge that the original gains over E3 of 0.8 and 0.9 percentage points are modest. However, with independent training runs and each method selecting its own checkpoint under the same development criterion, SE-GRPO achieves higher mean Micro accuracy than E3 for all five matched training seeds. Averaging the five training-seed means gives 35.62% for SE-GRPO and 34.87% for E3, a mean improvement of 0.75 percentage points. These results support stable gains from SE-GRPO, rather than an advantage arising solely from the original common-checkpoint comparison. See Appendix K.1 for details.