- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
LURE studies self-generated curricula for verifiable reasoning. A Challenger selects difficulty parameters for programmatic task generators, while separate Planner and Executor policies learn from verifier feedback. The Planner receives trajectory-level verified progress; the Executor receives immediate progress rewards and terminal correctness. All three policies use clipped GRPO with round-start KL regularization. Experiments show promising in-domain results across three environments and backbone families. Experiments across three environments and three backbone families demonstrate the effectiveness of the proposed method.
Critical strengths
- Using the same programmatic verifier for curriculum feedback and process supervision provides a coherent training framework without a separately trained process reward model.
- The PhantomWiki improvements are promising, and the in-domain experiments cover three environments, three backbone families, and unified/specialist settings.
Critical weaknesses
-
LURE uses different learning-rate, epoch, and KL settings from the baselines and trains on more retained tasks per round. Moreover, the “w/o planner credit” ablation changes the Planner’s reward but retains the model, leaving the benefit of a separate Planner over a single Solver untested.
-
The evaluation protocol does not explain several reported scores, and uncertainty is insufficiently documented. ZebraLogic reportedly uses 300 puzzles with one greedy rollout each, but Table 1 values such as 11.6% and 16.9% cannot directly arise under that protocol. The authors should clarify sample counts and aggregation, and report independent training repetitions and confidence intervals to establish whether small advantages, particularly on OOD benchmarks, are reliable.
Questions for authors
- Under comparable token/GPU cost, what benefit comes from separate Planner and Executor models over a single or parameter-sharing Solver?
- What exact sample counts, aggregation rules and independent training repetitions produce the reported ZebraLogic scores? Please provide per-run results and uncertainty for the principal comparisons.
- Which role’s weights are used for OOD evaluation: Planner or Executor? How are checkpoints selected, and why is OOD evaluation restricted to Qwen2.5-7B-Instruct despite the in-domain Llama and Gemma experiments?
- Will the code be publicly released?
3 comments
Authors' Rebuttal (2/2)
Continued: Authors’ Rebuttal (2/2)
Q3 [Weights, checkpoints, and backbones for OOD evaluation]
OOD evaluation uses the Executor weights, with checkpoints selected on a held-out validation set and never on OOD scores. The Executor produces complete answers, whereas the Planner outputs subgoals, each tied to a verifier predicate. The Tab. 3 caption describes the prompting: benchmark-specific prompts without a role prompt. The validation set comprises 240 generated PhantomWiki instances, 320 IFEval-style prompts generated by the same constraint builder with 16 constraint types, and 300 generated ZebraLogic puzzles. It is a fixed held-out set, separate from the 120, 541 and 300 reported test instances. Generator seed ranges are disjoint across training, validation and test, and training tasks identical to a validation or test instance are removed before training. Validation instances never enter Solver updates or Challenger feedback.
All eight-round comparisons, including the shared-recipe and verifier controls, use the same validation instances and the eight end-of-round checkpoints. Unified models maximize the mean of the three validation accuracies, and specialists use their own environment. Ties are broken in favor of the earlier round. Validation uses the same number of passes as the test evaluation. All 13 OOD benchmarks use the one selected unified checkpoint. We have added these details to App. B. The original OOD table uses Qwen because Qwen is the backbone of the component analyses (Figs. 4 and 5). Table R4 adds both other backbones on all 13 OOD benchmarks.
Table R4. OOD group averages (%) for Llama-3.1-8B-Instruct and Gemma-2-9B-it unified models. Overall is the mean over the 13 benchmarks.
| Backbone | Method | Near-Domain | Mathematics | Logic | Other Reasoning | Overall |
|---|---|---|---|---|---|---|
| Llama-3.1-8B | Base | 26.8 | 24.0 | 18.6 | 24.4 | 24.1 |
| Llama-3.1-8B | R-Zero | 29.1 | 27.1 | 19.2 | 23.9 | 26.0 |
| Llama-3.1-8B | LURE | 28.8 | 24.8 | 19.8 | 22.5 | 24.9 |
| Gemma-2-9B | Base | 35.6 | 28.7 | 24.2 | 26.9 | 29.9 |
| Gemma-2-9B | R-Zero | 37.6 | 28.3 | 24.7 | 28.4 | 30.6 |
| Gemma-2-9B | LURE | 36.1 | 28.0 | 26.0 | 27.3 | 30.1 |
On both added backbones, LURE remains above the base model overall and has the highest far-transfer logic score. In the Qwen three-run result, both methods reach 36.5 on OOD and LURE is 7.5 points higher in-domain. LURE's in-domain gains therefore come without loss of overall OOD performance relative to the base model, with transfer varying across benchmarks and backbones. We have revised the transfer claims in the abstract, the introduction, Sec. 4.2 and the conclusion, and added Table R4 to App. C.
Q4 [Code release]
Yes, we will release our code with the final version.
We hope this fully addresses your questions and comments, and remain happy to engage in further discussion for the rest of the rebuttal period.
Thank you for the detailed rebuttal, which addresses my concerns. However, I will maintain my score of 2 because LURE requires substantial environment-specific engineering and its practical gains remain limited. In my view, an agentic setting could broaden LURE’s practical applicability, and may be an interesting direction for future work.
Authors' Rebuttal (1/2)
Thank you for the valuable feedback. We address each point below.
W1 & Q1 [Optimization settings, retained tasks, and the separate Planner]
LURE still outperforms R-Zero under the shared recipe, and R-Zero equipped with our verifier needs four times the candidates and five times the rollouts to match LURE. The separate Planner leads the single Solver by 2.7 points at equal rounds and by 0.5 points at matched GPU hours. We address this in two parts: the optimization recipe with task exposure, and the separate Planner.
(1) Recipe and exposure. In the paper, every method receives 144 candidates per round, GRPO trains on the same 144 tasks per round as LURE, and R-Zero's lower count comes from its own filter (App. A, Fig. 7). We retrain R-Zero under LURE's recipe (learning rate 5×10−6, one PPO epoch, KL 0.1 to the round-start snapshot). R-Zero-V is R-Zero equipped with our generators and verifiers: its Challenger proposes task specifications, the shared generator and validity gate instantiate them, and verified capture replaces majority agreement in its reward and filter. R-Zero-V with four times as many candidates and subsampled LURE match exposure in both directions. Table R1 reports the results.
Table R1. Shared-recipe, verifier and exposure comparisons on the Qwen2.5-7B-Instruct unified model. Task counts are eight-round means.
Under the shared recipe, LURE leads R-Zero by 3.8 points. At matched exposure, LURE reaches 45.9 versus 44.4 for R-Zero-V. To reach LURE's accuracy, R-Zero-V uses approximately 4608+8×150.1≈5809 Solver rollouts per round for probing and training, and LURE uses 1152, which is 19.8% as many.
(2) Separate Planner versus a single Solver. LURE uses separate parameters θplan and θexec. The single Solver generates a plan and then executes it, with planner credit on plan tokens and executor credit on action tokens. The parameter-sharing variant uses one set of weights with role-specific prompts. All variants retain LURE's curriculum, verifier feedback, and training recipe. We also extend the single Solver's training to match LURE's GPU hours. Table R2 reports the results.
Table R2. Solver variants of the Qwen2.5-7B-Instruct unified model.
Separate parameters give the highest accuracy in every comparison. Parameter sharing uses fewer GPU hours at comparable token counts and offers a lower-cost alternative with 0.6 points lower accuracy. We have added Table R2 to App. C.
W2 & Q2 [ZebraLogic sample counts, aggregation, and uncertainty]
Each ZebraLogic score averages three evaluation passes over the same 300 puzzles, so 11.6 and 16.9 are 104/900 and 152/900. App. B omitted this aggregation step, and we have corrected it. With kr solved puzzles in pass r, A=31∑r=13300100kr, rounded after averaging, for all ZebraLogic results in Tabs. 1, 10 and 11. Each pass uses one greedy output per puzzle, and outputs can differ across passes since small numerical differences in the logits can change near-tied choices. The passes re-evaluate one checkpoint, and Table R3 reports three independent training runs.
Table R3. Three independent training runs of the Qwen2.5-7B-Instruct unified model.
We report the sample std and Welch-t intervals. In-domain means are 48.6 ± 0.7 versus 41.1 ± 0.2, with gap interval [5.9, 9.1], and the PhantomWiki, IFEval and in-domain intervals exclude zero. These runs use the configurations of Tab. 1, and with R-Zero under LURE's recipe the in-domain gap is 3.8 points (Table R1 above), more than five times LURE's run-to-run std.
Please see the subsequent Authors' Rebuttal (2/2) for remaining responses.