- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors explore whether curriculum learning can improve long horizon implicit reasoning in language models. They use a synthetic task, using a DAG for the structure and then translated into english, this allows them to control the generation of the training data. The curriculum gradually increases the maximum lookahead and the vocabulary of the attribute names in the data. They show this curriculum enables models to solve problems requiring greater lookahead, then those trained on a uniform distribution of lengths and vocabulary from the same dataset. They thoroughly investigate how multiple design choices affect their conclusions, including model size, initialisation and curriculum step size. They also demonstrate improvements on reasoning benchmarks compared without training with a curriculum.
Critical strengths
-
Strong benefits of curriculum learning: Section 4.1 provides clear evidence of the benefits of the authors' approach when compared to training without a curriculum. For a lookahead length of 96 and 128, there is a clear speedup of training with the curriculum. At length 256, the model with curriculum achieves 98%, whilst the model without curriculum only achieves 45% accuracy. These results show clear training efficiencies from their design, as well as increasing the model's capacity to correctly answer harder problems.
-
Thorough exploration of design choices: The authors dive deeply into investigating the impacts of the curriculum step size, vocabulary, model size, and the differences of their conclusions when comparing between its effectiveness fir starting from a pre-trained checkpoint and random initialisation. These findings and results are useful for future work on this topic.
-
Downstream Reasoning Accuracy: Section 4.6 extends the findings from being interesting in isolation about curriculum design for long horizon implicit reasoning, but shows that the benefits actually transfer to downstream reasoning tasks using Qwen3 1.7B. Although the benefits are spiky depending on the benchmark, this section adds relevance of the synthetic training findings to downstream reasoning tasks.
Critical weaknesses
- Robustness Across Training Seeds: The experiments in the paper only use one training seed, which raises some concerns about the robustness of the gains and convergence times. For Figure 2 and Table 1, I am concerned that the different training seeds may change the reported speedup gains and also could change how the model with no curriculum (which currently plateaus at 45%) performs on the harder problems (L = 256).
Questions for authors
- Could the authors perform their experiments across additional training seeds? This would address my robustness concern. Some important experiments to run over multiple seeds are Section 4.1, Figure 2, Table 1, and also the downstream reasoning in Table 4. If these could both be performed over multiple seeds, it would strengthen the paper.
4 comments
Response to Reviewer Q8ce (2/2)
For the downstream results in Table 4, we repeated the experiment with the same two additional training seeds to stand alongside the seed in the original submission. For each seed, we fine-tuned with curriculum learning to , following the procedure in Section 4.6, and without curriculum learning for the same training compute (137.1K, 155.0K and 128.0K PFLOPs for the three seeds). Table R1 reports accuracy as mean standard deviation over seeds, and the increase in accuracy due to the curriculum for each seed.
Table R1. Downstream reasoning accuracy (%) for at over three training seeds. Increase in accuracy is the accuracy with curriculum minus the accuracy without curriculum. *: the two models trained with the same seed differ significantly on the corresponding benchmark (two-sided exact McNemar test on paired items, ).
| Benchmark | With curriculum | Without curriculum | Increase in seed 1234 (paper) / 2345 / 3456 | Mean increase |
|---|---|---|---|---|
| ProofWriter OWA | 51.1 3.3 | 42.0 2.4 | +14.3* / +4.3* / +8.5* | +9.0 |
| ProofWriter CWA | 66.7 1.7 | 59.8 5.6 | +12.8* / +1.4 / +6.5* | +6.9 |
| ParaRules | 53.2 2.4 | 47.4 3.1 | +4.7* / +3.0* / +9.7* | +5.8 |
| ZebraLogic-MC | 53.5 3.7 | 47.2 1.3 | +5.2* / +11.5* / +2.2 | +6.3 |
| PrOntoQA | 58.2 1.7 | 54.0 2.0 | +5.1* / +3.3* / +4.3* | +4.2 |
| FOLIO | 46.5 0.8 | 40.7 0.8 | +4.4* / +5.4* / +7.4* | +5.7 |
| JustLogic | 53.7 0.4 | 47.2 1.3 | +7.4* / +7.5* / +4.9* | +6.6 |
| ProverQA | 46.6 1.6 | 48.3 0.9 | / 0.0 / * | |
| CLUTRR | 26.9 1.5 | 26.1 1.7 | +1.9 / / +2.1 | +0.8 |
| LogicBench-BQA | 73.2 0.5 | 70.5 1.5 | +2.4* / +1.8 / +3.8* | +2.7 |
| RuleTaker | 61.0 1.1 | 60.5 2.5 | +1.2 / * / +2.9 | +0.5 |
| Web of Lies | 57.3 0.6 | 55.8 2.5 | +0.2 / +0.4 / +4.0* | +1.5 |
| LogiQA | 43.0 0.5 | 42.4 1.0 | / +0.8 / +1.9 | +0.6 |
| NLGraph | 44.1 0.5 | 44.5 1.7 | * / +1.2 / +0.1 | |
| Mean of 14 | 52.5 0.5 | 49.0 0.8 | +3.9 / +2.6 / +3.9 | +3.5 |
In Section 4.6 (Lines 496-500), we replaced the single-seed summary with a summary of these results:
At , averaged over three training seeds, training with curriculum outperforms training without curriculum on 12 of the 14 benchmarks, with a mean accuracy of against . The increase in accuracy is positive in every seed on nine benchmarks, and it is statistically significant in every seed on ProofWriter OWA, ParaRules, PrOntoQA, FOLIO, and JustLogic. Accuracies on ProverQA and NLGraph are lower on average with curriculum.
The results are stable across training seeds. Averaged over the 14 benchmarks, the standard deviation of accuracy across seeds is percentage points for the curriculum-trained model and points for the model trained without curriculum, compared with an increase in accuracy of points. In each seed, the curriculum-trained model outperforms the model trained without curriculum on at least 11 of the 14 benchmarks.
A paired bootstrap over evaluation items, pooling the three seeds, yields a confidence interval of points for the increase in mean accuracy. After Holm correction over the 14 benchmarks, nine benchmarks show a significant increase in accuracy (ProofWriter OWA, ProofWriter CWA, ParaRules, ZebraLogic-MC, PrOntoQA, FOLIO, JustLogic, LogicBench-BQA and Web of Lies), and the other five show no significant difference.
Having replicated the downstream experiment with three training seeds, we also revised the paper as follows:
In Table 4 (Section 4.6, Lines 432-448), the accuracy values have been changed to be the average accuracy over three training seeds.
In Appendix F (Lines 1423-1440), we added a new Table 23, which reports the accuracy with and without curriculum and the increase in accuracy due to curriculum learning for each training seed.
Thank you for engaging with a thorough response and additional experiments! Overall, I consider my concerns largely resolved and thus I remain supportive of acceptance and will keep my original score.
Follow-up for Reviewer Q8ce: Fitted Ceiling with a Second Training Seed
We thank the reviewer for the continued support of our submission, and are glad to hear that we were able to largely address the concerns.
We wanted to provide some new results on the effect of the training seed on the fitted ceiling. We are repeating the training of from Section 4.2 with a second seed (2345), with all other hyperparameters and the curriculum configuration held fixed. This training run has now reached , where a ceiling can already be identified, and we report its current fit below.
For the second seed, the saturating exponential also fits better than the power law (), with the fitted equation
Its fitted asymptote is ( bootstrap interval ), compared with () for the original seed. The interval of the second seed contains the fitted asymptote of the original seed. Under the same Diebold-Mariano test (Appendix D.1), the saturating exponential fitted on the first of stages predicts the remaining stages significantly more accurately than the power law (), as was the case for the original seed. In Section 4.3 (Lines 397-402) of the revised paper, we also extended the training of to 1.19M PFLOPs and fitted both forms, and the saturating exponential again fits its trajectory better than the power law (), with . Together, these results suggest that, for a given model, a lookahead ceiling exists under different curriculum step sizes and different training seeds. At the same step size, the value of the fitted ceiling is also largely consistent across training seeds.
For the camera-ready version, we will extend this training run to 1.5M PFLOPs and report the fitted asymptote for both seeds.
Response to Reviewer Q8ce (1/2)
We thank the reviewer for the thoughtful feedback. Below we address each concern in turn and attempt to implement the reviewer's suggestions. Our changes and additions to the paper that are most relevant to the reviews are highlighted in blue in the revised PDF. If any concerns remain after our response, we would be grateful if the reviewer could raise them, so that we have the opportunity to address them before the end of the discussion period.
1. The comparison in Section 4.1 (Figure 2 and Table 1) and the downstream results in Table 4 use a single training seed and may change with the seed.
We agree that a single training seed cannot show whether the advantage of curriculum learning is robust to training randomness. To address this, we repeated the comparison of training with and without curriculum at maximum lookahead L=256 (Section 4.1, Figure 2) with two additional seeds (2345 and 3456) to stand alongside the seed in the original submission (1234), for a total of three seeds. We prioritized this comparison because we consider it the most impactful one in Section 4.1, since it is the only lookahead at which training without curriculum does not reach 98% at all in the given budget. We added a new Table 9 to Appendix C.1 (Lines 956-963), which reports each seed and the mean ± standard deviation.
Table 9 (Appendix C.1). Training Qwen3-0.6B with and without curriculum at maximum lookahead L=256.
In Section 4.1 (Lines 271-274), we revised the description of the L=256 comparison to report these results:
Training with curriculum is consistent across seeds. At the same cumulative training compute, evaluation accuracy with curriculum differs between any two seeds by 2.3 points on average and by at most 6.9 points.
Training compute is matched within each seed, so the training run without curriculum is given the same cumulative compute at which the training run with curriculum ended for that seed. At that point, the model trained without curriculum is at 48.6±3.5 evaluation accuracy on average.
We revised Figure 2, which shows evaluation accuracy vs. cumulative training compute for training with and without curriculum at L∈96,128,256. Its L=256 panel now shows all three seeds for both settings and their mean, each in the color of its setting.
We also agree that the speedups observed in Table 1 at the smaller lookaheads (L=96-128) may change with the training seed. Training without curriculum fixes the maximum lookahead from the start, so each entry in Table 1 for training without curriculum requires a separate training run. Replicating all of these runs with two additional seeds requires a large amount of compute, which was not possible within the length of the rebuttal period. For the camera-ready version, we will provide the full replication of Table 1. Given how consistent the L=256 comparison is across seeds, where the compute to reach 98 evaluation accuracy has a standard deviation of about 5 of its mean, we strongly suspect that the behavior reported in Table 1 will hold beyond slight seed-to-seed variation.