- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper tackles adaptive curriculum learning for LLM reinforcement learning finetuning. It introduces a method that organizes tasks into a hierarchical tree using their latent representations and then introduces a Bayesian batch sampling procedure over this structure. They compare different sampling procedures and find that maximizing learning signal through difficulty is not sufficient to ensure enough training diversity and to maximize downstream performance.
Critical strengths
The paper tackles an important issue in RL finetuning for LLMs, and the construction of a structured latent over training inputs over which sampling can be done is a novel and interesting idea.
Critical weaknesses
- No external baseline beyond Dynamic Sampling which suffers from high training time due to repeated sampling, this makes it hard to assess the benefit of the method namely the Latent Task Tree construction. The paper cites some alternative works using bandits but doesn't provide any training comparison.
- The benefit of BMC compared to Tree-only and Difficulty-only ablation is unclear form the results provided by the authors, results on different benchmarks seem close and are provided without any confidence interval.
Questions for authors
- How do you isolate the benefit of the Latent Task Tree construction itself?
- Could you provide a training comparison against the bandit-based methods you cite?
2 comments
Thank you for the response. I acknowledge that a comparison against MoPPS is included in Appendix D. However, it mainly uses MoPPS to argue that the Difficulty Only ablation is a representative stand-in for bandit-based curriculum methods. It is not a direct test of whether BMC improves on them.
My main concerns remain. The evidence that BMC outperforms existing methods is limited: against MoPPS, BMC’s only stated advantage on a OOD benchmark is on GPQA-Diamond, and without any confidence intervals or repeated runs, the statistical support for these differences is insufficient. The Tree Only ablation isolates the effect of difficulty-aware selection on the tree, but it does not show that the hierarchical latent structure itself matters compared with alternative structures. Finally, the response acknowledges that the productivity and diversity improvements translate into only modest performance differences, which, given the limited statistical support, leaves the practical benefit of BMC unestablished.
I therefore maintain my score.
Response to Reviewer
We thank the reviewer for their feedback. We address your comments and concerns below:
Concern 1: External Baselines
An external bandit-based baseline is already included. In Appendix D, we include comparisons to MoPPS [1], which is a representative method for adaptive curriculum learning. We also describe why Difficulty Only is representative of various other adaptive curriculum learning methods that abide by the same difficulty-protocol, which we refer to as the "Standard Bandit Pattern."
Concern 2: Tree Isolation
The Tree Only ablation isolates the role of the Latent Task Tree construction. Selections start from the root of the tree and descend randomly to select problems. When this random selection is replaced with difficulty-based selection, the method becomes BMC. If you take away the tree from BMC, the method becomes Difficulty Only. Tree Only is used throughout the paper.
Concern 3: Benchmark Interpretation
The core contribution of our work is the Productivity-Diversity-Utility decomposition for curriculum learning. We use this decomposition to explain differences between sampling methods that benchmark scores alone may not reveal.
Each axis shows different behavior between the ablations:
Productivity: Tree Only experiences a 40% reduction in effective problems compared to BMC and Difficulty Only (Figure 4). This is because Tree Only prioritizes diverse coverage of problems and ignores difficulty/productivity.
Diversity: BMC achieves approximately 3$\times$ the rarity-weighted exposure of Difficulty Only (Appendix B, Figure 7), indicating broader exposure to problem types. Difficulty Only prioritizes productive difficulty without explicitly accounting for problem type. Tree Only achieves even broader coverage because it prioritizes coverage without accounting for productivity.
Utility: The substantial differences in productivity and diversity do not translate directly into equally large benchmark differences, because the additional problem types encountered may differ in their relevance to the evaluations. BMC-T investigates this distinction: changing which problem types are prioritized improves targeted benchmark scores while maintaining similar productivity (Figure 6). Thus, similar benchmark scores do not imply similar curriculum behavior.
Concern 4: Confidence intervals
Quantifying variability across training seeds requires multiple independent runs per experimental configuration. We provide experiments across different datasets, model sizes, and RL algorithms, but did not repeat each configuration across independent training seeds due to the substantial cost at our scale.
[1] Qu et al., Can Prompt Difficulty be Online Predicted for Accelerating RL Finetuning of Reasoning Models?, KDD 2026 (https://arxiv.org/abs/2507.04632)