- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes TD-Grokking, which uses reference-solution-guided decomposition to turn empirically zero-reward problems into self-contained subproblems with verifiable answers, then jointly trains on the original problems and their subproblems using GRPO. Experiments across three model families and mathematical, scientific, and medical benchmarks report consistent improvements, while subproblem-only training and a dynamic variant examine transfer and training efficiency.
Critical strengths
- The paper addresses a practically relevant limitation of RLVR: the absence of informative rewards on consistently unsolved problems.
- The method is straightforward to implement and shows empirical gains across multiple model families and reasoning domains.
- The SFT, hint-based, generic-data, and subproblem-only controls provide useful evidence about training-time decomposition.
Critical weaknesses
-
The main comparisons do not establish an advantage under comparable training budgets. Table 1 uses 3,788 training records for vanilla GRPO and 18,505 for TD-Grokking, but training updates, rollout tokens, and checkpoint-selection rules are not reported for the Table 2 baselines. This particularly weakens the comparison with QuestA, where several gains are relatively small.
-
The reliability of generated subproblem answers is not quantified. Appendix A.3 describes format and compatibility checks rather than independent correctness validation, while Appendix D acknowledges incorrect targets and answer-normalization issues. Since these answers define the training rewards and diagnostic scores, a focused correctness audit is needed to assess the impact of such errors.
-
The recovery diagnostic does not distinguish local-skill transfer from simplified-root practice. Listing 2 requires the final subproblem to retain the original target and explicitly supplies solution-derived intermediate results. The recovery of 77 previously zero-reward roots therefore supports transfer within the same problem families, but does not isolate the contribution of local-skill practice.
-
Evaluation-set independence is insufficiently documented. Appendix D notes the absence of a documented overlap audit between training roots, generated subproblems, and evaluation benchmarks. Appendix A.6 also leaves the precise evaluation split and any model-selection use of the 1,000 PubMedQA items unclear, limiting confidence that the reported gains reflect held-out generalization.
Questions for authors
My main concern is the experimental validation, particularly the comparability of training budgets, evaluation-set independence, and whether the transfer analysis isolates the contribution of local-skill practice. Please address the Critical Weaknesses above.