- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
STEER studies reasoning RL with a mixture of labeled and unlabeled mathematics problems. It assigns pseudo-answers using student consensus, weights updates by confidence, and maintains a dynamic memory of accepted trajectories. TF-IDF retrieval over questions and trajectories supplies an additional reward for unlabeled examples. The experiments compare two model families and several label budgets on five mathematics benchmarks, including comparisons with TraPO and an ablation of the graph-reward weight.
Critical strengths
-
The semi-supervised setting is useful, and TraPO is a relevant comparison. Reporting several labeled/unlabeled budgets makes it possible to assess whether the gains persist beyond a single label allocation.
-
The graph-weight ablation provides a direct check of the proposed additional reward. In the Qwen3 1k+1k setting, the score increases from at to with the full method, so there is evidence that the graph term contributes within this training pipeline.
-
The memory reuses student-generated trajectories without a separate process verifier. Question retrieval followed by trajectory comparison is a relatively accessible design, and the paper evaluates it with both Qwen2.5-Math-7B and Qwen3-4B.
Critical weaknesses
-
The empirical gains over TraPO are mostly modest: three of the four reported model/data configurations improve the average score by less than one point. Without uncertainty estimates, it is difficult to assess how reliable these small gains are. The evaluation is also limited to 4B/7B models and mathematics benchmarks, and the lack of reward training curves makes the learning dynamics difficult to assess. Overall, the evidence for a consistent advantage is not yet convincing.
-
The consensus-based pseudo-labeling reinforces the model’s majority answers, which may improve performance largely by concentrating the answer distribution, similarly to entropy minimization. Although the paper includes entropy-based baselines, it remains unclear how much of the gain comes from this effect rather than useful reasoning information in the trajectory graph. An analysis of answer entropy, pseudo-label accuracy, and successful-trajectory diversity during training would help distinguish improved reasoning from stronger self-consistency.
-
The graph ablation is informative, but it also limits how much of the improvement can be attributed to the graph. In the Qwen3 1k+1k setting, already scores , compared with for STEER and for TraPO. The other reported gains over TraPO are mostly below one point. Thus, the 5.7-point gap to TraPO cannot all be credited to trajectory guidance. Can the authors separate the effects of confidence weighting and sample admission from the graph reward?
Questions for authors
Please see Critical Weaknesses.