- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes TraceLift, a framework for training a reasoning planner to guide a frozen executor. The planner is trained with GRPO using execution correctness and a reasoning-quality score weighted by the executor’s improvement over a no-plan baseline. The authors construct TRACELIFT-GROUPS, a dataset of reference and perturbed reasoning traces, to train the reasoning reward model. Experiments on coding and mathematics report improvements over execution-only training across three model families.
Critical strengths
-
The method measures whether a reasoning plan actually helps a fixed executor. This provides a concrete basis for rewarding useful plans beyond their apparent quality.
-
The experiments cover coding and mathematics across three model families. Reward ablations, including a constant-RM control, provide evidence that the proposed components contribute to performance.
-
Analyses of reasoning length and human preferences provide additional evidence about plan quality. The length results help address whether the gains simply come from producing longer traces.
Critical weaknesses
-
The training-cost comparison is incomplete. TraceLift requires reward-model training and repeated executor sampling to estimate uplift, yet no comparison under matched total training compute is reported. It remains unclear whether the gains persist when execution-only training receives a comparable budget.
-
The reward-component ablations focus on Qwen2.5-7B coding tasks. Similar ablations on mathematics and other model families would help establish whether each component consistently contributes to the reported improvements.
-
The reward model is trained on synthetically perturbed reasoning traces. The main experiments do not directly establish how reliably it evaluates naturally occurring planner errors, which may differ from the errors introduced during dataset construction.
-
The main results are averaged over three runs without reporting variability estimates. The large SVAMP improvement also warrants explanation: the gain over execution-only training is 21.00 points with LoRA, compared with 5.66 points under full-parameter training (Table 4). This indicates that the size of the benefit depends substantially on the adaptation setting.
Questions for authors
-
Could you report the total training cost and compare TraceLift with execution-only training under a matched compute budget?
-
Does the advantage over the constant-RM control also hold on mathematics and other model families?
-
How accurately does the RM evaluate naturally occurring errors in planner-generated traces, beyond the synthetic perturbations used to construct its training data?