- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies whether post-training gains in tool-using language models reflect greater dependence on tool returns. It distinguishes routing, execution, acquisition, integration, and termination, and introduces paired replay from the same recorded history and first action. Natural replay preserves ordinary returns, while matched-null replay replaces every executed return with a token-length-matched uninformative observation. The accuracy difference measures intervention-specific return dependence, and its change relative to the frozen base model measures the training-induced change.
The primary experiment crosses Code, Math, and Search training with SFT, GRPO, and on-policy distillation, using three training runs per cell. Additional experiments include fixed-initial-opportunity CodeDebug evaluation and a private-fact diagnostic. The results show that endpoint accuracy, acquisition, and return dependence can move separately.
Critical strengths
1.The paper addresses an important ambiguity with an explicit intervention rather than inferring information use from tool-call counts or answer agreement. Replaying a common first action and replacing subsequent returns prevents simple recovery through additional calls. The reported contrast is reproducible and has a clearly stated operational interpretation.
2.The supporting experiments go substantially beyond a single return-ablation result. CodeDebug equalizes the initial program, action, and diagnostic across policies, while the private-fact diagnostic shows that Search-SFT retrieves the answer on every item but answers only 7.3% correctly, versus 96.8% under direct placement. Three training runs per primary cell, additional SFT runs on CodeDebug, and comparator sensitivity analyses strengthen the distinction between receiving information and benefiting from it.
Critical weaknesses
1.The interpretation remains broader than semantic integration. The repeated-token null is visibly abnormal and changes subsequent behavior: the retrospective Search-to-Math audit reports 2.06 queries per item under null versus 1.46 under Natural. Because all subsequent returns are replaced and unscorable endings receive zero, the accuracy contrast includes changes in later calling and termination. Furthermore, the positive Search-SFT-to-Math effect is concentrated on Base-unrouted items, while the Base-routed estimate is approximately zero. The paper acknowledges these boundaries, but stronger separation of information use from intervention-induced behavioral disruption would improve the central interpretation.
2.The generality and statistical resolution are limited. The primary student is Qwen2.5-3B, evaluation panels contain approximately 200 items, and confirmatory tests use only three training-run effects with two degrees of freedom. The Math-SFT-to-CodeDebug adjusted p-value of .0489 should therefore be interpreted cautiously, although the additional SFT runs support the direction. Training procedures also differ in tokens, compute, and visited-state diversity, while the family and teacher extensions change coupled factors. These experiments support conditional observations rather than a general ranking of post-training methods.
Questions for authors
1.Can Help/Harm outcomes be cross-tabulated with changes in whether a scorable answer is submitted? This would clarify how much of the measured dependence is associated with termination changes.
2.Is a schema-preserving, uninformative comparator available for CodeDebug under the same frozen evaluation contract? Its effect on the Math-SFT result would materially inform the interpretation of the fixed-opportunity finding.
3.Can the Math-SFT-to-CodeDebug effect be reported separately for runtime-error and wrong-output diagnostics? This would help determine whether the result is concentrated in one feedback type.
Thank you for your detailed review and specific questions. We run each task twice from the same starting point: once with the real tool outputs, and once with every output replaced by filler (one token repeated to the same length; the "matched null" in our paper). We call these two runs "replays". If accuracy drops, the model was relying on the tool outputs. T is the trained model's accuracy drop minus the base model's drop.
W1: The measure is broader than integration. U, the accuracy drop under filler, includes effects on later calls and on whether the model answers (Section 3.2). For SFT and GRPO models trained on Math or Search, T stays positive in every training run on their own tool when filler is replaced by real outputs from other items. In the original filler comparison, most of T comes from items where both replays answer. On CodeDebug, Math-SFT's T stays positive under the format control (see Q2 below).
W2: Generality, statistical power and compute.
Q1: Help/Harm by whether a program is submitted. The table covers the 199 CodeDebug items. Columns group items by which replay submits a program. Each cell gives the number of items, then Help / Harm (Help: correct with the real diagnostic, wrong with filler; Harm: the reverse). For example, "76; 29/0" means that on 76 items only the real-output replay submits, and its program is correct on 29 of them.
Math-SFT's T on CodeDebug is 15.9 points (average of three training runs): 13.6 points come from items where only one replay submits, and 2.3 from items where both do. So with filler, Math-SFT submits a program less often. When we score the last program written, submitted or not, T stays positive in all three runs. The appendix will give these groups for every Math and Search cell.
Q2: An uninformative output that keeps the diagnostic format. Our format control replaces only the first output. It keeps the diagnostic's layout (output blocks, field names, success flag and Traceback header) and replaces its text with filler of about the same length; later outputs stay real. Math-SFT's T is +9.0 points (interval [3.2, 15.1]) and positive in all five training runs. So T stays positive when the first output keeps the diagnostic format.
Q3: Runtime errors and wrong outputs. T by diagnostic type, from the original filler comparison:
T is positive in all three training runs for both types and about three times larger for runtime errors.