- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper shows that SFT datasets with different tool-call depths lead to very different tool-use dynamics under identical outcome-only RL. Tool initiation rises for Low and collapses for High. The authors hypothesize that pre-call reasoning is the key factor, support this with an idealized analysis, and show that filtering High-depth data by pre-call length improves post-RL accuracy.
Critical strengths
- The idea is simple and cheap: one extra teacher-forced pass, with no hinted sampling and no hints at test time.
- The paper includes useful controls: hint-only, reassigned and generic procedures, and selector variants.
- The evaluation is broad and covers several external benchmarks.
Critical weaknesses
- The causal claim is not established. The datasets may differ in other properties, and length-based filtering may select on correlated features. Pre-call reasoning is never directly manipulated.
- The improvement is modest. LF25 improves pooled accuracy by about 3.5 points over High, and tool use after RL is rare. It is therefore unclear how much of the gain comes from better tool use.
- The theory adds limited insight. Pre-call reasoning does not appear in the analysis, and the ordering-preservation result follows directly from the idealized dynamics.
- The title does not fit the work well. The paper mainly filters SFT data by reasoning length. It does not really show that the model "thinks twice" before calling tools.
Questions for authors
- What are the source, tool and depth compositions of the filtered subsets?
- Can you directly lengthen or truncate pre-call reasoning while keeping trajectories otherwise fixed?
3 comments
Q1: Source, Tool and Depth Compositions of the Filtered Subsets
We characterize the Original High, LF50, and LF25 SFT datasets along three dimensions: (1) data source, (2) tool type, and (3) tool-call depth.
(1) Data-source composition
Table R3.3. Distribution of source groups (% of trajectories).
| Source group | High | LF50 | LF25 |
|---|---|---|---|
| cof.zoom | 15.20% | 3.47% | 2.19% |
| deepeyesv2.compute_only | 15.70% | 14.17% | 16.88% |
| mini_o3.zoom | 34.80% | 45.00% | 39.38% |
| thyme.compute_only | 34.30% | 37.36% | 41.56% |
Filtering changes the source mixture, particularly reducing CoF representation.
(2) Tool composition
During construction, we typically require the tool-type composition to remain identical across High, LF50, and LF25: each contains 37.5% Zoom, 37.5% Code, and 25% tool-free trajectories.
(3) Tool-call depth
Table R3.4. Tool-call depth distribution (% of all trajectories). Mean depth excludes tool-free examples.
| Number of calls | High | LF50 | LF25 |
|---|---|---|---|
| 0 (tool-free) | 25.00% | 25.00% | 25.00% |
| 4 | 35.95% | 40.28% | 45.62% |
| 5 | 18.60% | 13.06% | 8.44% |
| 6 | 10.70% | 11.25% | 11.56% |
| 7 | 6.05% | 7.22% | 8.12% |
| 8+ | 3.70% | 3.19% | 1.25% |
| Mean depth (tool-using only) | 5.02 | 4.97 | 4.82 |
Filtering slightly reduces average tool-call depth and changes the depth distribution. What filtering essentially does is selecting the SFT trajectories with the longest average pre-call reasoning, rather than explicitly favoring trajectories with fewer tool calls. Therefore, this phenomenon should be interpreted as an empirical observation instead of a natural outcome of the filtering strategy.
In summary, tool-type proportions are exactly preserved, but filtering also changes source composition and, to a lesser extent, call-depth distributions.
Q2: Directly Lengthen or Truncate Pre-Call Reasoning
To directly intervene on pre-call reasoning while retaining the selected examples, we truncate approximately half of each pre-call segment in LF50 without generating new text. We split each segment into sentences, preserve the final sentence before the tool call, and keep the earliest sentences until roughly 50% of the original pre-call tokens remain. The truncated data is trained for four SFT epochs, followed by the original RL procedure.
Table R3.5. Post-RL tool-on accuracy (%) for pre-call truncation.
| Initialization | V*Bench | ChartQA | MathVista | WeMath | CharXiv | Overall |
|---|---|---|---|---|---|---|
| Original High | 84.29 | 85.48 | 65.10 | 57.30 | 31.40 | 66.24 |
| Original LF50 | 84.82 | 85.36 | 67.70 | 58.79 | 37.20 | 67.92 |
| LF50-Truncated | 75.92 | 84.32 | 63.90 | 58.05 | 33.30 | 65.85 |
LF50-Truncated underperforms the original LF50 on all five benchmarks, reducing pooled accuracy from 67.92% to approximately 65.85%. It also falls slightly below Original High, despite using the higher-pre-call LF50 trajectory subset. This pattern suggests that the information removed from pre-call reasoning contributes to the benefit of the selected SFT demonstrations, beyond merely choosing those examples.
Dear Reviewer,
Thank you again for your thoughtful review. We have provided additional experiments and analyses to address your concerns, and we would greatly appreciate any further feedback you may have.
As the discussion period is approaching its end, we would be grateful if you could take another look at our responses and reconsider your overall assessment in light of the new evidence.
Thank you very much for your time and consideration!
W1: Unestablished Causal Claim
We agree that selecting complete trajectories by pre-call reasoning length can also change correlated data properties, so the original filtering results alone do not establish causality. To address this concern, we introduce a composition-matched random control that accounts for data source, tool type, call depth, and trajectory length.
Composition-matched random selection
We construct Matched-Random25 by matching LF25's distributions of data source, tool type, call depth, and trajectory length, while selecting examples randomly within the matched composition. This tests whether the gain can be explained by these dataset properties rather than the pre-call-length criterion.
Table R3.1. Post-RL tool-on accuracy (%) with composition matching.
Compared with the original Random25 (66.67% overall), Matched-Random25 improves to 68.39%, consistent with dataset composition contributing to the original random-selection gap. LF25 nevertheless remains 1.36 points higher than the matched control, with gains on four of five benchmarks. Thus, controlling the measured source, tool, depth, and length distributions reduces—but does not eliminate—the filtering advantage.
W2: Modest Improvement
We agree that the absolute improvement is moderate, but it is consistent across independent training runs. We repeated Original High and LF25 with three seeds under the same 32-step RL setting.
Table R3.2. Post-RL tool-on accuracy (%), mean ± SD across three seeds.
LF25 outperforms High in pooled accuracy in all three seeds, with an average gain of +2.11 percentage points and lower variation. Given the relatively small scale of our SFT data, we consider this consistent improvement from a simple filtering strategy—without modifying the model or RL procedure.
W3: Limited Theoretical Insight
We thank the reviewer for this comment. We agree that the ordering-preservation result follows from the assumed idealized dynamics. However, we believe its main value lies in the implication it provides for SFT data design, rather than the mathematical complexity of the result itself.
Specifically, the analysis shows that advantages in both overall and tool-conditioned accuracy at SFT initialization are preserved under the idealized RL dynamics, even though tool-initiation rates need not preserve their ordering. This suggests that improving the reliability and effectiveness of tool-use behaviors learned during SFT may be more important than simply increasing tool-call frequency or depth.
This motivates our data-selection strategy: we seek SFT demonstrations that teach more reliable and useful tool interactions. We use pre-call reasoning length as an empirical proxy for identifying such demonstrations, and evaluate its effectiveness through length-based filtering and additional controlled experiments.
We acknowledge that pre-call reasoning length is not explicitly modeled in the theoretical dynamics. We will clarify that the theory motivates the SFT optimization objective, while the filtering experiments investigate a practical proxy for achieving it. We will revise the presentation to make this theoretical-to-empirical connection more explicit.
W4: Improper Title
We agree that "Think Twice" may suggest an explicit reconsideration mechanism that our method does not implement. We will therefore revise the title to "Which Tool Behaviors Survive RL? Lessons from Pre-Call Reasoning in SFT." This better reflects our central question—how SFT initialization shapes the tool-use behaviors retained during RL—without implying that we explicitly introduce a second reasoning stage or establish a universal causal mechanism.