acceptodds
Under review as a conference paper at ICLR 2027

Think Twice Before Calling Tools: How Pre-Call Reasoning in SFT Shapes Tool-Use Retention under RL

Abstract

Tool use learned during supervised fine-tuning (SFT) is not always retained during reinforcement learning (RL). We study how properties of SFT data may affect this retention by constructing Low, Medium, and High SFT datasets with approximately matched token budgets but different tool-call depths. Under the same RL recipe, the three initializations reach similar RL training accuracy yet exhibit markedly different tool-use dynamics: tool initiation increases in Low, declines in Medium, and collapses in High. We observe that these datasets also differ substantially in the amount of reasoning preceding each tool call. In particular, High trajectories contain considerably less pre-call reasoning, motivating the hypothesis that pre-call reasoning may affect how well tool use is calibrated and, consequently, whether the corresponding behaviors are retained during RL. An idealized theoretical analysis provides a selective-retention view of this process: advantages in overall and tool-conditioned accuracy can be preserved under outcome-only optimization, while tool initiation increases only when tool-using trajectories are more successful than the tool-free alternative. Guided by this analysis, we rank the original High trajectories by average pre-call reasoning length and retain the highest-ranked examples while approximately matching cumulative SFT token exposure. This simple filtering strategy improves post-RL accuracy and tool-conditioned final-answer success without consistently increasing tool initiation across multiple benchmarks. These results suggest that pre-call reasoning is a useful dimension of SFT supervision for understanding and improving tool-use retention under downstream RL.

Primary area:
foundation or frontier models, including LLMs
Keywords:
tool-integrated reasoning, multimodal large language models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_uWptOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper shows that SFT datasets with different tool-call depths lead to very different tool-use dynamics under identical outcome-only RL. Tool initiation rises for Low and collapses for High. The authors hypothesize that pre-call reasoning is the key factor, support this with an idealized analysis, and show that filtering High-depth data by pre-call length improves post-RL accuracy.

Critical strengths
  • The idea is simple and cheap: one extra teacher-forced pass, with no hinted sampling and no hints at test time.
  • The paper includes useful controls: hint-only, reassigned and generic procedures, and selector variants.
  • The evaluation is broad and covers several external benchmarks.
Critical weaknesses
  1. The causal claim is not established. The datasets may differ in other properties, and length-based filtering may select on correlated features. Pre-call reasoning is never directly manipulated.
  2. The improvement is modest. LF25 improves pooled accuracy by about 3.5 points over High, and tool use after RL is rare. It is therefore unclear how much of the gain comes from better tool use.
  3. The theory adds limited insight. Pre-call reasoning does not appear in the analysis, and the ordering-preservation result follows directly from the idealized dynamics.
  4. The title does not fit the work well. The paper mainly filters SFT data by reasoning length. It does not really show that the model "thinks twice" before calling tools.
Questions for authors
  1. What are the source, tool and depth compositions of the filtered subsets?
  2. Can you directly lengthen or truncate pre-call reasoning while keeping trajectories otherwise fixed?
3 comments

W1: Unestablished Causal Claim

We agree that selecting complete trajectories by pre-call reasoning length can also change correlated data properties, so the original filtering results alone do not establish causality. To address this concern, we introduce a composition-matched random control that accounts for data source, tool type, call depth, and trajectory length.

Composition-matched random selection

We construct Matched-Random25 by matching LF25's distributions of data source, tool type, call depth, and trajectory length, while selecting examples randomly within the matched composition. This tests whether the gain can be explained by these dataset properties rather than the pre-call-length criterion.

Table R3.1. Post-RL tool-on accuracy (%) with composition matching.

InitializationV*BenchChartQAMathVistaWeMathCharXivOverall
High84.2985.4865.1057.3031.4066.24
Random2581.6882.2066.9060.3435.8066.67
Matched-Random2585.3487.1667.4057.1438.8068.39
LF2586.9187.7268.9060.5738.4069.75

Compared with the original Random25 (66.67% overall), Matched-Random25 improves to 68.39%, consistent with dataset composition contributing to the original random-selection gap. LF25 nevertheless remains 1.36 points higher than the matched control, with gains on four of five benchmarks. Thus, controlling the measured source, tool, depth, and length distributions reduces—but does not eliminate—the filtering advantage.

W2: Modest Improvement

We agree that the absolute improvement is moderate, but it is consistent across independent training runs. We repeated Original High and LF25 with three seeds under the same 32-step RL setting.

Table R3.2. Post-RL tool-on accuracy (%), mean ± SD across three seeds.

InitializationV*BenchChartQAMathVistaWeMathCharXivOverall
High83.42 ± 1.5186.15 ± 0.7566.10 ± 1.0558.22 ± 1.1335.20 ± 3.3667.47 ± 1.08
LF2585.51 ± 1.3287.56 ± 0.2468.73 ± 0.3859.54 ± 1.0139.90 ± 1.3169.58 ± 0.19

LF25 outperforms High in pooled accuracy in all three seeds, with an average gain of +2.11 percentage points and lower variation. Given the relatively small scale of our SFT data, we consider this consistent improvement from a simple filtering strategy—without modifying the model or RL procedure.

W3: Limited Theoretical Insight

We thank the reviewer for this comment. We agree that the ordering-preservation result follows from the assumed idealized dynamics. However, we believe its main value lies in the implication it provides for SFT data design, rather than the mathematical complexity of the result itself.

Specifically, the analysis shows that advantages in both overall and tool-conditioned accuracy at SFT initialization are preserved under the idealized RL dynamics, even though tool-initiation rates need not preserve their ordering. This suggests that improving the reliability and effectiveness of tool-use behaviors learned during SFT may be more important than simply increasing tool-call frequency or depth.

This motivates our data-selection strategy: we seek SFT demonstrations that teach more reliable and useful tool interactions. We use pre-call reasoning length as an empirical proxy for identifying such demonstrations, and evaluate its effectiveness through length-based filtering and additional controlled experiments.

We acknowledge that pre-call reasoning length is not explicitly modeled in the theoretical dynamics. We will clarify that the theory motivates the SFT optimization objective, while the filtering experiments investigate a practical proxy for achieving it. We will revise the presentation to make this theoretical-to-empirical connection more explicit.

W4: Improper Title

We agree that "Think Twice" may suggest an explicit reconsideration mechanism that our method does not implement. We will therefore revise the title to "Which Tool Behaviors Survive RL? Lessons from Pre-Call Reasoning in SFT." This better reflects our central question—how SFT initialization shapes the tool-use behaviors retained during RL—without implying that we explicitly introduce a second reasoning stage or establish a universal causal mechanism.

Q1: Source, Tool and Depth Compositions of the Filtered Subsets

We characterize the Original High, LF50, and LF25 SFT datasets along three dimensions: (1) data source, (2) tool type, and (3) tool-call depth.

(1) Data-source composition

Table R3.3. Distribution of source groups (% of trajectories).

Source groupHighLF50LF25
cof.zoom15.20%3.47%2.19%
deepeyesv2.compute_only15.70%14.17%16.88%
mini_o3.zoom34.80%45.00%39.38%
thyme.compute_only34.30%37.36%41.56%

Filtering changes the source mixture, particularly reducing CoF representation.

(2) Tool composition

During construction, we typically require the tool-type composition to remain identical across High, LF50, and LF25: each contains 37.5% Zoom, 37.5% Code, and 25% tool-free trajectories.

(3) Tool-call depth

Table R3.4. Tool-call depth distribution (% of all trajectories). Mean depth excludes tool-free examples.

Number of callsHighLF50LF25
0 (tool-free)25.00%25.00%25.00%
435.95%40.28%45.62%
518.60%13.06%8.44%
610.70%11.25%11.56%
76.05%7.22%8.12%
8+3.70%3.19%1.25%
Mean depth (tool-using only)5.024.974.82

Filtering slightly reduces average tool-call depth and changes the depth distribution. What filtering essentially does is selecting the SFT trajectories with the longest average pre-call reasoning, rather than explicitly favoring trajectories with fewer tool calls. Therefore, this phenomenon should be interpreted as an empirical observation instead of a natural outcome of the filtering strategy.

In summary, tool-type proportions are exactly preserved, but filtering also changes source composition and, to a lesser extent, call-depth distributions.

Q2: Directly Lengthen or Truncate Pre-Call Reasoning

To directly intervene on pre-call reasoning while retaining the selected examples, we truncate approximately half of each pre-call segment in LF50 without generating new text. We split each segment into sentences, preserve the final sentence before the tool call, and keep the earliest sentences until roughly 50% of the original pre-call tokens remain. The truncated data is trained for four SFT epochs, followed by the original RL procedure.

Table R3.5. Post-RL tool-on accuracy (%) for pre-call truncation.

InitializationV*BenchChartQAMathVistaWeMathCharXivOverall
Original High84.2985.4865.1057.3031.4066.24
Original LF5084.8285.3667.7058.7937.2067.92
LF50-Truncated75.9284.3263.9058.0533.3065.85

LF50-Truncated underperforms the original LF50 on all five benchmarks, reducing pooled accuracy from 67.92% to approximately 65.85%. It also falls slightly below Original High, despite using the higher-pre-call LF50 trajectory subset. This pattern suggests that the information removed from pre-call reasoning contributes to the benefit of the selected SFT demonstrations, beyond merely choosing those examples.

Dear Reviewer,

Thank you again for your thoughtful review. We have provided additional experiments and analyses to address your concerns, and we would greatly appreciate any further feedback you may have.

As the discussion period is approaching its end, we would be grateful if you could take another look at our responses and reconsider your overall assessment in light of the new evidence.

Thank you very much for your time and consideration!

Reviewer_bkm7OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates how supervised fine-tuning (SFT) data shapes tool-use behavior during subsequent outcome-only reinforcement learning (RL). Using Qwen2.5-VL-7B with Zoom and Code tools, the authors compare SFT datasets with different tool-call depths under approximately matched token budgets. Despite similar RL training accuracy, the resulting policies exhibit substantially different tool-initiation dynamics, with high-depth initialization showing a sharp decline in tool use. The authors hypothesize that reasoning preceding each tool call influences tool-use calibration and provide an idealized analysis relating tool retention to the relative success of tool-using and tool-free trajectories. Motivated by these observations, they filter high-depth SFT trajectories by average pre-call reasoning length while approximately matching cumulative supervision budgets. Across five benchmarks, this strategy improves post-RL accuracy and tool-availability gains over the original dataset and a matched random-selection baseline, without consistently increasing tool-call frequency.

Critical strengths

None

Critical weaknesses
  1. The role of pre-call reasoning is not sufficiently isolated. Filtering complete trajectories by average pre-call reasoning length may also change data-source composition, task difficulty, tool-call depth, and trajectory quality. Since the SFT data combines multiple datasets, potential differences in teacher reasoning styles introduce another confound. The random-selection baseline supports the usefulness of the selection criterion but does not establish whether the gains specifically arise from pre-call reasoning. The theoretical analysis does not explicitly model this factor either.

  2. The conclusions lack evidence of robustness across training conditions. All core results use Qwen2.5-VL-7B, Zoom/Code tools, and one GRPO configuration, while filtering is evaluated only on the High dataset. Without multiple seeds, longer training runs, or interaction-budget variations, it remains unclear whether the observed collapse and filtering gains reflect stable behavior or a particular training regime. Evaluation on five benchmarks broadens task coverage but does not resolve this limitation.

Questions for authors
  1. Does pre-call-length filtering still improve downstream RL performance when data source, task difficulty, and tool-call depth are controlled? A matched-selection experiment or a same-question trajectory-editing ablation would substantially strengthen the proposed explanation. How does it compare with filtering by total reasoning length?

  2. Do the tool-initiation differences and filtering gains persist across independent seeds and longer RL training? Can the authors additionally demonstrate the benefit under another backbone, RL configuration, or interaction budget to establish that it extends beyond the current setting?

5 comments

W1Q1: Controlled Selection and Total-Reasoning-Length Filtering

Thank you for raising this insightful question. To address the concern that pre-call filtering may select trajectories with different source, tool composition, or depth, we evaluate Matched-Random25, drawn from the High pool to match LF25's distributions of data source, tool type, call depth, and trajectory length. We also evaluate Total-Reasoning25, selecting trajectories by total natural-language reasoning length rather than average pre-call length per call, under the same nominal 25% supervision-token budget.

Table R2.1. Post-RL tool-on accuracy (%) for selection controls.

InitializationV*BenchChartQAMathVistaWeMathCharXivOverall
Original High84.2985.4865.1057.3031.4066.24
Random2581.6882.2066.9060.3435.8066.67
Matched-Random2585.3487.1667.4057.1438.8068.39
Total-Reasoning2576.4483.5666.2054.5734.3065.15
LF2586.9187.7268.9060.5738.4069.75

The original Random25 is substantially below LF25 (66.67% versus 69.75%), but this difference may partly reflect correlated changes in source, tool type, call depth, and trajectory length. After matching these properties, the random baseline improves to 68.39%, yet LF25 still leads by 1.36 points overall and on four of five benchmarks. This indicates that the measured composition factors explain part, but not all, of the filtering advantage. LF25 also outperforms Total-Reasoning25 on all five benchmarks, with a pooled advantage of 4.61 points, supporting average pre-call length as a more informative selection proxy than total reasoning length in this setting.

W2Q2: Robustness across Training Conditions

Thank you for raising the concern that our observations may depend on a particular training run or configuration. We investigate this through three complementary checks: independent seeds, longer RL training, and an alternative backbone.

(1) Multiple training seeds

We evaluate Original High and LF25 under three independent training seeds with the same 32-step RL configuration. Table R2.2 reports tool-on accuracy as mean ± sample standard deviation.

Table R2.2. Post-RL tool-on accuracy (%) across three seeds.

InitializationV*BenchChartQAMathVistaWeMathCharXivOverall
High83.42 ± 1.5186.15 ± 0.7566.10 ± 1.0558.22 ± 1.1335.20 ± 3.3667.47 ± 1.08
LF2585.51 ± 1.3287.56 ± 0.2468.73 ± 0.3859.54 ± 1.0139.90 ± 1.3169.58 ± 0.19

LF25 outperforms High in overall accuracy for all three seeds. Its average advantage is +2.11 percentage points, while its overall accuracy also varies less across runs.

(2) Longer RL training

To assess whether the improvement persists beyond the original 32 updates, we extend High and LF25 to 64 RL steps using the same training setting.

Table R2.3. Post-RL tool-on accuracy (%) after 64 updates.

InitializationV*BenchChartQAMathVistaWeMathCharXivOverall
High83.7786.1665.3050.8640.1066.13
LF2584.2988.1268.9058.0539.7069.35

LF25 retains an advantage over High after 64 RL updates, with a pooled gain of +3.22 percentage points and higher accuracy on four of the five benchmarks.

Table R2.4. Tool initiation rate (%) throughout RL training. The seven checkpoints correspond to the start and steps 8, 16, 24, 32, 48, and 64.

InitializationStartStep 8Step 16Step 24Step 32Step 48Step 64
High59.8012.346.294.922.110.200.12
LF2558.4836.022.897.115.208.7919.10

Both models show an overall decline in tool initiation, but the severity and persistence of collapse differ. High falls almost continuously from 59.80% to just 0.12% at step 64, indicating a sustained, near-complete collapse. LF25 also drops sharply at first, but subsequently fluctuates and partially rebounds, reaching 19.10% at step 64. Thus, the longer run confirms the persistent collapse of High, whereas LF25 does not remain collapsed throughout later training.

(3) Alternative backbone

To examine whether the filtering benefit extends beyond Qwen2.5-VL-7B, we also compare High and LF25 using Qwen3.5-4B. As shown in Table R2.5, LF25 improves accuracy on all five benchmarks, suggesting that the benefit is not exclusive to the original backbone.

Table R2.5. Qwen3.5-4B post-RL tool-on accuracy (%).

InitializationV*BenchChartQAMathVistaWeMathCharXivOverall
High79.0686.2871.7065.7150.5072.67
LF2581.6886.8473.7067.0555.1074.35

In summary, LF25 outperforms High in pooled tool-on accuracy across all three seeds, after 64 RL steps, and on all five evaluated benchmarks with Qwen3.5-4B. The extended training run further shows that High's tool-initiation collapse persists to near zero, while LF25 declines initially but later fluctuates and partially recovers. Together, these results address the reviewer's concerns about the stability of the filtering gains and the persistence of tool-use collapse under longer training.

Thanks for the additional experiments. They address some of my concerns, but I’m still not convinced that pre-call reasoning itself explains the gains. The gap becomes much smaller with Matched-Random25, and LF25 may still be selecting better data in other ways, such as reasoning quality or task difficulty. The filtering does seem to help, but it remains unclear whether pre-call reasoning is actually the key factor.

Reviewer_py7UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how properties of SFT data may affect tool-use retention during RL. The authors construct Low, Medium, and High SFT datasets with approximately matched token budgets but different tool-call depths. Under the same RL recipe, the three initializations reach similar RL training accuracy yet exhibit markedly different tool-use dynamics. An idealized theoretical analysis provides a selective-retention view of this process. Guided by this analysis, the authors rank the original High trajectories by average pre-call reasoning length and retain the highest-ranked examples. This filtering strategy improves post-RL accuracy and tool-conditioned final-answer success without consistently increasing tool initiation across multiple benchmarks. These results suggest that pre-call reasoning is a useful dimension of SFT supervision for understanding and improving tool-use retention under downstream RL.

Critical strengths

1.This paper investigates whether tool-use behaviors learned during SFT are retained during RL. By distinguishing overall accuracy, tool-initiation rate, and tool-conditioned accuracy, it provides a more fine-grained perspective than evaluating final task performance alone.

2.Figure 1 shows that the Low, Medium, and High initializations converge to similar RL training accuracy while exhibiting markedly different tool-initiation dynamics, with the rate for High eventually dropping to approximately 5%. This observation highlights that similar task performance does not necessarily imply that models have learned the same tool-use strategies.

Critical weaknesses

1.This paper evaluates only two tool types, Zoom and Code, using a single base model, leaving the generalizability of its findings unclear. Although these tools represent visual evidence acquisition and programmatic computation, respectively, they differ substantially in pre-call planning, information acquisition, and tool-output utilization. Consequently, the findings may not generalize to other multimodal tool-use settings. Figure 2(a) further highlights this concern. The mean pre-call reasoning length for Code decreases sharply as tool-call depth increases, from 991 tokens for Low to 423 for Medium and 134 for High. In contrast, the corresponding values for Zoom vary much less and do not follow the same monotonic trend. This discrepancy suggests that the overall pattern may be strongly influenced by the specific tool type. The authors should therefore report tool-use dynamics and the effects of length-based filtering separately for Zoom and Code to establish whether the main findings hold consistently across both tools. Evaluating additional tool types would further strengthen the paper's broader claims about multimodal tool-use agents. If such experiments are not feasible, the authors should more clearly delimit the scope of their conclusions.

2.The distributional differences among the SFT datasets are insufficiently characterized. Although the authors match the number of trajectories, tool-type composition, and approximate total token budgets, they do not adequately report trajectory-length distributions or token allocation across reasoning text, tool-call instructions, and tool observations. Given the differences in tool-call depth across datasets, these factors may jointly affect pre-call reasoning length and the effective supervision received by the model. Moreover, length-based filtering may alter the original data distribution. For example, if Low and High trajectories have comparable per-call reasoning lengths, High trajectories may be substantially longer due to their greater number of tool calls. Without detailed statistics on the selected trajectories, it remains unclear whether the filtering process disproportionately excludes such High trajectories, potentially biasing the resulting data distribution and limiting the validity of the conclusions.

3.The analysis of the models after SFT remains relatively coarse. Although the authors report aggregate metrics such as accuracy and tool-initiation rates, they do not sufficiently investigate what tool-use behaviors the models have learned from the respective SFT datasets. In particular, it remains unclear whether differences in pre-call reasoning length translate into differences in the models' ability to determine when tools are useful, select appropriate tools, and effectively utilize tool outputs. This gap makes it difficult to establish a clear connection between SFT data properties, the behaviors acquired by the resulting models, and subsequent tool-use dynamics under RL, thereby weakening the evidence supporting the paper's central claims.

Questions for authors

1.Could the authors report tool-use dynamics and length-filtering results separately for Zoom and Code, given their substantially different pre-call reasoning trends in Figure 2(a)? Can the main findings be validated on additional tool types, or can the claims be more clearly scoped to the current setting?

2.Could the authors provide detailed trajectory-length distributions and token allocations across reasoning, tool calls, and tool observations for each SFT dataset? Could they also clarify whether length-based filtering disproportionately excludes longer High trajectories and conduct additional controls to disentangle the effects of pre-call reasoning length, tool-call depth, and trajectory composition?

3.Could the authors provide a more fine-grained analysis of the behaviors learned by the SFT models, particularly their ability to determine when tools are useful, select appropriate tools, and utilize tool outputs effectively? How do these behaviors relate to the observed differences in tool-use dynamics during subsequent RL?

4 comments

W1Q1: Tool-Specific Dynamics and Length-Filtering Effects

We thank the reviewer for raising the concern that our findings may depend on the specific tool type. We agree that the original paper did not sufficiently distinguish the behaviors of Zoom and Code. To address this, we provide additional tool-specific analyses in three aspects: (1) the effects of length filtering on SFT data, (2) the evolution of pre-call reasoning during RL, and (3) the tool-initiation dynamics of High and LF25.

(1) The length filtering effects on the SFT datasets

We first examine the mean pre-call reasoning length for Zoom and Code in the High, LF50, and LF25 SFT datasets. As shown in Table R1.1, despite their different original pre-call reasoning patterns, length filtering consistently increases pre-call reasoning length for both tools. From High to LF25, the mean length increases from 184 to 297 tokens for Zoom and from 134 to 216 tokens for Code. This confirms that the filtering effect on SFT data is not driven exclusively by Code.

Table R1.1. Mean pre-call reasoning tokens per call in the SFT datasets.

DatasetOverallZoomCode
Original High161184134
LF50231262189
LF25263297216

(2) The comparison of mean pre-call reasoning tokens for Zoom and Code

As shown in Table R1.2, both Zoom and Code exhibit longer first-call pre-call reasoning after RL, but the changes differ in magnitude. In High, Zoom increases by 152 tokens, compared with 107 for Code; in LF25, the increases are 123 and 39, respectively. Thus, LF25 starts with longer first-call reasoning for both tools and reaches comparable final lengths with a smaller shift, especially for Code. The later calls reveal clearer tool-specific differences: under High, Zoom increases more than Code at both the second and third calls; under LF25, Code increases at both positions, whereas Zoom is nearly unchanged or slightly decreases. Overall, the shared first-call trend supports our aggregate observation, while later-call dynamics are tool-dependent.

Table R1.2. Mean pre-call reasoning tokens at RL start → end, separated by call position.

InitializationCall positionOverallZoomCode
High1st266 → 385196 → 348284 → 391
High2nd147 → 162202 → 256130 → 142
High3rd156 → 179226 → 322126 → 139
LF251st312 → 377231 → 354339 → 378
LF252nd185 → 202252 → 256158 → 197
LF253rd195 → 198280 → 268147 → 187

(3) The tool initiation rate for Zoom and Code

As shown in Table R1.3, tool initiation decreases substantially for both Zoom and Code during RL, with a more pronounced decline for Zoom. For High, Zoom initiation decreases from 30.55% to 1.68%, while Code decreases from 32.66% to 6.29%. LF25 exhibits the same pattern, with Zoom decreasing from 33.44% to 0.22% and Code from 26.02% to 5.40%. These results demonstrate that the overall decline in tool use holds for both tool types, with different magnitudes.

Table R1.3. Tool initiation rate (%) at RL start → end.

InitializationOverallZoomCode
High62.27 → 7.9730.55 → 1.6832.66 → 6.29
LF2557.15 → 5.5733.44 → 0.2226.02 → 5.40

Taken together, although Zoom and Code exhibit different pre-call reasoning patterns in the original SFT datasets, our additional analyses demonstrate that the main qualitative findings hold for both tools. These consistent trends indicate that our overall observations are not driven solely by Code, although the magnitude of the changes and the detailed dynamics differ between tools. They also provide further insight into how pre-call supervision in SFT relates to the selective retention of tool-use behaviors during RL.

We also acknowledge that our empirical scope is currently limited to Zoom and Code with the evaluated backbone and RL setting, and will state this explicitly in the revised manuscript. Nevertheless, observing the same broad distinction between pre-call supervision and selective tool-use retention across two functionally distinct tools offers useful insight and motivates testing whether it extends to other tool families.

W2Q2: Trajectory-Length Distributions and Token Allocations

We address the reviewer's concerns in three parts: (1) trajectory-length distributions and what filtering retains, (2) token allocation across reasoning, tool calls, and observations, and (3) a composition-matched random control to assess whether the gains are explained by correlated data properties.

(1) Trajectory-length distributions and filtering

Length filtering favors trajectories with more substantial supervised content. As shown in Table R1.4, both the mean and central length quantiles increase from High to LF50 and LF25, consistent with the longer trajectories observed for Zoom and Code separately.

Table R1.4. Distribution of assistant-supervised tokens per trajectory. P0/P100 indicate the minimum/maximum; P25, P50, and P75 denote the 25th, 50th (median), and 75th percentiles. Mean is the arithmetic average.

DatasetMeanP0P25P50P75P100
Low804.3262005251,358.255,058
Medium863.426349.75753.51,3982,749
High930.8264969211,3193,867
LF501,220.1278071,2751,726.753,059
LF251,353.3289291,354.51,945.253,059

(2) Token allocation

We separately report full multimodal and text-only token allocation. Table R1.5 shows that tool observations occupy a larger share as interaction depth increases, reflecting the longer context introduced by tool returns and images. These observations provide context and are masked from the SFT loss.

Table R1.5. Native interaction token allocation (%), including Zoom image tokens.

DatasetPre-call reasoningTool actionTool observationFinal reasoning + answer
Low47.08%9.60%18.94%24.39%
Medium40.29%10.54%36.73%12.44%
High26.44%9.34%59.34%4.88%
LF5027.05%6.93%61.71%4.31%
LF2531.90%7.66%54.52%5.92%

Table R1.6 compares textual composition without image-token expansion. The pre-call text share rises from 59.04% in High to approximately 65.6% in both filtered subsets. The contrast with Table R1.5 largely reflects different observation-token proportions.

Table R1.6. Text-only token allocation (%), excluding image-token expansion.

DatasetPre-call reasoningTool actionTool observationFinal reasoning + answer
Low56.57%11.52%3.04%28.87%
Medium60.35%15.82%5.50%18.33%
High59.04%21.08%9.19%10.68%
LF5065.52%16.83%7.42%10.23%
LF2565.61%15.77%6.60%12.02%

(3) Composition-Matched Random Selection

To assess whether LF25's gains arise from changes in dataset composition, we construct Matched-Random25 from High, approximating LF25's distributions of data source, tool type, tool-call depth, and trajectory length, and apply the same SFT and RL recipe.

Table R1.7. Post-RL tool-on accuracy (%) for matched selection.

InitializationV*BenchChartQAMathVistaWeMathCharXivOverall
High84.2985.4865.1057.3031.4066.24
Random2581.6882.2066.9060.3435.8066.67
Matched-Random2585.3487.1667.4057.1438.8068.39
LF2586.9187.7268.9060.5738.4069.75

Matching improves the random baseline from 66.67% to 68.39%, indicating that dataset composition contributes to the original gap. LF25 remains 1.36 points ahead of Matched-Random25 overall and leads on four of five benchmarks, supporting the value of pre-call-length selection beyond the matched source, tool, depth, and trajectory-length characteristics.

W3Q3: Fine-Grained Analysis of Tool-Use Behavior

Thank you for pointing out that overall accuracy and tool-initiation rate alone do not reveal what tool-use behaviors are learned during SFT. We therefore performed a fine-grained analysis of Original High and LF25 checkpoints at both SFT initialization and after RL on MathVista and WeMath. We randomly sampled tool-using trajectories from each checkpoint–benchmark combination without selecting by answer correctness or tool success. We used GPT-6.1-Sol as the judge for all three behavioral assessments at both SFT initialization and RL checkpoints. Specifically, the judge assessed (i) whether the first call was justified by the information and task needs available to Qwen2.5-VL-7B, (ii) whether the tool type (Zoom or Code) and the tool arguments were appropriate, and (iii) whether the output was correctly interpreted and used. For decisions (i) and (ii), the judge did not see future observations, ground truth, or checkpoint identity.

Table R1.8. Fine-grained behavior at SFT initialization. Percentages are based on the relevant observed calls or successful Code returns.

BenchmarkSFT initializationFirst-call justificationAppropriate tool type (all calls)Correct use of successful Code outputs
MathVistaHigh31.2%33.3%7.7%
MathVistaLF2534.4%34.9%16.0%
WeMathHigh28.1%37.6%64.3%
WeMathLF2546.9%42.9%72.4%

Compared with High, LF25 shows higher first-call justification and appropriate-tool rates on both benchmarks. LF25 also makes better use of successful Code outputs: utilization increases from 7.7% to 16.0% on MathVista and from 64.3% to 72.4% on WeMath. These SFT results suggest that the filtered demonstrations help improve both tool-use decisions and the use of returned evidence.

Table R1.9. Fine-grained behavior at RL checkpoints.

BenchmarkRL checkpointFirst-call justificationAppropriate tool type (all calls)Correct use of successful Code outputs
MathVistaHigh62.5%21.5%41.9%
MathVistaLF2546.9%35.5%71.3%
WeMathHigh18.2%57.1%58.5%
WeMathLF2543.8%65.6%84.2%

Table R1.9 points to tool selection and evidence utilization as persistent bottlenecks in High. LF25 has higher appropriate-tool rates on both MathVista (35.5% vs. 21.5%) and WeMath (65.6% vs. 57.1%). Its advantage is especially clear in the correct use of successful Code outputs: 71.3% vs. 41.9% on MathVista and 84.2% vs. 58.5% on WeMath. The LF25–High utilization gap is larger after RL than at SFT initialization on both benchmarks, suggesting that the difference in evidence use remains important at the RL endpoint.

First-call justification is benchmark-dependent: LF25 exceeds High on WeMath (43.8% vs. 18.2%), while High is higher on MathVista (62.5% vs. 46.9%). LF25's consistent advantage therefore lies in tool choice and evidence use among the observed interactions. This complements the initiation-rate curves by showing how well the remaining tool interactions are executed.

Dear Reviewer,

Thank you again for your thoughtful review. We have provided additional experiments and analyses to address your concerns, and we would greatly appreciate any further feedback you may have.

As the discussion period is approaching its end, we would be grateful if you could take another look at our responses and reconsider your overall assessment in light of the new evidence.

Thank you very much for your time and consideration!

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers,

We sincerely thank you again for your thoughtful feedback and constructive suggestions. Throughout the rebuttal period, we have conducted substantial additional experiments and analyses to address your concerns, including:

  • Tool-specific RL dynamics and length-filtering effects (Reviewer py7U, W1Q1).
  • SFT trajectory-length distributions, token allocations, and data-composition analyses (Reviewer py7U, W2Q2; Reviewer uWpt, Q1).
  • Composition-matched random selection (Reviewer py7U, W2Q2; Reviewer bkm7, W1Q1; Reviewer uWpt, W1).
  • Total-reasoning-length filtering and pass@8 task-difficulty analysis (Reviewer bkm7, W1Q1 and follow-up).
  • Direct pre-call reasoning truncation on fixed trajectories (Reviewer uWpt, Q2; Reviewer bkm7, W1Q1 follow-up).
  • Multiple training seeds, extended RL training, and evaluation on an additional backbone (Reviewer bkm7, W2Q2; Reviewer uWpt, W2 for multiple seeds).
  • Fine-grained tool-use behavior analysis (Reviewer py7U, W3Q3).

We believe these new results have substantially strengthened our empirical evidence and clarified the central contributions of our work.

As the discussion period is approaching its end, we would greatly appreciate any final feedback you may have. We would also be grateful if you could revisit your assessments in light of the additional evidence and clarifications we have provided.

Thank you very much for your time and consideration!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.