- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes BranchWise, a post-training framework for tool-use agents that converts successful trajectories into state-conditioned positive actions and verified counterfactual negative branches. The method combines rule and execution-evidence-derived counterfactuals with errors sampled from the initial policy, then trains with evidence-weighted SFT and a localized, margin-aware ranking loss.
Critical strengths
The paper identifies a concrete limitation of trajectory-only SFT: a successful trajectory specifies an observed action but not the local alternatives that a policy must reject. The proposed framework is useful and well connected to tool-use settings.
Critical weaknesses
- Table 8 has severe layout issues.
- The article only conducted experiments using Qwen3-4B and Qwen3-8B. It cannot be determined whether this method is only effective on Qwen-series LLMs.
- This article mentions many related works such as CausalFlow, but there is no comparison.
Questions for authors
- Can you provide more experimental results besides Qwen-series LLMs?
- Can you provide more baseline results such as CausalFlow?
29 comments
Main Response — Part 02/07
Main Response — Part 02/07
| Control | Setting |
|---|---|
| Backbone and initialization | Original Llama-3.1-8B-Instruct; SFT and BranchWise both initialize from this model |
| Correct trajectories | Same 10,200: 9,000 ToolMind mixed tool-use trajectories + 1,200 Tau2 training-domain state trajectories |
| SFT objective | Standard SFT loss on correct completions only, retaining history as context, without evidence weights or error-branch loss |
| BranchWise objective | Evidence-reliability-weighted correct-branch SFT plus margin-aware joint local ranking over correct and incorrect branches sharing the same state |
| Update budget | Both use 1,200 optimizer steps, effective batch size 32, maximum length 8,192 |
| Optimization | Learning rate 2e-4; LoRA , , dropout |
| Templates and parsing | Llama-compatible multi-tool-call history template and tool-call parser |
| Evaluation | BFCL V3, Tau2, ToolHop, ToolTalk-Hard |
Model-specific error construction. We retain verifiable rule/execution-grounded branches but do not treat Qwen's model errors as Llama errors. For the full method, we resample from the original Llama four times at each of 4,800 training-side candidate states, yielding 19,200 responses. Verification and deduplication yield 1,870 distinct errors across 1,185 states, which are then fused with evidence-constructed branches. The data checks pass, and no test examples from the four benchmarks are read during construction.
How the full method learns. At each decision state, the same history and tool context are preserved, and ranking scores are computed only in the local regions where correct and incorrect actions diverge. Correct-branch SFT weights are determined by evidence reliability. One correct action is jointly ranked against several errors, pushing insufficiently separated branches toward the target score margin. E5 and E6 therefore differ in error supervision, local comparison, and the training objective. Section 1.3 compares the complete methods; Section 2.3 tests the objective and error sources through controlled experiments.
Compatibility and evaluation checks. All three settings use the formal Llama-compatible evaluation pipeline. The remaining evaluation settings are the same as in the paper.
1.3 New Training Experiments E5–E6 and Base Evaluation: Results
| Metric | Llama Base | E5 Vanilla SFT | E6 BranchWise | E6 − E5 |
|---|---|---|---|---|
| BFCL V3 Overall | 26.36 | 41.17 | 45.94 | +4.76 pp |
| Tau2 Pass@1 | 15.11 | 14.75 | 22.66 | +7.91 pp |
| ToolHop Direct | 7.14 | 7.24 | 8.14 | +0.90 pp |
| ToolHop Mandatory | 7.34 | 21.31 | 36.78 | +15.48 pp |
| ToolHop Free | 10.45 | 20.50 | 34.27 | +13.77 pp |
| ToolTalk-Hard Success | 16.00 | 18.00 | 20.00 | +2.00 pp |
| ToolTalk-Hard Soft Success | 47.39 | 47.55 | 47.60 | +0.05 pp |
1.4 Analysis: Cross-model gains and sustained tool interaction
The following Tau2 domains and three BFCL components further show where the improvements arise:
| Component | Llama Base | E5 Vanilla SFT | E6 BranchWise |
|---|---|---|---|
| Tau2 Airline | 30.00 | 6.00 | 18.00 |
| Tau2 Retail | 7.02 | 27.19 | 33.33 |
| Tau2 Telecom | 16.67 | 6.14 | 14.04 |
| BFCL Non-Live Overall | 39.18 | 46.27 | 49.07 |
| BFCL Live | 33.41 | 62.37 | 69.61 |
| BFCL Multi-Turn | 6.50 | 14.88 | 19.12 |
-
BranchWise's benefits extend to Llama under the same training conditions. E5 and E6 share the backbone, 10,200 correct trajectories, and update budget. Full BranchWise exceeds standard SFT on all seven aggregate metrics. This directly addresses whether the method is effective only on Qwen.
-
Tasks requiring sequential decisions benefit most. Mandatory and Free improve by 15.48 and 13.77 pp, Tau2 by 7.91 pp, and BFCL Multi-Turn by 4.25 pp, with Long Context increasing from 15.00% to 23.50%. These tasks require continued action selection over multi-step interactions, for which local error branches provide supervision beyond successful paths.
-
BranchWise mitigates the regressions of standard SFT in some domains. SFT reduces Airline from 30.00% for Base to 6.00%, and Telecom from 16.67% to 6.14%. BranchWise recovers them to 18.00% and 14.04%, while further increasing Retail from 27.19% to 33.33%. The three-domain success count grows from 41/278 for SFT to 63/278, indicating better decisions in other domains while learning from training data.
Main Response — Part 03/07
Main Response — Part 03/07
- Aggregate and component results support each other. BFCL Non-Live Overall, Live, and Multi-Turn all improve over SFT, increasing Overall from 41.17% to 45.94%. Direct and ToolTalk-Hard also improve, though less than the tool-interaction settings. These results support cross-family gains in this controlled same-backbone comparison; Airline, Telecom, and parallel calls still exhibit gaps relative to Base.
Revision plan. We will include the full non-Qwen comparison table, BFCL and Tau2 components, and explanations of Llama-specific error construction and compatible templates in the paper.
2. Response to Question 2: CausalFlow comparison and controlled training tests of BranchWise's design
Your original question: Can you provide more baseline results such as CausalFlow?
2.1 Summary and comparison approach
Thank you for suggesting a more direct comparison with related work. We add two components: first, an account of CausalFlow's commonalities and technical differences relative to BranchWise (Section 2.2); second, four new training experiments—E1: Same-Data Action-DPO, E2: Same-Data SimPO, E3: Same-Data Uniform Listwise, and E4: Action-DPO Using Only Model-Generated Errors—to test optimization objectives and error sources separately (settings and results in Section 2.3).
2.2 Analysis of CausalFlow
We analyzed CausalFlow's design and implementation. CausalFlow starts from failed executions, intervenes by replacing candidate steps, and verifies through subsequent execution and feedback whether a failure was repaired, yielding paired incorrect and repaired actions. These pairs can also be used for offline preference learning or reward modeling. Both approaches therefore exploit local error information; the distinction concerns the starting point of supervision, the verification question, and the learning method.
| Comparison | CausalFlow | BranchWise |
|---|---|---|
| Supervision starting point | Existing failed executions and their repairs | Successful reference trajectories, tool/execution evidence, and initial-model errors |
| Verification question | Whether replacing a candidate step repairs the eventual result | Whether an alternative action under the same state violates checkable conditions |
| Learning object | Verified incorrect/repaired action pairs | Local branch groups fusing multi-source errors with weighted SFT and margin-aware ranking |
Role of our design. At locations with sufficient local evidence, BranchWise can recover failure directions from successful trajectories without first requiring complete failed executions and successful repairs. It then combines these alternative actions with actual model errors and specifies ranking positions and objectives. Thus, supervision construction and model updates focus on decision differences at the same state.
Conclusion: A complete CausalFlow comparison would involve failure collection, intervention-based repair, verification, and training, with a final score reflecting the entire pipeline. We instead prioritize E1–E4 to distinguish the respective effects of error construction and the ranking objective in our study.
2.3 New Training Experiments E1–E4: Separate tests of optimization objectives and error sources
We add four training experiments:
- E1: Same-Data Action-DPO. Retains the complete BranchWise error pool but replaces ranking with a local pairwise preference objective relative to the initial model.
- E2: Same-Data SimPO. Retains the same local positive/negative actions but uses a reference-free pairwise preference objective.
- E3: Same-Data Uniform Listwise. Retains joint comparison across multiple errors, sets the positive target margin to zero, and uses ordinary softmax contrast.
- E4: Action-DPO Using Only Model-Generated Errors. Uses the same local DPO as E1 but removes evidence-constructed errors and retains only responses actually generated by the initial model and verified as incorrect, for comparison against the complete pool.
Experimental objective. Test separately whether the full method's gains can be explained by an existing local preference objective or by simply adding model failures.
Experimental setup.
Main Response — Part 01/07
** Main Response — Part 01/07**
During this rebuttal, we added extensive controlled experiments, ablations, and in-depth analyses and redrew the paper's analysis figures, further improving the experimental validation, technical argumentation, and presentation of BranchWise. We believe this additional evidence makes the paper more complete, rigorous, and persuasive. We sincerely hope that the reviewer will consider these improvements when reassessing the paper's score. Your recognition would also encourage us to continue pursuing more rigorous, interesting, and valuable research.
Thank you for recognizing the issue studied in this paper: successful trajectories show only the actions actually taken, without specifying which alternative local actions the model should reject. We agree that the original submission's evaluation on the Qwen family alone is insufficient to establish transfer to other models, and that the related-work comparison and presentation of Table 8 also need improvement.
In response, we conducted six new training experiments during the rebuttal:
We also completed one Base evaluation of the untrained Llama backbone as the shared starting point for E5 and E6 (Sections 1.2–1.3). We additionally include a methodological comparison with CausalFlow and one new analysis, A5: Evaluation Protocol and Aggregation Audit (Sections 3.1–3.3).
We first answer the two Questions for Authors, then address the three Critical Weaknesses individually, for a total of five issues. Overlapping issues are cross-referenced by section number within this document.
1. Response to Question 1: New Llama experiments show that multi-step tool-use gains are not restricted to Qwen
1.1 Summary of our response
Yes. We conducted two new Llama-3.1-8B-Instruct training experiments, E5: Standard SFT and E6: Full BranchWise, and completed one Base evaluation using the original untrained Llama model as the common starting point. The three settings use the same model-family-compatible tool-call templates and scoring pipeline and are evaluated on four benchmarks. Full settings and results appear in Sections 1.2–1.4.
Compared with SFT on the same backbone, BranchWise improves all seven aggregate metrics across the four benchmarks. Tau2 and ToolHop Mandatory/Free increase by 7.91, 15.48, and 13.77 pp, respectively, and BFCL Overall increases by 4.76 pp. This directly supports the effectiveness of local decision supervision on another model family.
1.2 New Training Experiments E5–E6 and Base Evaluation: Objectives, data, and setup
Experimental objective. Determine whether local error supervision beyond successful trajectories and the complete training objective remain helpful when changing the model family.
New comparisons. This section includes two new training experiments: E5, Llama Standard SFT, which learns only correct trajectories; and E6, Llama Full BranchWise, which adds verified local error branches, evidence-weighted supervision, and margin-aware ranking to the same correct trajectories. A Base evaluation of the original untrained Llama serves as their common starting point. All three use Llama-3.1-8B-Instruct. Full results are in Section 1.3 and detailed analyses in Section 1.4.