acceptodds
Under review as a conference paper at ICLR 2027

Bench2Robust: Scenario-Controlled Recovery for Tool-Using Agents

Abstract

Tool-using LLM agents are typically developed and post-trained in controlled environments with reliable tools, but deployment exposes them to tool failures, API and schema drift, and third-party services outside the developer's control. We introduce BENCH2ROBUST, a framework for scenario-controlled solvability that augments existing benchmarks with episodes requiring retrying the original path, switching to a fallback-equivalent tool, or recognizing that no viable path remains. By controlling recoverability, BENCH2ROBUST separates recovery-policy quality from randomness in whether failures persist or resolve. Across 7 models from 4 families and 10 task/domain slices spanning customer-service and function-calling benchmarks, 69 of 70 model-subset pairs degrade under tool failures, by up to 46.7 percentage points. We propose Bayesian Tool Memory (BTM) for structured runtime recovery context and use reinforcement learning (RL) for learned recovery behavior. On held-out Retail tasks using the benchmark's original tool set, BTM improves robustness by 16.8 percentage points without retraining, while RL retains a 6.3-point gain without inference-time BTM. Combining RL with BTM yields a 20.7-point improvement over the base model while preserving clean performance. These results show that robust tool use benefits from combining structured runtime recovery context with learned recovery behavior.

Primary area:
datasets and benchmarks
Keywords:
tool-using agents, agent robustness, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_67FaOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Bench2Robust augments tool-use benchmarks with retry-solvable, switch-required, and unsolvable scenarios, alongside explicit failures and silent response corruptions. Bayesian Tool Memory (BTM) supplies recovery instructions, fallback relations, and empirical recovery statistics, and is combined with RL training. On 402 held-out Combined Retail tasks without added alternatives, online BTM raises the 4B base model's injected pass rate from 20.1% to 36.9%; RL without inference-time BTM reaches 26.4%, and the full system reaches 40.8%. S3 is a training condition, not a demonstrated independently learned abstention capability.

Critical strengths
  1. Controlling whether the original path remains viable or switching is required is a useful way to study recovery. The proposed design distinguishes choosing an appropriate recovery strategy from benefiting when a random failure disappears. The distinction between execution-suppressing failures and post-execution observation corruption also makes the modeling assumptions explicit.

  2. Tables 2 and 12 provide useful held-out evidence for runtime recovery context and behavior retained after training. Both 4B and 8B improve without inference-time BTM, with further gains when it is supplied. Clean-performance results, severity sweeps, and inference-cost reporting characterize practical trade-offs. The paper appropriately acknowledges that online BTM uses completed-episode outcomes and that evaluation seeds do not measure training-run variability.

Critical weaknesses

The supplied implementation conflicts with the central scenario definition. Section 2.1 and Algorithm 1 place episode-persistent blocking outside the stochastic injection budget and consecutive-failure limits. However, GenericInjectionEngine.should_inject passes blocked tools through InjectionPolicy.can_inject, which enforces both limits. The supplied test test_permanent_block_respects_write_protection_and_budget explicitly expects blocking to stop once the budget is exhausted, consistent with the README. If these semantics were used experimentally, a supposedly switch-required episode could instead be solved by retrying, changing what the evaluation measures. The README says the modified training fork is omitted, so this discrepancy does not establish that the reported experiments used incorrect semantics. Nevertheless, the connection between the stated intervention and the experimental implementation remains unresolved. This implementation gap limits how confidently the gains can be interpreted as recovery under the stated scenario controls.

Questions for authors

Which implementation and revision generated the scenario-controlled results? Please reconcile it with the supplied engine and test, and provide a trace showing that an S2/S3-blocked tool remains unavailable after both stochastic limits are reached. A separate experimental implementation enforcing Algorithm 1 could resolve the concern; if the supplied semantics were used, which scenario labels and results are affected?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 67Fa

We thank the reviewer for recognizing the value of our scenario-controlled evaluation and empirical results.

We traced the implementation discrepancy issue to an error introduced during anonymization and code packaging. In our original implementations, both the evaluation harness (eval/src/run_inference.py:367-388) and the RL training environment (training/bench2robust_rl/rollout.py:982-994) check the blocked set before consulting the stochastic policy, and never call it for blocked tools. Blocking therefore cannot lapse after B or K_max is reached, and the issue in the released GenericInjectionEngine does not affect our reported results. We verified this with execution traces: under B=1, K_max=1, all blocked calls across both implementations remained blocked.

We have corrected the packaged library and provide an updated Anonymous GitHub repository with the original source, the corrected library, tests, and verification traces.

AuthorsOpenReview ↗

Follow-up on our responses to your review

Dear Reviewer 67Fa,

We want to thank you again for your thoughtful review and constructive feedback. As the interactive discussion period is currently active, we wanted to gently follow up to see if our replies and proposed revisions have successfully addressed your concerns. If there are any remaining questions or areas where further clarification would be helpful, please don't hesitate to let us know. We would be more than happy to discuss them further!

Best regards,

The Authors

Dear reviewer, please respond to the author's comment to further facilitate the discussion as the author-reviewer discussion period ends at November 18 at 11:59pm AoE

Best, AC

AuthorsOpenReview ↗

Follow-up on our responses to your review

Dear Reviewer 67Fa,

With approximately 12 hours remaining in the discussion period, we would greatly appreciate your feedback on whether our implementation clarification, corrected code, and verification traces have resolved your concern.

If so, we would be grateful if you could consider updating your rating to reflect this resolution. We are happy to provide any further clarification.

Thank you again for your time and consideration!

Best regards, The Authors

Reviewer_hY4uOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

Paper Summary

The paper studies tool-failure robustness in multi-turn tool-using LLM agents. It introduces BENCH2ROBUST. BENCH2ROBUST is a wrapper that injects tool-response failures into existing benchmarks. The benchmarks are τ²-bench, Retail-3I, Airline-3I, and BFCL. The wrapper has nine failure types. Six are explicit errors. Three are silent corruptions. Each episode gets one of three scenarios. In S1, the original path stays viable. In S2, one path is blocked and the agent must use a fallback-equivalent tool. In S3, all paths are blocked.

The paper proposes two interventions. The first is Bayesian Tool Memory (BTM). BTM is a system-prompt block. It contains per-tool and per-error recovery rates from base-model rollouts, a fallback map, and six recovery rules. The rates can stay fixed or update online from episode outcomes. The second is DAPO-based RL on Qwen3-4B-Thinking with a five-phase S1/S2/S3 curriculum. BTM is the prompt during RL rollouts.

The experiments have four parts. First, a robustness-gap evaluation across seven models and ten task slices. Second, held-out results on a 402-task Retail split for base, BTM, RL, and RL with BTM. Third, ablations of BTM components and a comparison with vanilla SFT and GRPO. Fourth, transfer of the Retail-trained policy to Airline-3I, BFCL, and Telecom.

Critical strengths

Strengths

  1. The problem is important. Tool failures are common in deployment, and most benchmarks ignore them.
  2. The scenario control (S1, S2, S3) separates task solvability from agent behavior. This gives a cleaner training signal than random injection.
  3. The failure taxonomy is broad. It covers six explicit errors and three silent corruptions. Prior benchmarks cover fewer types (Table 11).
  4. The robustness-gap evaluation is wide. It covers 7 models, 4 families, and 10 task slices.
  5. BTM gives a large gain with no training. This is a cheap and practical intervention.
  6. The ablations are clear. They separate BTM, RL, and the combination. They also separate static structure, dynamic priors, and online updating.
  7. The clean-task performance does not change. The method does not harm the normal case.
  8. The sensitivity analysis covers four failure rates and three injection budgets.
  9. The paper reports five evaluation seeds with standard deviations for the main 4B results.
  10. The limitations section is honest. It names the missing abstention reward, the simulated failures, and the single training run.
  11. The appendix gives the full training configuration, the noise distribution, and the BTM prompt template.
Critical weaknesses

Weaknesses

  1. There is no comparison with prior methods. The related work names PALADIN, NoisyAgent, RobustBench-TC, and ToolCritic as the closest work. None of them appear in Table 5. Vanilla SFT and GRPO are the only baselines.
  2. The held-out set is not out-of-distribution. It is a 402-task split of the same Retail pool. It shares the same tools, users, and product database as the training set. The paper does not say how the split is made.
  3. The online BTM variant uses test labels. Section 3.1 states that the Beta counts update from the benchmark outcome after each evaluation episode. This feeds pass/fail labels from the test set into the prompt. The main numbers in Table 2 use this variant.
  4. The BTM prompt contains the fallback map. This is a solution hint. The gain measures prompt quality, not agent reasoning.
  5. The main gain comes from the prompt, not from training. BTM alone gives +16.8pp. RL alone gives +6.3pp. On Airline-3I, BFCL, and Telecom, the RL-only gains are +1.5 to +2.7pp, which is within the noise floor.
  6. The paper does not report results by scenario. Table 2 gives one pass rate. There is no S1 versus S2 breakdown, and S3 has no metric. The main framing of the paper is not tested.
  7. Each RL model has one training run. The five seeds cover evaluation only.
  8. The training-recipe comparison is not fair. "Ours" uses BTM, DAPO, a KL term, and a curriculum. Vanilla SFT and GRPO use none of these. There is no "GRPO with BTM prompt" row.
  9. There is no token-matched baseline. BTM uses 50 to 80 percent more tokens under injection (Table 14).
  10. BTM hurts on factual errors (−4.6pp) and gives small gains on stale and partial data. The silent-error case is not analyzed.
Questions for authors

I do not have any questions.

5 comments
AuthorsOpenReview ↗

Response to Reviewer hY4u

We thank the reviewer for recognizing the importance of tool-failure robustness and the contributions of our scenario-controlled evaluation and BTM. We address the concerns with additional experiments and clarifications below.

W1: Comparison with Prior Methods

Direct empirical comparisons with PALADIN, NoisyAgent, RobustBench-TC, and ToolCritic are constrained by differences in evaluation protocols and implementation availability.

Specifically, NoisyAgent and ToolCritic lack publicly available official implementations. NoisyAgent trains agents under stochastic environmental noise, whereas our framework explicitly controls episode-level solvability, distinguishing when agents should retry, switch, or escalate. This separates recovery behavior from whether failures happen to resolve. ToolCritic introduces an additional inference-time critic for detecting and correcting tool-use errors, making it potentially complementary to our approach. PALADIN relies on ToolBench-specific recovery training and exemplar retrieval, while RobustBench-TC focuses primarily on single-turn perturbations without episode-level solvability control.

Although these studies address closely related problems, their differences in implementation availability, training mechanisms, and evaluation protocols make direct reproduction as baselines in our scenario-controlled, multi-turn setting impractical. We will clarify these distinctions and our methodological contribution in Related Work.

W2: Held-Out Split and Generalization

We clarify that the 402 held-out Retail tasks were randomly sampled from the 1,339-task pool (seed 42). The training and held-out sets contain distinct tasks, with different task objectives and required actions, although they share the same Retail environment, tools, and underlying database. Thus, this evaluation measures generalization to unseen tasks within the same domain, rather than out-of-distribution generalization.

We also evaluate cross-domain transfer in Table 13, where the RL policy is trained exclusively on Retail with BTM and evaluated on three unseen target benchmarks: BFCL, Airline-3I, and Telecom. Without inference-time BTM, the Retail-trained policy improves injected success over the base model by +1.5, +2.7, and +0.9 percentage points, respectively. With target-domain BTM at inference, the same BTM-trained policy improves over the BTM-only baseline by +2.0, +3.1, and +0.8 points, respectively. These results provide additional evidence of recovery behavior beyond the Retail training domain.

W3: Online BTM and Test-Label Leakage

Online BTM is an intentional feedback-driven adaptation mechanism. Following the standard online learning paradigm, BTM updates its beliefs using feedback from previously completed episodes. Each episode is executed without access to its own or future outcomes. Only after completion does BTM incorporate the evaluator's pass/fail feedback into its posterior statistics for subsequent episodes. This preserves strict temporal ordering and prevents access to current- or future-task labels.

To assess robustness, we report mean ± standard deviation across five evaluation seeds (Table 12), with posterior counts reset to the same training-derived initialization at the start of each run.

We will clarify the feedback protocol and temporal ordering in Section 3.1 and acknowledge the assumption of reliable post-episode feedback in deployment.

W4/W5: Fallback Information and the Contributions of BTM and RL

We clarify that BTM's fallback map intentionally provides structured recovery guidance rather than a content-neutral prompt. Our goal is to improve tool-failure recovery through runtime guidance and learned recovery behavior, not to isolate improvements in the agent's intrinsic reasoning ability. Providing actionable recovery information without retraining is itself one of BTM's contributions.

The held-out Retail evaluation (Table 2) compares the base and RL-trained policies under matched inference-time BTM settings. RL improves No-alt success from 20.1% to 26.4% (+6.3 pp) without inference-time BTM, and from 36.9% to 40.8% (+3.9 pp) when both models receive BTM. Thus, although runtime guidance provides the larger gain, RL-trained policies retain additional recovery capability.

Regarding cross-domain results (Table 13), Airline-3I, BFCL, and Telecom are transfer evaluations, not domains used for RL training. The policy was trained exclusively on Retail and evaluated on these unseen benchmarks without further fine-tuning. The modest positive gains (+0.9–2.7 pp without inference-time BTM) suggest some transfer of recovery behavior.

AuthorsOpenReview ↗

Response to Reviewer hY4u

W6/W8: Scenario-Level Results and Training-Recipe Comparisons

We conducted additional scenario-level evaluations and trained the requested GRPO+BTM control, alongside vanilla DAPO. The table below reports the available results.

MethodCleanNo-altAlt-S1Alt-S2
Vanilla DAPO63.223.932.631.8
GRPO + train w/ BTM65.928.942.547.8
Ours63.940.843.350.0

Our full training recipe improves No-alt performance by 11.9 percentage points over GRPO+BTM (40.8% vs. 28.9%) and 16.9 points over vanilla DAPO (40.8% vs. 23.9%). These results demonstrate the effectiveness of our combined training approach beyond the additional BTM-controlled baseline. The S1/S2 results further characterize recovery under different solvability conditions. We will include the scenario-level recovery analysis and clarify the evaluation settings in the revision.

W7: Training-Seed Variability

We trained three independent seeds of the Full recipe using the same curriculum and hyperparameters. Across training seeds, the mean ± standard deviation of pass rates was 61.3 ± 8.6% (Clean), 38.0 ± 2.5% (No-alt), and 44.7 ± 2.4% (Alt).

Injected performance shows relatively limited variation across seeds, while clean performance varies more substantially. We will report these results separately from the evaluation-seed variance in Table 12 and acknowledge that three training runs are insufficient to establish training stability.

W9: Token-Matched Baseline

We clarify that all models were evaluated under identical context-window, output-token, episode-step, and tool-call limits, without imposing any constraints on their actual token consumption beyond these shared limits. The baseline models were equally free to use more tokens; differences in token usage reflect the models' behavior under the same evaluation constraints rather than unequal token budgets.

Although higher token consumption may contribute to the observed performance gains, it is not an additional budget granted to BTM. We report actual token usage, steps, and tool calls alongside task success to make this performance–cost trade-off transparent.

W10: Factual and Silent Errors

We conducted an additional analysis of silent-corruption events. Among episodes containing silent errors, only 3–11% involved repeated queries with the same tool and arguments, indicating that agents rarely re-query to verify potentially corrupted observations.

This helps contextualize BTM's limited effectiveness on silent errors. Unlike explicit failures, silent corruptions provide no direct error signal, making recovery guidance alone less actionable. Consistent with this distinction, Table 4 shows that BTM provides limited gains on partial (+8.5 pp) and stale (+2.7 pp) errors and decreases performance on factual errors (−4.6 pp). In contrast, RL+BTM improves factual-error performance by +10.5 pp over BTM alone, suggesting that training contributes recovery capabilities beyond explicit guidance.

We will include this behavioral analysis and clarify that repeated querying captures only one possible verification strategy, rather than all ways an agent may detect or recover from silent corruption.

AuthorsOpenReview ↗

Follow-up on our responses to your review

Dear Reviewer hY4u,

We want to thank you again for your thoughtful review and constructive feedback. As the interactive discussion period is currently active, we wanted to gently follow up to see if our replies and proposed revisions have successfully addressed your concerns. If there are any remaining questions or areas where further clarification would be helpful, please don't hesitate to let us know. We would be more than happy to discuss them further!

Best regards,

The Authors

Reviewer_MAL2OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper focuses on scenario-controlled solvability of tool-using LLMs and provides the Bench2Robust framework, introducing multiple failure scenarios during tool response. It also introduces an integrated training process, including Bayesian tool memory and curriculum-based DAPO training. BTM provides information for recoverability, and curriculum-based DAPO training provides a way for the model to learn to handle different scenarios. Multiple experiments show the existence of the robustness gap and further demonstrate the effectiveness of the proposed method.

Critical strengths
  1. This paper provides a good perspective on the robustness of tool-using LLMs, showing the existence of the robustness gap during tool-response failure.

  2. Based on the tool-response failure scenarios, this paper proposes three different solvability settings, supporting the thorough analysis of experiments.

  3. The experiment section provides multiple baselines, ablation studies, and parameter studies to show the gap existence and effectiveness of the method.

Critical weaknesses
  1. Different scenarios are introduced to simulate different solvability in Section 2.1. However, S3, as a fully blocked scenario, cannot provide any success signals during training. Meanwhile, incomplete task scenarios can occur in S2. Therefore, the necessity of S3 data should be more carefully evaluated.

  2. The design of the different parts in the proposed method is not fully assessed. Generally speaking, curriculum learning is introduced in DAPO training without any experimental results, which show the effectiveness of curriculum learning. Meanwhile, the experimental section also didn’t provide the performance of vanilla DAPO to show its effectiveness.

  3. The design of reward is somehow ambiguous. Since an agentic task can be solved through multiple paths or reasoning steps, there could be multiple valid actions and paths, which could not be fully listed. Therefore, the definition of “matched actions” should be more explicit to explain coverage of different valid paths.

Questions for authors
  1. How necessary is the S3 data? Since S3 data can exist in S2, an extra ablation study on S3 data can help the reader better understand the effectiveness of S3 data.

  2. How much improvement is introduced by curriculum learning? An extra ablation study on curriculum learning can better show the performance improvement. Meanwhile, the performance after different curricula can better show the effectiveness of curriculum design.

  3. A more detailed introduction to the reward design can better explain the incomplete task signal, especially the definition of “matched action”.

6 comments
AuthorsOpenReview ↗

Response to Reviewer MAL2

We thank the reviewer for recognizing the importance of robustness gaps in tool-using agents and the value of our scenario-controlled evaluation and empirical analysis. We provide a more detailed discussion about the reviewer's feedback below:

W1/Q1: Necessity of S3 Training

S3 explicitly targets the no-solution-to-escalation recovery path, where escalation is the appropriate outcome rather than task completion. To assess the contribution of S3 exposure, we conducted a No-S3 ablation, replacing S3 episodes with S1/S2 episodes while preserving the phase-specific S1/S2 ratios specified in the original curriculum (Appendix B.2, Table 9), renormalized to exclude S3. We maintained the curriculum structure and matched training iterations and rollout budget. Both models were evaluated on the same 402 held-out tasks.

ModelTrain BTMInfer BTMS1S2S3 Esc.
FullYesNo35.242.586.3
No-S3YesNo34.342.077.6
FullYesYes43.350.087.3
No-S3YesYes40.044.373.6

S3 is unsolvable by construction. The escalation rate is reported as a behavioral diagnostic.

S3 exposure increases the escalation rate on unsolvable tasks by 8.7–13.7 percentage points, with and without inference-time BTM, while maintaining comparable or better S1/S2 task success. These results support the value of explicit S3 exposure for encouraging escalation behavior. Since S3 does not provide a positive abstention reward, we treat escalation as a behavioral diagnostic rather than a measure of learned abstention correctness.

W2/Q2: Curriculum Learning and Vanilla DAPO

To evaluate curriculum learning, we conducted a Non-curriculum ablation by randomly shuffling the same S1/S2/S3 training episodes used in the Full curriculum, preserving total scenario exposure and training compute while removing the staged ordering. All results are percentages: S1/S2 report task pass rates, while S3 Esc. reports the escalation rate on unsolvable tasks.

ModelTrain BTMInfer BTMS1S2S3 Esc.
Full (staged)YesNo35.242.586.3
Non-curriculumYesNo24.424.478.4
Full (staged)YesYes43.350.087.3
Non-curriculumYesYes35.641.564.2

At matched scenario exposure and training compute, staged training outperforms shuffled training on S1/S2 task success and S3 escalation. The improvement is particularly pronounced with inference-time BTM, supporting the benefit of curriculum ordering.

We also evaluated the requested vanilla DAPO baseline, removing training-time BTM, the KL term, and curriculum ordering while preserving training compute. Our combined training approach improves No-alt and Alt performance over vanilla DAPO.

MethodCleanNo-altAlt
Vanilla DAPO63.223.932.3
Ours63.940.845.5

W3/Q3: Reward Design and Matched Actions

Our training reward uses τ²-bench's reference actions [1,2]. An action is matched if any tool call has the same tool name and matching values for all specified arguments, regardless of order. The completion-reward branch in Eq. (1) applies when all reference actions are matched. Otherwise, partial credit is proportional to the matched actions, scaled by the repetition penalty.

In our scenario-controlled S2 setting, we explicitly account for fallback-equivalent tool actions in reward matching, allowing agents to receive credit for switching to a viable alternative when the original path is blocked. Although reference-action matching may under-reward alternative valid execution paths, it provides a tractable training signal beyond binary completion rewards, consistent with prior work leveraging verifiable tool-call trajectories for training [3]. Without partial credit, 38% of training tasks have zero reward variance (Section 3.2).

We will clarify that "task completed" in Eq. (1) denotes reference-action completion. This process-based reward is used only for training rewards. Reported task success relies on τ²-bench's official outcome-based evaluator, which accommodates multiple valid execution paths. We will clarify this distinction and acknowledge the limitation in the revision.

References

[1] Barres et al. (2025). τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment.

[2] τ²-Bench Official Evaluation Documentation.

[3] Wang et al. (2026). Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents. ACL 2026.

AuthorsOpenReview ↗

Follow-up on our responses to your review

Dear Reviewer MAL2,

We want to thank you again for your thoughtful review and constructive feedback. As the interactive discussion period is currently active, we wanted to gently follow up to see if our replies and proposed revisions have successfully addressed your concerns. If there are any remaining questions or areas where further clarification would be helpful, please don't hesitate to let us know. We would be more than happy to discuss them further!

Best regards,

The Authors

Dear reviewer, please respond to the author's comment to further facilitate the discussion as the author-reviewer discussion period ends at November 18 at 11:59pm AoE

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.