- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates the question: “How can we train an agent to achieve consistently strong performance under any budget constraint?” The authors reformulate this problem as finding the minimum budget required for each task to achieve a target reliability level (). Based on this formulation, they propose BABLE (Bilevel Any-Budget Learning for Agents), which trains the model by alternately optimizing two components:
-
Task-Adaptive Budget Controller (TABC): Dynamically tightens the budget required for each task based on observed task success.
-
Likelihood-Guided Conditional Advantage Optimization (LCAO): Enhances the learning signal in sparse-success scenarios.
The authors evaluate BABLE using the Qwen3-8B model on four benchmarks: MuSiQue, GAIA, WebWalkerQA, and FRAMES. They consider two types of budget constraints, token-level and tool-call-level, and evaluate performance under three budget configurations: Any (average), Max (maximum), and Low (low). BABLE demonstrates clear advantages under the Any and Max budget configurations.
Critical strengths
The problem addressed by the authors is highly relevant to real-world applications. Given the widespread use of agents, developing models that achieve high success rates at low cost is of significant practical importance.
The paper is exceptionally clear and well-organized. The background, motivation, and methodology are presented in an accessible manner, and the experimental results are reasonably interpretable. The authors also provide further explanations and insights into the results, which significantly enhance the overall reading experience.
The proposed alternating optimization scheme for TABC and LCAO is clearly presented and reasonably interpretable. The approach also has the potential for broad applicability.
Critical weaknesses
The discussion of the reliability target ( $ \alpha $ ) requires further elaboration. The paper sets ($ \alpha $ = 0.25), but the rationale for this choice is unclear. The authors should also explain how varying ($ \alpha $) affects the method's performance.
The assumptions underlying practical deployment require further validation. In particular, the paper assumes that the success rate increases monotonically with the budget, but provides insufficient empirical evidence to support this assumption. The authors should discuss whether the proposed method remains applicable when this assumption does not hold.
The paper could further discuss how models trained under different budget constraints can deliver tangible cost savings in real-world applications. In particular, it is unclear how a deployed model identifies the task category corresponding to the training setup and consequently determines how to allocate its budget.
Questions for authors
The paper assumes that task success rates increase monotonically with the budget. Is there empirical evidence supporting this assumption? Would the algorithm remain effective if the assumption does not hold, or holds only in expectation?
Qwen3-8B supports both /think and /no_think modes. Which mode was used in the experiments? In practice, different budget constraints often correspond to different reasoning efforts and reasoning lengths. Would the proposed method remain effective in such settings?
How can the proposed method translate into tangible benefits in real-world applications and inference scenarios?
What is the rationale for choosing ()? How sensitive is the overall algorithm to this hyperparameter?
In Table 1, BABLE underperforms LCPO under the Low budget configuration. What accounts for this gap? Is there room to improve the algorithm's performance in low-budget settings?
4 comments
Tool call budgets
| MuSiQue | GAIA | WebWalkerQA | FRAMES | |
|---|---|---|---|---|
| 0.25 | 10.68 / 37.57 / 48.97 | 16.94 / 31.12 / 36.75 | 2.93 / 9.82 / 12.40 | 8.34 / 26.85 / 36.15 |
| 0.50 | 4.29 / 34.06 / 46.49 | 15.65 / 27.90 / 32.67 | 2.16 / 7.93 / 10.62 | 4.97 / 24.51 / 34.50 |
| 0.75 | 7.10 / 33.48 / 43.78 | 16.52 / 29.76 / 35.35 | 3.10 / 7.66 / 9.57 | 5.77 / 23.90 / 33.12 |
| 1.00 | 3.50 / 32.44 / 44.34 | 15.72 / 29.68 / 35.03 | 2.08 / 7.58 / 9.75 | 4.46 / 24.08 / 33.31 |
The reference from Table 1 gives the highest Anytime accuracy in all eight combinations of dataset and budget type, exceeding the best result at a higher target in each combination by 1.04–4.84 percentage points. Among the three higher targets, gives the highest Anytime accuracy under both token and tool call budgets on MuSiQue, WebWalkerQA, and FRAMES. On GAIA, the best Anytime results under token and tool call budgets among the three occur at and , respectively.
Q3&W3. Deployment under varying inference budgets
An agent service may receive the same type of request throughout the day, while the compute and API capacity available to each user changes with traffic. For example, during daytime peaks, a search agent may have a smaller budget for tokens or API calls per request to serve more users within the service’s latency target. At night, the same service can raise the cap so the agent has more room to check additional sources or refine its answer when the task calls for it. BABLE aims to make the same trained policy perform well across these budget levels. The service can keep one checkpoint loaded, set a cap for each request based on current load, and enforce it during execution. A tight cap limits how much work each request can consume at peak times. A looser cap allows longer executions to finish when spare capacity is available. The service can change this cap as traffic changes while serving both cases with the same model.
Q5. Performance at low budgets relative to LCPO
LCPO subtracts a scaled token cost penalty from the binary correctness reward . This favors shorter executions even among correct trajectories and gives cheaper failures a higher reward. The short refusals observed on LLaMA illustrate the resulting incentive for reward hacking (Appendix E). BABLE gives a reward of 1 to correct trajectories within each task's TABC budget and uses LCAO to reinforce them. Training a single policy across budgets requires the policy to finish early when a short solution suffices and to keep reasoning or using tools when the task requires more steps. To maintain strong performance at larger budgets, BABLE gives up some accuracy under tight budgets on some benchmarks. LCPO leads by 0.11 to 0.52 points at 3K on GAIA, WebWalkerQA, and FRAMES. BABLE leads by 2.73 to 10.68 points at Max on those benchmarks and achieves higher Anytime Accuracy under token budgets on all four (Table 1).
We will incorporate your valuable feedback into the revised PDF and upload the updated version shortly.
Thank you for the response. Most of my concerns have been addressed.
Thank you for carefully considering our responses and for your thoughtful follow-up. We are glad that most of your concerns have been addressed. Your constructive feedback has helped us clarify the theoretical arguments and strengthen the experimental analysis. We sincerely appreciate your time and the valuable discussion.
W2&Q1. Budget monotonicity and its scope
Thank you for asking about budget monotonicity. Appendix D establishes monotonicity for each task under the truncation protocol of Eq. (2) with a fixed policy. For a given task and the same rollout randomness, an execution that finishes with a correct answer within the smaller budget still finishes with that answer under the larger budget. The larger cutoff can also admit additional successful executions, as illustrated in Fig. 5. The success event at the smaller budget is therefore a subset of the success event at the larger budget. Taking probabilities yields pθ(x,b1)≤pθ(x,b2) for every task x and b1≤b2.
We evaluated Qwen3-8B Vanilla under token cutoffs from 1K to 10K. Accuracy was nondecreasing with budget on all four benchmarks.
Q2. Evaluation with and without thinking
Our main Qwen3-8B BABLE evaluation uses thinking mode. We also evaluated eight configurations on the same four benchmarks with thinking disabled in the chat template.
Each cell reports Low / Any / Max accuracy (%). Max is defined as accuracy on the full saved rollouts.
Token budgets. Low uses 3K tokens. Any averages the results from 2K to 8K in steps of 500 tokens.
Tool call budgets. Low allows one call. Any averages the results at the limits of one through five calls.
With thinking disabled, BABLE achieves the highest Any accuracy under token budgets on all four benchmarks and under tool call budgets on GAIA and FRAMES. It also achieves the highest Max accuracy on MuSiQue and WebWalkerQA in both tables.
W1&Q4. Choice of reliability threshold and sensitivity
We chose α=0.25 to require two successes out of eight attempts.
We evaluated Qwen3-8B in thinking mode at α=0.50, 0.75, and 1.00. Each cell reports Low / Any / Max accuracy (%) at the Appendix C cutoffs.
Token budgets