- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies active visual agents operating under varying interaction budgets. It introduces FA-DAPO which improve on top of standard DAPO and adapt it towards a turn-budget aware setting. The paper also present a simple practical nudge to improve training efficiency under varied turn budgets. In experiments, proposed pipeline show descent improvement under controlled budget comparing to vanilla training pipeline.
Critical strengths
-
This work is focusing on a practical yet oftenly overlooked problem, and the addressing the issue does carry certain practical benefits.
-
The propose method is simple and effective. Propose method shows strong edge comparing to vanilla training method.
Critical weaknesses
-
Reviewer hates to say so, but there is very limited technical and science innovation. The method presented in the paper is an incremental change or adaption on top of DAPO plus a engineering tricks. Also work like AnytimeReasoner [1] already trains on answers elicited from interrupted reasoning under sampled budgets. AdaTurn extends this general idea to visual tool-use trajectories, but the experiments should validate what its particular training choices add.
-
Turn budget does not fully reflect comparable inference cost. Eq. 7 first generates an inadmissible final-turn tool request, discards it, and generates a replacement answer. A retained trajectory of T turns can therefore require T+1 model generations. Table 6 provides useful six-turn token statistics without clearly explaining whether discarded generations and repeated processing are counted.
[1] Optimizing Anytime Reasoning via Budget Relative Policy Optimization
Questions for authors
-
Can you compare fixed versus randomized training budgets and ordinary DAPO versus FA-DAPO with matched initialization, data, and inference-time final-answer handling?
-
How are discarded generations, replacement answers, and repeated prefill counted? Does AdaTurn retain its advantage over budget-prompted Mini-o3 when accuracy is compared against actual model invocations or measured latency?
2 comments
Response to Reviewer Yn7v [Part 2]
Q1: Fixed versus randomized budgets, and ordinary DAPO versus FA-DAPO. Given the limited rebuttal period, we cannot provide per-benchmark ablation results for a full quantitative comparison. From our preliminary experiments, training with a dynamic budget range outperforms a fixed budget because it exposes the model to diverse training conditions: different budget levels demand qualitatively different behaviors, such as aggressive search compression under a tight budget versus extended verification under a generous one. Training across this range teaches the model to maximize performance for whatever budget it receives.
Regarding ordinary DAPO versus FA-DAPO, the budget-prompted Mini-o3 evaluated in Appendix E (Table 5) can be viewed as an approximation of ordinary DAPO combined with inference-time final-answer handling, since both Mini-o3 and AdaTurn use the same backbone and training data, and the budget prompt replicates the forced-answer mechanism at inference time without RL training. The comparison below shows that AdaTurn (FA-DAPO) still outperforms this combination on six of the seven benchmarks, demonstrating that explicitly training the budget boundary via FA-DAPO provides a clear advantage over ordinary DAPO with inference-time handling.
| Model (4 turns) | VP-Hard | VP-Medium | VP-Easy | V* | HR-4K | HR-8K | MME-RW |
|---|---|---|---|---|---|---|---|
| Mini-o3 w/ budget prompt | 36.1 | 47.7 | 58.1 | 81.9 | 73.1 | 67.7 | 59.7 |
| Ours (FA-DAPO) | 39.0 | 47.6 | 61.2 | 85.2 | 75.7 | 71.1 | 64.0 |
Response to Reviewer Yn7v [Part 1]
We sincerely thank the reviewer for the careful reading and constructive feedback. We are encouraged that the reviewer finds the problem "practical yet often overlooked," the method "simple and effective," and that it "shows strong edge comparing to vanilla training method." Below we address each concern in detail.
W1: Novelty and relation to AnytimeReasoner. AdaTurn contributes FA-DAPO, which converts over-budget tool requests into trainable final-answer steps and optimizes the full trajectory with group-normalized reward, and a load-balanced rollout scheduler that reduces worker imbalance under dynamic budgets and yields a 1.34× throughput improvement. Both designs are driven by concrete failure modes in real deployment.
We also explored more "novel" alternatives in Appendix C. Masking the prefix turns so that only the forced-answer step receives a gradient degrades performance, suggesting that budget-aware behavior is a trajectory-level property that must be learned jointly with the preceding search decisions rather than as an isolated correction at the last turn. Restricting RL to only the final forced-answer step, where all trajectories in a group share the same prefix, performs worse still: without signal flowing back through the full rollout, the policy cannot learn to allocate its turns wisely. Taken together, these ablations directly validate what each of AdaTurn's particular training choices contributes: forced-answer regeneration over masking or penalizing (FA-DAPO vs. alternatives), full-rollout optimization over final-step-only training, and FA-DAPO over ordinary DAPO with inference-time final-answer handling (Q1 below). These experiments show that although our default design may appear less "novel" and "simple," it is the most effective approach, and that adding complexity does not improve performance.
We thank the reviewer for sharing AnytimeReasoner. AnytimeReasoner trains a language model to produce good answers at any point during its reasoning chain by sampling token-level budgets, truncating the thinking trace at the sampled budget, and supervising the model to summarize a best answer from the truncated prefix. It is a token-level, text-only study evaluated on mathematical reasoning tasks, with no visual inputs, no external tool calls, and no multi-turn agent loop. AdaTurn addresses a complementary setting: turn-level budgets, multimodal inputs, and a multi-turn agent loop where each turn involves an external image-grounding tool call. The budget boundary in AdaTurn is a discrete decision point between tool use and final-answer generation, rather than a truncation inside a single reasoning trace, which requires a different training formulation (FA-DAPO) and different systems support (load-balanced scheduling). We will discuss AnytimeReasoner in the related work section in the revision.
W2 and Q2: Token cost of discarded generations. The token counts in Table 6 include all tokens generated during the rollout, including the discarded final tool request. The discarded generation is thus already accounted for in the reported output token count, and the efficiency advantage of AdaTurn holds even when these additional tokens are counted. Furthermore, the extra cost of the discarded generation is limited compared to a full extra turn. When FA-DAPO detects an over-budget tool call and rolls back to the history before the last turn, the prefix up to that point is already in the KV cache. The re-generation of the forced-answer step therefore only needs to decode from the cached prefix without re-prefilling the full context, so the additional latency and compute are limited compared to a full extra turn. We will clarify these points explicitly in the revision.