- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper addresses two problems in tool-using language agents: efficient evaluation of agent harness modifications and effective rollout allocation for reinforcement learning. For the first problem, the authors propose DISCORD, which evaluates candidate harness edits by comparing paired task outcomes between the incumbent and candidate configurations. It uses exact statistical testing and early stopping to reduce evaluation costs while preserving the retention decision. For the second problem, the authors introduce BRACE, which leverages historical rollout outcomes to predict the likelihood of mixed-reward groups. Based on these predictions, it assigns sampling weights to eligible tasks before generating fixed-size rollout groups, aiming to reduce uninformative training rollouts caused by identical within-group rewards.
Critical strengths
- The paper investigates two practically important issues in tool-using agents: the high cost of evaluating harness modifications and the inefficient use of rollout budgets during RL training. Both are relevant to improving agent performance under limited computational resources.
- The experiments include comparisons against multiple baselines and evaluations across three Qwen3-VL model sizes.
Critical weaknesses
- Although the paper identifies two relevant problems, the motivation for addressing them together is not sufficiently developed. DISCORD focuses on evaluating and retaining harness modifications without changing model parameters, whereas BRACE focuses on allocating training tasks to optimize model parameters. Their connection appears primarily conceptual, through the use of outcome disagreement, rather than methodological or operational.
- The proposed approaches build on relatively intuitive principles: using paired outcome disagreements to evaluate competing configurations and prioritizing tasks expected to produce informative reward variation. However, the paper does not sufficiently clarify which methodological components constitute the principal novelty relative to existing paired statistical testing, early-stopping procedures, and adaptive task-sampling strategies. The authors should more explicitly explain why the proposed mechanisms are preferable to simpler alternatives and which components are essential to the reported improvements. A stronger discussion of the limitations of existing approaches and the specific mechanisms by which DISCORD and BRACE overcome them would improve the technical contribution.
- Although BRACE achieves the best solve rates across the three model scales, its improvements are less consistent on other evaluation metrics. In Table 2, BRACE obtains the best result on only three of the nine reported model-scale/metric combinations. These observations indicate potential trade-offs between task-solving performance, tool-use reliability, and interaction efficiency that are not sufficiently explained.
Questions for authors
What is the substantive connection between the two proposed methods beyond their shared reliance on reward disagreement? Are they intended to form a unified optimization framework, or are they independent approaches addressing different problems? If the latter, what motivates presenting them as a single contribution?
Thank you for identifying the need for a clearer connection, a more precise novelty claim, and an explicit discussion of metric trade-offs. We have revised the framing and added controls to address these points.
Why present DISCORD and BRACE together? Both address expensive interactions during development of the same agent, but at different decision points: harness retention and policy training. The conceptual progression is from measuring useful contrast to predicting where it will occur. In DISCORD, paired agreement cannot distinguish the candidate from the incumbent; in BRACE, unanimous rewards cannot produce a group-centered reward signal. DISCORD motivates BRACE through this principle, rather than supplying a required intermediate output. The Introduction, Figure 1, and Sections 2–4 now distinguish the common motivation from the different statistical objects. We make no claim of a jointly optimized pipeline or an experimentally established gain from composing the two procedures.
Novelty relative to simpler alternatives. We now explicitly credit paired exact testing and curtailment as established tools. DISCORD's contribution is their operational use and evaluation for harness retention, including stopping without changing the prescribed full-test decision. BRACE addresses prospective allocation through the combination of capped historical task evidence, posterior uncertainty, correlated-group forecasts, and eligibility checks. Uniform sampling ignores task evidence; post-generation filtering cannot recover the cost of groups already generated. PLR, TSCL, TRACE, and VIP provide comparisons to other adaptive rules. The revised baseline description also makes clear that TSCL uses historical evidence, while its bank snapshot and eligible pool differ from BRACE's.
Which components matter? Table 3 reports that removing the bank lowers solve rate by up to 4.3 percentage points. The prospective correlation-aware forecast reduces degeneracy-prediction error from 0.591 to 0.056 over 83 evaluation arm-cycles, excluding the cycles used to fit correlation. Proposition 1 states the important boundary: at fixed group size and known task success probability, correlation correction preserves ranking, so calibration alone does not establish improved task ordering or policy performance.
The original VIP-style row changed score and eligibility together. Section 5.5 now adds a 4B matched-eligibility control: BRACE achieves 14.9% versus 13.2% for VIP-style allocation. Without the mask, the rates remain 14.9%/13.2%. Thus the 1.7-point gap persists with eligibility held fixed; the interaction is 0.0 points at reported precision. This supports the score comparison at 4B without attributing the full gap specifically to correlation correction. The main text also reports early-window mixed-group yields of 55.8/50.2/43.3 per 1,000 attempts for BRACE versus 26.2/21.9/21.9 for uniform GRPO. These measure realized allocation yield, not full-training compute savings.
Solve rate, call reliability, and turns. We agree that these metrics capture different outcomes. Section 5.3 now states explicitly that BRACE leads the three solve-rate columns, but not the six invalid-call/turn columns. Relative to the base, invalid calls fall from 6.5% to 4.6% at 4B but rise from 16.6% to 19.3% at 2B and from 6.8% to 9.0% at 8B. BRACE targets the availability of a training reward signal, not an explicit call-validity or trajectory-length objective. Short trajectories may also end in failure, so mean turns alone do not establish successful-task efficiency. We retain the full metrics and scope the claim to task completion, rather than claiming uniform improvement in tool-use reliability.
Presentation and additional validation. The main 10 pages now contain the organizing principle, component evidence, cost trade-offs, and a new text-only Qwen3-4B-Instruct-2507 control (Table 4). BRACE achieves 13.6% mean AWM solve rate versus 12.4% for PLR, 12.1% for uniform GRPO, and 11.4% for TRACE. The revised text distinguishes descriptive gains from corrected paired significance. The appendix supplies detailed protocols and limitations rather than carrying the main answers to the reviews.
We hope these changes make both the contribution and the strength of its supporting evidence substantially easier to assess.