- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
CARE-PO trains tool-using agents to adjust their execution effort with a single set of model weights. Its reward combines correctness and evidence coverage with a bounded cost multiplier based on a task-specific reference budget estimated before training. A successful source trajectory is added to each GRPO group with a nonnegative advantage, while an effort tag in the prompt adjusts the cost assigned to tool calls. The evaluation covers three model sizes on GAIA and four held-out multi-hop benchmarks, alongside analyses of token costs and training dynamics.
Critical strengths
-
The method addresses a practical deployment concern by substantially reducing tool calls and accounting for token costs. Its bounded reward preserves the stated ordering between successful and unsuccessful trajectories. This design is useful even without a stronger guarantee at the policy level.
-
The evaluation covers three model sizes and several tasks, includes ablations of effort conditioning and evidence coverage, and reports both improvements and the transfer losses at 9B. The supplementary records support recomputation of the main tables and reported ablations, making the results easier to verify.
Critical weaknesses
-
The experiments do not isolate which components prevent collapse. The comparison with OTC-PO changes the reference budget, evidence shaping, and injection of successful source trajectories simultaneously. Source injection also provides a demonstration update with positive advantage, which could stabilize training independently of the fixed budget. The existing effort and coverage ablations do not distinguish these effects. This limits the conclusions about why collapse is avoided at 27B, even if the combined method performs well.
-
The evidence for controlling effort is stronger than the evidence for controlling capability. On GAIA, the 27B model achieves higher success with fewer calls under low effort than under high effort: 57.4% success with 5.20 calls versus 54.7% with 6.12 calls. At 4B, both settings achieve the same mean success rate despite substantially different call counts. Both 9B variants reduce success on the held-out multi-hop tasks. These results support control over computational expenditure, but do not consistently establish the claimed high-effort capability setting or a transferable trade-off between quality and cost.
Questions for authors
-
Could you isolate the contributions of the fixed reference budget and the nonnegative advantage floor for source trajectories while keeping the remaining reward design and training procedure unchanged? A CARE-PO ablation without source injection, or an OTC-PO baseline with the same source injection, would help clarify the collapse result. If these comparisons are unavailable, please limit the attribution to the combined method.
-
What do paired comparisons on individual questions reveal about low and high effort? Do the three GAIA runs evaluate the same trained checkpoint repeatedly, or do they include independent training runs? Please distinguish these sources of uncertainty and clarify when users should expect higher effort to improve quality.
Response to Reviewer 4Rgd
We thank the reviewer for the clear and precise questions.
Q1. Isolating the fixed budget and the source-trajectory floor. We ran both controls named in your review at 27B under matched settings (Tables R1 and R2 in the general comment; Appendix E, Tables 5–6): CARE-PO without source injection, and OTC-PO with the same source injection. Removing injection from CARE-PO leads to partial collapse later in training (training success 0.58, 18.8% zero-call rollouts in steps 901–1000), whereas adding the same injection to OTC-PO prevents collapse altogether (training success 0.89, 0.5% zero-call rollouts vs. 99.8% without it). Because these controls toggle the injected member as a unit—combining the successful demonstration and its zero-clipped advantage floor—we scope our anti-collapse attribution strictly to the source-trajectory floor mechanism as a whole (Sections 3.4 and 6.3). Above that floor, the fixed per-task budget with evidence pricing governs the quality–cost equilibrium: replacing the fixed budget inside CARE-PO with a moving group-minimum anchor still trains stably under injection, yet compresses further at a substantial accuracy cost on GAIA (low tier: 41.6% SR at 1.96 calls vs. 53.3% at 3.22 calls; Table R2).
Q2. Uncertainty in GAIA runs, paired per-question tests, and when higher effort improves quality. Each configuration in the main tables uses a single training run (one training seed), and the three GAIA evaluation rounds repeatedly evaluate that same checkpoint across three sessions (Table 1 and Section 4 captions updated for clarity); the reported ± s.d. therefore reflects evaluation variance (live web search non-stationarity and decoding) rather than cross-seed training variance, which Section 7 states explicitly as a limitation. Paired per-question comparisons between high and low effort (Appendix E, Table 8; detailed in our response to Reviewer hSat, item 2) separate expenditure control from accuracy gains:
Users should therefore expect the low tag to deliver substantial call savings when the model has sufficient capacity for the task, and select the high tag on smaller models or retrieval-critical multi-hop queries to mitigate—though at 9B not fully eliminate (60.8% vs. 43.2% on Bamboogle, vs. 76.0% for the base model)—accuracy loss from skipping a required hop.