- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Quality-Guided Cost Optimization (QGC), a modification to GRPO reward construction for learning agent recruitment policies. It observes that within-group normalization can effectively cancel the cost coefficient when trajectories have identical quality. QGC introduces different reward rules depending on whether all trajectories fail, achieve equal positive quality, or have varying quality. On compositional programming tasks, QGC reports a 1.46% success improvement over Quality-Only on Qwen3-8B, with lower estimated cost.
Critical strengths
The observation about cost coefficient cancellation under within-group normalization is mathematically reasonable and potentially useful. The paper also includes ablation comparisons to investigate different reward constructions.
Critical weaknesses
- The recruitment problem and cost model are insufficiently motivated.
The benchmark consists of programming tasks with relatively explicit subtask decompositions. In many cases, assigning workers according to the number of independent subtasks appears to be a natural heuristic. Yet no simple structure-aware recruitment baseline is evaluated. It is therefore unclear whether learning is necessary or offers meaningful advantages.
The cost model is also inadequately justified. It uses the maximum estimated text length across execution slots plus a fixed penalty of 50 per recruited worker. The scheduling assumptions, the choice of this penalty, and its relationship to actual execution costs are not established. Since this proxy determines both the training rewards and reported efficiency gains, these limitations are consequential.
- The experimental evidence does not convincingly establish the proposed method's advantage.
The reported gain is very modest, only 1.46 percentage points, with considerable variation across three training seeds. One seed performs worse than both Quality-Only and GDC. No paired statistical tests or confidence intervals are provided.
More importantly, the tie-suppressed control (TSC) achieves comparable success to QGC (45.76% versus 45.48%). Thus, the benefit of the proposed positive-quality tie-handling mechanism remains unclear. The reported cost reduction is also largely achieved by GDC, suggesting that much of the improvement comes from introducing a cost penalty rather than the proposed method.
The main comparisons additionally use different cost coefficients (QGC at 0.03 versus GDC/NC at 0.1), making it difficult to attribute the observed differences to reward construction.
Questions for authors
-
How does QGC compare against simple structure-aware recruitment heuristics?
-
Can the authors provide statistically supported comparisons under matched resource budgets and appropriately tuned cost coefficients?
-
How sensitive are the results to the worker penalty, and how does the cost proxy relate to actual token consumption or latency?