- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes SRPO for training LLM-based multi-agent systems. An active set consists of newly sampled outputs generated from the same logical pre-action state and consumed together by one transition, including a transition in shared orchestration memory. Sequential routing, verification, and aggregation are separate events. SRPO sums the sequence log-ratios within an event, divides by the square root of the active-set cardinality, exponentiates this quantity, and applies one clipped group-relative objective with a shared event advantage. It averages the loss over events. The theoretical analysis explains the variance and local score scale of square-root normalization under stated independence or covariance conditions. Evaluation covers mathematical reasoning with Qwen3-4B/8B and multi-turn search with Qwen2.5-3B/7B, reporting Avg@16 and Pass@16.
Critical strengths
The optimization unit is defined precisely enough to be useful in an implementation. Sections 3.1 and 4.3 distinguish simultaneous outputs from later decisions that observe them, include shared memory in the state, and require complete event records in each batch. This matters because joint updates can otherwise accidentally combine dependent outputs or omit a member. The explicit treatment of singleton router and aggregator events also makes the formulation applicable to mixed workflows.
Critical weaknesses
- The headline comparisons do not isolate an algorithmic improvement. Sections 5.1–5.2 and Tables 1–2 compare SRPO runs with published baseline values. Appendix A explicitly states that the baselines are cross-paper references rather than paired reruns. Matching metric definitions does not establish matching orchestration, training data processing, decoding, retrieval resources, or computation. As can be seen from the result, the 0.2-point macro gains are smaller than the reported SRPO seed standard deviations, while comparable baseline uncertainty is unavailable. Appendix A.1 measures changes within SRPO traces and tool calls, rather than a matched budget against competing methods.
- Table 3 compares three jointly clipped reductions and K = 1 versus K = 5, but contains no K = 5 agent-wise clipping control using the same advantage estimator and event weighting.
- No comparison for joint update (JointPPO) vs agent-wise update.
Questions for authors
- Which baseline results in Tables 1–2 are compatible with SRPO in training split, checkpoint selection, prompts, decoding settings, and retrieval configuration? Can you identify the differences that remain after harmonizing the metric definitions?
- Under the same orchestration and matched training and inference budgets, what are the results and paired uncertainty for SRPO versus the strongest baseline.
- At K = 5, how does SRPO compare with independent member clipping when the event advantage, event weighting, optimizer, and generated trajectories are held fixed? Does the benefit persist after tuning the clipping scale for each variant?
- hich exact event-reduction implementation produced each headline result, and how do those runs relate to the token-uniform and event-uniform diagnostics in Sections 5.4–5.5 and Appendix F?
Dear authors and reviewers,
The author-reviewer discussion period closes on November 18 (AoE). Please engage in response periods so that we have our issues discussed thoroughly.
AC