- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes RPO-MAS, a receiver-conditioned intervention-selection framework for multi-agent communication. For each receiver state, the method considers three possible interventions: SEND, where the receiver updates its answer based on a peer response; SELF, where the receiver performs an additional autonomous refinement step; and KEEP, where the receiver preserves its current state.
The method constructs preference data by executing all candidate interventions from the same receiver checkpoint and comparing their resulting answer correctness. A lightweight scorer is then trained with a pairwise ranking objective to predict which intervention should be selected at inference time.
Overall, the paper presents a reasonable formulation of receiver-aware communication selection, but the empirical gains are modest and the main methodological contribution appears incremental. More importantly, the method relies on task- and backbone-specific preference supervision constructed using ground-truth correctness, while providing no convincing evidence that the learned intervention policy generalizes beyond the specific task distributions used to train it.
Critical strengths
-
The intervention decomposition is conceptually clear.
Separating communication intoSEND, autonomous refinement throughSELF, and no intervention throughKEEPis a useful way to distinguish peer information from additional computation and state preservation. -
The matched-counterfactual construction is reasonable.
Evaluating candidate interventions from the same receiver checkpoint is a sensible attempt to isolate the effect of different intervention choices. The inclusion ofSELFandKEEPis also preferable to comparing communication against an insufficient baseline. -
The paper is generally well organized.
The problem formulation, scoring model, and inference procedure are described relatively clearly, and the experimental protocol is sufficiently specified to understand the proposed system.
Critical weaknesses
1. The empirical gains are small and may not be statistically reliable
The improvement over majority voting is only 0.81 percentage points for Qwen2.5-7B-Instruct and 0.71 points for Llama-3.1-8B-Instruct. Several individual results are unchanged or worse than majority voting. Given the small evaluation subsets and stochastic LLM generation, the paper should report results across multiple seeds, at least with mean-std.
2. The method relies on oracle-like, ground-truth-derived supervision
The controller is trained by executing SEND, SELF, and KEEP, then using the ground-truth correctness of their outputs to construct preference labels. This is a valid supervised policy-learning setup, but it is close to hindsight or answer-aware imitation and weakens the claim that the method learns a generally available communication principle. The paper should more clearly position RPO-MAS as task-specific supervised intervention selection.
3. There is no evidence of cross-task or out-of-distribution generalization
Separate controllers are trained for every dataset and backbone, while evaluation remains within the same task and model family. There are no experiments on unseen tasks, cross-dataset transfer, cross-backbone transfer, prompt changes, different team sizes, or distribution shifts. Thus, the results may reflect benchmark-specific pattern fitting rather than a transferable receiver-conditioned policy.
4. The algorithmic novelty is limited and relevant baselines are missing
SEND, SELF, and KEEP are standard communication, self-refinement, and no-op operations. The main addition is a lightweight scorer trained on outcome-derived pairwise labels. The paper does not compare against simple disagreement/confidence heuristics, learned communication gates, contextual bandits, or other reward-based selectors, making it difficult to isolate the value of the proposed preference objective.
5. Preference labels are noisy and based on limited rollouts
Candidate interventions appear to be judged from single sampled outcomes, even though LLM generation is stochastic. A single correct or incorrect completion is a noisy estimate of an intervention's expected utility. The paper should analyze label stability across seeds and repeated rollouts, as well as sensitivity to the amount of preference data.
6. The evaluation protocol is narrow and the receiver-level metrics are difficult to interpret
The experiments use five homogeneous agents, two interaction rounds, two backbones, and task-specific controllers trained from only 400 examples. In addition, low harm can be achieved by selecting KEEP frequently, so correction and harm rates should be interpreted jointly with intervention frequency, computation cost, and final team accuracy. The very low selection rate of SELF also raises the question of whether the full three-way intervention space is necessary.
Questions for authors
1. Does the learned policy generalize? Please evaluate cross-task, cross-backbone, or out-of-distribution transfer instead of training a separate controller for every dataset and backbone.
2. How stable are the preference labels? Since labels appear to be based on single sampled outcomes, please report results using repeated rollouts and analyze preference-label stability.
3. How much of the gain comes from the proposed objective rather than adaptive gating? Please compare against simple disagreement/confidence heuristics and a learned KEEP/UPDATE or contextual action-selection baseline.
4. Is SELF necessary? Given its very low selection frequency, please report an ablation using only SEND and KEEP, together with performance as the number of agents and interaction rounds changes.
5. What is the cost-benefit trade-off? Please report the cost of constructing the preference data and relate intervention frequency, correction rate, harm rate, computation cost, and final team accuracy.