- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to train language-model-based multi-agent systems (MAS) without a centralized trainer that collects agents' trajectories, learning signals, parameters, and optimization states. The authors formulate collaborative MAS learning as a consensus-constrained finite-sum optimization problem, where each role-specific agent keeps a local copy of a shared LoRA adapter and optimizes a local objective based on its own role, prompts, messages, and rollout outcomes, with consensus ADMM reconciling these overlapping local copies through decentralized communication (mainly ring all-reduce). They find that distributed ADMM training substantially reduces harmful behavior in compromised MAS configurations while nearly matching centralized training.
Critical strengths
-
The paper identifies the role-induced coupling in collaborative MAS learning and recasts it as a consensus-constrained optimization problem.
-
The method also has certain general applicability; e.g., the local objectives of the roles can, in principle, be extended to other fine-tuning targets beyond the DPO-style loss described in the text.
-
The ablation in Table 2 is especially useful. Local-only training and consensus averaging without dual updates are substantially worse than full ADMM, whereas full ADMM nearly matches centralized training.
Critical weaknesses
-
Most reported results appear to be based on single runs, and the paper explicitly notes that some compromised-position configurations are evaluated only once.
-
The main trained setting uses Qwen3-8B with rank-16 LoRA and four-agent sequential/hierarchical architectures. The scaling appendix mostly studies untrained systems at different numbers of agents, while only one additional trained experiment is reported at N=3. Thus, the claims about scalability and broad applicability should be moderated.
-
The paper demonstrates locality of trajectories and optimization states, but this should not be conflated with a formal privacy guarantee. Agents still exchange model/adapter variables during consensus, which can potentially leak information. The paper does not analyze parameter leakage, membership inference, malicious update attacks, Byzantine behavior, or communication failures.
-
The “compromised” agent is an obliterated backbone that remains trainable under the shared optimization procedure, rather than an actively adversarial participant that sends poisoned updates or strategically deviates from ADMM.
-
Some claims in the introduction and conclusion should be calibrated more carefully: “scalable,” “resilient,” and “self-organizing” are broader than what the current experiments directly establish.
-
I noticed minor proofreading issues, e.g., “centralized v.s. distributed,” which should be “vs.” or “versus”. A final copy-editing pass would be beneficial.
Questions for authors
-
What exact information is exchanged during ring all-reduce or gossip, and what trust assumptions are made about communication peers? Please clarify whether the privacy claim is intended only as data/trajectory locality, rather than protection against leakage from exchanged adapter parameters.
-
Does the compromised-agent setting assume that the compromised node follows the prescribed ADMM protocol and accepts fine-tuning updates? How would the method behave if a compromised node sent poisoned or inconsistent consensus messages/updates?
6 comments
Response to Reviewer jxJS (Part II)
[W2] The main trained setting uses Qwen3-8B with rank-16 LoRA and four-agent sequential/hierarchical architectures. The scaling appendix mostly studies untrained systems at different numbers of agents, while only one additional trained experiment is reported at N=3. Thus, the claims about scalability and broad applicability should be moderated.
Response:
We added training at a larger system size, on a smaller backbone, and at other LoRA ranks. ADMM reduces ASR and closely matches centralized training in each tested setting. These experiments extend coverage but do not establish training scalability at much larger system sizes.
Scope of the scaling evidence. Appendix D reports the untrained compromised-agent effect for several sizes from to and training at (L1140–1176). Training cost grows with , and our compute budget limits substantially larger trained systems (L1165–1167). Table R9 lists the added trained configurations.
The training comparison uses the remaining settings of Table A2; ASR is measured on HarmBench under compromised initialization.
Table R9. Additional trained configurations under compromised initialization (HarmBench ASR, %). Unless varied, use Qwen3-8B, four roles, and LoRA rank 16; see paper Table A4.
| Setting | Base (untrained) | Centralized | ADMM |
|---|---|---|---|
| S-MAS, , Qwen3-8B (Table A4) | 40.25 | 35.22 | 37.11 |
| S-MAS, , Qwen3-8B (new) | 40.88 | 29.56 | 30.19 |
| S-MAS, , Qwen3-4B (new) | 48.43 | 30.82 | 32.08 |
| H-MAS, , LoRA rank 8 (new) | 52.83 | 27.04 | 27.67 |
| H-MAS, , LoRA rank 32 (new) | 52.83 | 25.16 | 26.42 |
The base values come from Tables A3 and A5. ADMM reduces ASR in every setting and stays within 1.89 pp of centralized training (L1163–1165), extending the conclusion of Table 1 beyond the main setting.
Per-node communication. Each node holds its backbone and local LoRA copy. Ring all-reduce sends times the trainable parameter size per node per round, below twice that size for any . Traffic is 134.5 MB at and 149.5 MB at (L1178–1180). Total rollout and training cost still grows with the number of roles.
Backbone and adapter. The role-local objective can be instantiated for other compatible backbones, LoRA ranks, or differentiable losses. Our tests cover Qwen3-8B and Qwen3-4B with ranks 8, 16, and 32. Appendix E retains broader heterogeneous architectures, interaction structures, and larger systems as future evaluation (L1199–1209).
Appendix D, Table A4, reports the additional trained configurations (L1163–1176); Appendix F.3 gives the matched-work and communication comparison (L1290–1310).
[W5] Some claims in the introduction and conclusion should be calibrated more carefully: "scalable," "resilient," and "self-organizing" are broader than what the current experiments directly establish.
Response:
The abstract presents scalable, resilient, and self-organizing MAS as a longer-term direction, using the phrase "paving the way toward" (L017–018). The evidence supporting that direction is narrower: bounded per-node consensus traffic, no central trainer, repair of the tested compromised systems, and gossip-based coordination.
Scalability. Per-node consensus traffic stays below twice the trainable parameter size (L1307–1308). The added trained configurations extend the evidence to six roles (Table A4, L1169–1176); they do not establish effectiveness at arbitrary system sizes.
Resilience. The framework avoids a central trainer and improves safety in the tested compromised-agent and active-update settings. This does not imply tolerance of arbitrary failures or attacks.
Self-organization. Consensus can use neighbour-to-neighbour gossip (L473–482). The present experiments still assume synchronous training with a fixed participant set (L1213–1214).
The abstract retains this directional wording. The conclusion (L513–515) and Appendix E (L1199–1221) clarify the scope and limitations of the evidence; we do not claim that large-scale resilience or dynamic self-organization has been established.
[W6] I noticed minor proofreading issues, e.g., "centralized v.s. distributed," which should be "vs." or "versus". A final copy-editing pass would be beneficial.
Response:
We thank the reviewer for the careful reading. In the revision, we proofread the paper.
Response to Reviewer jxJS (Part III)
[W3] The paper demonstrates locality of trajectories and optimization states, but this should not be conflated with a formal privacy guarantee. Agents still exchange model/adapter variables during consensus, which can potentially leak information. The paper does not analyze parameter leakage, membership inference, malicious update attacks, Byzantine behavior, or communication failures.
[Q1] What exact information is exchanged during ring all-reduce or gossip, and what trust assumptions are made about communication peers? Please clarify whether the privacy claim is intended only as data/trajectory locality, rather than protection against leakage from exchanged adapter parameters.
Response:
Our claim is data, trajectory, and training-state locality, not a formal privacy guarantee. The paper states it this way: training proceeds "without centralizing agents' trajectories, data, or optimization states" (L097–098), and the motivation is that a central trainer must access all role-local information, which may not be available across devices, organizations, or trust domains (L212–214). The paper makes no differential-privacy or leakage claim.
What is exchanged
- One adapter-sized vector per node per round. In the -step, each node contributes (Algorithm A1, L986–987), whose size is that of the trainable up-projection (22.4M values, 89.7 MB in fp32; L1054). The down-projection is frozen and identical on all nodes, so it is never sent (L1051–1054).
- Ring all-reduce computes the exact average of the , at 134.5 MB per node per round for (L480–481, L1307–1308).
- Gossip exchanges only with the two ring neighbours, for rounds per consensus step (Fig. 5, L497–498).
- Not sent in the consensus step: prompts, rollouts, rewards, gradients, and optimizer states. Role messages are passed only during rollouts, as in any MAS.
Trust assumptions
Peers are honest-but-curious, and communication is synchronous with a fixed set of nodes; asynchronous consensus and nodes that join or leave are listed as future work (Appendix E, L1206–1214). Malicious updates are addressed in our response to W4, Q2.
Leakage from exchanged parameters
Any collaborative training must exchange some information, and exchanged parameters can leak information, as in federated learning. Relative to centralized training, which receives every role's prompts, trajectories, rewards, and states, ADMM exposes only one adapter-sized vector per round. The -step needs only the sum of the , so it is compatible with secure aggregation [R1], which hides individual vectors, and with differentially private local updates (L1216–1218).
Appendix E explicitly states the trust assumptions and distinguishes locality from a formal privacy guarantee (L1213–1221).
Response to Reviewer jxJS (Part I)
We thank the reviewer for recognizing the role-induced coupling formulation, the flexibility of role-local objectives, and the value of the consensus and dual ablations.
We address each concern below (W: weakness; Q: question). Section, table, appendix, and line (L) references are to the revised paper, with changes in blue; R-prefixed tables and references appear in this response.
Response:
We added independent training seeds for the main comparison. ADMM reduces ASR and improves on NoDual in every seed. We also repeated the compromised-position evaluation with different sampling seeds; the smaller differences between the remaining positions are not interpreted as a stable ordering.
Table R7 summarizes the main comparison over three independent training seeds, including the original runs in Tables 1 and 2 (Table A6, L1242–1251). Each run uses the evaluation protocol of Appendix C.4.
Table R7. Main results over three independent training seeds (HarmBench ASR, %, mean ± standard deviation).
The additional seeds confirm the conclusions of Sec. 5.2: ADMM reduces ASR in every seed, differs from centralized training by −0.42 pp on S-MAS and +2.52 pp on H-MAS in mean ASR, and improves on NoDual in every seed (L1254–1258).
The compromised-agent effect recurs across configurations. One compromised agent raises HarmBench ASR in every configuration we evaluated:
The training effect recurs across settings and benchmarks. ADMM lowers ASR in all four settings of Table 1 and on all four safety benchmarks, with little change in utility (L415–427).
Prompt-level uncertainty. Table R8 reports the differences for the original Table 1 and 2 runs, with approximate 95% intervals over the 159 HarmBench test prompts. Training-seed variation is reported separately in Table R7.
Table R8. Prompt-level ASR differences in percentage points with approximate 95% intervals over 159 HarmBench test prompts. The final row gives the magnitude of the observed gap; paper Table A7 uses the signed ADMM minus centralized contrast.
The intervals of the compromised-agent effect, the training effect, and the dual-update gain exclude zero. ADMM and centralized training remain close: across training seeds, their mean ASRs differ by −0.42 pp on S-MAS and +2.52 pp on H-MAS (Table R7; Appendix F.1, L1257).
Compromised positions. We repeated the paper Table A1 evaluation three times with different sampling seeds (Appendix A, L965–971). The Solver and Synthesizer remain the worst single compromised positions in every repetition, with ASR 42.14 ± 2.88% and 53.46 ± 3.50%, respectively. Every other single position stays at most 7.55 percentage points above the benign MAS. We do not interpret the smaller differences among those other positions as a stable ordering, which motivates training all roles rather than assuming which one is compromised.
Appendix F.1 reports the three-seed summaries (L1229–1258), Appendix F.2 reports prompt-level intervals (L1266–1283), and Appendix A reports the repeated compromised-position evaluation (L965–971).