- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
NEXA is a response-conditioned training policy designed to optimize the collaboration topology of Multi-Agent Systems. The framework initiates with all agents generating initial drafts in parallel and subsequently utilizes semantic embeddings to dynamically determine if sequential communication is required to refine the final output. By employing a learnable sparse DAG prediction mechanism, this approach achieves accuracy comparable to state-of-the-art baselines without the need for external LLM judges or complex hand-crafted topology search. This method significantly reduces token consumption and computational latency, providing substantial practical value for multi-agent systems where inference costs must be strictly controlled.
Critical strengths
S1. The paper focuses on the critical question of whether communication is necessary. By deciding whether to further collaborate based on initial responses, it offers significant practical value for multi-agent systems that require strict control over inference costs.
S2. The method determines the topological order based on the contribution scores of current responses and only learns the forward communication edges within that order. This design avoids the complexity of simultaneously learning both sequence and connectivity while ensuring the communication graph is acyclic by construction. It is a clean and easily executable design.
Critical weaknesses
W1. NEXA inherits the contribution-based ordering from SelfOrg with its primary modification being the replacement of the original stochastic self-organization rules with a trainable communication edge prediction policy. Overall, the contribution appears to be an incremental improvement over existing frameworks. The unique value of the new learning mechanism compared to simple sparse communication strategies needs more thorough justification.
W2. NEXA’s consensus-based ordering may place correct minority responses downstream when agents share similar errors, preventing corrective feedback to upstream agents. Without targeted evaluation of such cases, the impact on final-answer accuracy remains unclear.
Questions for authors
Q1. Since NEXA fixes the communication graph based on the initial drafts, how does it handle errors that may arise during sequential refinement? Beyond the reported harm rates, could the authors analyze whether downstream agents correct or propagate these errors, and discuss when replanning based on updated responses might justify its additional communication cost?
3 comments
Response to Reviewer cWy6 [2/2]
(continued)
The most restrictive case, where every correct draft appears only in the final two positions, therefore occurs in only 22/623 = 3.5% of complete draft pools. Moreover, these cases produce three rescues and no harms relative to same-draft Parallel aggregation. Thus, while the fixed order creates a genuine worst-case restriction, these downstream-only cases are rare in this audit and do not reduce accuracy relative to same-draft Parallel.
Correct downstream responses also remain available to the final aggregation and may inform later agents; they are not removed from the candidate pool.
We will add this analysis and make the existing limitation more prominent in the main text. Jointly learning the ordering and connectivity, or allowing reverse correction, remains a natural extension of the current one-pass design.
Q1 - During sequential refinement, do downstream agents correct or propagate newly introduced errors, and when would dynamic replanning be worthwhile?
Thank you for raising this concern. Nexa fixes the graph from the initial response pool, but executes it sequentially: each downstream agent receives the most recent available responses from all of its upstream parents. Thus, a response corrected earlier in the graph is already available to later agents within the same propagation pass.
The possibility of error propagation is already explicitly acknowledged in the submitted paper (the original submission). Section 4.4 reports rescue, harm, and preservation rates, while the Ethics and Limitations sections note that communication may propagate shared errors and that semantic agreement does not guarantee correctness. We now complement these aggregate results with a targeted analysis of how parent correctness relates to downstream refinement.
The analysis uses Nexa's actual multiple-parent setting. For each refined target, we associate its outcome with every incoming parent response. A target with several parents therefore contributes one parent-target observation per incoming edge and may appear in both columns when it receives a mixture of correct and incorrect responses.
| Target before refinement | Outcome after refinement | Correct-parent instance | Incorrect-parent instance |
|---|---|---|---|
| Wrong | Becomes correct | $\approx$78% | $\approx$24% |
| Correct | Becomes wrong | $\approx$3.2% | $\approx$38% |
Correct upstream evidence is therefore associated with substantially more downstream repair and substantially less corruption. These are descriptive multi-parent associations rather than isolated causal effects, since several parent responses may jointly influence one target.
This result should also be interpreted together with the preceding in-expectation analysis of contribution ordering. Among 623 complete draft pools, only 22/623 (3.5%) place every correct minority response in the final two ranks; these cases produce three rescues and zero harms relative to same-draft Parallel. Across the complete AQUA-RAT evaluation, communication produces 27 rescues and 6 harms. Thus, individual adverse propagation paths can occur, as already stated in the paper, but in expectation, communication remains net positive in the evaluated distribution.
The current method uses a single propagation pass and does not replan after each update. Replanning could be beneficial when refinement materially changes response agreement or the apparent reliability of the agents, for example, when updated responses sharply disagree or would substantially alter the contribution ordering. However, another planning and refinement stage also introduces additional calls, tokens, and latency. A selective trigger based on the updated response state is therefore a promising extension, whereas indiscriminate replanning would weaken the efficiency advantage of the current one-pass design.
We will add this multiple-parent analysis and representative correction and regression cases, while making clear that dynamic replanning is outside the current formulation.
References
- Tastan, N., Horvath, S., and Nandakumar, K. "Stochastic Self-Organization in Multi-Agent Systems." ICLR, 2026.
Dear Reviewer,
Thank you again for your feedback. We would greatly appreciate it if you could let us know whether our rebuttal addressed your concerns so we can address any remaining ones.
Response to Reviewer cWy6 [1/2]
We thank the reviewer for recognizing Nexa's practical value, clean design, and acyclic response-conditioned orchestration. We address the concerns below.
Thank you for raising this concern. We agree that Nexa inherits the contribution-based ordering from SelfOrg, and we will make this lineage more explicit. The contribution is not a new ordering rule. Rather, Nexa replaces SelfOrg's hand-specified stochastic graph-construction rule with a trainable response-to-execution policy.
Both methods are response-conditioned, acyclic, and judge-free. The key distinction is that SelfOrg reapplies a predefined self-organization rule, whereas Nexa is optimized for final-answer correctness and maps each realized response pool to a complete execution decision: whether refinement is needed, which agents should be updated, and which upstream responses they should receive.
To directly test whether the learned mechanism provides value beyond simple sparse routing, we add the following comparison under the same communication budget:
Nexa achieves the strongest mean performance among the random and fixed sparse alternatives. Since the communication budget is held fixed, this experiment evaluates the learned response-conditioned graph rather than differences in refinement budget.
Importantly, this is not obtained by retraining a graph for each deployment. The same frozen controller is reused across tasks, team sizes, model generations, 14B/32B agents, and the new four-family heterogeneous evaluation. Thus, the learned object is not a single benchmark-specific topology, but a reusable rule for mapping realized response pools to selective refinement plans.
We will revise the manuscript to distinguish the inherited and new components more precisely: the contribution-based order is inherited from SelfOrg, while Nexa contributes a reusable, task-trained response-to-execution policy that adaptively selects the communication structure and refinement budget.
Thank you for raising this concern. This failure mode is already explicitly identified in the submitted paper (the original submission). Section 3.2 states that contribution ordering is a distributional prior rather than a correctness verifier, and notes that a rare correct response may be placed downstream. Appendix A further explains that, under the single forward pass, such a response cannot provide reverse feedback to earlier agents.
The relevant claim is therefore in expectation, not that the ordering is correct for every individual example. When agent errors are dispersed, agreement tends to concentrate around the correct response, so contribution ordering generally places useful responses earlier. The question is whether exceptions occur frequently enough to materially affect average accuracy.
We add a targeted AQUA-RAT analysis over the 623 examples for which all five initial drafts can be assigned correctness labels:
(continued)