acceptodds
Under review as a conference paper at ICLR 2027

Response-Conditioned Parallel-to-Sequential Orchestration for Multi-Agent Systems

Abstract

Multi-agent systems can solve complex tasks through collaboration between multiple Large Language Model agents. Existing collaboration frameworks typically operate in either a parallel or a sequential mode. In the parallel mode, agents respond independently to queries followed by aggregation of responses. In contrast, sequential systems allow agents to communicate via a directed topology and refine one another step by step. However, both modes are inadequate for achieving the desired objectives of minimizing communication and latency while simultaneously maximizing the accuracy of the final response. In this work, we introduce a hybrid paradigm called Nexa, a trainable response-conditioned policy that bridges the gap between the two modes. Nexa begins with a parallel execution stage, embeds the resulting responses into a shared semantic space, and then predicts a sparse directed acyclic communication graph. If the graph is empty, the system remains purely parallel; if it is non-empty, the system performs one sequential message propagation. The policy is a lightweight transformer model, and the method avoids the need for external LLM judges or reward models, as well as hand-crafted test-time topology search. We formalize this hybrid execution problem, show that the resulting graph is acyclic by construction, and present a training procedure based on policy-gradient optimization. Results demonstrate that Nexa attains accuracy comparable to the strongest baseline while using 35.4% fewer tokens; the frozen policy also transfers across agent counts, tasks, and model scales, including 14B and 32B evaluations on GSM-Hard and AIME-2024.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent systems, llms

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_cWy6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

NEXA is a response-conditioned training policy designed to optimize the collaboration topology of Multi-Agent Systems. The framework initiates with all agents generating initial drafts in parallel and subsequently utilizes semantic embeddings to dynamically determine if sequential communication is required to refine the final output. By employing a learnable sparse DAG prediction mechanism, this approach achieves accuracy comparable to state-of-the-art baselines without the need for external LLM judges or complex hand-crafted topology search. This method significantly reduces token consumption and computational latency, providing substantial practical value for multi-agent systems where inference costs must be strictly controlled.

Critical strengths

S1. The paper focuses on the critical question of whether communication is necessary. By deciding whether to further collaborate based on initial responses, it offers significant practical value for multi-agent systems that require strict control over inference costs.

S2. The method determines the topological order based on the contribution scores of current responses and only learns the forward communication edges within that order. This design avoids the complexity of simultaneously learning both sequence and connectivity while ensuring the communication graph is acyclic by construction. It is a clean and easily executable design.

Critical weaknesses

W1. NEXA inherits the contribution-based ordering from SelfOrg with its primary modification being the replacement of the original stochastic self-organization rules with a trainable communication edge prediction policy. Overall, the contribution appears to be an incremental improvement over existing frameworks. The unique value of the new learning mechanism compared to simple sparse communication strategies needs more thorough justification.

W2. NEXA’s consensus-based ordering may place correct minority responses downstream when agents share similar errors, preventing corrective feedback to upstream agents. Without targeted evaluation of such cases, the impact on final-answer accuracy remains unclear.

Questions for authors

Q1. Since NEXA fixes the communication graph based on the initial drafts, how does it handle errors that may arise during sequential refinement? Beyond the reported harm rates, could the authors analyze whether downstream agents correct or propagate these errors, and discuss when replanning based on updated responses might justify its additional communication cost?

3 comments
AuthorsOpenReview ↗

Response to Reviewer cWy6 [1/2]

We thank the reviewer for recognizing Nexa's practical value, clean design, and acyclic response-conditioned orchestration. We address the concerns below.

W1 - What is Nexa's contribution beyond SelfOrg, and does the learned mechanism provide value over simple sparse routing?

Thank you for raising this concern. We agree that Nexa inherits the contribution-based ordering from SelfOrg, and we will make this lineage more explicit. The contribution is not a new ordering rule. Rather, Nexa replaces SelfOrg's hand-specified stochastic graph-construction rule with a trainable response-to-execution policy.

Both methods are response-conditioned, acyclic, and judge-free. The key distinction is that SelfOrg reapplies a predefined self-organization rule, whereas Nexa is optimized for final-answer correctness and maps each realized response pool to a complete execution decision: whether refinement is needed, which agents should be updated, and which upstream responses they should receive.

ComponentSelfOrgNexa
Response orderingContribution-basedContribution-based, inherited from SelfOrg
Graph constructionHand-specified stochastic ruleLearned from the globally contextualized response pool
Topology training signalNoneFinal-answer correctness with sparsity regularization
Execution decisionApplies the predefined communication procedureSelects active edges, refined agents, and communication budget
DeploymentReapplies the heuristicReuses one frozen policy across settings

To directly test whether the learned mechanism provides value beyond simple sparse routing, we add the following comparison under the same communication budget:

Communication graphAccuracy (%)Difference from Nexa
Nexa: learned, response-conditioned57.74 ± 2.31-
Random55.64 ± 2.90-2.10
Chain53.02 ± 2.02-4.72
Star53.94 ± 2.73-3.80
Tree56.43 ± 0.82-1.31

Nexa achieves the strongest mean performance among the random and fixed sparse alternatives. Since the communication budget is held fixed, this experiment evaluates the learned response-conditioned graph rather than differences in refinement budget.

Importantly, this is not obtained by retraining a graph for each deployment. The same frozen controller is reused across tasks, team sizes, model generations, 14B/32B agents, and the new four-family heterogeneous evaluation. Thus, the learned object is not a single benchmark-specific topology, but a reusable rule for mapping realized response pools to selective refinement plans.

We will revise the manuscript to distinguish the inherited and new components more precisely: the contribution-based order is inherited from SelfOrg, while Nexa contributes a reusable, task-trained response-to-execution policy that adaptively selects the communication structure and refinement budget.


W2 - Can contribution ordering place a correct minority downstream and prevent it from correcting upstream agents? What is the impact on final accuracy?

Thank you for raising this concern. This failure mode is already explicitly identified in the submitted paper (the original submission). Section 3.2 states that contribution ordering is a distributional prior rather than a correctness verifier, and notes that a rare correct response may be placed downstream. Appendix A further explains that, under the single forward pass, such a response cannot provide reverse feedback to earlier agents.

The relevant claim is therefore in expectation, not that the ordering is correct for every individual example. When agent errors are dispersed, agreement tends to concentrate around the correct response, so contribution ordering generally places useful responses earlier. The question is whether exceptions occur frequently enough to materially affect average accuracy.

We add a targeted AQUA-RAT analysis over the 623 examples for which all five initial drafts can be assigned correctness labels:

Correct-minority diagnosticResult
Complete five-draft pools623/762
Strict correct-minority cases164/623 (26.3%)
All correct drafts confined to ranks 4-522/164 (13.4%)
Rescues / harms in these 22 cases3 / 0

(continued)

AuthorsOpenReview ↗

Response to Reviewer cWy6 [2/2]

(continued)

The most restrictive case, where every correct draft appears only in the final two positions, therefore occurs in only 22/623 = 3.5% of complete draft pools. Moreover, these cases produce three rescues and no harms relative to same-draft Parallel aggregation. Thus, while the fixed order creates a genuine worst-case restriction, these downstream-only cases are rare in this audit and do not reduce accuracy relative to same-draft Parallel.

Correct downstream responses also remain available to the final aggregation and may inform later agents; they are not removed from the candidate pool.

We will add this analysis and make the existing limitation more prominent in the main text. Jointly learning the ordering and connectivity, or allowing reverse correction, remains a natural extension of the current one-pass design.


Q1 - During sequential refinement, do downstream agents correct or propagate newly introduced errors, and when would dynamic replanning be worthwhile?

Thank you for raising this concern. Nexa fixes the graph from the initial response pool, but executes it sequentially: each downstream agent receives the most recent available responses from all of its upstream parents. Thus, a response corrected earlier in the graph is already available to later agents within the same propagation pass.

The possibility of error propagation is already explicitly acknowledged in the submitted paper (the original submission). Section 4.4 reports rescue, harm, and preservation rates, while the Ethics and Limitations sections note that communication may propagate shared errors and that semantic agreement does not guarantee correctness. We now complement these aggregate results with a targeted analysis of how parent correctness relates to downstream refinement.

The analysis uses Nexa's actual multiple-parent setting. For each refined target, we associate its outcome with every incoming parent response. A target with several parents therefore contributes one parent-target observation per incoming edge and may appear in both columns when it receives a mixture of correct and incorrect responses.

Target before refinementOutcome after refinementCorrect-parent instanceIncorrect-parent instance
WrongBecomes correct$\approx$78%$\approx$24%
CorrectBecomes wrong$\approx$3.2%$\approx$38%

Correct upstream evidence is therefore associated with substantially more downstream repair and substantially less corruption. These are descriptive multi-parent associations rather than isolated causal effects, since several parent responses may jointly influence one target.

This result should also be interpreted together with the preceding in-expectation analysis of contribution ordering. Among 623 complete draft pools, only 22/623 (3.5%) place every correct minority response in the final two ranks; these cases produce three rescues and zero harms relative to same-draft Parallel. Across the complete AQUA-RAT evaluation, communication produces 27 rescues and 6 harms. Thus, individual adverse propagation paths can occur, as already stated in the paper, but in expectation, communication remains net positive in the evaluated distribution.

The current method uses a single propagation pass and does not replan after each update. Replanning could be beneficial when refinement materially changes response agreement or the apparent reliability of the agents, for example, when updated responses sharply disagree or would substantially alter the contribution ordering. However, another planning and refinement stage also introduces additional calls, tokens, and latency. A selective trigger based on the updated response state is therefore a promising extension, whereas indiscriminate replanning would weaken the efficiency advantage of the current one-pass design.

We will add this multiple-parent analysis and representative correction and regression cases, while making clear that dynamic replanning is outside the current formulation.

References

  • Tastan, N., Horvath, S., and Nandakumar, K. "Stochastic Self-Organization in Multi-Agent Systems." ICLR, 2026.

Dear Reviewer,

Thank you again for your feedback. We would greatly appreciate it if you could let us know whether our rebuttal addressed your concerns so we can address any remaining ones.

Reviewer_eFvpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed a framework (NEXA) that learns when to communicate, who communicates with whom based on their initial responses. They uses a lightweight transformer to predict a sparse communication graph, allowing agents to either remain parallel or perform sequential refinement. The experiments show that their method could have comparable accuracy to the baselines but reducing the token usage. The authors also evaluate if the learned communication method could transfer across different settings.

Critical strengths
  • I like the idea of combining parallel and sequential collaboration, allowing agents to decide if extra communication is needed based on their actual responses.

  • Even though later in the weakness part I argued that the current experimental results cannot fully support the paper's core ideas, but the evaluations direction is good, e.g., comparing the accuracy and the token usage, and the latency, this could show potential of adaptive communication for improving efficiency in multi-agent systems. Then the authors also evaluate the learned policy in different settings to see if the policy could transfer, this is also a practical problem to consider in multi-agent communication problems.

Critical weaknesses

This paper actually studied three problems: 1. does the method actually learn when communication is necessary? 2. learning of who should communicate with whom; 3. if the learned method really optimize communication cost. But I don't think the current results could support their claims.

  • one of the main claims is that NEXA can decide if the agents need extra communication. However, figure 6 only reports the percentage of sparse graphs, not how often the policy chooses an empty graph and skips communication entirely, these are different things, so the paper does not clearly show if the policy actually learns to avoid communication when it is unnecessary.

  • for the second question, the paper used the semantic similarity to rank agents and only learns the communication edges. Tab. 3 shows that contribution-based ordering is better than random or reversed ordering, but this does not demonstrate that the learned edges are better than simple, heuristic comm graphs. I would like to see comparisons with random, similarity-based, and fixed sparse graphs under the same comm budget.

  • then for the third question, the training objective penalizes the number of communication edges rather than actual token cost or latency. These are not necessarily the same. For example, in the heterogeneous-agent experiment, NEXA uses about three times as many tokens as pure parallel execution. This raises questions about if the method can consistently reduce communication costs in more realistic settings.

Questions for authors
  1. How often does NEXA actually skip communication? Does it learn when communication is not necessary?

  2. How do the learned graphs compare with random or simple heuristic graphs under the same communication budget?

  3. Why the authors choose to optimize the number of edges but not the actual token cost or latency? (especially given the higher token usage with heterogeneous agents)

2 comments
AuthorsOpenReview ↗

Response to Reviewer eFvp

We thank the reviewer for recognizing the value of adaptive communication, the accuracy-cost evaluation, and the transferability of the learned policy. We address the three concerns below.

W1 / Q1 - How often does Nexa actually choose an empty graph and skip communication entirely?

Thank you for bringing this to our attention. We now directly report the rate at which the policy selects an empty graph, thereby skipping sequential refinement entirely.

Empty graphsSame-pass Parallel → Nexa
46/762 (6.04%)54.98% → 57.74%

Thus, on AQUA-RAT, Nexa skips communication entirely for 6.04% of examples, while improving over the corresponding same-pass Parallel baseline by 2.76 accuracy points.

We will revise the manuscript to report empty-graph frequency in addition to edge sparsity.


W2 / Q2 - How do Nexa's learned communication graphs compare with random or simple fixed sparse graphs under the same communication budget?

Thank you for this suggestion. Table 3 evaluates the contribution-defined ordering, but does not directly compare Nexa's learned graph with simple heuristic topologies. SelfOrg already provides the response-similarity/contribution-based heuristic comparison, while the new experiment below complements it with random and fixed sparse graphs under the same communication budget and evaluation setting:

Communication graphAccuracy (%)Difference from Nexa
Nexa: learned, response-conditioned57.74 ± 2.31-
Random55.64 ± 2.90-2.10
Chain53.02 ± 2.02-4.72
Star53.94 ± 2.73-3.80
Tree56.43 ± 0.82-1.31

Nexa achieves the strongest mean performance among the random and fixed sparse alternatives. Notably, random communication is a competitive, rather than trivial, control: MacNet also reports strong average performance for irregular/random topologies among fixed-graph structures (Qian et al., 2025).

These results support constructing the communication graph conditionally from the realized response pool rather than using a random or fixed topology.

We will add this experiment alongside Table 3 and clarify that the two analyses are complementary: Table 3 evaluates the ordering prior, while the new experiment evaluates graph construction under the same communication budget.


W3 / Q3 - Why optimize edge count rather than actual token cost or latency?

Thank you for raising this point. We agree with this distinction and had already explicitly acknowledged it in Appendix A and Appendix F.4 of the original submission: edge count is a provider-independent structural proxy and does not directly optimize critical-path latency, monetary cost, or prompt-cache behavior.

We use edge count only as a training-time proxy for communication burden because it is immediately available for every sampled graph and directly regularizes the policy's communication decisions. In contrast, realized token usage and latency are observed only after graph execution and depend on response length, tokenizer, model family, hardware, batching, and serving configuration. Directly optimizing these quantities would tie the controller to a particular deployment environment.

To verify that this structural proxy translates into physical savings, we measure calls, prompt-and-completion tokens, and end-to-end latency in the mixed-family GSM-Hard evaluation:

MethodMean callsMean tokensMean latency
Same-draft Parallel4.001,82119.73 s
Nexa6.004,80634.26 s
One-round SelfOrg8.007,44346.33 s

Nexa and SelfOrg achieve similar accuracy, while Nexa uses 25% fewer calls, 35.4% fewer tokens, and 26.1% lower latency.

Thus, edge count provides a readily available, deployment-independent training signal, while the realized measurements above validate that the learned sparse plans reduce physical communication cost relative to full one-round collaboration. We will make this distinction more prominent in the main text rather than leaving it only in the Limitations and appendix.

References

  • Qian, C., Xie, Z., Wang, Y., et al. "Scaling Large Language Model-based Multi-Agent Collaboration." ICLR, 2025.

Dear Reviewer,

Thank you again for your feedback. We would greatly appreciate it if you could let us know whether our rebuttal addressed your concerns, so we can address any remaining ones.

Reviewer_CsKSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

NEXA is a lightweight controller for multi-agent LLM systems that dynamically decides whether to stop after a parallel pass or run one targeted round of sequential refinement. First, all agents generate answers independently. Text embeddings establish an agent priority order, and a tiny policy samples an acyclic communication graph. If refinement is triggered, agents update their responses based on upstream feedback, and the answer closest to a contribution-weighted centroid is chosen as the final output without an external judge.

On math and coding benchmarks, NEXA achieves accuracy on par with methods like SelfOrg while using 35.4% fewer tokens and running about 2.5× faster. The authors further report that the frozen controller transfers without retraining across varying team sizes (5 to 20 agents), larger model scales (7B to 32B), and harder tasks like AIME-2024, with minimal controller overhead.

Critical strengths
  1. Simple design with useful efficiency gains. NEXA uses a small controller and needs no LLM judge or test-time graph search. It matches one-round SelfOrg in accuracy while using 35.4% fewer tokens. At five agents, reported latency falls from 19.89 to 7.89 seconds. This accuracy–cost trade-off could make multi-agent systems more practical.

  2. Good controls and clear claims. The paper compares NEXA with parallel aggregation using the same drafts, helping show whether communication adds value. It also reports when revisions help or hurt. The authors focus on saving cost while keeping similar accuracy, and the controls make the results easier to judge.

Critical weaknesses
  1. Limited model coverage. All experiments use Qwen models, and the trainable-topology baselines are tested only at 1.5B. At larger sizes, NEXA’s gains are small or absent. The AIME-2024 result is compared only with Single, so it is unclear whether the learned policy adds value beyond using multiple drafts. The heterogeneous team also mixes only Qwen sizes, leaving cross-family transfer untested. Comparisons with Parallel and SelfOrg at larger sizes, plus a mixed-family test, would help clarify these claims.

  2. Unusually weak HumanEval baselines. Several multi-agent methods score far below Single on HumanEval, especially self-consistency (12.52 versus 50.41). This raises questions about their prompts, code extraction, aggregation, or training setup. HumanEval also appears to drive much of NEXA’s reported advantage: on AQUA-RAT, NEXA and AgentPrune are nearly tied (57.74 versus 57.58).

Questions for authors
  1. What settings were used for the HumanEval baselines, and why do several perform far below Single?
  2. How does NEXA compare with Parallel and SelfOrg at larger model sizes?
3 comments
AuthorsOpenReview ↗

Response to Reviewer CsKS [1/2]

We thank the reviewer for recognizing Nexa's simple design, same-draft controls, and practical accuracy-cost trade-off. We address the two concerns below.

W1 - Limited model coverage: How does Nexa compare with Parallel and SelfOrg at larger model sizes, and does the policy transfer across model families?

Thank you for raising this concern. We agree that the AIME-2024 comparison should have included Parallel and SelfOrg. We would also like to clarify the extent of the larger-model evaluation already included in the original submission. Section 4.3 and Figure 3 report model-scale generalization, while Appendix F.2 provides the complete 14B/32B comparisons: Table 8 for AQUA-RAT, Table 9 for GSM8K, and Table 10 for GSM-Hard. These tables already compare Nexa against Single, CoT, same-draft Parallel, and one-round SelfOrg. Across these settings, Nexa is either the strongest method or remains on par with the strongest result within the reported variation.

For AIME-2024, however, Appendix F.2, Table 11 reported only Single and Nexa. We agree that this did not distinguish the benefit of response-conditioned refinement from the benefit of generating multiple initial drafts. We therefore added the requested controls under the same evaluation protocol. Results report mean accuracy ± standard deviation across 3 seeds:

AIME-2024 / model scaleSingleSame-draft ParallelIndependent SelfOrgNexa
Qwen2.5-14B11.11 ± 1.9211.11 ± 3.8514.44 ± 1.9217.78 ± 1.92
Qwen2.5-32B20.00 ± 0.0014.44 ± 5.0917.78 ± 3.8524.44 ± 1.93

Parallel aggregates the exact initial drafts used by Nexa, while SelfOrg is evaluated using independent one-round collaborative generations. Nexa outperforms same-draft Parallel by +6.67 points at 14B and +10.00 points at 32B, and SelfOrg by +3.33 and +6.67 points, respectively. Thus, the AIME improvement cannot be attributed merely to drawing multiple initial responses: response-conditioned refinement provides an additional gain.

The magnitude of a multi-agent gain should also be interpreted relative to the remaining task headroom, rather than model size in isolation. On GSM8K, Single already reaches 93.93% at 14B and 95.20% at 32B (Appendix F.2, Table 9), leaving little room for any collaborative method to improve. In contrast, GSM-Hard remains substantially more difficult: Single reaches only 60.73% and 60.87%, while Nexa reaches 63.13% and 64.67% at 14B/32B and outperforms both Parallel and SelfOrg at both scales. AIME-2024 similarly retains substantial headroom.

This is consistent with prior multi-agent findings that collaboration gains are strongly related to the capability of the underlying agents and the remaining task difficulty: as single-agent performance approaches saturation, the available gain from additional collaboration naturally decreases, while harder tasks retain greater opportunity for multi-agent improvement (Li et al., 2024; Kim et al., 2026).

Therefore, our scaling claim is not that collaboration gains must increase monotonically with parameter count. Rather, Nexa remains effective at 14B and 32B, is strongest or on par with Parallel and SelfOrg across the evaluated settings, and provides the clearest gains when the task retains meaningful headroom.

The efficiency advantage over one-round SelfOrg also persists at scale: in the reported 14B/32B AQUA-RAT and GSM8K evaluations, Nexa requires approximately 7.8 calls versus 10 calls for SelfOrg, together with approximately 34-37% fewer tokens.

AuthorsOpenReview ↗

Response to Reviewer CsKS [2/2]

Cross-family transfer

We agree that the heterogeneous-agent experiment in the submitted Appendix F.4 varies model scale within the Qwen family and therefore does not directly establish transfer across model families. We now add a mixed-family evaluation on GSM-Hard using:

  • Qwen2.5-7B-Instruct
  • Mistral-7B-Instruct-v0.3
  • Falcon3-7B-Instruct
  • Llama-3.1-8B

Importantly, we use the same controller trained and selected with homogeneous Qwen2.5-1.5B agents on GSM8K, without any retraining. The controller receives no model names, family indicators, or agent identities; it operates only on semantic representations of the agents' responses. This experiment therefore simultaneously changes the task, model scale, and model family.

Results below report mean accuracy across 3 seeds.

GSM-HardAccuracyMean calls
Stochastic Single44.20%1.00
Same-draft Parallel46.60%4.00
Independent SelfOrg47.67%8.00
Nexa47.60%6.00

For Single, one of the four models is selected stochastically for each question, independently of correctness. Parallel aggregates the exact four initial drafts used by Nexa, directly controlling for heterogeneous draft diversity. SelfOrg uses the same four-model team but generates an independent one-round collaborative trajectory.

Nexa improves over stochastic Single by +3.40 accuracy points and over same-draft Parallel by +1.00 point. Nexa also achieves a virtually identical observed accuracy to one-round SelfOrg (47.60% versus 47.67%).

The corresponding efficiency measurements are:

Measured costSame-draft ParallelNexaIndependent SelfOrg
Mean calls4.006.008.00
Mean tokens1,8214,8067,443
Mean latency19.73 s34.26 s46.33 s

Relative to one-round SelfOrg, Nexa uses 25% fewer calls, 35.4% fewer tokens, and 26.1% lower latency, while maintaining similar accuracy.

These results show that the learned orchestration policy is not tied to a single Qwen response distribution. A controller trained entirely with small, homogeneous Qwen agents transfers unchanged to a heterogeneous team spanning four distinct model families, improves over same-draft Parallel, and matches SelfOrg with substantially lower communication cost.

We will add the full mixed-family setup and results to the appendix and distinguish explicitly between heterogeneous-scale and heterogeneous-family transfer.


W2 / Q1 - Why are several HumanEval baselines unusually weak, especially self-consistency?

Thank you for highlighting the anomalously low HumanEval self-consistency result. We audited the HumanEval evaluation and reran every method using the same baseline implementations and configurations used in the submission. The evaluation uses all 164 HumanEval tasks, HumanEval-specific code-generation/refinement prompts, and execution-based scoring.

The issue was isolated to self-consistency. Its aggregation path reused answer extraction and voting intended for short-form reasoning outputs. This is unsuitable for code because functionally equivalent programs may differ textually, while code fences and surrounding explanations can interfere with string-based aggregation. We corrected this path to preserve complete programs and evaluate them by execution.

Methodpass@1 (%)
Self-consistency48.98$\pm$1.41

This replaces the submitted 12.52% pass@1 result and removes the apparent self-consistency collapse. The remaining methods reproduced the values reported in Table 1 within the original run-to-run variation and therefore remain unchanged.

Thus, the lower results of some graph-based baselines are not due to the self-consistency issue or a shared failure of the HumanEval execution harness; they are their reproduced performance under the common evaluation setting.

We also agree that Nexa and AgentPrune are effectively on par on AQUA-RAT (57.74% versus 57.58%), and we will not present this 0.16-point difference as meaningful. The relevant distinction is efficiency: across Table 1, Nexa uses 18.36M tokens, compared with 31.44M for AgentPrune, corresponding to 41.6% fewer tokens. We will therefore characterize this result as comparable AQUA-RAT accuracy at substantially lower communication cost.

We will replace the affected self-consistency entry, recompute the dependent aggregate statistics, and document the corrected settings for HumanEval aggregation and execution.

References

  • Li, J., Zhang, Q., Yu, Y., Fu, Q., and Ye, D. "More Agents Is All You Need." Transactions on Machine Learning Research, 2024.
  • Kim, Y., Gu, K., Park, C., et al. "Capable language models can outgrow the benefits of collaboration." Nature Machine Intelligence, 8:1157-1172, 2026.

Dear Reviewer,

Thank you again for your feedback. We would greatly appreciate it if you could let us know whether our rebuttal addressed your concerns, so we can address any remaining ones.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

We thank the reviewers for the constructive feedback. In response, we ran several additional analyses. The main findings are summarized below.

ConcernWhat we found
Larger-model coverageThe original submission already includes 14B/32B results on AQUA-RAT, GSM8K, and GSM-Hard. We additionally evaluated Parallel and SelfOrg on AIME-2024. Nexa reaches 17.78 ± 1.92 at 14B and 24.44 ± 1.93 at 32B, compared with 11.11 ± 3.85 / 14.44 ± 5.09 for same-draft Parallel and 14.44 ± 1.92 / 17.78 ± 3.85 for SelfOrg.
Cross-family transferWe evaluated the frozen controller, without retraining, on a mixed team of Qwen2.5-7B, Mistral-7B, Falcon3-7B, and Llama-3.1-8B. Nexa reaches 47.60%, compared with 44.20% Single, 46.60% same-draft Parallel, and 47.67% SelfOrg. Relative to SelfOrg, it uses 25% fewer calls, 35.4% fewer tokens, and 26.1% lower latency.
HumanEvalWe identified an issue specific to the self-consistency aggregation path. The corrected result is 48.98 ± 1.41% pass@1; the remaining baselines reproduce the submitted results.
Communication stoppingNexa selects an empty graph on 46/762 (6.04%) AQUA-RAT examples, while improving the corresponding parallel baseline.
Graph constructionNexa achieves 57.74 ± 2.31, compared with Random 55.64 ± 2.90, Chain 53.02 ± 2.02, Star 53.94 ± 2.73, and Tree 56.43 ± 0.82.
Correct-minority casesCases where all correct responses fall in ranks 4-5 occur in only 22/623 (3.5%) AQUA-RAT pools. In these cases, Nexa produces 3 rescues and 0 harms relative to same-draft Parallel.
Error propagationIn the multiple-parent analysis, correct-parent instances are associated with ≈78% repair and ≈3.2% corruption, compared with ≈24% repair and ≈38% corruption for incorrect-parent instances. Full details are provided in the corresponding reviewer response.

Key takeaway. Taken together, these results support the main claim of the paper: Nexa learns a reusable, response-conditioned policy for selective refinement that transfers across tasks, model scales, and model families, while preserving strong accuracy and reducing communication cost relative to full one-round SelfOrg.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.