acceptodds
Under review as a conference paper at ICLR 2027

Who Else Should See It? Group-Aware Evidence Routing for Multi-Agent LLMs

Abstract

In multi-agent search systems, deciding whether evidence is useful to one agent does not determine how widely it should be shared. We present MERGE (Multi-Agent Evidence Routing with Group Effects), a router that learns how the benefit of sharing evidence with an additional agent depends on how many agents already hold it. Using paired interventions that differ by one additional agent, MERGE combines this learned signal with agent-specific usefulness and communication cost, then selects routing decisions jointly. Across nine QA dataset–backbone settings, MERGE achieves the highest mean F1 and EM scores. On the primary HotpotQA/Qwen2.5-14B setting, it reaches 0.716 F1, improving by 2.4 points over the strongest controlled baseline while using 11.8% fewer tokens. Analysis reveals early diminishing returns from redundant sharing and a late-stage effect tied to the fixed voting rule. Together, these results show that routing quality depends on both how information is distributed across the team and how the team combines its final decisions. Code is available at https://anonymous.4open.science/r/merge-routing-053E/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
multi-agent LLM, Evidence Routing

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_nWvLOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes MERGE, a group-aware evidence routing framework for multi-agent LLM systems. MERGE learns how the marginal benefit of sharing evidence changes with the number of agents already holding it, using paired interventions to estimate its effect on final team performance. The learned group effect is combined with agent-specific evidence utility and communication cost through a joint routing procedure. Experiments across multi-hop QA benchmarks, multiple LLM backbones, larger teams, and fact verification demonstrate consistent improvements in answer quality and communication efficiency over existing and controlled baselines.

Critical strengths

[#1] Well-motivated formulation of group-aware evidence routing.

The paper identifies an important limitation of agent-specific evidence routing: the value of sharing information with an additional agent depends not only on its individual usefulness but also on how widely that information has already been distributed. MERGE provides a principled framework for incorporating holder-dependent sharing effects into routing decisions. The paired-intervention approach is particularly interesting because it derives supervision from downstream team outcomes rather than relying solely on heuristic relevance scores.

[#2] Comprehensive empirical evaluation with meaningful controlled comparisons.

The paper provides extensive experiments across multiple benchmarks, LLM backbones, and team sizes, including transfer to fact verification. Importantly, the authors conduct controlled comparisons against Separable and Fixed-Shape variants, investigate the contributions of group-effect modeling and joint routing, and provide predictor diagnostics and sensitivity analyses. These experiments offer substantial evidence that the proposed components improve routing performance within the evaluated setting.

Critical weaknesses

[#1] Limited evidence that the learned group effect generalizes beyond the fixed aggregation mechanism.

The paper's central contribution is to model how the marginal value of sharing evidence changes with the number of existing holders. However, the supervision signal is derived from final team-level F1 under a fixed majority-based aggregation rule. Consequently, the learned group effect may reflect not only the informational value of evidence dissemination but also whether additional recipients can change the final voting outcome.

The authors partially investigate this distinction in Section 6.4 and Appendix B.12. Notably, the early-stage diminishing-return pattern persists under individual-agent F1, whereas the sharp late-stage increase largely disappears. This suggests that an important component of the learned sharing dynamics is specific to the aggregation mechanism.

Although optimizing majority-vote outcomes is legitimate, all main transfer experiments retain the same aggregation rule. Therefore, the results do not establish whether MERGE learns generalizable information-sharing dynamics or primarily exploits the structure of the chosen collective decision procedure. Evaluating routing performance under alternative aggregation mechanisms would help clarify the scope of the proposed contribution.

[#2] Insufficient validation of single-recipient marginal estimates for joint routing decisions.

MERGE learns sharing benefits from paired interventions that differ in whether one additional agent receives a selected piece of evidence. However, its inference-time procedure jointly optimizes multiple routing decisions, potentially involving different evidence items and recipients. The effects of these decisions may not be additive, as sharing one piece of evidence can change the usefulness of other evidence through complementary reasoning, redundancy, or subsequent search behavior.

The authors provide useful ablations demonstrating that joint routing and group-effect modeling are complementary, including a factorial analysis in Appendix B.9. Nevertheless, these results establish the effectiveness of the combined procedure rather than directly validating whether the estimated routing objective accurately represents the downstream effects of jointly selected actions.

This distinction is important because the proposed predictor omits cross-cluster interactions, and its supervision is based on single-recipient interventions under a fixed continuation policy. Consequently, the extent to which these estimates remain reliable for compound routing actions and states induced by the learned policy remains unclear. A direct comparison between predicted joint-routing benefits and observed outcomes under matched multi-decision interventions would strengthen the methodological justification.

Questions for authors

Q1. Distinguishing information-sharing value from aggregation-specific effects

The paper's central motivation is that the marginal value of sharing evidence depends on how many agents already possess that information. However, the intervention-based supervision measures changes in final team F1 under a fixed majority-voting mechanism. As the authors acknowledge in Section 6.4 and Appendix B.12, the sharp late-stage increase in marginal sharing value becomes substantially weaker when evaluated using individual-agent F1 instead of the final aggregated outcome.

This raises an important question about what the group-effect predictor is actually learning. Is it primarily estimating how additional evidence improves the team's collective information state, or is it learning when additional evidence holders are likely to change the majority-vote outcome?

Could the authors further disentangle these two effects? In particular:

  • Would the learned group-effect predictor retain its advantage under alternative aggregation rules, such as confidence-weighted voting or evidence-based answer selection?
  • If the aggregation rule changes, does the existing predictor remain effective without retraining, or must the group-effect model be refitted?
  • Can the authors quantify how much of MERGE's improvement comes from better individual-agent reasoning versus changes in the final collective decision?

Addressing these questions would clarify whether MERGE captures a broadly applicable principle of group-aware evidence sharing or a more specialized, albeit useful, strategy for optimizing majority-based multi-agent systems.

Q2. Validity of marginal estimates for compound routing actions

MERGE learns the marginal benefit of sharing evidence through paired interventions that differ in whether one additional agent receives a selected piece of evidence. However, the proposed joint routing procedure simultaneously considers multiple sharing decisions, whose effects may interact.

For example, two evidence items may be individually insufficient but jointly necessary to complete a multi-hop reasoning chain. Conversely, two individually useful items may become redundant when delivered together. In such cases, the actual benefit of jointly sharing both items may differ substantially from the benefit implied by combining their independently estimated marginal effects.

The factorial analysis in Appendix B.9 demonstrates that group-effect modeling and joint routing are complementary, but it does not directly establish whether the estimated routing objective accurately predicts the outcomes of compound actions.

Could the authors provide a more direct validation of this approximation? Specifically:

  • For a representative set of intermediate team states, how closely does the predicted utility of a jointly selected routing action correspond to its observed downstream improvement?
  • How frequently do complementary or redundant interactions cause the combined outcome to differ substantially from the prediction based on individual marginal effects?
  • Would explicitly modeling pairwise interactions between evidence-sharing decisions improve routing quality, or does the current approximation already capture most of the relevant dependencies?

These analyses would help establish whether the proposed objective is a reliable approximation to the joint routing problem, rather than merely an effective heuristic within the evaluated environment.

Q3. Reliability of intervention-based estimates under the learned routing policy

The group-effect predictor is trained using paired continuations under a fixed continuation policy, while MERGE subsequently makes adaptive routing decisions that can alter agents' future queries, retrieved evidence, and information-sharing opportunities. The paper acknowledges this discrepancy in Section 7 and provides a continuation-policy sensitivity analysis in Appendix B.8.

Nevertheless, the existing analysis does not fully establish whether the learned predictor remains calibrated on intermediate states generated by MERGE itself. This matters because an evidence-sharing decision may have a different downstream effect when subsequent communication is determined adaptively rather than replayed according to a fixed schedule.

Could the authors investigate this distributional discrepancy more directly? For example:

  • How do predicted marginal sharing benefits compare with observed intervention outcomes when the intermediate states are sampled from actual MERGE trajectories?
  • Does predictor calibration deteriorate as the learned router progresses through multiple search rounds?
  • How sensitive is routing performance to the difference between the fixed continuation policy used for training and the adaptive continuation induced by MERGE?
  • Would collecting additional intervention data from on-policy trajectories or iteratively updating the predictor improve performance?

Such experiments would strengthen the connection between the controlled intervention-based training objective and the actual sequential decision-making process at inference time.

Reviewer_LZzvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MERGE, a framework for evidence routing in multi-agent systems. It adds group-effect modeling and joint routing selection to collaborative search and evidence exchange, aiming to estimate the benefit of sharing evidence with an additional agent and decide how widely and to whom evidence should be shared. The method is evaluated across multiple QA datasets and model backbones.

Critical strengths

The evaluation is broad relative to the narrowly defined hypothesis and reports useful limits. MERGE has the highest mean F1 in all nine main QA settings, transfers without refitting to larger teams and HoVer, and improves the primary online quality-cost trade-off.

Critical weaknesses
  1. The practical motivation for the fixed-team routing problem remains insufficiently connected to the broader evidence-coordination problem. The controlled study demonstrates routing effects under homogeneous search and fixed final voting, but does not establish when decentralized evidence access is necessary or what concrete constraints prevent a lead agent from consolidating evidence. Central aggregation is not assumed to solve the problem automatically. Rather, without a clearer account of the relevant bottleneck and deployment conditions, the useful within-protocol findings offer a narrower practical contribution than I find sufficient for acceptance.
  2. The evidence on complex tasks and the presentation of gains remain insufficient. The QA benchmarks are relatively established; although they include multi-hop tasks, I would like to see more complex, open-ended tasks requiring multiple rounds of tool interaction to determine whether the framework remains effective. The primary training setting uses Qwen2.5-14B. Although the main table and appendix include other and newer models, I remain concerned about generality to more modern models and tasks. The 15.1% in Table 1 is a relative improvement over NO-COMM. In the primary setting, MERGE scores 0.716 versus 0.692 for FIXED-SHAPE and 0.680 for SEPARABLE, corresponding to absolute gains of 2.4 and 3.6 F1 points. Appendix B.1 measures gains against the strongest displayed comparator. The different reference baselines and units should be made more directly comparable. In my assessment, the practical gains over stronger controlled baselines still do not sufficiently support the overall contribution.
Questions for authors
  1. How does the proposed problem arise in an architecture where a lead agent plans, selects the number of subagents, and aggregates their findings? What benefit does MERGE provide beyond direct evidence aggregation by the lead agent?
  2. Request for evidence: Can the framework be evaluated on more complex, open-ended tasks requiring multiple rounds of tool interaction, and do the conclusions hold with newer models?
5 comments

Dear Reviewer LZzv,

Thank you again for your review. As the discussion period ends on Nov 18 (AoE), we wanted to check whether our response addressed your concerns. In brief:

  1. Lead-agent setting: in a manager pilot (150 held-out questions), Manager + MERGE beats a local-utility relay by +1.2 F1 [+0.3, +2.1] at the same budget, and matches broadcast quality with 35% fewer tokens.

  2. Complex tool tasks: on all 165 GAIA validation tasks with the frozen router, MERGE solves 64 vs. 59 for FIXED-SHAPE with 8.9% fewer tokens. The CI includes zero, so we report this as preliminary.

  3. Newer models: the frozen router gives positive gains on Qwen3.6-27B, GPT-5.6 Sol and Claude Opus 5.

  4. Gain reporting: we will replace relative numbers in Table 1 with absolute differences.

If these address your concerns, we would be grateful if you would consider updating your assessment. We are happy to clarify anything before the deadline.

Reviewer_LZzvOpenReview ↗

Updated assessment after author response

Thank you for the response and follow-up. The manager pilot substantially addresses my concern about the value of routing in a lead-agent setting, particularly the comparison against local-utility relay at a comparable token budget. I also appreciate the clarification of gain reporting and the appropriately qualified presentation of the preliminary GAIA results.

Two clarifications would help finalize my assessment. Does the manager-pilot token accounting include planner/manager and online routing overhead under matched search and context budgets? Also, how are final answers aggregated in this pilot, and what dependence on majority voting remains?

The response has substantially alleviated my main concerns, and I am inclined to revise my recommendation from weak rejection to weak acceptance. Please incorporate the experimental protocols, results, and limitations into the revised manuscript.

Thank you for the follow-up and for reconsidering your assessment.

Token accounting. The reported tokens include all LLM calls in the pilot: the lead planner, all worker calls, and the relayed evidence placed in worker contexts. The planner, search budget (3 rounds, 5 documents per query), and per-worker context cap are identical across all four conditions, so planner tokens enter every row equally. MERGE’s routing itself makes no LLM calls; the local scorer and group predictor are a frozen text encoder and small MLPs, so routing adds no tokens (and negligible CPU/GPU latency of a few milliseconds per step). The token differences (+78 vs. terminal consolidation, −12 vs. local-utility relay) therefore reflect only which evidence is relayed.

Aggregation. Final answers use the same strict-majority vote among workers as in the main experiments, so the pilot retains that dependence. Our evidence on alternative aggregators is in Appendix B.15, where the frozen router maintains advantages of +1.5 and +1.4 F1 points under confidence-weighted voting and an evidence-based LLM judge, respectively.

We will add these protocol details and this limitation to Appendix B.17.

Reviewer_fwn5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MERGE learns how the benefit of routing evidence changes with the number of agents already holding it. It combines this signal with local evidence utility and communication costs, then selects routing actions jointly. Paired interventions provide the training targets, with the agent team and final voting rule otherwise fixed.

Critical strengths
  • The central distinction is useful: evidence can help an individual agent without being equally valuable to every additional recipient. The holder-dependent adjustment turns this into an explicit routing objective.
  • The controlled comparisons are convincing. Removing the group term loses 3.6 F1 points, and the group-effect × joint-routing ablation reports a positive 2.5-point interaction with a paired confidence interval. Constant-effect, lookup-table, and state-shuffle controls further help isolate the contribution.
  • MERGE has the highest mean F1 and EM across all nine QA settings. On the primary setting, it improves over FIXED-SHAPE by 2.4 F1 points, with a paired 95% CI of [1.2, 3.3], while using 11.8% fewer online tokens. The paper separates decoding repetitions from training-seed variation and reports both.
Critical weaknesses
  1. The demonstrated transfer retains a fairly specific decision rule. The predictor explicitly encodes distance to a majority threshold, and the late-stage peak largely disappears under mean individual-agent F1. That diagnostic is informative, but it does not test the deployed router under a different aggregator. Backbone, dataset, and HoVer transfer all retain voting. The role-specialized pilot helps, but remains a small fixed-composition experiment.
  2. The intervention targets differ from the deployed routing dynamics. Labels are collected under a frozen replay schedule, with later delivery of the selected cluster to the selected agent blocked. These are controlled effects under that continuation protocol. MERGE subsequently makes new routing decisions as states change. The 50-question continuation-policy check addresses two frozen schedules, but does not establish within-state action ranking under MERGE’s own trajectories. This matters for interpreting the predictor as a deployed marginal-value estimate.
  3. The practical total-cost advantage remains unquantified. Appendix D.3 counts 67,800 fitting/selection continuations and gives an amortization formula, but does not measure the offline token cost or GPU time. The online savings are clear; the deployment volume needed to recover collection and fitting costs is still unknown.
Questions for authors
  1. How much of the routing advantage remains with an alternative final aggregation rule, and would the group predictor need refitting?
  2. Can you report the measured offline collection/fitting cost and the resulting break-even deployment volume?
  3. How accurately does the surrogate rank routing actions within states encountered by MERGE, and how sensitive is this to the semantic-clustering threshold?
1 comment

Dear Reviewer fwn5,

Thank you for your positive review. As the discussion period ends on Nov 18 (AoE), we wanted to check whether our response addressed your three points:

  1. Alternative aggregators: +1.5 / +1.4 F1 frozen, +1.9 / +1.8 refitted.

  2. Offline cost: 388.5M tokens and 440.4 GPU-hours in total; MERGE needs no extra offline collection relative to FIXED-SHAPE.

  3. On-policy ranking: Spearman 0.48 on 360 MERGE states, stable across clustering thresholds (0.47–0.51).

If these resolve your concerns, we would be grateful if you would consider reflecting this in your score. Thank you for your time.

Reviewer_GAbUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MERGE, a group-aware evidence routing method for multi-agent LLM systems. The main idea is that the benefit of sharing evidence with an additional agent depends on how many agents already have access to the same information. The authors use paired counterfactual interventions to estimate this benefit and incorporate the learned signal into a joint routing objective, together with agent-specific utility and communication costs.

Critical strengths

The paper highlights a limitation of independent, agent-specific evidence routing: the usefulness of information to one agent does not necessarily determine how broadly it should be shared. This is a meaningful distinction in multi-agent communication.

The evaluation covers multiple datasets and LLM backbones, with controlled baselines that isolate the contributions of group-aware scoring and joint routing.

Critical weaknesses

1.The central observation that the marginal value of sharing information depends on how many agents already possess it is fairly intuitive. Information redundancy and diminishing returns are already well-established considerations in communication and information selection. The paper does not convincingly establish why this requires a fundamentally different routing formulation, rather than extending existing utility-based routing methods with a redundancy-aware term. 2. The group predictor primarily operates on a small set of handcrafted features, including the holder count, search stage, evidence relevance, and distance to the majority threshold. It does not explicitly model interactions between different agents, their beliefs, or dependencies between evidence clusters.

Questions for authors
  1. How sensitive are the improvements to the final answer aggregation rule? Would the same group-aware routing strategy remain effective under a judge-based or confidence-weighted aggregation mechanism?
  2. Have the authors evaluated whether the predictor correctly ranks competing routing actions within the same intermediate team state, rather than only measuring prediction error across interventions?
1 comment

Dear Reviewer GAbU,

Thank you again for your review. As the discussion period ends on Nov 18 (AoE), we wanted to check whether our response addressed your concerns. In brief:

  1. On redundancy-aware baselines: FIXED-SHAPE is such a baseline, and MERGE leads it by +2.4 F1 [+1.2, +3.3]. Shuffling the learned values across states loses 2.4 points, so the gain comes from state-dependent shape, not from redundancy alone.

  2. On richer features: adding agent identity/disagreement or cross-cluster interaction features changes F1 by only +0.2 / +0.3.

  3. On aggregation rules: the frozen router keeps +1.5 / +1.4 F1 under confidence-weighted voting and an LLM judge.

  4. On within-state ranking: on 360 on-policy states, Spearman is 0.48 (vs. 0.31 for local utility), and mean decision regret is 1.56 F1 (vs. 2.34 for local utility).

If these address your concerns, we would be grateful if you would consider updating your assessment. We are happy to clarify anything before the deadline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.