acceptodds
Under review as a conference paper at ICLR 2027

MedDAC: Learning Directional Communication via Correction and Harm Modeling for Medical Multi-Agent Reasoning

Abstract

Multi-agent reasoning relies on communication among agents, yet peer messages can either correct an erroneous receiver or mislead an initially correct one. Moreover, agents may revise their answers through self-reflection alone, making it difficult to distinguish peer influence from ordinary second-pass reasoning. We present MedDAC, a framework for learning directional agent communication policies by modeling both corrective and harmful peer effects. MedDAC evaluates receiver-specific sender-subset interventions against matched self-review to estimate the incremental local and system-level effects of communication actions. MedDAC-Static aggregates these effects to learn a fixed sparse communication graph under a message budget, while MedDAC-Dynamic predicts question-conditioned communication actions from the current multi-agent state. Across five medical QA benchmarks, the complete MedDAC-Static pipeline achieves 74.57% macro-average accuracy, compared with 69.77% for single-agent inference. Under the same Majority Vote aggregation, its six-message policy achieves 73.13%, comparable to the 72.99% obtained with 12-message Full Communication. These results show that modeling both corrective and harmful peer effects enables selective communication while maintaining competitive reasoning accuracy with substantially fewer directed messages.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Language Models, Medical Multi-Agent Reasoning, Directional Communication, Correction–Harm Modeling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Q6Z7OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies an information exchange method among multiple agents for medical datasets, which can slightly improve the system's performance under a limited budget.

Critical strengths

The paper's structure is complete, multiple medical datasets are used, and the proposed method can also improve the baselines to some extent.

Critical weaknesses

Compared with majority voting, the improvement of the proposed method is too small, even less than 1%. The paper lacks confidence intervals for the different methods. Adding confidence intervals is necessary to determine whether the proposed method is statistically significantly better.

Introducing different personas for different agents may lead to better performance, and additional experiments are needed to investigate this possibility.

Furthermore, the paper excludes all samples for which every agent initially provides an incorrect answer. However, these samples may still yield correct answers after interaction. The reviewer would like to see the results on this subset.

Questions for authors

The paper focuses on medical datasets, but the proposed pipeline seems applicable to all types of datasets. Has the method been specifically adapted for medical datasets?

On PubMedQA, the proposed method seems to perform worse than the baseline. What are the possible explanations for this?

Recently, several second-order information aggregation methods have been proposed. Would these methods outperform majority voting in your setting?

3 comments
AuthorsOpenReview ↗

Reply to Reviewer Q6Z7 (1/2)

We thank the reviewer for the careful assessment and valuable questions concerning modest gains, agent diversity, and calibration coverage.

W1. Performance improvements and statistical reliability

Table 1 evaluates the complete MedDAC pipeline with Consensus Refinement (Static: 74.57%; Vanilla: 69.77% across five benchmarks). Table 2 instead fixes Majority Vote to focus on communication selection: Static reaches 73.13%, Random-K6 72.52%, and Full Communication 72.99%. These comparisons answer different questions; the complete-pipeline gain should not be attributed solely to communication selection.

We additionally conducted paired bootstrap analyses (10,000 resamples) and stratified paired permutation tests.

Table R1. Paired Statistical Analysis of Communication Policies

ComparisonΔ Accuracy (pp)95% CI (pp)p-value
Static − Random-K6+0.75[+0.13, +1.36]0.0165
Static − Full-12+0.22[−0.37, +0.81]0.4740
Dynamic − Random-K6+0.60[+0.01, +1.21]0.0561
Dynamic − Full-12+0.07[−0.53, +0.66]0.8258

Static shows a positive paired gain over Random-K6 (unadjusted p=0.0165). Dynamic's improvement is positive but not statistically significant under the permutation test. Both MedDAC policies are constrained to six directed messages, compared with twelve for Full Communication. The contribution is learned selective communication under a limited budget, rather than maximizing communication. Statistical results will be included in Appendix E.1 (Table E1).

W2. Sampling diversity versus specialist agents

We deliberately keep the same backbone and task instructions across four agents, varying sampling temperatures (0.1, 0.4, 0.7, 1.0). This isolates directional communication effects from heterogeneous backbones and specialist prompting; assigning a specialist persona does not itself establish verified clinical expertise. Each receiver has seven nonempty peer subsets, giving 28 candidate receiver-specific actions whose joint effects are estimated directly. The formulation can accommodate heterogeneous agents after appropriate recalibration, but we do not claim to have validated that extension.

W3. Initially all-wrong cases

Mixed-correctness selection is restricted to labeled calibration, where having both correct and incorrect initial answers helps identify helpful corrections and harmful revisions relative to matched self-review. No ground-truth correctness labels or all-correct/mixed/all-wrong classification are used at test time; all test questions remain in evaluation. An all-wrong group might recover through new reasoning induced by communication, but such recovery is not guaranteed, and the current calibration subset does not directly validate utilities in that regime. We will clarify this scope.

AuthorsOpenReview ↗

Reply to Reviewer Q6Z7 (2/2)

W4. PubMedQA performance decrease

We examined group-level correctness changes during communication on 1,000 PubMedQA questions with Qwen3.8-27B. There were 10 wrong-to-correct (W→C) transitions but 15 correct-to-wrong (C→W) transitions, a net loss of five correct answers (0.5 percentage points). This directly motivates modeling both correction and harm: peer messages can repair mistakes but can also overturn correct group decisions.

Downstream aggregation also shows task dependence. In Table 4, Consensus Refinement changes PubMedQA accuracy from 79.00% to 77.90% for Dynamic and 78.80% to 77.60% for Static, relative to Majority Vote. Thus, additional communication or answer revision is not uniformly beneficial. We will clarify these observations alongside the component discussion in Appendix F.1 (Table F1).

W5. Alternative aggregation and medical applicability

Table 4 compares Majority Vote, Weighted Vote, Final Judge, and Consensus Refinement while holding MedDAC communication policies fixed. Consensus Refinement raises overall macro-average accuracy from 73.13% to 74.57% for Static and 72.90% to 74.56% for Dynamic. These are aggregation benefits, not effects attributable solely to graph selection.

To address the suggestion about additional aggregation, we evaluated ISP and OW-I alongside Majority Vote and Weighted Vote:

Table R2. Additional Aggregation Strategy Evaluation

AggregationDynamic Acc. (%)Static Acc. (%)
Majority Vote73.0273.16
Weighted Vote73.2373.36
ISP73.0973.33
OW-I73.1073.32

Weighted Vote performs best in this supplementary evaluation, improving upon Majority Vote by 0.21 percentage points for Dynamic and 0.20 for Static. ISP and OW-I also exceed Majority Vote but do not outperform Weighted Vote. This indicates that added aggregation complexity need not improve accuracy. We will include the new analysis in Appendix G.2 (Table G2).

Finally, MedDAC uses no handcrafted medical ontology or diagnosis-specific rules: it learns the corrective and harmful consequences of receiver-specific peer interventions. The general intervention and graph-selection framework could be adapted to other tasks with suitable utility calibration, but five medical QA benchmarks do not establish clinical safety or cross-domain transfer. We will clarify that distinction.

We thank the reviewer again for helping us more clearly separate communication selection, harmful peer influence, and downstream aggregation.

Dear Reviewer Q6Z7,

Thank you again for your thoughtful review. As the discussion deadline is approaching, could you please let us know whether our responses address your concerns? We would be happy to provide further clarification before the discussion closes if any issues remain.

Thank you for your time and consideration.

Warm regards, The Authors

Reviewer_F1ocOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MedDAC, a framework that learns selective communication strategies for medical multi-agent reasoning. The method estimates beneficial corrections and harmful revisions by comparing peer interventions with matched self-reviews. It introduces two variants: MedDAC-Static, which learns a fixed communication graph, and MedDAC-Dynamic, which selects communication actions adaptively for each question. Experiments on five medical QA benchmarks show improved reasoning accuracy while reducing the number of targeted messages.

Critical strengths
  • Clear motivation and an interesting question. The paper addresses an important issue in multi-agent reasoning: communication can both correct wrong answers and mislead agents that were already correct. Modeling both effects is valuable for improving multi-agent collaboration.

  • Carefully designed communication framework. The matched self-review control helps separate genuine peer influence from ordinary answer changes. The receiver-specific sender-subset formulation also offers a reasonable way to model directed communication, instead of treating all agent interactions as equal.

Critical weaknesses
  • Marginal improvements over strong baselines. Although MedDAC achieves the highest average accuracy, it improves on ReConcile by only 0.37 percentage points. Under the controlled communication setting, MedDAC-Static improves on Random-K6 by only 0.61 points. A statistical significance analysis would help establish how reliable these gains are.
Questions for authors
  • Limited evidence for the benefit of harm modeling. The paper combines correction and harm estimates in its communication utility, but does not provide enough ablations to isolate their individual contributions. Comparisons without harm modeling or without the matched self-review would better justify the main design choices.

  • Limited benefit of dynamic communication. MedDAC-Dynamic introduces question-conditioned communication prediction, but it does not always outperform the simpler static variant. The paper should further explain why adaptive communication does not yield stronger performance, and under what conditions it is beneficial.

  • Unclear scalability. The framework is evaluated only on four-agent systems, where exhaustive communication graph selection remains tractable. However, the number of possible sender subsets grows exponentially with the number of agents. Experiments with larger groups of agents would help demonstrate scalability.

Reviewer_BKv8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes MedDAC, a framework for improving communication among LLM agents in medical question answering. The main idea is that communication between agents is not always beneficial, as it can help correct wrong answers but may also cause agents to change initially correct ones. To address this, MedDAC evaluates the effects of peer communication against self-review and learns which agents should communicate. The authors introduce two variants: MedDAC-Static, which learns a fixed communication structure, and MedDAC-Dynamic, which adjusts communication based on the current question and agent responses. Experiments on five medical QA benchmarks show that MedDAC achieves better overall accuracy than single-agent inference and competitive performance against existing multi-agent methods. The results also show that MedDAC can maintain performance comparable to full communication while using fewer messages, although the improvement over random communication selection is relatively small.

Critical strengths
  1. Clear motivation: The paper addresses an important issue in multi-agent reasoning, where communication can both correct errors and mislead agents that initially have the correct answer.

  2. Communication efficiency: MedDAC achieves accuracy comparable to full communication while using only half the directed messages, suggesting that many interactions in multi-agent systems may be unnecessary.

  3. Evaluation across multiple benchmarks: The experiments cover five medical QA benchmarks with different characteristics, providing a reasonably broad evaluation of the method.

  4. Detailed analysis of communication behavior: The paper examines how communication utilities vary across agents and how the dynamic policy changes its communication patterns based on agent disagreement, offering some insight into how the learned policies behave.

Critical weaknesses
  1. Missing statistical significance analysis: The reported improvements over strong baselines are relatively small, but the paper does not provide confidence intervals or statistical significance tests. It is therefore unclear whether these improvements are reliable.

  2. Limited scalability evaluation: The experiments use only four agents, and the static policy relies on searching over possible communication configurations. It is unclear how well this approach would scale to larger multi-agent systems.

  3. Incomplete calibration coverage: The calibration process excludes questions where all agents initially agree, but the dynamic policy still makes communication decisions for these cases during inference. This raises concerns about the reliability of its predictions on unanimous cases.

  4. Unclear generalizability beyond medical multiple-choice QA: All five benchmarks focus on medical question answering in a multiple-choice format. It remains unclear whether the proposed communication strategy would work similarly for open-ended reasoning tasks or other domains.

Questions for authors

The paper mainly evaluates performance using accuracy on medical QA benchmarks, without assessing clinical reliability or patient safety. How do the authors ensure that self-review and peer communication do not introduce potentially harmful medical recommendations in real-world clinical settings?

3 comments
AuthorsOpenReview ↗

Reply to Reviewer BKv8

We thank the reviewer for the careful evaluation and constructive suggestions. We address the concerns regarding statistical reliability, agent design, and scalability below.

W1. Statistical reliability and communication efficiency

We conducted question-level paired bootstrap analyses (10,000 resamples) and stratified paired permutation tests to examine the observed improvements.

Table R1. Paired Statistical Analysis of Communication Policies

ComparisonΔ Accuracy (pp)95% CI (pp)p-value
Static − Random-K6+0.75[+0.13, +1.36]0.0165
Static − Full-12+0.22[−0.37, +0.81]0.4740
Dynamic − Random-K6+0.60[+0.01, +1.21]0.0561
Dynamic − Full-12+0.07[−0.53, +0.66]0.8258

Static improves over Random-K6 by 0.75 percentage points (unadjusted p=0.0165) under the same six-message budget. It also achieves a small positive observed difference relative to Full Communication while using half as many directed messages. Dynamic shows a positive but statistically inconclusive improvement over Random-K6. We do not interpret these results as evidence that every comparison is significant; rather, they support the accuracy–communication trade-off motivating selective directional interactions. Details will be included in Appendix E.1 (Table E1).

W2. Four-agent design and scalability

We intentionally use four agents sharing the same backbone and instructions, with sampling temperatures 0.1, 0.4, 0.7, and 1.0 to induce controlled response diversity. This helps isolate communication effects without confounding from heterogeneous models or unverified specialist personas. Four agents already yield 28 receiver-specific sender-subset actions (four receivers × seven nonempty peer subsets), each scored as a joint intervention rather than a sum of independent edges.

We also tested the graph-selection solver with precomputed synthetic utilities. Exact dynamic programming matched exhaustive search for four and five agents and solved an eight-agent instance with 1,016 candidate actions in 8.687 ms. This establishes the computational feasibility of graph selection given action utilities, not the scalability of collecting exhaustive intervention data or end-to-end clinical performance. We will make this distinction clearer.

W3. Mixed-correctness calibration versus test-time inference

MedDAC uses labeled mixed-correctness calibration cases—at least one initially correct and one initially incorrect agent—to observe both useful corrections and harmful revisions. The goal is to estimate communication effects relative to matched self-review, not to classify the correctness of agents during inference. Static aggregates these effects into a frozen graph; Dynamic predicts receiver- and group-level action utilities using observable confidence, agreement, voting, and action-history features.

Crucially, mixed-correctness selection applies only to calibration. Both policies evaluate all test questions without gold labels, including unanimous and initially all-wrong cases. Figure 4(b) shows that Dynamic selects 2.71 directed messages on average for 4:0 voting patterns, versus roughly 5.6–5.8 for disagreement patterns. This demonstrates behavioral adaptation, although unanimously correct or incorrect cases lie outside the labeled mixed-correctness calibration support. We will clarify this distinction in Appendix G.1 and the method description.

W4. Open-ended reasoning and cross-domain generalization

Structured medical QA provides the reproducible correctness signal needed to quantify correction and harm under matched intervention conditions. Open-ended tasks may admit multiple valid answers and therefore require a validated, more flexible outcome or utility metric. The intervention formulation and budgeted graph selection do not require a medical ontology, but estimated utilities depend on the task distribution and agent error patterns. Our five medical QA benchmarks support cross-benchmark evaluation within medicine, not zero-shot transfer to unrelated domains. Target-domain calibration and evaluation would be needed for such transfer.

W5. Clinical reliability

MedDAC addresses selective communication within a multi-agent reasoning pipeline; it is not a clinically validated diagnostic system. Real-world workflows additionally require triage, evidence acquisition, specialist coordination, longitudinal records, imaging interpretation, and safety verification. Benchmark accuracy cannot establish patient safety. We will clarify this scope and investigate integration into broader clinical workflows in future work.

We thank the reviewer again for helping us sharpen both the empirical claims and the applicability of MedDAC.

Dear Reviewer BKv8,

Thank you again for your thoughtful review. As the discussion deadline is approaching, could you please let us know whether our responses address your concerns? We would be happy to provide further clarification before the discussion closes if any issues remain.

Thank you for your time and consideration.

Warm regards, The Authors

Thanks for your reply!

But I am still confused about W3, which is about calibration coverage. The calibration procedure excludes both all-correct and all-wrong cases. But I am wondering the motivation to excludes all-correct and all-wrong cases.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Reply & Rebuttal Period Paper Revision Summary

Dear Area Chairs and Reviewers,

We sincerely thank all three reviewers for their careful evaluations and constructive suggestions. Their feedback has helped us strengthen the empirical analysis and clarify the methodological contributions and scope of MedDAC.

Our central contribution is a correction- and harm-aware framework for selective directional communication in medical multi-agent reasoning. MedDAC evaluates receiver-specific sender-subset interventions against matched self-review, estimates their corrective and harmful effects, and selects valuable interactions under a constrained message budget.

In our original experiments, MedDAC-Static achieves 74.57% macro-average accuracy across five medical QA benchmarks, compared with 69.77% for Vanilla inference. Under controlled Majority Vote aggregation, Static reaches 73.13%, compared with 72.52% for Random-K6 and 72.99% for Full Communication, while using six rather than twelve directed messages.

During the rebuttal period, we conducted additional experiments and analyses to address the reviewers' concerns:

  1. Statistical reliability: In response to Reviewers BKv8, F1oc, and Q6Z7, we conducted paired bootstrap analyses with 10,000 resamples and stratified paired permutation tests to assess the reliability of communication-policy comparisons. We will include the results in Appendix E.1 (Table E1).

  2. Component ablations: We conducted a 2×2 ablation of harm modeling and matched self-review, examining their effects on communication-policy construction and downstream accuracy across five medical QA benchmarks. The results will be presented in Appendix F.1 (Table F1).

  3. Adaptive communication: We clarified the complementary objectives of Static and Dynamic. Static learns a fixed high-utility communication graph, whereas Dynamic adapts communication allocation to observable agent states. Additional discussion will appear in Appendix G.1, complementing Figure 4(b).

  4. Aggregation extensions: In response to Reviewer Q6Z7, we evaluated ISP and OW-I alongside Majority Vote and Weighted Vote. The results show that more elaborate aggregation does not necessarily outperform weighted voting. We will include these experiments in Appendix G.2 (Table G2).

We also clarified the motivation for the controlled four-agent design, the computational scalability of graph selection, and the limitations concerning calibration coverage, domain generalization, and clinical deployment.

Together, these results reinforce the central objective of MedDAC: learning which directional interactions are useful or harmful, rather than assuming that more communication necessarily leads to better reasoning.

We sincerely thank the reviewers and Area Chairs again for their thoughtful consideration.

Authors of MedDAC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.