acceptodds
Under review as a conference paper at ICLR 2027

MedECHO: Privacy-Aware Multi-Agent Collaboration for Cross-Hospital Diagnosis

Abstract

Medical multi-agent systems combine diagnostic opinions, but their shared-case workflows do not directly fit cross-hospital diagnosis: raw cases remain at the originating hospital, while remote hospitals hold heterogeneous private knowledge. We identify two sequential failures in this setting: restricting disclosure can lose diagnostic signal, and simple voting can miss a correct candidate already returned by collaborators. We propose MEDECHO to address these stages. Controlled Disclosure (CD) builds, reviews, and audits a closed-schema collaboration packet, with local fallback when review fails. Evidence Adoption (EA) gathers source-tagged remote opinions and lets the originating hospital compare candidates against its complete case before issuing the final diagnosis. Extensive experiments on six datasets, three base LLM backbones, and three seeds show that MEDECHO improves dataset-equal accuracy over the strongest evaluated baseline by 15.94, 15.88, and 9.23 percentage points on DeepSeek V4 Flash, Qwen 3.7 Flash, and GPT-5.6-Luna, respectively. These consistent backbone-level gains support the robustness and cross-backbone generalizability of MEDECHO within the evaluated diagnosis setting.

Primary area:
foundation or frontier models, including LLMs
Keywords:
medical multi-agent system, cross-hospital diagnosis, privacy-preserving collaboration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 5 reviews, above 27% of the others.
Reviewer_1z6LOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper investigates privacy-constrained multi-agent collaboration for cross-hospital diagnosis. The authors construct a simulated cross-hospital setting in which each hospital agent has access to its own private knowledge resources, while patient information shared across hospitals is restricted. They identify two key challenges: the disclosure gap, where restricted information sharing may reduce diagnostic accuracy, and the adoption gap, where correct diagnoses proposed by collaborating agents may not be selected in the final decision. To address these challenges, the authors propose MedECHO, which combines Controlled Disclosure (CD) and Evidence Adoption (EA). CD generates and validates structured collaboration packets to limit information exposure while preserving clinical signals. EA integrates remote diagnostic opinions and multiple evidence sources through a Candidate Ledger, Retrieval-aware Selection, a centroid-based ranker, and a Promotion Guard to improve final diagnostic decisions. Experiments on six medical diagnosis datasets using three LLM backbones and three random seeds show that MedECHO improves average diagnostic accuracy over the strongest evaluated baselines by 15.94, 15.88, and 9.23 percentage points, respectively.

Critical strengths
  1. The paper identifies two meaningful challenges in privacy-constrained multi-agent diagnosis: the disclosure gap and the adoption gap. The authors provide empirical observations to illustrate these problems, including the performance difference between raw-context and packet-based voting, and the gap between candidate discovery and final diagnostic accuracy. I think these observations provide a reasonable motivation for the proposed framework.

  2. The paper reports substantial improvements in diagnostic accuracy across six medical datasets, three LLM backbones, and multiple random seeds. The authors also provide several ablation studies to examine the contributions of different components. In particular, the results help illustrate the importance of candidate expansion and auxiliary retrieval in improving diagnostic performance.

Critical weaknesses
  1. From my understanding, the practical motivation for the proposed cross-hospital multi-agent setting could be further clarified. Although privacy constraints are important in cross-hospital collaboration, it is not entirely clear to me whether direct collaboration between independently deployed hospital agents represents a realistic clinical need, or why this problem specifically requires a multi-agent framework rather than conventional cross-institutional information exchange. In addition, the experimental environment is constructed by partitioning public datasets into simulated hospital nodes, rather than using real institutional data and communication constraints. While this is reasonable for controlled experiments, I am not sure how well the resulting setting reflects the challenges of real-world cross-hospital collaboration.

  2. Although the paper includes several ablation studies, I feel that the experiments do not fully separate the contributions of different mechanisms. For example, the authors compare Raw-Context Voting (RV), Packet Voting (PV), and MedECHO, but I could not find a comparison where MedECHO uses the full patient case while keeping the same Evidence Adoption (EA) pipeline. Such a comparison would help clarify how much diagnostic performance is lost due to restricted information sharing under the proposed method. Furthermore, the ablation results suggest that Candidate Ledger and Support RAG contribute substantially to the overall improvement. It would be helpful to include a local-only baseline with comparable retrieval resources and the same EA mechanism to better understand how much of the improvement actually comes from cross-hospital multi-agent collaboration.

Questions for authors
  1. Could the authors provide a comparison where MedECHO uses the full patient case instead of the restricted packet, while keeping the same EA pipeline and other experimental settings? The current RV and PV comparison evaluates the effect of restricted information sharing under majority voting. I think this additional experiment would help clarify how much diagnostic performance is lost due to information restrictions under the proposed method, and whether EA can actually reduce this loss.

  2. Could the authors include a local-only baseline that uses the same auxiliary retrieval resources, Candidate Ledger, and EA decision mechanism, but without remote agent collaboration? From the ablation results, Candidate Ledger and Support RAG seem to contribute substantially to the overall improvement. Such a comparison would help clarify how much of the performance gain comes specifically from cross-hospital multi-agent collaboration, rather than from stronger retrieval and candidate selection.

  3. Could the authors further clarify the practical motivation for the proposed cross-hospital multi-agent setting? From my understanding, the experimental environment is constructed using simulated hospital nodes with partitioned retrieval resources. I wonder whether there are existing or potential clinical applications that specifically require independently deployed hospital agents to collaborate under such information-sharing constraints, and how this setting differs from conventional cross-institutional information exchange.

  4. Could the authors provide more details or additional experiments regarding the privacy guarantees of CD? The current privacy evaluation appears to rely on a relatively small set of synthetic canary cases. I wonder whether the proposed mechanism has been evaluated against more realistic privacy risks, such as indirect information disclosure or patient re-identification from combinations of clinical features.

  5. I noticed a possible inconsistency between Figure 7 and Table 1. Figure 7 reports an average accuracy of 51.98% for MedECHO in the same-family setting, whereas Table 1 reports 65.80% for what appears to be the corresponding configuration. Could the authors clarify whether these results use different experimental settings or whether there is an error in the reported values?

6 comments
AuthorsOpenReview ↗

Author Response(1/4)

Q1. Matched disclosure utility

We completed matched-disclosure comparisons with DeepSeek-V4.1-Flash-1017, Qwen 3.7 Flash and GPT-5.6-Luna. Within each comparison, input variants share the local opinion, retrieval, Support and frozen EA; the disclosure protocol is evaluated separately from the resource-factorial experiment.

The earlier DeepSeek-V4.1-Flash-1017 matched experiment:

Remote input + fixed EAS2D Acc (%)K-Sym Acc (%)MAC-302 Acc (%)Macro Acc (%)
Full raw case77.7853.9143.7558.48
Full CD packet79.3753.9143.7559.01
Minimal clinical packet77.7853.0443.7558.19

Full CD is +0.53 macro points above raw input and +0.82 above the minimal packet. The minimal packet retains complaints, clinical concepts and abnormal findings while removing contextual fields.

The new disclosure comparison adds identity masking and length-matched summaries. Values are equal-weight S2D/K-Sym/MAC-302 macro Acc (%):

Remote input + fixed EAQwen 3.7 FlashGPT-5.6-Luna
Full raw case60.2849.81
Full CD packet61.1050.63
Minimal clinical packet59.9949.52
Identity masking57.0946.62
Length-matched summary59.9949.52

Full CD gains 0.82 points over raw input and 1.11 points over the minimal packet on both Qwen and GPT.

The separate earlier DeepSeek V4 Flash factorial control, on the same dataset scope, retains the input-by-decision comparison:

Remote inputVoting macro Acc (%)EA macro Acc (%)
Raw58.2856.73
CD packet55.3855.91

The raw-minus-packet gap is 2.90 points with voting and 0.82 with EA; on MAC-302, raw accuracy changes from 50.00% to 43.75%, while packet accuracy stays 43.75%. The earlier V4 identity-masking and length-matched-summary EA controls reach 55.09% and 57.70%.

Conclusion. With retrieval and EA fixed, full CD preserves diagnostic utility and exceeds raw input by 0.53 macro points on V4.1-Flash-1017 and 0.82 points on Qwen/GPT, while retaining a useful contribution from contextual fields.

AuthorsOpenReview ↗

Author Response(2/4)

Q2. Resource-matched Local-EA

We extended the resource-factorial controls across backbones. Local-EA retains local processing and frozen EA while removing remote opinions and derived evidence. Values are equal-weight S2D/K-Sym/MAC-302 macro Acc (%); the historical V4 column is separate from the V4.1 disclosure experiment.

SystemDeepSeek V4 Flash (earlier)Qwen 3.7 FlashGPT-5.6-Luna
Local RAG54.0948.3945.56
Local-EA without Support55.4856.8046.33
Local-EA with Support54.8058.2047.73
MedECHO without Support56.8862.3651.90
MedECHO with Support55.9161.3950.92
Budget-matched local reasoning55.6261.1050.63

The remote-evidence gain over Local-EA with Support is 1.11 points on V4 and 3.19 points on Qwen/GPT. MedECHO exceeds the configured budget-matched local control by 0.29 points on each backbone. The new Qwen/GPT costs and times are estimates. The no-Support MedECHO variant reaches 62.36%/51.90% on Qwen/GPT, above the full configuration, so the factorial comparison separates Support effects from collaboration.

Conclusion. Remote evidence improves macro Acc over same-Support Local-EA across the tested backbones, with a 3.19-point gain on Qwen/GPT and a smaller 0.29-point margin over budget-matched local reasoning.

AuthorsOpenReview ↗

Author Response(3/4)

Q3. Hospital agents and privacy

The agent formulation makes the responsibilities of asynchronous consultation explicit. The originating institution retains the full patient context and the final diagnostic decision; a consulting institution supplies an interpretation grounded in evidence it controls. The reviewed packet governs information sharing, and the source-tagged candidate ledger lets the originator decide whether to adopt a remote proposal. This adds reasoning and adoption decisions to the record-transfer function of conventional information exchange.

The matched Non-IID comparison is complete. Under fixed private-knowledge capacities, we compared LR, RV, PV, MAC, MedChain, Local-EA and MedECHO across random and Dirichlet allocations. MedECHO has the highest S2D/K-Sym macro Acc in each backbone/allocation; its results are:

BackboneRandom Acc (%)alpha=1.0 Acc (%)alpha=0.1 Acc (%)Local-EA at alpha=0.1 (%)
DeepSeek V4.1 Flash61.9960.3258.2354.90
Qwen 3.7 Flash60.8459.1757.0753.75
GPT-5.6-Luna60.7659.1057.0053.67

UCI retains the original Cleveland, Hungarian, Switzerland and VA Long Beach institutions. New DeepSeek/Qwen and earlier GPT pooled Acc (%) are:

BackboneLocal RAGLocal-EAMedECHO
DeepSeek V4.1 Flash81.8285.5686.63
Qwen 3.7 Flash81.8285.5686.10
GPT-5.6-Luna81.8287.7087.70

Synthetic privacy attacks evaluate direct disclosure and auxiliary-information inference in a closed-world identity pool containing the true identity:

Linkage channelSuccess (%)
Direct identifiers in raw input100.00
Direct-identifier channel after CD0.50
CD plus demographic auxiliary information33.88
CD plus demographic and exactly matching clinical information99.67

CD reduces direct-identifier linkage to the 0.50% random prior. The currently released deterministic case hash is vulnerable to dictionary linkage when candidate IDs and the salt are known. Removing the released hash would block this channel; clinical-feature linkage remains. Attribute-inference balanced accuracy is 50.59%, 69.87%, and 91.02% at generator signal-strength parameters 0, 0.5, and 1.

Synthetic disclosure stress tests give both packet variants 0% identity-canary exposure, 91.30% clinical concept recall, and 100% negation polarity error. Direct disclosure control, auxiliary-information linkage, and negation handling are distinct requirements.

Conclusion. The matched Non-IID and UCI results support hospital-scoped evidence processing across backbones; CD controls direct-identifier exposure, while clinical-feature inference and negation remain explicit protection requirements.

Reviewer_YhBfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MedECHO, a privacy-aware multi-agent collaboration framework for cross-hospital medical diagnosis. The method addresses two challenges: preserving diagnostic information under restricted cross-hospital disclosure and effectively incorporating diagnostic opinions from collaborating hospitals. To address these issues, MedECHO introduces Controlled Disclosure (CD), which converts raw clinical cases into structured, auditable collaboration packets, and Evidence Adoption (EA), which integrates remote diagnostic opinions through candidate aggregation, evidence-based reranking, and guarded decision-making at the originating hospital.

Critical strengths
  1. The paper addresses an important challenge in cross-hospital medical collaboration, where patient information must be protected while enabling effective diagnostic knowledge sharing. Separating the problem into information disclosure and evidence adoption provides a reasonable perspective.

  2. MedECHO introduces a clear collaboration workflow in which the originating hospital retains the raw clinical case and final diagnostic authority.

  3. The proposed method achieves substantial accuracy improvements over the evaluated baselines across multiple datasets and LLM backbones.

Critical weaknesses
  1. The proposed components combine established mechanisms, including structured information filtering, schema validation, retrieval-augmented generation, candidate aggregation, feature-based reranking, and threshold-based decision rules. In particular, EA relies on a centroid-based ranker and a handcrafted promotion guard, rather than introducing a fundamentally new multi-agent learning mechanism. The authors should clarify the distinctive methodological contributions of MedECHO beyond integrating existing privacy-aware communication and evidence-fusion strategies.

  2. Several datasets are relatively small after sampling, particularly MAC-302, which contains only 16–17 test cases per seed. The evaluation relies largely on public symptom-disease classification datasets rather than real multi-institutional clinical cohorts.

  3. The experiments simulate hospitals by partitioning public symptom-disease datasets into separate nodes, while the main evaluations use the same LLM backbone across hospitals. This setting does not adequately represent real institutional heterogeneity in clinical data distributions, documentation practices, medical expertise, and independently maintained knowledge resources.

  4. The computational analysis primarily reports normalized token workloads reconstructed through offline replay, rather than actual inference latency, API cost, or memory consumption. Given that MedECHO involves multiple LLM agents, retrieval operations, and evidence aggregation, an end-to-end efficiency comparison is necessary to establish its practical advantages over existing collaboration frameworks.

Questions for authors

See Weakness.

4 comments
AuthorsOpenReview ↗

Author Response(1/4)

Q1. Candidate-adoption workflow

We added related-method comparisons and fixed-evidence interventions to evaluate the candidate-adoption design. CD specifies what collaborators receive; EA specifies how their proposals influence the originating hospital's diagnosis. The originator retains the full case, records candidate provenance, and reassesses proposals against retrieval evidence. A minority opinion remains eligible for selection even when it loses a vote.

The contribution is this disclosure-constrained diagnostic workflow, built from established retrieval and validation tools. We compare diagnosis-task adaptations of PAPILLON and CARE under the same auxiliary resources. Entries are dataset-equal mean Acc (%) over S2D, K-Sym, and MAC-302:

MethodQwen 3.7 FlashGPT-5.6-Luna
PAPILLON, diagnosis adaptation60.2849.81
CARE, diagnosis adaptation57.0946.62
MedECHO61.3950.92

Source-selection interventions keep model replies and retrieval evidence fixed:

BackboneWithout source selectionFull EA
DeepSeek V4 Flash46.6355.91
Qwen 3.7 Flash58.3561.39
GPT-5.6-Luna47.8950.92

The added Qwen/GPT component experiments assign gains of 1.93 points to candidate expansion, 3.04 to source selection, and 1.11 to the final promotion guard. Removing dataset routing instead improves Acc by 0.82 points, distinguishing the useful selection mechanisms from that routing step.

Conclusion. MedECHO improves on the evaluated delegation adaptations, and fixed-evidence ablations demonstrate a repeatable contribution from originator-side candidate selection across three backbones.

AuthorsOpenReview ↗

Author Response(2/4)

Q2. Matched controls and repeated runs

We extended the resource-matched comparison to Qwen and GPT. Local-EA keeps the local case, Support RAG, candidate expansion, and frozen EA, removing remote opinions and their derived evidence. Acc (%) is averaged equally over S2D, K-Sym, and MAC-302:

SystemDeepSeek V4 FlashQwen 3.7 FlashGPT-5.6-Luna
Local RAG54.0948.3945.56
Local-EA with Support54.8058.2047.73
Comparable-budget local reasoning55.6261.1050.63
MedECHO55.9161.3950.92

Remote evidence adds 1.11, 3.19, and 3.19 percentage points over matched Local-EA. MedECHO also exceeds the comparable-budget local reasoning configuration by 0.29 points on each backbone.

The main experiment reports mean and standard deviation across three independent runs, separately from these supplementary controls. On MAC-302, the main MedECHO-minus-RV differences are +22.30 points with DeepSeek, +10.29 with Qwen, and -6.00 with GPT, indicating backbone-dependent performance. The added original-center UCI evaluation below broadens the data setting.

Conclusion. Resource-matched controls show a positive remote-evidence contribution across three backbones, while repeated main runs and original-center evaluation address complementary sources of variation.

AuthorsOpenReview ↗

Author Response(3/4)

Q3. Institutional heterogeneity

We completed a seven-method Non-IID comparison with DeepSeek V4.1 Flash, Qwen 3.7 Flash, and GPT-5.6-Luna. Random capacity allocation and two Dirichlet conditions use the same test cases, total private knowledge, Support resource, and originator assignment. The intervention changes which diseases each hospital's private knowledge covers.

Entries are S2D/K-Sym mean Acc (%). The strongest comparator is selected within each condition from LR, RV, PV, MAC, MedChain, and Local-EA:

BackboneAllocationStrongest comparatorMedECHO
DeepSeek V4.1 FlashRandomRV: 61.4261.99
DeepSeek V4.1 FlashDirichlet alpha = 1.0RV: 59.3260.32
DeepSeek V4.1 FlashDirichlet alpha = 0.1Local-EA: 54.9058.23
Qwen 3.7 FlashRandomLocal-EA: 59.1760.84
Qwen 3.7 FlashDirichlet alpha = 1.0Local-EA: 57.5159.17
Qwen 3.7 FlashDirichlet alpha = 0.1Local-EA: 53.7557.07
GPT-5.6-LunaRandomRV: 60.5560.76
GPT-5.6-LunaDirichlet alpha = 1.0RV: 58.4559.10
GPT-5.6-LunaDirichlet alpha = 0.1Local-EA: 53.6757.00

MedECHO has the highest mean Acc in each listed setting. At alpha = 0.1, its advantage over the strongest comparator is 3.33 percentage points on each backbone.

UCI Heart Disease retains the original Cleveland, Hungarian, Switzerland, and VA Long Beach centers. Pooled Acc (%) is:

BackboneLocal RAGLocal-EAMedECHO
DeepSeek V4.1 Flash81.8285.5686.63
Qwen 3.7 Flash81.8285.5686.10
GPT-5.6-Luna81.8287.7087.70

Model-composition experiments address model differences; these added evaluations separately address private-knowledge distributions and original institutional data.

Conclusion. MedECHO retains the leading mean accuracy under the tested Non-IID conditions and improves on Local RAG with all three backbones on original-center UCI data, extending evaluation beyond random resource-size differences.

AuthorsOpenReview ↗

Author Response(4/4)

Q4. Resource trade-offs

We added multi-backbone accuracy and workload comparisons for local, voting, discussion, and referral methods. The Qwen/GPT figures below are company-reported aggregate workloads. Acc is dataset-equal across S2D, K-Sym, and MAC-302; input-plus-output tokens are case-weighted. Costs use a common estimate of USD 0.30 per million input tokens and USD 1.20 per million output tokens.

MethodQwen Acc (%)Qwen tokens/caseQwen estimated USDGPT Acc (%)GPT tokens/caseGPT estimated USD
Local RAG48.392,7940.248545.562,8490.2533
Local-EA58.205,8130.517047.735,4030.4807
RV + Support51.6210,8070.960752.879,8390.8751
PV + Support45.5510,7680.958539.4911,0880.9861
MAC + Support46.8510,2500.911746.0910,1860.9063
MedChain + Support44.1011,0200.980947.9011,1190.9887
Comparable-budget local reasoning61.1013,2371.177950.6313,2951.1818
MedECHO61.3914,3771.278750.9212,7171.1313

These cost entries are standardized workload estimates, not provider bills. The new batch's wall-time values are also formula-based estimates; observed latency is kept separate.

The earlier DeepSeek V4 Flash S2D return provides per-case timing and local process memory under cold-cache, concurrency-4 execution:

MethodMedian case time (s)P95 case time (s)Peak local RSS (MB)
Local RAG3.205.37887
Local-EA5.007.25924
MedECHO15.0824.331035
Comparable-budget local reasoning14.8520.311035

The corresponding earlier three-dataset workload is 13,465 tokens/case for MedECHO versus 13,542 for the local reasoning control, with Acc of 55.91% versus 55.62%. Local RAG and Local-EA remain cheaper alternatives.

Conclusion. The comparisons quantify the cost of remote consultation and evidence processing. Against the local reasoning control, GPT MedECHO improves mean Acc by 0.29 points with fewer reported tokens; Qwen obtains the same accuracy increment with a larger workload. These results support choosing a collaboration strategy according to its accuracy and resource requirements.

Reviewer_SRdVOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper focuses on how multiple hospitals can collaborate in diagnosis, leveraging their respective medical expertise while protecting patient privacy and improving diagnostic accuracy. This paper identifies two key challenges, namely "Disclosure Loss" and "Adoption Gap". To address these challenges, this paper proposes MedECHO, combining Controlled Disclosure (CD) and Evidence Adoption (EA). Experiments across various LLM backbones and diagnosis datasets show consistent improvements in diagnostic accuracy.

Critical strengths
  1. The paper identifies two issues in privacy-constrained cross-hospital diagnosis: information loss during disclosure and the failure to adopt correct diagnostic candidates. Through a comparative analysis, the author quantifies these issues, providing specific evidence on information disclosure and evidence adoption.
Critical weaknesses
  1. The proposed method fails to adequately address the two failure modes mentioned in the paper. The core discussion of the paper focuses on the two bottlenecks of the disclosure gap and the adoption gap, and respectively proposes CD and EA as solutions. However, the experiments don't clearly confirm this correspondence. Although PV uses the reviewed cooperative data packets, its average accuracy rate decreases compared to RV. The paper does not explain how CD alleviates the loss of diagnostic information. Therefore, the connection between the identified disclosure issues and the proposed CD solution still lacks sufficient justification.

  2. There are several inconsistencies between the experimental results and the analysis. For example, the authors state that MedECHO's ECR on MAC-302 remains zero across all three agent compositions, whereas Figure 1 and Table 1 report nonzero values. Additionally, in Section 4.4, "Efficiency against multi-agent baselines.", it reports 51.98% accuracy for MedECHO, but it is 65.80% in Table 1.

Questions for authors
  1. Could the authors clarify the inconsistencies between Table 1, Figure 7, and the discussion in Section 4.4?
  2. How much of MedECHO's performance improvement can be attributed specifically to cross-hospital collaboration, rather than additional components such as Support RAG?
  3. What evidence demonstrates that CD effectively mitigates disclosure-induced diagnostic information loss?
9 comments
AuthorsOpenReview ↗

Author Response(1/3)

Q1. CD utility

We added matched-input experiments across three backbones to test CD within the same EA pipeline. Each input variant shares its group's local opinion, private retrieval, Support RAG, and frozen EA. Entries are dataset-equal mean Acc (%) over S2D, K-Sym, and MAC-302:

Remote inputDeepSeek V4.1 Flash-1017Qwen 3.7 FlashGPT-5.6-Luna
Full raw case58.4860.2849.81
Full CD packet59.0161.1050.63
Minimal clinical packet58.1959.9949.52

The minimal packet retains complaints, clinical concepts, and abnormal findings while removing contextual fields. Full CD improves over the minimal packet by 0.82, 1.11, and 1.11 points, and over the raw case by 0.53, 0.82, and 0.82 points.

The added Qwen/GPT experiments also compare ordinary de-identification and a length-matched clinical summary:

Remote inputQwen 3.7 FlashGPT-5.6-Luna
Identity masking57.0946.62
Length-matched summary59.9949.52
Full CD packet61.1050.63

The RV-to-PV drop diagnoses the effect of restricted information under a fixed voting rule. The fixed-EA comparisons directly evaluate CD's solution: retaining clinically useful context inside the disclosure boundary. Separate synthetic stress tests record 0% identity-canary exposure and 91.30% positive clinical-concept recall. The negation probes expose a polarity-handling failure, evaluated separately from positive-concept retention.

Conclusion. With retrieval and EA fixed, CD achieves higher mean Acc than raw and minimal inputs across three backbones, and exceeds ordinary masking and summary controls on Qwen and GPT. This supports clinically structured disclosure as a useful collaboration interface.

AuthorsOpenReview ↗

Author Response(2/3)

Q2. EA and collaboration

We separated remote evidence from local processing using matched Local-EA controls across three backbones. Local-EA retains the local case, Support RAG, candidate processing, and frozen EA, removing only remote opinions and their derived evidence. Entries are mean Acc (%) over S2D, K-Sym, and MAC-302:

BackboneLocal RAGLocal-EAMedECHOGain over Local-EA (pp)
DeepSeek V4 Flash54.0954.8055.91+1.11
Qwen 3.7 Flash48.3958.2061.39+3.19
GPT-5.6-Luna45.5647.7350.92+3.19

Source-selection ablation fixes model opinions and evidence:

BackboneWithout source selectionFull EA
DeepSeek V4 Flash46.6355.91
Qwen 3.7 Flash58.3561.39
GPT-5.6-Luna47.8950.92

The first comparison measures the value of remote consultation; the second measures the originator's selection among available evidence.

UCI Heart Disease retains its original institutional split:

BackboneLocal RAGLocal-EAMedECHO
DeepSeek V4.1 Flash81.8285.5686.63
Qwen 3.7 Flash81.8285.5686.10
GPT-5.6-Luna81.8287.7087.70

We also extended the raw-to-packet comparison beyond medicine. The Acc drops are:

BackboneBANKING77 (pp)Medical dataset mean (pp)
DeepSeek V4.1 Flash12.994.01
Qwen 3.7 Flash12.994.01
GPT-5.6-Luna12.9911.98

Disclosure loss occurs in both domains. The clinical motivation is the restriction on patient information sharing and the originator's responsibility for the final diagnosis.

Conclusion. Matched controls establish a positive remote-evidence contribution across three backbones, while selection ablations identify the decision step that makes available evidence useful. MedECHO also improves on Local RAG under the original UCI institutional split.

AuthorsOpenReview ↗

Author Response(3/3)

Q3. Reporting corrections

We will align Section 4.4 with Table 1's same-family mean Acc of 65.80%. Figure 7's efficiency point and ordering will be withdrawn because its resource coordinate is not linked to that run; the matched supplementary resource comparison will be presented separately.

We will remove the blanket zero-ECR statement for MAC-302 and inconsistent composition-level Acc/ECR comparisons. The main three-run results remain separate from the composition study.

Conclusion. The revision will consistently report 65.80% for the same-family aggregate and associate correction metrics and efficiency with their corresponding experimental settings.

Reviewer_xotLOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies cross-hospital LLM-based diagnosis where the originating hospital retains the raw case while remote hospitals contribute knowledge through structured messages. It proposes MEDECHO, which combines Controlled Disclosure (CD) for reviewing schema-constrained packets with Evidence Adoption (EA), an originator-side candidate ledger and guarded reranking step. Experiments across six datasets, three LLM backbones, and three seeds compare MEDECHO with local (LR), voting (RV, PV), and medical multi-agent (MAC, MedChain) baselines.

Critical strengths

The paper isolates a real failure mode in collaborative diagnosis. Section 3.2 and Figure 2 compare RV and PV under the same voting rule, showing a consistent drop in accuracy when switching from raw context to packets across all three backbones. They also separate candidate discovery from final Top-1 selection. Identifying this concrete decision bottleneck—which EA is designed to address—is much more useful than just showing that a new protocol improves overall accuracy.

Critical weaknesses
  1. Limited novelty and missing comparisons with closely related work. The paper does not show a clear methodological advance over existing privacy-preserving delegation and medical multi-agent systems. PAPILLON already combines locally generated privacy-preserving requests, remote assistance, and local aggregation using the private input. CARE pairs remote structured guidance with local patient-specific decision-making in healthcare, and the MedAgentNet preprint uses structured queries and restricted disclosure for medical-agent collaboration. While MEDECHO combines closed-schema packets, candidate tracking, retrieval, centroid ranking, and threshold-based promotion, the paper does not explain what specific technical problem this combination solves that prior methods cannot. System integration could still be a valid contribution, but without direct comparisons and resource-matched controls, the paper does not prove that this integration adds value.

  2. Major numerical and reporting contradictions make the experimental results unreliable.

  • Table 1 vs. Section 4.4 and Figure 7: Under the same-family, seed-42, three-dataset setting (S2D, K-Sym, MAC-302), Table 1 reports a MEDECHO average accuracy of 65.80%, whereas Section 4.4 and Figure 7 report 51.98%. At 51.98%, MEDECHO falls behind both RV (56.09%) and MAC (55.81%).
  • Section 4.3 vs. Table 1: Section 4.3 misstates which methods lead in Table 1 across settings and claims that MEDECHO’s ECR on MAC-302 is zero in all three compositions, even though Table 1 lists 18.18%, 27.27%, and 27.27%.
  • Impossible S2D accuracy values in Table 1: Two S2D entries violate Equation (23), which defines accuracy in terms of LR accuracy and ECR. Given the reported LR accuracy (77.78%) and MEDECHO’s ECR values (71.43% and 57.14%), the maximum possible accuracy—even with zero regressed cases—is 93.65% and 90.48%. Yet Table 1 reports 95.24% and 93.65%. Rounding cannot explain this.
  • Table 1 vs. Tables 9–10: Several homogeneous seed-42 results in Table 1 are mathematically incompatible with the three-seed averages in Tables 9 and 10 if they follow the same evaluation protocol. For example, MEDECHO’s seed-42 K-Sym accuracy of 53.04% cannot be one of the three runs summarized as 34.49 ± 2.66% in Table 10.

Together, these contradictions make it impossible to verify the paper’s accuracy gains, method rankings, or efficiency claims.

Questions for authors
  1. What specific technical capability does MEDECHO add beyond existing privacy-preserving delegation and medical-agent collaboration methods, and which controlled experiment isolates its effect?
  2. Which set of numbers reflects the final system? Please reconcile the discrepancies across Table 1, Figure 7, Section 4.3, and the Table 10 caption. Did these results come from different system versions or different case subsets? Please provide the configuration IDs, anonymized case IDs, and per-seed counts for the corrected results.
  3. How can the reported metrics be mutually consistent? Please explain how two S2D entries in Table 1 exceed the upper bounds implied by LR accuracy and ECR, and why the homogeneous seed-42 numbers conflict with the three-seed statistics in Tables 9–10. Did all runs use the exact same cases and LR references? Reporting per-seed counts of correct, corrected, and regressed predictions would clarify this.
  4. What does “pre-filter” mean in Table 2? Does this cohort include cases used during policy development? Please reconcile Table 2 with the reproducibility statement and show whether the ablation findings still hold on the final development-disjoint test cohort.
  5. How much of MEDECHO’s accuracy gain remains when remote opinions are removed while the originator keeps the same Support RAG, candidate expansion, ranker, and inference budget? Conversely, how do the baselines perform when given those same auxiliary resources?
6 comments
AuthorsOpenReview ↗

Author Response(1/4)

Q1. Candidate adoption

We added comparisons with diagnosis-task adaptations of PAPILLON and CARE, together with interventions on EA. PAPILLON addresses privacy-conscious delegation, CARE combines remote guidance with local decisions, and MedAgentNet studies restricted-disclosure medical collaboration. MedECHO focuses on the next decision: how the originating hospital evaluates and adopts a diagnostic candidate after it has been proposed.

Remote primary and differential diagnoses enter a source-tagged candidate ledger alongside local evidence. The originator retains the complete case and evaluates candidates using retrieval support and a frozen selection policy. This preserves minority proposals for a second assessment instead of eliminating them through a vote.

The added comparisons use the same diagnostic tasks and auxiliary retrieval resources. Entries are dataset-equal mean Acc (%) over S2D, K-Sym, and MAC-302:

MethodQwen 3.7 FlashGPT-5.6-Luna
PAPILLON, diagnosis adaptation60.2849.81
CARE, diagnosis adaptation57.0946.62
MedECHO61.3950.92

With model opinions and retrieved evidence fixed, removing source selection reduces Acc from 61.39% to 58.35% with Qwen and 50.92% to 47.89% with GPT. The earlier DeepSeek V4 Flash intervention gives 55.91% to 46.63%.

Conclusion. MedECHO improves on the evaluated delegation adaptations, while fixed-evidence interventions identify an accuracy contribution from originator-side source selection across all three backbones. This supports candidate adoption beyond generating additional opinions.

AuthorsOpenReview ↗

Author Response(2/4)

Q2. Resource-matched controls

We extended matched Local-EA and enhanced-baseline experiments to Qwen and GPT. Local-EA retains the full local case, Support RAG, candidate expansion, and frozen EA, removing remote opinions and all remote-derived evidence. The local reasoning control additionally receives a comparable-budget configuration. Entries are mean Acc (%) over S2D, K-Sym, and MAC-302:

SystemDeepSeek V4 FlashQwen 3.7 FlashGPT-5.6-Luna
Local RAG54.0948.3945.56
Local-EA without Support55.4856.8046.33
Local-EA with Support54.8058.2047.73
MedECHO without Support56.8862.3651.90
MedECHO with Support55.9161.3950.92
Comparable-budget local reasoning55.6261.1050.63

With Support fixed, remote evidence improves mean Acc by 1.11, 3.19, and 3.19 percentage points. Without Support, the corresponding gains are 1.40, 5.57, and 5.57 points. Support itself has mixed effects, as shown by the factorial comparison.

We also give RV, PV, MAC, and MedChain the same Support resource:

Enhanced baselineQwen 3.7 FlashGPT-5.6-Luna
RV + Support51.6252.87
PV + Support45.5539.49
MAC + Support46.8546.09
MedChain + Support44.1047.90
MedECHO61.3950.92

MedECHO leads this comparison with Qwen; enhanced RV leads with GPT. The Local-EA control isolates remote contribution separately from this overall ranking.

Stronger local retrieval. With the same reranked retrieval configuration, S2D/K-Sym mean Acc is:

BackboneLocal-RAG-plusMedECHO
DeepSeek V4.1 Flash58.5961.99
Qwen 3.7 Flash53.1660.84
GPT-5.6-Luna58.3060.76

Conclusion. MedECHO improves on resource-matched Local-EA across all three backbones and retains an average-accuracy advantage over enhanced local retrieval. These controls demonstrate a contribution from remote evidence beyond additional Support RAG.

AuthorsOpenReview ↗

Author Response(3/4)

Q3. Reporting and protocols

We will align Section 4.4 with Table 1's same-family Macro Acc of 65.80%. The corrections are:

ItemCorrection
Table 1 and Section 4.4Retain 65.80% and align the corresponding accuracy statement.
Composition-level Acc/ECRRemove the inconsistent S2D metric pairs and the blanket MAC-302 zero-ECR statement.
Figure 7Withdraw the efficiency point and ordering not linked to the Table 1 run; present matched supplementary resources separately.
Table 10 captionDescribe actual dataset-level results instead of claiming a lead on both metrics throughout Difficult and Rare datasets.

GPT K-Sym uses two originator protocols:

ProtocolAcc (%)
Main experiment, mean +/- sample SD34.49 +/- 2.66
Separate homogeneous-composition experiment38.26

The main experiment distributes originators across hospitals; the composition experiment fixes the originator. The corrected composition entry is 38.26%, replacing 53.04%, and is reported separately from the main three-run statistic.

Conclusion. The revision will use 65.80% consistently for the same-family aggregate and separate the main, composition, and efficiency protocols, resolving conflicting accuracy statements and metric comparisons.

Reviewer_bqxFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MedECHO, a privacy-aware multi-agent framework for cross-hospital diagnosis. It introduces two components: Controlled Disclosure, which restricts patient information sharing through structured clinical packets, and Evidence Adoption, which integrates remote diagnostic opinions with local evidence to improve the final diagnosis. Experiments across six datasets and three LLM backbones demonstrate improvements in diagnostic accuracy over existing baselines.

Critical strengths
  1. The paper addresses two challenges in cross-hospital diagnosis: preserving diagnostic information under privacy constraints and effectively adopting correct diagnoses from collaborating hospitals.
  2. The experiments cover multiple datasets and LLM backbones, with ablation studies and efficiency comparisons demonstrating the effectiveness of the proposed framework.
Critical weaknesses
  1. Unrealistic cross-hospital experimental setting. The experiments simulate hospitals by partitioning the same datasets into three nodes with a 50/30/20 allocation, without explicitly modeling differences in disease distributions, patient populations, or clinical practices. As highlighted by recent multi-institutional medical research [1], such heterogeneity is an important challenge in cross-hospital collaboration. The current evaluation is therefore insufficient to demonstrate the framework's effectiveness under realistic non-IID settings.
  2. Insufficient evaluation of the core contributions. The framework relies heavily on private RAG, auxiliary Support RAG, and a trained evidence-based ranker. However, the experiments lack comparisons with stronger medical RAG approaches and do not sufficiently isolate the benefits of cross-hospital collaboration from additional retrieval and ranking components. This makes it difficult to determine whether the improvements primarily arise from the proposed collaboration mechanism or the enhanced retrieval and candidate selection process.

Reference
[1] Toward federated large language models in medicine: a parameter-efficient framework for privacy-preserving, multi-institutional adaptation, npj Digital Medicine, 2026.

Questions for authors
  1. Can the authors evaluate MedECHO on real multi-institutional datasets or introduce more realistic non-IID settings with heterogeneous disease distributions and patient populations? How does the framework perform as the degree of inter-hospital heterogeneity increases?
  2. Can the authors compare MedECHO against stronger medical RAG baselines and a local RAG framework equipped with the same Support RAG, candidate expansion, and ranking mechanisms? Such comparisons would help isolate the actual contribution of multi-agent collaboration from additional retrieval and ranking components.
  3. The privacy analysis is limited to six canary cases with 13 predefined clinical signals. Can the authors provide more comprehensive evaluations against re-identification or sensitive attribute inference attacks, particularly when structured clinical features are shared across hospitals?
  4. How sensitive is MedECHO to other important hyperparameters, such as RAG retrieval depth, candidate pool size, and the number of collaborating hospitals?
4 comments
AuthorsOpenReview ↗

Author Responses (1/4)

Q1. Institutional heterogeneity

** We completed matched Non-IID comparisons across backbones and extended the UCI source-institution evaluation.** The Non-IID experiment changes private-knowledge allocation at fixed 50/30/20 capacities, retaining the evaluation cases, originators and each method's remaining settings. Values below are equal-weight S2D/K-Sym macro Acc (%).

BackboneAllocationLRRVPVMACMedChainLocal-EAMedECHO
DeepSeek V4.1 FlashRandom56.1361.4253.3952.7350.8660.3261.99
DeepSeek V4.1 Flashalpha=1.054.4759.3251.2951.0647.9658.6660.32
DeepSeek V4.1 Flashalpha=0.151.1453.0346.7446.1543.4154.9058.23
Qwen 3.7 FlashRandom50.7055.5646.4548.4044.2859.1760.84
Qwen 3.7 Flashalpha=1.049.0453.4644.3546.7441.3957.5159.17
Qwen 3.7 Flashalpha=0.145.7147.1739.8041.8236.8353.7557.07
GPT-5.6-LunaRandom55.8560.5549.8656.6453.1159.1060.76
GPT-5.6-Lunaalpha=1.054.1858.4547.7654.9850.2157.4359.10
GPT-5.6-Lunaalpha=0.150.8652.1643.2050.0645.6653.6757.00

UCI Heart Disease retains Cleveland, Hungarian, Switzerland and VA Long Beach as source institutions. The new DeepSeek/Qwen results and earlier GPT result give the following pooled Acc (%):

BackboneLocal RAGLocal-EAMedECHO
DeepSeek V4.1 Flash81.8285.5686.63
Qwen 3.7 Flash81.8285.5686.10
GPT-5.6-Luna81.8287.7087.70

Conclusion. MedECHO has the highest S2D/K-Sym macro Acc in each matched allocation and backbone, with a 3.33-point advantage over Local-EA at alpha=0.1. On UCI it improves over Local RAG across backbones, matching Local-EA under GPT Luna.

AuthorsOpenReview ↗

Author Responses (2/4)

Q2. Collaboration and resources

We extended the resource-factorial controls to Qwen 3.7 Flash and GPT-5.6-Luna. Local-EA retains local processing and frozen EA while removing remote opinions and derived evidence. The following tables report equal-weight S2D/K-Sym/MAC-302 macro Acc (%); the earlier DeepSeek V4 Flash results remain a separate model-version column.

MethodDeepSeek V4 Flash (earlier)Qwen 3.7 FlashGPT-5.6-Luna
Local RAG54.0948.3945.56
Local-EA, without Support55.4856.8046.33
Local-EA, with Support54.8058.2047.73
MedECHO, without Support56.8862.3651.90
MedECHO, with Support55.9161.3950.92
Budget-matched local reasoning55.6261.1050.63

With Support fixed, MedECHO adds 3.19 macro points over Local-EA on both Qwen and GPT, and 0.29 points over the configured budget-matched control. The new Qwen/GPT costs and times are estimates. The no-Support MedECHO configuration is higher by 0.97 points on both backbones, showing that Support's contribution depends on the configuration.

Support-enhanced baselines and diagnosis-adapted PAPILLON/CARE:

MethodQwen 3.7 FlashGPT-5.6-Luna
RV + Support51.6252.87
PV + Support45.5539.49
MAC + Support46.8546.09
MedChain + Support44.1047.90
PAPILLON-adapted60.2849.81
CARE-adapted57.0946.62
MedECHO61.3950.92

MedECHO leads this comparison on Qwen; RV + Support reaches 52.87% versus MedECHO's 50.92% on GPT.

Matched retrieval controls compare Local-RAG-plus and MedECHO under dense, hybrid and reranked retrieval. This table uses S2D/K-Sym macro Acc (%):

BackboneRetrievalLocal-RAG-plusMedECHO
DeepSeek V4.1 FlashDense57.3659.53
DeepSeek V4.1 FlashHybrid58.5960.76
DeepSeek V4.1 FlashReranked58.5961.99
Qwen 3.7 FlashDense51.9358.38
Qwen 3.7 FlashHybrid53.1659.61
Qwen 3.7 FlashReranked53.1660.84
GPT-5.6-LunaDense57.0758.30
GPT-5.6-LunaHybrid58.3059.53
GPT-5.6-LunaReranked58.3060.76

Conclusion. Matched controls isolate a 3.19-point remote-evidence gain over Local-EA on Qwen/GPT, while stronger retrieval also benefits MedECHO; the advantage over budget-matched local reasoning is 0.29 points.

AuthorsOpenReview ↗

Author Responses (3/4)

Q3. Parameter sensitivity

We extended retrieval-depth, hospital-number and candidate-cap controls to Qwen and GPT. Values are equal-weight S2D/K-Sym/MAC-302 macro Acc (%); depth 5 and K=3 use the full resource-matched configuration.

FactorValueQwen 3.7 FlashGPT-5.6-Luna
Retrieval depth260.2849.81
Retrieval depth561.3950.92
Retrieval depth1061.3950.92
Hospitals K149.4946.67
Hospitals K361.3950.92
Hospitals K560.5750.10
Candidate cap359.4648.99
Candidate cap560.2849.81
Candidate cap1061.1050.63
Candidate cap2061.3950.92

Hospital-number controls repartition a fixed total private corpus; candidate caps are offline interventions holding model replies and retrieval fixed, constraining the whole EA chain and fallback to the eligible set.

The earlier DeepSeek V4 Flash candidate replay gives complementary evidence on candidate availability, with the same dataset scope:

Candidate capMacro Acc (%)
346.29
546.63
1048.08
2054.17
Uncapped55.91

In that replay, K-Sym candidate coverage rises from 11.30% at cap 3 to 84.35% at cap 20 and 100% uncapped.

Conclusion. Qwen/GPT accuracy is stable from retrieval depth 5 to 10, peaks at K=3 among the tested hospital settings, and improves as the candidate cap increases from 3 to 20. Candidate breadth and retrieval depth have distinct effects.

AuthorsOpenReview ↗

Author Responses (4/4)

Q4. Privacy and clinical retention

We added identity-linkage, hash-dictionary, sensitive-attribute, and clinical-retention tests. Linkage uses closed-world candidate pools containing the true identity. The clinical-auxiliary condition supplies exactly matching synthetic clinical features:

Attack conditionLinkage success (%)
Random prior0.50
Direct identifiers, raw input100.00
Direct identifiers, CD output0.50
Demographic auxiliary information, CD output33.88
Demographic and matching clinical auxiliary information, CD output99.67

The currently released deterministic case hash is vulnerable to dictionary linkage when candidate IDs and the salt are known. Removing the released hash would block this channel; clinical-feature linkage remains. Attribute-inference balanced accuracy is 50.59%, 69.87%, and 91.02% at synthetic signal strengths 0, 0.5, and 1.

Both full and minimal packets have 0% identity-canary exposure and 91.30% positive clinical-concept recall.

Conclusion. CD achieves 0% identity-canary exposure in the stress tests.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.