- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper investigates privacy-constrained multi-agent collaboration for cross-hospital diagnosis. The authors construct a simulated cross-hospital setting in which each hospital agent has access to its own private knowledge resources, while patient information shared across hospitals is restricted. They identify two key challenges: the disclosure gap, where restricted information sharing may reduce diagnostic accuracy, and the adoption gap, where correct diagnoses proposed by collaborating agents may not be selected in the final decision. To address these challenges, the authors propose MedECHO, which combines Controlled Disclosure (CD) and Evidence Adoption (EA). CD generates and validates structured collaboration packets to limit information exposure while preserving clinical signals. EA integrates remote diagnostic opinions and multiple evidence sources through a Candidate Ledger, Retrieval-aware Selection, a centroid-based ranker, and a Promotion Guard to improve final diagnostic decisions. Experiments on six medical diagnosis datasets using three LLM backbones and three random seeds show that MedECHO improves average diagnostic accuracy over the strongest evaluated baselines by 15.94, 15.88, and 9.23 percentage points, respectively.
Critical strengths
-
The paper identifies two meaningful challenges in privacy-constrained multi-agent diagnosis: the disclosure gap and the adoption gap. The authors provide empirical observations to illustrate these problems, including the performance difference between raw-context and packet-based voting, and the gap between candidate discovery and final diagnostic accuracy. I think these observations provide a reasonable motivation for the proposed framework.
-
The paper reports substantial improvements in diagnostic accuracy across six medical datasets, three LLM backbones, and multiple random seeds. The authors also provide several ablation studies to examine the contributions of different components. In particular, the results help illustrate the importance of candidate expansion and auxiliary retrieval in improving diagnostic performance.
Critical weaknesses
-
From my understanding, the practical motivation for the proposed cross-hospital multi-agent setting could be further clarified. Although privacy constraints are important in cross-hospital collaboration, it is not entirely clear to me whether direct collaboration between independently deployed hospital agents represents a realistic clinical need, or why this problem specifically requires a multi-agent framework rather than conventional cross-institutional information exchange. In addition, the experimental environment is constructed by partitioning public datasets into simulated hospital nodes, rather than using real institutional data and communication constraints. While this is reasonable for controlled experiments, I am not sure how well the resulting setting reflects the challenges of real-world cross-hospital collaboration.
-
Although the paper includes several ablation studies, I feel that the experiments do not fully separate the contributions of different mechanisms. For example, the authors compare Raw-Context Voting (RV), Packet Voting (PV), and MedECHO, but I could not find a comparison where MedECHO uses the full patient case while keeping the same Evidence Adoption (EA) pipeline. Such a comparison would help clarify how much diagnostic performance is lost due to restricted information sharing under the proposed method. Furthermore, the ablation results suggest that Candidate Ledger and Support RAG contribute substantially to the overall improvement. It would be helpful to include a local-only baseline with comparable retrieval resources and the same EA mechanism to better understand how much of the improvement actually comes from cross-hospital multi-agent collaboration.
Questions for authors
-
Could the authors provide a comparison where MedECHO uses the full patient case instead of the restricted packet, while keeping the same EA pipeline and other experimental settings? The current RV and PV comparison evaluates the effect of restricted information sharing under majority voting. I think this additional experiment would help clarify how much diagnostic performance is lost due to information restrictions under the proposed method, and whether EA can actually reduce this loss.
-
Could the authors include a local-only baseline that uses the same auxiliary retrieval resources, Candidate Ledger, and EA decision mechanism, but without remote agent collaboration? From the ablation results, Candidate Ledger and Support RAG seem to contribute substantially to the overall improvement. Such a comparison would help clarify how much of the performance gain comes specifically from cross-hospital multi-agent collaboration, rather than from stronger retrieval and candidate selection.
-
Could the authors further clarify the practical motivation for the proposed cross-hospital multi-agent setting? From my understanding, the experimental environment is constructed using simulated hospital nodes with partitioned retrieval resources. I wonder whether there are existing or potential clinical applications that specifically require independently deployed hospital agents to collaborate under such information-sharing constraints, and how this setting differs from conventional cross-institutional information exchange.
-
Could the authors provide more details or additional experiments regarding the privacy guarantees of CD? The current privacy evaluation appears to rely on a relatively small set of synthetic canary cases. I wonder whether the proposed mechanism has been evaluated against more realistic privacy risks, such as indirect information disclosure or patient re-identification from combinations of clinical features.
-
I noticed a possible inconsistency between Figure 7 and Table 1. Figure 7 reports an average accuracy of 51.98% for MedECHO in the same-family setting, whereas Table 1 reports 65.80% for what appears to be the corresponding configuration. Could the authors clarify whether these results use different experimental settings or whether there is an error in the reported values?
6 comments
Author Response(2/4)
Q2. Resource-matched Local-EA
We extended the resource-factorial controls across backbones. Local-EA retains local processing and frozen EA while removing remote opinions and derived evidence. Values are equal-weight S2D/K-Sym/MAC-302 macro Acc (%); the historical V4 column is separate from the V4.1 disclosure experiment.
| System | DeepSeek V4 Flash (earlier) | Qwen 3.7 Flash | GPT-5.6-Luna |
|---|---|---|---|
| Local RAG | 54.09 | 48.39 | 45.56 |
| Local-EA without Support | 55.48 | 56.80 | 46.33 |
| Local-EA with Support | 54.80 | 58.20 | 47.73 |
| MedECHO without Support | 56.88 | 62.36 | 51.90 |
| MedECHO with Support | 55.91 | 61.39 | 50.92 |
| Budget-matched local reasoning | 55.62 | 61.10 | 50.63 |
The remote-evidence gain over Local-EA with Support is 1.11 points on V4 and 3.19 points on Qwen/GPT. MedECHO exceeds the configured budget-matched local control by 0.29 points on each backbone. The new Qwen/GPT costs and times are estimates. The no-Support MedECHO variant reaches 62.36%/51.90% on Qwen/GPT, above the full configuration, so the factorial comparison separates Support effects from collaboration.
Conclusion. Remote evidence improves macro Acc over same-Support Local-EA across the tested backbones, with a 3.19-point gain on Qwen/GPT and a smaller 0.29-point margin over budget-matched local reasoning.
Author Response(3/4)
Q3. Hospital agents and privacy
The agent formulation makes the responsibilities of asynchronous consultation explicit. The originating institution retains the full patient context and the final diagnostic decision; a consulting institution supplies an interpretation grounded in evidence it controls. The reviewed packet governs information sharing, and the source-tagged candidate ledger lets the originator decide whether to adopt a remote proposal. This adds reasoning and adoption decisions to the record-transfer function of conventional information exchange.
The matched Non-IID comparison is complete. Under fixed private-knowledge capacities, we compared LR, RV, PV, MAC, MedChain, Local-EA and MedECHO across random and Dirichlet allocations. MedECHO has the highest S2D/K-Sym macro Acc in each backbone/allocation; its results are:
| Backbone | Random Acc (%) | alpha=1.0 Acc (%) | alpha=0.1 Acc (%) | Local-EA at alpha=0.1 (%) |
|---|---|---|---|---|
| DeepSeek V4.1 Flash | 61.99 | 60.32 | 58.23 | 54.90 |
| Qwen 3.7 Flash | 60.84 | 59.17 | 57.07 | 53.75 |
| GPT-5.6-Luna | 60.76 | 59.10 | 57.00 | 53.67 |
UCI retains the original Cleveland, Hungarian, Switzerland and VA Long Beach institutions. New DeepSeek/Qwen and earlier GPT pooled Acc (%) are:
| Backbone | Local RAG | Local-EA | MedECHO |
|---|---|---|---|
| DeepSeek V4.1 Flash | 81.82 | 85.56 | 86.63 |
| Qwen 3.7 Flash | 81.82 | 85.56 | 86.10 |
| GPT-5.6-Luna | 81.82 | 87.70 | 87.70 |
Synthetic privacy attacks evaluate direct disclosure and auxiliary-information inference in a closed-world identity pool containing the true identity:
| Linkage channel | Success (%) |
|---|---|
| Direct identifiers in raw input | 100.00 |
| Direct-identifier channel after CD | 0.50 |
| CD plus demographic auxiliary information | 33.88 |
| CD plus demographic and exactly matching clinical information | 99.67 |
CD reduces direct-identifier linkage to the 0.50% random prior. The currently released deterministic case hash is vulnerable to dictionary linkage when candidate IDs and the salt are known. Removing the released hash would block this channel; clinical-feature linkage remains. Attribute-inference balanced accuracy is 50.59%, 69.87%, and 91.02% at generator signal-strength parameters 0, 0.5, and 1.
Synthetic disclosure stress tests give both packet variants 0% identity-canary exposure, 91.30% clinical concept recall, and 100% negation polarity error. Direct disclosure control, auxiliary-information linkage, and negation handling are distinct requirements.
Conclusion. The matched Non-IID and UCI results support hospital-scoped evidence processing across backbones; CD controls direct-identifier exposure, while clinical-feature inference and negation remain explicit protection requirements.
Author Response(1/4)
Q1. Matched disclosure utility
We completed matched-disclosure comparisons with DeepSeek-V4.1-Flash-1017, Qwen 3.7 Flash and GPT-5.6-Luna. Within each comparison, input variants share the local opinion, retrieval, Support and frozen EA; the disclosure protocol is evaluated separately from the resource-factorial experiment.
The earlier DeepSeek-V4.1-Flash-1017 matched experiment:
Full CD is +0.53 macro points above raw input and +0.82 above the minimal packet. The minimal packet retains complaints, clinical concepts and abnormal findings while removing contextual fields.
The new disclosure comparison adds identity masking and length-matched summaries. Values are equal-weight S2D/K-Sym/MAC-302 macro Acc (%):
Full CD gains 0.82 points over raw input and 1.11 points over the minimal packet on both Qwen and GPT.
The separate earlier DeepSeek V4 Flash factorial control, on the same dataset scope, retains the input-by-decision comparison:
The raw-minus-packet gap is 2.90 points with voting and 0.82 with EA; on MAC-302, raw accuracy changes from 50.00% to 43.75%, while packet accuracy stays 43.75%. The earlier V4 identity-masking and length-matched-summary EA controls reach 55.09% and 57.70%.
Conclusion. With retrieval and EA fixed, full CD preserves diagnostic utility and exceeds raw input by 0.53 macro points on V4.1-Flash-1017 and 0.82 points on Qwen/GPT, while retaining a useful contribution from contextual fields.