acceptodds
Under review as a conference paper at ICLR 2027

Learning What to Share and What to Complement for Multi-Agent Collaboration

Abstract

Multi-agent collaboration sharing strategies facilitate knowledge exchange across agents, yet struggle to balance privacy protection with effective collaboration when agents hold different private knowledge. We redesign the communication boundaries around these knowledge differences and propose a privacy-aware multi-agent collaboration framework that uses them as structural signals throughout the reciprocal communication process. Relevance-aware Adaptive Disclosure (RAD) governs outgoing task sharing through a self-calibrated disclosure boundary, moving beyond task fragmentation. Complementary Expertise Integration (CEI) governs returned peer knowledge through geometry-guided complementarity, preserving useful expertise while suppressing redundant reasoning. Across medical and legal tasks on MedQA and LegalQA, our framework consistently improves both open-source and closed-source LLMs, achieving up to 15.3% relative gains in task utility and 9.3% in privacy–utility trade-off over baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Multi-Agent Collaboration, Multi-Agent System

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_keYfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes privacy-aware collaboration among LLM agents with heterogeneous private memories. RAD selectively abstracts sensitive query spans using memory-relevance losses and a query-local Gaussian mixture; CEI ranks peer evidence by geometric complementarity before generating reasoning summaries. Experiments on constructed MedQA and LegalQA tasks with three LLM backbones evaluate accuracy, fact-reconstruction protection, and their trade-off, alongside component and sensitivity analyses.

Critical strengths
  1. The two communication directions receive distinct, useful treatment. Separating outgoing disclosure from incoming evidence selection addresses both unnecessary exposure and redundant collaboration. The modular comparisons help identify their different effects on utility and privacy.
  2. There is concrete evidence of useful collaboration. With Qwen3-30B, the full method improves accuracy over a single agent from 81.88% to 84.79% on MedQA and from 42.86% to 45.36% on LegalQA (Table 1). These gains support the practical value of exploiting heterogeneous memories.
Critical weaknesses
  1. The privacy boundary is incomplete at the central CEI step. Equations (10)–(12) and Algorithms 1/3 require joint access to target- and peer-memory representations, yet the paper does not specify where this computation occurs or what representations cross institutions. Appendix C evaluates attacks on the protected query and returned summaries only. Without accounting for the information needed to compute CEI, neither distributed implementability nor the reported protection is established for the complete protocol.
  2. The aggregate metric does not establish a convincing privacy–utility advantage. OP pools query and retrieved-memory facts, allowing 94.86% protection despite QP = 0 for the Qwen single-agent baseline (Table 1). This weighting can obscure query exposure. Moreover, the full method's DeepSeek average TO is below RAD alone (79.19 versus 79.98), and uncertainty is unreported. Thus, the benefit of adding collaboration to disclosure protection remains insufficiently resolved.
Questions for authors
  1. Protocol clarification: Where is CEI computed, what exact messages or embeddings are exchanged, and what protection results hold when the attacker sees that complete transcript?
  2. Evaluation clarification: Which models and prompts perform reconstruction and entailment judging? Does the conclusion hold when query and memory protection are assessed separately using a common source-fact universe across methods?
  3. Evidence needed for assessment: What are the evaluation sizes and paired uncertainty estimates for the full method versus RAD alone and RAD + PrivCollab-Agent across memory partitions and inference runs?
5 comments
AuthorsOpenReview ↗

Response to Reviewer keYf (1/3)

W1&Q1: Clarification on privacy boundary at CEI and RAD

We sincerely appreciate your insightful observation and apologize for the insufficient clarification of CEI's communication protocol. To clarify the communication boundaries of our framework, we summarize the complete workflow below.

Table A. Communication protocol of our framework.

StageTarget AgentPeer Agent
Query ProtectionRAD—
Task SharingSend protected queryReceive protected query
Reasoning PreparationSend geometric descriptorsCEI
Output SharingReceive reasoning summarySend reasoning summary

Importantly, CEI operates on the peer side without requiring centralized access to raw private memories. As formulated in Sec. 4.2 (Eqs. 9–12), each agent locally computes query-relative geometric descriptors (θ,u)(\theta,\mathbf{u}) for its retrieved memories. The target shares its descriptors with peers, which locally compute pairwise complementarity scores and prioritize complementary evidence for reasoning. To further examine the privacy–utility trade-offs of RAD and CEI, we conduct additional comparisons on MedQA using DeepSeek-V4-Flash.

Table B: Comparison of query abstraction strategies for RAD.

MethodTU (↑)QP (↑)MP (↑)OP (↑)TO (↑)
All Abstraction91.2657.5694.9191.6691.43
No Abstraction (CEI Only)97.090.0097.2090.5193.68
Random Abstraction95.4746.4994.6990.3292.82
Ours96.4448.1196.6591.9694.15

Table C: Comparison of CEI deployment strategies.

MethodTU (↑)QP (↑)MP (↑)OP (↑)TO (↑)
Target-side CEI95.4748.1196.2491.4693.42
Ours96.4448.1196.6591.9694.15
  • Task Sharing (RAD). RAD protects the original query, with QP measuring sensitive information recoverable from the protected query. As shown in Table B, All Abstraction maximizes QP but reduces TU, while No Abstraction preserves TU without query protection. Our RAD adaptively preserves expertise-relevant information while protecting sensitive details, achieving a better privacy–utility balance.

  • Output Sharing (CEI). CEI enables complementary reasoning sharing without exchanging raw private memories, with MP measuring information leakage from peer summaries. As shown in Table C, Target-side CEI reweights only local memories after receiving unguided peer reasoning. In contrast, Peer-side CEI directly guides complementary peer reasoning, improving TU, MP, and TO while keeping raw memories local.

We have added these experiments in Paper’s 30th page.\textcolor{#5B9BD5}{\text{We have added these experiments in Paper's 30th page.}}

W2.1: Clarification on OP Aggregation

We sincerely thank you for highlighting this important point. To further examine the effect of privacy aggregation, we consider an alternative Mean-based OP alongside our Fact-weighted OP:

OPmean=QP+MP2\mathrm{OP}_{\mathrm{mean}} = \frac{\mathrm{QP}+\mathrm{MP}}{2}

OPours=1−∣F^Q∣+∣F^M∣∣FQ∣+∣FM∣\mathrm{OP}_{\mathrm{ours}} = 1 - \frac{|\widehat{F}^Q|+|\widehat{F}^M|}{|F^Q|+|F^M|}

The key distinction lies in the unit of privacy evaluation. Mean-based OP assigns equal weight to QP and MP, regardless of the number of private facts in each. In contrast, our OP assigns equal weight to each evaluated private fact rather than each privacy dimension.

This distinction is particularly relevant to multi-agent collaboration. A single query may involve multiple peer agents, each retrieving several private memories containing distinct atomic facts [1, 2]. Consequently, query and memory facts may differ substantially in quantity. While Mean-based OP always assigns 50% weight to each privacy dimension, our Fact-weighted OP accounts for the actual number of facts involved. We further compare the two aggregation strategies using Qwen3-30B and DeepSeek-V4-Flash on MedQA:

Table D: Performance under different OP aggregation strategies.

BackboneMethodTUQPMPMean OPMean TOOur OPOur TO
Qwen3-30BSingle Agent81.880.00100.0050.0062.0994.8687.89
Ours84.7950.2196.3773.2978.6293.0388.72
DeepSeek-V4-FlashSingle Agent95.470.00100.0050.0065.6393.0794.25
Ours96.4448.1196.6572.3882.7091.9694.15

Under Mean-based OP, our framework surpasses Single Agent by 16.53 and 17.07 TO points on Qwen3-30B and DeepSeek-V4-Flash, respectively. These larger gains arise because Mean-based OP equally weights QP and MP, eliminating the influence of unequal fact counts. Nevertheless, we retain Fact-weighted OP to reflect actual fact-level privacy exposure, under which our framework still outperforms Single Agent on Qwen3-30B and achieves comparable TO on DeepSeek-V4-Flash, with higher TU on both.

AuthorsOpenReview ↗

Response to Reviewer keYf (2/3)

W2.2: Effectiveness of Collaboration Mechanisms

We thank you for raising this important point regarding the benefits of collaboration.

First, collaboration improves task utility. As shown in Table E, introducing peer collaboration consistently improves TU across different query protection strategies.

Table E: Effect of collaboration on TU using DeepSeek-V4-Flash.

Privacy MechanismCollaboration StrategyMedQA TU (↑)
MNI-GateWithout Collaboration89.97
MNI-Gate+ Reasoning Share92.23 (+2.26)
PromptObfusWithout Collaboration92.56
PromptObfus+ Reasoning Share94.17 (+1.61)
RADWithout Collaboration94.50
RAD+ CEI (Ours)96.44 (+1.94)

Second, effective collaboration requires complementary expertise utilization. Unlike direct Reasoning Share, CEI explicitly prioritizes complementary peer evidence beyond the target's existing knowledge. For example, RAD + CEI achieves 96.44 TU on MedQA, outperforming RAD + PrivCollab-Agent (94.82), demonstrating the benefit of complementarity-aware reasoning.

Q2: Clarification on Reconstruction Evaluation

We sincerely thank you for highlighting this point. We clarify our reconstruction protocol from two aspects.

  • (1) Reconstruction Model and Prompts.

    Both Query and Memory Reconstruction use DeepSeek-V4-Flash as the attack model, with prompts tailored to their respective communication boundaries:

    • Query Reconstruction: The attacker reconstructs private query facts from the peer-visible protected query. The key prompt instructions are:

    You are a protocol-aware privacy reconstruction attacker auditing a protected medical QA query. ... provide exactly one best-guess replacement sentence or clause using its surrounding context ... Do not enumerate alternatives. Do not select or reveal the correct answer option...

    • Memory Reconstruction: The attacker reconstructs private-memory facts from the target-visible original query and peer reasoning summaries. The key prompt instructions are:

    You are a privacy reconstruction attacker. The user message contains exactly the task context known to a target client and the messages it received from two peers. ... Reconstruct every atomic, case-specific fact about those hidden peer memories that the target can state or reasonably infer.

  • (2) Entailment Judging.

    We use a fixed microsoft/deberta-base-mnli classifier to evaluate reconstruction success, with reconstructed statements as premises and source facts as hypotheses. A fact is recovered only when the argmax prediction is ENTAILMENT. This fixed criterion reduces evaluation variability and ensures consistent assessment across methods.

Q3: Evaluation Robustness Across Memory Partitions and Inference Runs

We thank you for suggesting additional uncertainty estimates. To assess robustness, we conduct additional experiments across random seeds and memory heterogeneity levels using DeepSeek-V4-Flash

  • (1) Robustness Across Random Seeds.

MedQA and LegalQA contain 103 and 133 test questions, respectively. With three agents each serving as the target, we obtain 309 and 399 target–question evaluations per run. We report results with 3 and 5 additional seeds (4 and 6 runs in total).

Table F: Performance across additional random seeds at α=0.5\alpha=0.5 (mean ± SD).

DatasetAdded SeedsMethodTU (↑)OP (↑)TO (↑)
MedQA3RAD95.31 ± 0.8195.04 ± 0.0395.17 ± 0.39
RAD + PrivCollab-Agent96.52 ± 1.4494.30 ± 0.1795.40 ± 0.79
Ours97.17 ± 0.6793.98 ± 0.0995.55 ± 0.33
5RAD95.42 ± 0.6695.04 ± 0.0395.23 ± 0.32
RAD + PrivCollab-Agent97.30 ± 1.2794.33 ± 0.1495.79 ± 0.69
Ours97.36 ± 0.6393.92 ± 0.1295.61 ± 0.28
LegalQA3RAD48.37 ± 2.4495.22 ± 0.2064.15 ± 2.15
RAD + PrivCollab-Agent46.49 ± 4.6793.54 ± 0.3562.11 ± 4.17
Ours49.94 ± 0.8791.04 ± 0.0664.50 ± 0.73
5RAD48.12 ± 2.4595.25 ± 0.1663.94 ± 2.16
RAD + PrivCollab-Agent48.08 ± 3.7793.56 ± 0.2863.52 ± 3.29
Ours49.87 ± 1.3890.93 ± 0.1864.42 ± 1.17

Our framework achieves the batter mean TU across all settings and the completive mean TO in three of four cases, with consistently lower reported variability in TU and TO. Although RAD + PrivCollab-Agent achieves slightly higher TO on MedQA with five additional seeds, its substantially larger standard deviation indicates less stable performance. These results highlight our framework's stronger task utility and greater robustness across random runs.

AuthorsOpenReview ↗

Response to Reviewer keYf (3/3)

Q3: Evaluation Robustness Across Memory Partitions and Inference Runs

  • (2) Robustness Across Memory Heterogeneity.

We further evaluate different Dirichlet parameters (α=0.1,0.5,1.0\alpha=0.1, 0.5, 1.0), using the original experimental results at α=0.5\alpha=0.5.

Table G: TU under different private-memory heterogeneity levels on MedQA.

Methodα=0.1\alpha=0.1α=0.5\alpha=0.5α=1.0\alpha=1.0
RAD93.5394.5092.56
RAD + PrivCollab-Agent93.8594.8293.53
Ours96.1296.4496.44

As α\alpha decreases from 0.5 to 0.1, stronger data heterogeneity limits each agent's knowledge coverage, causing larger TU drops for RAD and RAD + PrivCollab-Agent than for Ours. CEI mitigates this degradation by identifying complementary peer expertise beyond the target's knowledge. At α=0.5\alpha=0.5 and 1.01.0, broader knowledge coverage provides more stable expertise, while CEI suppresses redundant evidence, maintaining consistently high TU.

We have added these experiments in Paper’s 32th page.\textcolor{#5B9BD5}{\text{We have added these experiments in Paper's 32th page.}}

[1] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS, 2020.

[2] MASS-RAG: Multi-Agent Synthesis Retrieval-Augmented Generation. Findings of ACL, 2026.

All Ciations are added in revision.\textcolor{#5B9BD5}{\text{All Ciations are added in revision.}}

Reviewer_7CbjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies multi-agent collaboration strategies when different agents hold private knowledge, which leads to a trade-off between privacy exposure and task performance. Specifically, a target agent distributes partially abstracted task information to peer agents, collects processed evidence from peers, and then uses the aggregated information to derive the final answer for the task. To ensure privacy protection while maximizing the expertise utilization of peer agents, this paper proposes two modules: the RAD module compares the similarity of the raw task span and the replacement candidate to the target agent's private memory, and uses this expertise loss to decide which spans are disclosed to peers; the CEI module identifies the complementary expertise of peer agents relative to the target agent. Experiments on two benchmarks report improvement in task utility and privacy utility trade-off metric over baselines.

Critical strengths
  1. The paper models the two communication boundaries in multi-agent collaboration separately, i.e., the task information sent from the target agent to peers and the knowledge returned from peers to the target agent. This decomposition is clear and well illustrated, and it makes it easy to see which part of the pipeline each module is responsible for. The topic is also worthwhile, since collaboration between agents deployed in different organizations with private knowledge is a practical problem.

  2. RAD uses the target agent's own private memory as a training-free proxy for the utility of each sensitive span, by comparing the similarity of the raw span and its abstraction to the local memory. This signal only requires local information and no additional training. In addition, the per-query GMM avoids manually setting a global threshold.

  3. Both empirical and theoretical insights are provided to support the proposed framework.

  4. Privacy is evaluated on both communication boundaries rather than only on the outgoing query.

Critical weaknesses
  1. Potential privacy exposure. In Algorithm 1, each peer agent will conduct a step called "Score complementary peer evidence", which utilizes the top-K private memory of the target agent Rt(q)\mathcal{R}_t(q). If this memory is directly exchanged between the target agent and peer agents, could this expose the target agent’s private information (or only the embeddings are exchanged)? This potential privacy risk should be further clarified in the main paper.

  2. Experimental evaluation. For the non-collaborative settings, no query is sent to peers, so it is unclear how QP is defined: Single Agent is assigned QP = 0 (full exposure), while MNI-Gate without peers report non-zero QP. In addition, the TU of these rows differs from Single Agent (e.g., 69.90 for MNI-Gate vs. 81.88), which suggests that the target agent reasons on the protected query in this setting, whereas in the full framework the target keeps the original query. The evaluation protocol of these rows should be clarified.

  3. Lack of in-depth analysis. The experimental results require more detailed analysis. Although the proposed method improves TO (the trade-off metric) compared with the baselines, it performs noticeably worse on some other metrics. To better highlight the contributions of the proposed method, there could be a more thorough analysis of these results, explaining the underlying trade-offs and clarifying the advantages and limitations of the proposed method relative to the baselines.

  4. Missing baselines. Several important baselines are missing from the evaluation, including trivial references such as abstracting all information, retaining all information, and random disclosure at the same disclosure rate. Additionally, none of the paradigms criticized in Fig. 1, such as task decomposition and evidence exchange, are evaluated. These missing comparisons make it difficult to assess the effectiveness and advantages of the proposed method over existing approaches.

Some minor issues:

  1. For figure 4, the marker colors are not consistent with the legend.

  2. In section 5.2, "We vary the number of agents from 3 to 5 and observe performance across different collaboration scales, with TU ranging only from 83.25 to 83.50.", while TU is 84.79 with 3 agents.

  3. Also in section 5.2, "For Ours, K = 3 achieves the highest TO (87.9),", but this result cannot be found in Table 1 or Fig. 6. The only occurrence of this number appears in Fig. 5, which does not seem to correspond to the discussion in this paragraph.

Questions for authors
  1. How are the hyperparameters selected?

  2. How are the evaluation sets constructed? Reporting the size of the training split (private memory) and the test split for each dataset would also be helpful.

  3. How will the conclusion change when compared with some simple baselines, such as abstracting all sensitive spans, retaining all sensitive spans, and randomly disclosing the same number of spans as RAD?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 7Cbj (1/3)

W1: Clarification on Private Memory Exchange in CEI

We sincerely appreciate your careful observation. Importantly, CEI does not require exchanging raw private memories between agents. Specifically, as formulated in Sec. 4.2 (Eqs. 9–12), in which pairwise complementarity depends only on two query-relative geometric descriptors (θ\theta and u\mathbf{u}). Each agent independently computes these descriptors from its local memories. The target then shares only its geometric descriptors with the peer, allowing the peer to estimate complementarity without directly accessing the target's raw memories.

To further examine this design, we compare peer-side CEI with a target-side variant that avoids transmitting target-memory descriptors directly (Protecting memory more strictly). In the target-side variant, peers return geometric descriptors and unguided reasoning summaries, while the target locally computes CEI scores to reweight its own memories.

Table A: Comparison of CEI deployment strategies.

DatasetCEI PlacementTU (↑)QP (↑)MP (↑)OP (↑)TO (↑)
MedQATarget-side CEI95.4748.1196.2491.4693.42
Peer-side CEI (Ours)96.4448.1196.6591.9694.15
LegalQATarget-side CEI49.1220.9295.9891.2263.86
Peer-side CEI (Ours)49.6220.9295.5791.0364.23

As shown in Table A, peer-side CEI consistently improves TU and TO while maintaining high memory protection. Unlike target-side CEI, which reweights only target memories after receiving unguided peer summaries, peer-side CEI directly guides peer reasoning using complementary evidence. These results highlight peer-side CEI as a competitive design that preserves raw-memory locality while effectively leveraging complementary peer expertise. Thanks again for your suggestion and we added clarificationion in 5-6th page and experiment in 30th page.\textcolor{#5B9BD5}{\text{Thanks again for your suggestion and we added clarificationion in 5-6th page and experiment in 30th page.}}

W2: Clarification on Evaluation Settings and Communication Protocols

We sincerely appreciate your observation and apologize for the ambiguity in Table 1. Specifically, "Privacy Protecting (without peer)" and "Collaboration (with peer)" would be more accurately described as "Privacy Protection (without Collaboration)" and "Collaboration (without Privacy Protection)", respectively. We clarify these three settings below (also summarized in Table B).

  • (1) Privacy Protection (without Collaboration).

    This setting evaluates query protection independently, where each agent reasons on its received query, and QP is assessed through reconstruction attacks. Consider a scenario where Single Agent receives an unprotected query yet achieves both 100% QP and uncompromised task utility. This would unfairly disadvantage privacy-preserving methods that sacrifice task performance for query protection. Therefore, we assign Single Agent QP = 0 to indicate the absence of protection, while MP = 100% since no peer memories are exchanged. This explains why MNI-Gate achieves stronger QP but reduces TU from 81.88 to 69.90 due to information loss. This motivates RAD to protect sensitive information while preserving expertise-relevant cues for task utility.

  • (2) Collaboration (without Privacy Protection).

    This setting evaluates collaboration without query protection. Since peers receive unprotected queries, QP = 0, while potential memory leakage through peer reasoning results in MP < 100%. Reasoning Share improves TU from 81.88 to 82.20, but lacks explicit complementarity identification, motivating CEI to prioritize complementary peer evidence.

  • (3) Privacy Protection + Collaboration).

    This setting reflects, to some extent, realistic cross-organizational collaboration requiring both query protection and external expertise. For example, a hospital may share a privacy-protected patient query with other hospitals to obtain complementary clinical expertise without disclosing the patient's sensitive information. Accordingly, the target retains the original query for local reasoning, shares only its protected version with peers, and receives complementary reasoning summaries without directly exchanging raw private memories.

Table B: Communication protocols across three evaluation settings.*

SettingPhaseTarget AgentPeer Agent
Privacy Protection (without Collaboration)Task Sharing PhaseSend protected queryReceive protected query
Output Sharing Phase--
Collaboration (without Privacy Protection)Task Sharing PhaseSend raw query (+ descriptors)Receive raw query (+ descriptors)
Output Sharing PhaseReceive peer outputSend output
Privacy Protection + CollaborationTask Sharing PhaseSend protected query (+ descriptors)Receive protected query (+ descriptors)
Output Sharing PhaseReceive peer outputSend output
AuthorsOpenReview ↗

Response to Reviewer 7Cbj (2/3)

W3: In-depth Analysis of Privacy–Utility Trade-offs

We sincerely appreciate your recognition of our improvements in TO and your suggestion for a more detailed analysis.

  • (1) Regarding QP: Without collaboration, MNI-Gate achieves the highest protection through information minimization, but its restrictive disclosure limits task-relevant information, resulting in lower TU than RAD. Then, under collaboration, introducing Reasoning Share (MNI-Gate + Reasoning Share) allows the target to benefit from expert reasoning summaries, improving TU. However, MNI-Gate's restrictive disclosure limits these collaborative gains, making it difficult to offset the utility loss caused by information minimization. In contrast, our RAD preserves expertise-relevant information through adaptive disclosure, enabling more effective peer collaboration and ultimately achieving higher TU and TO.

  • (2) Regarding MP: Without query protecting, PrivCollab-Agent achieves strong memory protection through privacy-filtered claim-level sharing, but its limited use of complementary expertise leads to lower TU than CEI. This is also reflected in RAD + PrivCollab-Agent under query proecting: due to RAD's expertise-aware abstraction limits sensitive query exposure and reduces unnecessary private-memory disclosure, PrivCollab-Agent achieves stronger MP on Qwen3-30B. However, its conservative claim-level sharing still lacks complementarity-aware reasoning, limiting TU gains and ultimately yielding lower TO than our full framework.

  • (3) Regarding TU: our framework achieves the strongest performance within the joint privacy–collaboration setting. This improvement stems from two complementary aspects: RAD preserves task utility by retaining expertise-relevant information during query abstraction, while CEI further enhances TU by prioritizing complementary peer knowledge for collaborative reasoning.

To further analyze these trade-offs under different evaluation preferences, we introduce a weighted TO metric:

TOλ=(λTU+1−λOP)−1\mathrm{TO}_{\lambda} = \left(\frac{\lambda}{\mathrm{TU}} + \frac{1-\lambda}{\mathrm{OP}}\right)^{-1}

Here, λ=0.5\lambda=0.5 recovers the original TO, while λ>0.5\lambda>0.5 prioritizes task utility and λ<0.5\lambda<0.5 prioritizes privacy protection.

Table C: Weighted privacy–utility trade-off under different preferences (averaged across MedQA and LegalQA).

BackboneMethodλ=0.3\lambda=0.3λ=0.5\lambda=0.5λ=0.7\lambda=0.7
Qwen3-30BMNI-Gate + Reasoning Share78.3071.4866.27
PromptObfus + Reasoning Share77.9671.3866.34
RAD + PrivCollab-Agent80.3473.7768.65
Ours80.1974.6670.26
DeepSeek-V4-FlashMNI-Gate + Reasoning Share82.9878.3074.64
PromptObfus + Reasoning Share83.4978.8175.21
RAD + PrivCollab-Agent83.6679.1575.67
Ours83.0379.1976.26

Under privacy-oriented weighting (λ=0.3\lambda=0.3), our framework ranks second on Qwen3-30B and third on DeepSeek-V4-Flash, while the best-performing RAD + PrivCollab-Agent achieves higher TO through stronger QP and MP, benefiting in part from our RAD module's adaptive query protection. Under utility-oriented weighting (λ=0.7\lambda=0.7), our advantage becomes more pronounced, outperforming RAD + PrivCollab-Agent by 1.61 and 0.59 points, respectively. This further highlights CEI's effectiveness in leveraging complementary peer expertise.

We have added these results in Paper’s 30th page.\textcolor{#5B9BD5}{\text{We have added these results in Paper's 30th page.}}

W4&Q3: Additional Baselines for Disclosure Strategies

We sincerely thank you for suggesting additional baselines. We conduct further comparisons across three dimensions using DeepSeek-V4-Flash.

  • (1) Disclosure Strategies.

We compare RAD against all, none, and random abstraction (at the same disclosure rate).

Table D: Comparison of disclosure strategies on MedQA.

MethodTU (↑)OP (↑)TO (↑)
All Abstraction91.2691.6691.43
No Abstraction (CEI Only)97.0990.5193.68
Random Abstraction95.4790.3292.82
Ours96.4491.9694.15

All Abstraction provides stronger QP but sacrifices TU, while No Abstraction achieves higher TU without query protection. Our RAD adaptively preserves expertise-relevant information while protecting sensitive queries, achieving the highest TO.

Table E: Comparison of task-sharing paradigms using DeepSeek-V4-Flash.

DatasetMethodTask SharingTU (↑)OP (↑)TO (↑)
MedQAP2Skill + CEISub-task88.3594.2791.21
MNI-Gate + CEIFull-task90.2993.6691.94
OursFull-task96.4491.9694.15
LegalQAP2Skill + CEISub-task36.3494.7152.53
MNI-Gate + CEIFull-task49.1292.3964.14
OursFull-task49.6291.0364.23
AuthorsOpenReview ↗

Response to Reviewer 7Cbj (3/3)

W4&Q3: Additional Baselines for Disclosure Strategies

  • (2) Task-Sharing Paradigms.

We compare P2Skill + CEI (Sub-task Sharing), MNI-Gate + CEI (Full-task Sharing), and Ours (Full-task Sharing), keeping CEI fixed to isolate the impact of different task-sharing strategies. As shown in Tab.E, Sub-task Sharing provides stronger privacy protection but may lose essential task context, while MNI-Gate's restrictive disclosure limits task-relevant information. In contrast, RAD adaptively protects sensitive queries while preserving expertise-relevant semantics, enabling CEI to better exploit complementary peer knowledge and achieve the highest TU and TO on both datasets.

  • (3) Output-Sharing Paradigms.

We compare RAD + RAGentA (Answer Sharing), RAD + PrivCollab-Agent (Evidence Sharing), and Ours (Reasoning Sharing).

Table F: Comparison of output-sharing paradigms on MedQA and LegalQA using DeepSeek-V4-Flash.

DatasetMethodOutput SharingTU (↑)OP (↑)TO (↑)
MedQARAD + RAGentAAnswer94.1892.6093.38
RAD + PrivCollab-AgentEvidence94.8294.0494.43
OursReasoning96.4491.9694.15
LegalQARAD + RAGentAAnswer48.1292.7363.36
RAD + PrivCollab-AgentEvidence48.6293.0763.87
OursReasoning49.6291.0364.23

Answer Sharing provides limited reasoning information, restricting the utilization of complementary peer expertise. RAD + PrivCollab-Agent performs well on MedQA by combining our RAD's expertise-aware disclosure with conservative claim-level sharing, achieving stronger protection and slightly higher TO. However, its lack of complementarity-aware reasoning limits effective expertise utilization, leading to less consistent gains across datasets. On LegalQA, despite higher OP, its lower TU results in inferior TO. In contrast, our CEI consistently leverages complementary peer expertise, achieving higher TU on both datasets and the best average TO.

We have added these results in Paper’s 30th and 32nd pages.\textcolor{#5B9BD5}{\text{We have added these results in Paper's 30th and 32nd pages.}}

W5&W6&W7: Clarification on Presentation and Typographical Issues

We sincerely thank you for identifying these issues. And We have modified these mistake in Paper’s 8-9th page.\textcolor{#5B9BD5}{\text{And We have modified these mistake in Paper's 8-9th page.}}

Q1: Clarification on Hyperparameter Selection

We thank you for the question regarding hyperparameter selection. Importantly, our method does not rely on manually tuned decision thresholds. The parameter settings are summarized in the table below.

Table G: Summary of experimental configurations.

ParameterSettingRationale
GMM Components (RAD)2Binary preserve/abstract decisions
GMM Parameters (RAD)Automatically estimatedQuery-specific expertise-loss distribution
Numerical Constant ϵ\epsilon10−610^{-6}Numerical stability
Complementarity Scoring (CEI)Fixed rulesQuery-relative memory geometry
Retrieval Depth KK3Sensitivity analysis in Fig. 6
Number of Agents NN3Sensitivity analysis in Fig. 7 and Appendix G
Dirichlet α\alpha0.5Heterogeneity analysis in Fig. 10(b)

Q2: Clarification on Dataset Construction and Spliting

We thank you for the suggestion to clarify our dataset construction.

For MedQA, we construct a challenging evaluation subset primarily based on MedAgentsBench [1], supplemented with original MedQA test questions to maintain medical diversity. We select six categories from nine candidates to avoid sparse category-level evaluation. For LegalQA, we organize legal questions into six categories and retain their natural distribution to preserve legal-domain diversity.

For both datasets, the training splits (824 for MedQA and 783 for LegalQA) are used exclusively to construct private memories, without fine-tuning the LLMs. We partition these memories across three agents using Dirichlet allocation (α=0.5\alpha=0.5). Each agent serves as the target for every test question, yielding 309 and 399 target–question evaluations, respectively.

Furthermore, we have already investigated the impact of different private-memory heterogeneity levels in Fig. 10(b), demonstrating CEI's effectiveness across different memory partitions (α=0.1\alpha=0.1 and α=1.0\alpha=1.0). Under stronger specialization (α=0.1\alpha=0.1), CEI exploits distinct peer expertise through directional novelty and radial gain. Under less skewed distributions (α=1.0\alpha=1.0), CEI adaptively prioritizes complementary evidence while suppressing redundancy.

We have added these clarificationion in Paper’s 16th–18th pages for dataset construction and 19th page for the heterogeneity results.\textcolor{#5B9BD5}{\text{We have added these clarificationion in Paper's 16th–18th pages for dataset construction and 19th page for the heterogeneity results.}}

[1] Medagentsbench: Benchmarking thinking models and agent frameworks for complex medical reasoning. arXiv, 2025.

Reviewer_TM1XOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • The paper proposes a framework on how multiple agents should collaborate when they have access to different pieces of information without exposing sensitive information.
  • It proposes 2 methods: 1.RAD which decides which sensitive details in a query should be shared with other agents and which should be generalized based on the task in-hand. 2. Once the peer agent receives the privacy-filtered query from the target agent, CEI compares the peer agent's retrieved memories with the target agent's memeory and decides to share only complementary information which is not already present in the target agent's memory
  • The paper evaluates their approach on medical and legal QA datasets and finds improvement in task-performance and privacy-utility trade-off.
Critical strengths
  • The paper tackles an important problem of multi-agent collaboration without exposing unnecessary sensitive information.
  • The evaluation of multiple open-source and closed-source LLMs and legal and medical domain show performance improvement making it a promising approach
Critical weaknesses
  • If the peer agent and target agent memories are private, how does CEI assume that it can find complementary information?
  • The description of CEI and its implementation. The paper describes CEI as filtering out information present in target's memeory. However, the actual method ranks the peer's retrieved memories before the peer generates its response. There is no filter at the target agent.
Questions for authors
  • Can the authors clarify whether the improvement in the privacy-utility metric comes from better task performance or better privacy protection?
3 comments
AuthorsOpenReview ↗

Response to Reviewer TM1X (1/2)

W1: Clarification on Privacy-Preserving Estimation in CEI

We sincerely thank you for raising this important question and apologize for any confusion regarding CEI's privacy mechanism. Importantly, CEI does not require either agent to access the other's raw private memories. As formulated in Sec. 4.2 (Eqs. 9–12), pairwise complementarity depends only on two types of query-relative geometric descriptors:

\mathbf u= \frac{\hat{\mathbf m}-(\hat{\mathbf q}^{\top}\hat{\mathbf m})\hat{\mathbf q}} {\|\hat{\mathbf m}-(\hat{\mathbf q}^{\top}\hat{\mathbf m})\hat{\mathbf q}\|}.$$ **Each agent computes geometric descriptors locally, and the target shares only these descriptors with peers for complementarity estimation, without exchanging raw private memories. This design is motivated by the observation that semantic differences are reflected in embedding-space geometry [1–3].** CEI thus captures complementary knowledge through query-relative directional novelty and radial gain between target and peer memories. To further validate our peer-side design, we compare LLM Judge-based CEI and Geometry-based CEI on MedQA and LegalQA using DeepSeek-V4-Flash. *Table A: Comparison of complementarity estimation strategies.* | Dataset | Method | TU (↑) | QP (↑) | MP (↑) | OP (↑) | TO (↑) | |---|---|---:|---:|---:|---:|---:| | MedQA | LLM Judge-based CEI | 96.12 | 48.11 | 95.77 | 91.10 | 93.54 | | | Geometry-based CEI (Ours) | 96.44 | 48.11 | 96.65 | 91.96 | **94.15** | | LegalQA | LLM Judge-based CEI | 49.37 | 20.92 | 95.41 | 91.15 | 64.05 | | | Geometry-based CEI (Ours) | 49.62 | 20.92 | 95.57 | 91.03 | **64.23** | The results show that geometry-based CEI consistently improves both Task Utility(TU) and Memory Protection(MP) on MedQA/LegalQAP, leading to higher Trade-Off(TO). These findings demonstrate the dual benefits of geometric complementarity modeling in enhancing task utility and reducing private-memory exposure through returned reasoning summaries. $\textcolor{#5B9BD5}{\text{We have modified in Paper's 5th, 6th and 29th page for clarification on CEI privacy preserving and LLM Judge-based CEI comparision.}}$ ## W2: Discussion on Peer-side CEI Deployment We sincerely appreciate your thoughtful observation and apologize for any confusion regarding CEI's implementation. In fact, CEI operates at the peer side but doesn't need to filter redundant information at the target side. Due to heterogeneous expertise, the target primarily retrieves knowledge within its own expertise, potentially missing complementary information from peers. **To address this, our peer-side CEI uses target-memory geometric descriptors ($\theta$ and $\mathbf{u}$) to prioritize complementary peer memories before reasoning, without directly exchanging raw private memories.** Therefore, our peer-side design enables complementary evidence to directly guide peer reasoning. *Imagine instead deploying CEI at the target side: even with peer geometric descriptors, the target could identify which peer memories are complementary but could not directly reason over their underlying content.* This motivates our peer-side design: **target memory serves as a reference for redundancy, while peer memory provides complementary knowledge.** By combining locally accessible peer memories with CEI scores, the peer can generate complementary reasoning before sharing it with the target. **To further validate this design, we compare target-side and peer-side CEI deployment:** target-side CEI guides reasoning using target-accessible memories, whereas our peer-side CEI directly prioritizes complementary peer memories for reasoning. *Table B: Comparison of CEI deployment strategies.* | Dataset | CEI Placement | TU (↑) | QP (↑) | MP (↑) | OP (↑) | TO (↑) | |---|---|---:|---:|---:|---:|---:| | MedQA | Target-side CEI | 95.47 | 48.11 | 96.24 | 91.46 | 93.42 | | | Peer-side CEI (Ours) | 96.44 | 48.11 | 96.65 | 91.96 | **94.15** | | LegalQA | Target-side CEI | 49.12 | 20.92 | 95.98 | 91.22 | 63.86 | | | Peer-side CEI (Ours) | 49.62 | 20.92 | 95.57 | 91.03 | **64.23** | As shown in Table B, peer-side CEI consistently achieves higher TU and overall TO across both datasets, when facing the same protected query. These results support our design choice that applying CEI directly to peer memories allows complementary evidence to guide peer reasoning more effectively than applying CEI at the target side. $\textcolor{#5B9BD5}{\text{We have added in Paper's 30th page for experiment on CEI deployment.}}$ [1] Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arxiv, 2019. [2] SimCSE: Simple Contrastive Learning of Sentence Embeddings. EMNLP, 2021. [3] Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere. ICML, 2020. $\textcolor{#5B9BD5}{\text{All Ciations are added in revision.}}$
AuthorsOpenReview ↗

Response to Reviewer TM1X (2/2)

Q1: Discussion on the source of TO improvements

We sincerely appreciate your recognition of our improvements and your thoughtful question regarding the TO metric. As defined in Appendix D, TO equally weights TU and OP through their harmonic mean. Our improvements primarily arise from stronger task utility while maintaining substantial privacy protection. To further examine this trade-off under different preferences, we introduce a weighted TO metric:

TOλ=(λTU+1−λOP)−1\mathrm{TO}_{\lambda} = \left(\frac{\lambda}{\mathrm{TU}} + \frac{1-\lambda}{\mathrm{OP}}\right)^{-1}

Here, λ=0.5\lambda=0.5 recovers the original TO, while λ>0.5\lambda>0.5 prioritizes task utility and λ<0.5\lambda<0.5 prioritizes privacy protection.

Table C: Weighted privacy–utility trade-off under different preferences.

BackboneMethodλ=0.3\lambda=0.3λ=0.5\lambda=0.5λ=0.7\lambda=0.7
Qwen3-30BMNI-Gate + Reasoning Share78.3071.4866.27
PromptObfus + Reasoning Share77.9671.3866.34
RAD + PrivCollab-Agent80.3473.7768.65
Ours80.1974.6670.26
DeepSeek-V4-FlashMNI-Gate + Reasoning Share82.9878.3074.64
PromptObfus + Reasoning Share83.4978.8175.21
RAD + PrivCollab-Agent83.6679.1575.67
Ours83.0379.1976.26

Under privacy-oriented weighting (λ=0.3\lambda=0.3), our framework remains competitive, trailing the best-performing RAD + PrivCollab-Agent by only 0.15 and 0.63 points on Qwen3-30B and DeepSeek-V4-Flash, respectively. Notably, RAD + PrivCollab-Agent also incorporates our RAD module and achieves stronger protection through conservative claim-level sharing, but at the cost of lower task utility.

Under utility-oriented weighting (λ=0.7\lambda=0.7), our framework achieves the best performance on both backbones, outperforming RAD + PrivCollab-Agent by 1.61 and 0.59 points, respectively. These results highlight that our framework not only protects sensitive queries through RAD's expertise-aware abstraction but also substantially improves task utility by leveraging complementary peer expertise through CEI, achieving a favorable privacy–utility trade-off.

Dear Reviewers,

With less than one day remaining in the rebuttal discussion period, we truly hope to make the most of this valuable opportunity to communicate with you. We would be deeply grateful if you could kindly take another look at our responses and revised manuscript.

We fully understand your busy schedules and remain more than happy to address any remaining questions or concerns.

Thank you again for your valuable time, patience, and consideration.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Response and Revisions

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all three reviewers for your time, effort, and thoughtful feedback on our manuscript. We are particularly grateful to all three reviewers, 7Cbj, keYf, and TM1X, for recognizing the strengths of our work and providing valuable suggestions that have helped us further strengthen the manuscript.

To facilitate further assessment, we summarize the key concerns raised by the reviewers and our responses as follows:

  1. Privacy-preserving CEI and its deployment (TM1X: W1&W2, 7Cbj: W1, keYf: W1&Q1).

    We clarified the complete communication protocol, emphasizing that CEI operates on the peer side and exchanges query-relative geometric descriptors rather than raw private memories. Additional comparisons with target-side CEI and LLM Judge-based CEI further support our design choices.

  2. Privacy–utility trade-offs and evaluation protocols (TM1X: Q1, 7Cbj: W2&W3, keYf: W2).

    Our work aims to provide a more comprehensive evaluation of privacy protection in multi-agent collaboration by considering both outgoing query disclosure and returned peer information. We sincerely appreciate the reviewers' insightful comments, which helped us further clarify the evaluation protocols and better explain the privacy–utility trade-offs across both communication boundaries.

  3. Additional baselines and collaboration strategies (7Cbj: W4&Q3, keYf: Q3).

    We introduced All, None, and Random Abstraction baselines, comparisons of task-sharing and output-sharing strategies, additional multi-seed and knowledge-heterogeneity experiments. These experiments further demonstrate the complementary roles of RAD in preserving expertise-relevant task information and CEI in exploiting complementary peer knowledge.

All corresponding clarifications, additional experiments, and corrections have been incorporated into the revised manuscript. The major revisions and supporting experiments are summarized below.

ConcernRaised ByAdditional Experiments and ClarificationsLocation
CEI Privacy and DeploymentTM1X (W1&W2) ; 7Cbj (W1) ; keYf (W1&Q1)Privacy-preserving communication protocol; Geometry-based vs. LLM Judge-based CEI; Peer-side vs. Target-side CEIPages 5–6, 29–30
Privacy–Utility EvaluationTM1X (Q1) ; 7Cbj (W2&W3) ; keYf (W2)Evaluation protocol clarification; Mean-based OP; Weighted privacy–utility trade-offsPage 30
Additional Baselines and Collaboration Strategies7Cbj (W4&Q3) ; keYf (Q3)All/None/Random disclosure; Task-sharing and output-sharing comparisonsPages 30–32
Robustness and Reproducibility7Cbj (Q1&Q2) ; keYf (Q2&Q3)Multi-seed evaluation; Knowledge heterogeneity; Dataset and implementation detailsPages 16–19, 32

We are truly grateful to the Area Chair and all three reviewers for your thoughtful consideration and constructive insights. Their feedback has been invaluable in strengthening our work, and we sincerely hope that our revisions and additional experiments have addressed your concerns.

AuthorsOpenReview ↗

Follow-Up on Our Rebuttal Responses

Dear Reviewers,

We understand that reviewers may receive many OpenReview emails during the ICLR rebuttal period, and we sincerely apologize for adding another message. However, as the discussion period is coming to an end, we truly value this opportunity to communicate with such dedicated and thoughtful reviewers.

In response to your constructive feedback, we have conducted additional experiments and submitted a revised manuscript. We would be deeply grateful if you could revisit our responses when your schedule permits.

Once again, we apologize for the interruption and sincerely appreciate your time and understanding.

Area_Chair_zcpMOpenReview ↗

A Gentle Reminder: About One Day Left in the Discussion Period

Dear Reviewers,

This is a gentle reminder that approximately one day remains in the discussion period. Please review the authors' responses and other reviewers' comments, and engage in the discussion to clarify which concerns have been addressed and which remain unresolved.

Thank you for your time and effort.

Best,
AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.