acceptodds
Under review as a conference paper at ICLR 2027

How Much Can Tool Calls Reveal About You? Privacy Leakage from Tool-Call Traces

14 views

Abstract

Large language model (LLM)-based agents interact with external tools through protocols such as the Model Context Protocol (MCP), producing tool-call traces that may expose behavioral signals about users. In this work, we systematically study the risk of user-profile inference from MCP tool-call traces, where observers infer user attributes from protocol-valid tool interactions without access to user conversations. We introduce PrivacyTrace, a benchmark containing 1,000 synthetic user profiles and 4,000 agent trajectories across four domains, generated with real MCP tools from 25 servers. Experiments with four mainstream LLM attackers show that tool-call traces enable a 26.25% attack success rate for profile inference, compared with 17.97% from prior-based guessing. Further analysis reveals that tool arguments and returned results provide the main trace-grounded evidence, while linking traces across sessions further amplifies profiling risk. These findings demonstrate that MCP tool-call traces can act as effective behavioral side channels for user profiling, motivating the development of privacy-aware mechanisms for agent-tool interactions. Code and data are available at https://anonymous.4open.science/r/PrivacyTrace-04B7.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Agents, Agent Privacy, Privacy Inference Attacks, Tool-Call Traces, Model Context Protocol (MCP)

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_9TNrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose PrivacyTrace, which evaluates user profile inference from Model Context Protocol (MCP) tool-call traces without access to user dialogues. Using 1,000 synthetic profiles with 17 attributes, it generates 4,000 agent trajectories and 67,950 tool calls across four domains with 25 MCP servers. Observed traces increase Attack Success Rate (ASR) from a prior baseline (a attacker using only prior knowledge) of 17.97% to 26.25% (single-domain) and 32.77% (linked cross-domain). Tool arguments alone reach 27.71% ASR, slightly outperforming full traces.

Critical strengths
  1. The paper identifies a novel privacy side-channel exposing user attributes directly from legitimate MCP tool-call traces.
  2. The benchmark provides a statistically calibrated dataset of 1,000 synthetic profiles and 4,000 real MCP agent trajectories.
  3. The paper offers systematic ablation studies across trace channels, observer scopes, task domains, and multi-stage attacker protocols.
Critical weaknesses
  1. The prior-based baseline in the current evaluation is potentially weak and confounded. While the prior attacker only sees the 17-attribute schema, trace attackers additionally observe implicit context such as the task domain, tool types, call volume, and parameters. Thus, the comparison merely proves that the observation trace as a whole is informative, rather than isolating the marginal privacy leakage driven specifically by tool arguments or returned results given a specific domain or server context.
  2. The paper fails to justify how accurately its synthetic benchmark reflects real-world user behaviors and agent trajectories. Although the authors apply Iterative Proportional Fitting (IPF) to align demographic attributes with official aggregate statistics, demographic alignment does not guarantee behavioral realism (at individual conversation/trajectory level). Because both the scenarios and user interactions are fully driven by LLM simulators, they inevitably introduce synthetic artifacts—such as unnaturally structured user prompts and overly explicit task parameters.
  3. The paper fails to distinguish between genuine latent attribute inference from behavioral side channels and direct text extraction from explicit tool parameters. In Section 4.1, both the scenario planner and user simulator are given access to the ground-truth profile PjP_j to construct profile-conditioned tasks. Consequently, profile attributes or strong proxies are naturally converted into explicit task constraints and subsequently passed directly into tool invocation arguments (aia_i) by the agent.
  4. Including strict personal identifiers (e.g., Name, Phone, Email, Govt ID, Birthplace) in the target attribute set is unmotivated and distorts the evaluation metrics. According to Section 4.1 and Appendix D.2, the benchmark's scenario generator explicitly sanitizes all personal identifiers from the task inputs given to agents. Consequently, the near-zero prediction accuracy for these attributes (e.g., only 35 correct identifier predictions out of 16,000 slots) is an obvious artifact of prompt design rather than an empirical discovery about tool-trace privacy.
  5. The paper omits any explanation for a prominent anomaly in Figure 5(a): for several target attributes—most noticeably Relationship Status, but also Mental Health and Sex—the schema-only prior-based baseline significantly outperforms the full-trace attack.
Questions for authors
  1. How do you account for the Metadata + Sequence condition yielding an ASR of 14.53%, which is lower than the schema-only prior baseline of 17.97%? Does this indicate that prompt complexity and input formatting introduce noise that degrades attacker LLM reasoning?
  2. Could you provide domain-conditioned and server-conditioned prior baselines to isolate the true marginal privacy leakage offered specifically by tool arguments and returned results?
  3. How do you validate that the micro-level conversational interactions generated by your LLM simulator reflect genuine human behavior rather than synthetic artifacts like over-engineered user prompts? Have you conducted any empirical validation comparing your synthetic agent trajectories against a dataset of consented real-user traces or human evaluators?
  4. Is there any human-verified guarantee for the benchmark ground truth?
  5. How much trace leakage remains if simple heuristic client-side sanitizers are applied to redact explicit entities in tool arguments prior to transmission?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 9TNr (Part 1/2)

We thank Reviewer 9TNr for the thorough review and for recognizing the significance of this privacy channel, our calibrated benchmark, and our ablation studies. We have expanded our controls to separate prior knowledge from trace-content leakage, and all revisions have been incorporated into the revised manuscript with highlights.

W1 & Q1 & Q2: arguments, returned results, prompt framing, and server priors

To isolate the contribution of payload content, we hold tools, metadata, call count, and order fixed while adding arguments or returned results, evaluated via balanced accuracy (BA) [Brodersen et al., 2010]. The results are as follows:

ConditionBAGain over metadata + sequence (95% CI)
Metadata + sequence13.29%—
+ arguments29.68%+16.38 pp [15.74, 17.00]
+ returned results24.49%+11.19 pp [10.59, 11.78]
  • Domain and server priors (Q2): Because every profile executes one task per domain, each domain's attribute distribution equals the global population (domain-conditioned mode = population-based guessing at 16.45% BA). Using 5-fold profile splits, the server-conditioned mode reaches 16.39% BA versus 24.56% for the local-server attack (Appendix C.7). Thus, domain or server majority priors do not explain the trace gains.
  • Metadata + sequence drop (Q1): Neither condition suffers from generation, parsing, refusal, or truncation failures. On the 55 empty views with no visible calls, the trace-formatted prompt scores 4.74 pp lower in BA than the paired schema-only prompt (22.67% vs. 27.41%), showing that prompt framing shifts predictions even without trace content and accounts for part of the drop (Appendix C.9).

W2 & Q3: behavioral realism of the synthetic benchmark

To complement IPF population calibration, we conducted a blinded human naturalness evaluation (1 to 5 rubric) on 50 paired inputs comparing original public-benchmark queries against our simulated users' first turns (Appendix C.11). The results are as follows:

Public benchmarkPairsOriginal inputSimulated first turnDifference
Conversational Weather134.924.920.00
WANDS134.924.69−0.23
PubMedQA124.503.33−1.17
QASPER124.754.25−0.50
Overall504.784.32−0.46

Simulated first turns are nearly indistinguishable from original inputs on conversational weather and shopping tasks. The gap is larger on PubMedQA primarily because simulated users phrase requests as full conversational questions rather than terse retrieval queries.

AuthorsOpenReview ↗

Response to Reviewer 9TNr (Part 2/2)

W3 & W4: literal recovery, latent inference, and strict identifiers

We audited 500 correctly predicted slots across 484 views for exact ground-truth string presence in the trace. The results are as follows:

AttributeShare with the ground-truth string present
Citizenship187/200 (93.5%)
Religious belief21/200 (10.5%)
Non-null physical condition100/100 (100%)

While literal exposure is common for citizenship and physical condition, religious belief gains +18.51 pp in BA despite only 10.5% literal presence, showing semantic inference beyond exact copying. Similarly, in our two-stage experiment, conditioning only on previously inferred attributes (without traces) improves BA by +8.18 pp over schema-only guessing via cross-attribute correlations. For strict identifiers (W4), we included them to test whether prompt-level sanitization holds during execution, while our 10-attribute BA excludes them to avoid metric distortion. Of the 35 correct identifier predictions, only 2 names appear in plaintext (in BioMCP.think), whereas the rest have no literal match and reflect common-name collisions (Appendix C.10).

W5: the reversal on three attributes

Accounting for class imbalance via BA resolves the apparent ASR reversal on relationship status and mental condition, which was driven by schema-only guessing overwhelmingly predicting the majority class ("married" in 95.4% of views vs. 30.0% with traces; "None" in 100% vs. 84.4% with traces). Only sex remains lower under BA. The results are as follows:

AttributeSchema-only BAFull single-domain BADifference (95% CI)
Relationship status19.76%20.07%+0.31 pp [−0.89, 1.58]
Mental condition14.29%15.45%+1.17 pp [0.37, 2.07]
Sex50.16%46.23%−3.93 pp [−5.85, −2.05]

Q4: verification of profile ground truth and the semantic judge

An item-by-item audit of 100 sampled synthetic profiles found 99 fully consistent (one had a birthplace mismatch with its government ID). Separately, in a manual audit of 300 stratified judge decisions, GLM-5.2 agreed with human labels on 292 cases (Cohen's κ=0.94\kappa = 0.94; Appendix C.11).

Q5: leakage remaining after pre-transmission redaction

Stripping only the BioMCP.think reasoning field from the 100 traces where it is non-empty lowers BA from 41.07% to 31.73%, which still exceeds metadata + sequence alone by +12.05 pp (retaining ~56% of the argument gain; Appendix C.9). Thus, standard functional arguments still carry substantial profile leakage. Because live pre-transmission redaction of task-relevant entities (e.g., medical terms or destinations) can alter tool outputs and degrade utility, we highlight evaluating online privacy-utility trade-offs in Section 6.

Reference

Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M. (2010). The balanced accuracy and its posterior distribution. In Proceedings of the 20th International Conference on Pattern Recognition (pp. 3121–3124). IEEE.

We wanted to kindly follow up before the discussion period closes. In response to your review, we added domain and server-conditioned priors (16.39% vs. 24.56% local-server BA), controlled channel ablations, a blinded human naturalness evaluation, a literal-presence audit separating inference from copying, a BA analysis resolving the Figure 5(a) reversal, and human verification of ground truth and the judge. We would greatly appreciate your feedback.

Reviewer_peujOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether MCP tool traces act as a privacy side channel. Even without seeing the chat itself, passive observers like MCP servers or gateways can infer user attributes such as nationality, religion, and health status from tool arguments and returned values. The authors built PrivacyTrace, generating agent trajectories from synthetic users on MCP servers and found that tool arguments are the main source of leakage.

Critical strengths

The figures look good.

Critical weaknesses
  1. As a benchmark, the evaluation is too narrow. The attacks rely solely on the authors' own prompting technique and omit other standard baselines. Beyond testing four models on the one-shot single-domain setup, all other experiments evaluate only a single model, making the conclusions unconvincing.

  2. The threat model assumes an observer can see all tool calls and link them across sessions without seeing user dialogues. In reality, any entity that routes all calls and tracks persistent identities across sessions is typically the agent-hosting platform itself, which already has dialogue access. This leaves the global observer scenario without a realistic counterpart.

Questions for authors

Both the scenario planner and user simulator see the full sensitive profile and are prompted to create tasks that make sense for it. For example, health-related tasks are often directly built around a specific medical condition. Part of the measured ASR therefore reflects how heavily the generator bakes the profile into tasks rather than natural leakage from real user behavior. Could the authors include a control group with profile-independent tasks or tasks tied to only a subset of attributes to test whether leakage persists?

3 comments
AuthorsOpenReview ↗

Response to Reviewer peuj

We thank Reviewer peuj for praising the clarity and presentation of our figures. We have expanded the baseline and attacker comparisons, clarified the deployment assumptions behind the global observer, and added a profile-independent task control. All revisions have been incorporated into the revised manuscript with highlights.

W1: attack baselines and cross-model coverage

Because this tool-side privacy channel had not previously been quantified (as noted by Reviewer rv4F), prior attacks on agent disclosures or network metadata target different observations and cannot be directly carried over. We therefore added explicit no-trace baselines, including schema-only LLM guessing, population-based guessing from public margins, and a 5-fold server-conditioned mode, evaluated under balanced accuracy (BA) [Brodersen et al., 2010] (Appendix C.7). We also added Gemini-3-Flash to the evaluation across all three observer scopes. The results are as follows:

BaselineAverage BA
Schema-only16.05%
Population-based guessing16.45%
Server-conditioned mode16.39%
AttackerLocal-server BA [95% CI]Single-domain BA [95% CI]Linked four-domain BA [95% CI]
Gemini-3-Flash25.73% [25.27%, 26.19%]30.65% [30.05%, 31.22%]40.93% [39.73%, 42.08%]
GPT-5.5—30.64% [30.01%, 31.27%]—
DeepSeek-V4-Flash24.56% [24.14%, 24.97%]26.75% [26.21%, 27.27%]34.51% [33.12%, 35.84%]
MiniMax-M3—25.35% [24.79%, 25.92%]—

All tested models substantially outperform the no-trace baselines and exhibit the same consistent scaling across scopes: local-server < single-domain < linked four-domain.

W2: real-world counterparts of the tool-side observers

Our global observer models a separately operated MCP gateway: the agent host retains the conversation, while the gateway observes tool requests and responses and links sessions if a persistent user identifier is present. Existing gateways support this architecture:

  • Kong AI Gateway aggregates upstream MCP tools and logs request/response payloads alongside Consumer IDs, which enable cross-session linkage when mapped per user (Kong MCP aggregation; Kong MCP audit logs).
  • Azure API Management supports opt-in logging of MCP tool arguments/results and recording JWT claims or custom identifiers via trace policies (Azure MCP logging).

Crucially, our local-server threat requires no cross-session linkage: an individual server observing only calls routed to its own endpoint already achieves 24.56% BA (+8.16 pp over the server-conditioned mode).

Q1: the profile-independent task control

We agree that profile-conditioned tasks may make profile information especially relevant to the interaction. To examine this dependence, we fixed 400 tasks (100 per domain) using public datasets and resources (e.g., QASPER, PubMedQA, DailyMed, IANA) before assigning them to 100 profiles. Neither selection nor construction used the target profile, removing profile conditioning from task creation. The results are as follows:

ScopeSchema-only BAOne-shot attacker BAGain (95% CI)
Local-server22.77%24.43%+1.66 pp [0.66, 2.73]
Single-domain22.82%24.55%+1.72 pp [0.89, 2.60]
Linked four-domain22.79%25.28%+2.49 pp [1.18, 3.89]

Although generic tasks naturally yield smaller gains than profile-conditioned tasks, all confidence intervals remain strictly above zero, confirming that tool traces leak user attributes even when tasks are created independently of user profiles (Appendix C.8).

Reference

Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M. (2010). The balanced accuracy and its posterior distribution. In Proceedings of the 20th International Conference on Pattern Recognition (pp. 3121–3124). IEEE.

As the discussion period nears its end, we would appreciate your feedback on our response. We added three no-trace baselines, extended Gemini-3-Flash across all observer scopes, documented real MCP gateways (Kong, Azure APIM) matching the global-observer setting, and added the profile-independent task control you requested, where leakage remains significant. We would welcome the chance to address any remaining concerns.

Thank you for author's response.

I think W2 is solved. However, for W1 and Q1, there still exists very strong problems.

For W1, you only add 2 models. Actually this is not enough for a benchmark. For a benchmark paper, we sould consider different size model from different families, so that we can get some conclusion from the results. What's more, the attack method looks like also very simple.

For Q1, we can see that if remove the relationship of user profile during generation. The gain of your porposed method will decrease from ~10 to 1. It proves that the high leakage problem is from your data generaion's issue, not from a real world scenerio.

I will keep my score.

Reviewer_rv4FOpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies a privacy question about tool-using language model agents. When an agent works with external tools through the Model Context Protocol, it leaves behind tool-call traces. These traces contain tool names, call arguments, returned results, and call order. A tool server or a gateway can see these records even without seeing the user conversation or the agent reasoning. The paper asks how much an observer in this position can infer about the person behind the calls, for example religion, health conditions, income, or location.

To study this, the authors build PrivacyTrace, a benchmark of 1,000 synthetic user profiles and 4,000 agent trajectories. Each profile has 17 attributes with ground truth labels. The profiles are sampled and reweighted with iterative proportional fitting so that their margins match public population statistics from sources such as the United Nations, the World Bank, and the ILO across 26 countries. For each profile, the authors generate tasks in four domains (travel, health, shopping, and career learning) and run a real agent against 25 real MCP servers, producing 67,950 tool calls. Using real servers rather than simulated ones makes the traces close to what a real observer would collect.

The evaluation compares three inference protocols. A prior baseline predicts all attributes from the attribute schema alone. A one-shot attacker predicts all attributes from the trace. A two-stage attacker first picks the attributes it believes the trace supports, then fills in the rest from priors. Scoring uses deterministic rules per attribute, with a separate language model judging semantic cases. The main result is that trace access raises the average attack success rate from 17.97% to 26.25%. The authors then break this down along several axes. Call arguments and returned results carry most of the signal, while metadata and call order alone score below the prior baseline. Linking traces across the four domains raises the rate to 32.77%. Attributes such as citizenship, religion, and ethnicity are recovered often, while exact identifiers such as names, emails, phone numbers, and government IDs are almost never recovered (35 correct out of 16,000 slots). The two-stage attack nearly matches the one-shot attack, and conditioning on the trace-supported attributes improves completion of the remaining ones by 3.30 points over prior guessing. Results stay in a narrow band across four attacker models, so the effect does not appear tied to one model.

The contributions are threefold. First, the paper formalizes profile inference from tool-call traces as a privacy risk, separating it from prior work that asked what agents acquire or disclose. Second, it provides a benchmark with population-calibrated synthetic users and real tool executions, with code and data released. Third, it reports a systematic measurement across observer scopes, trace channels, domains, attributes, and attacker models.

The validation is fairly thorough for the setup used. The authors run a channel ablation, a paired comparison between server-level and domain-level views, a call-count analysis, a serialization check, and cost and failure accounting, and they repeat the main attack with four different models. The parts of the validation that carry the most weight in my reading are the paired scope comparison and the two-stage decomposition, since they try to separate what the trace adds from what priors already give. The parts that carry less weight are the absolute success rates, since the tasks are generated from the profiles themselves, the prior baseline is schema-only guessing rather than a calibrated population prior, and no confidence intervals or repeated runs are reported.

Critical strengths

Strength 1. The paper measures a privacy channel that exists in real deployments and had not been quantified before. MCP servers and gateways see tool names, arguments, returned results, and call order as part of normal operation, and nothing in the protocol stops them from keeping these records. Earlier work asked what agents acquire, what they disclose, or what network metadata gives away, but it did not ask what a server can infer about a persistent user profile from the semantic content of legitimate calls. This paper answers that question with 25 real MCP servers and 67,950 real tool executions instead of simulated APIs, so the traces look like what a real observer would actually collect. This matters for my assessment because it shows the risk is concrete and current rather than hypothetical, and it gives the community a measured phenomenon with released code and data that defense work can build on.

Strength 2. The experimental design works hard to separate what the trace adds from what an attacker already knows. Every trace condition is paired with a matched prior baseline under the same scoring rules. The channel ablation shows metadata and call order alone fall below the prior, which tells the reader where the signal actually lives. The server-to-domain comparison is done as a paired analysis, not only pooled averages. The two-stage attack decomposes the result and makes visible how much of the final accuracy comes from priors rather than the trace. The main attack is repeated with four different models and lands in a narrow band, so the effect is not an artifact of one attacker. I also checked the accounting slot by slot, including the 68,000-slot decomposition, the correctness overlap table, the call-count bins, and the cost figures, and everything is internally consistent. This matters because the central claim of the paper is a comparison, that traces beat priors, and the design makes that comparison credible even where the exact size of the gain can be debated.

Critical weaknesses

Weakness 1. The headline comparison rests on a weak prior baseline, so the size of the trace effect is not established. The main claim is that trace access raises the attack success rate from 17.97% to 26.25%. But the prior baseline only receives the attribute schema, and the paper's own released statistics suggest a guessing strategy built on public population margins would do far better with no trace at all. Mental condition is None for 81.4% of profiles, married is 68.8%, no occupation is 39%, the sex split is near even, and China accounts for 26.8% of citizenship. Averaging the modal values across the 17 attributes gives roughly 24% accuracy, which is close to the full trace rate. These margins come from public United Nations and World Bank tables, so a real attacker could use the same strategy. This is consequential because the paper's central claim is comparative. Without a calibrated prior baseline and per-attribute trace-over-prior deltas, the reader cannot tell how much of the 26.25% is caused by the trace and how much a strong guesser would get anyway. The fix is simple in principle, add an oracle marginal baseline and report the deltas, but until that is done the headline number is hard to interpret.

Weakness 2. The tasks are generated from the profiles, so the measured leakage is conditional on the benchmark design rather than on natural usage. The scenario planner receives the full profile and writes a task that is plausible for that profile, and the user simulator is privately conditioned on the profile during the conversation. The correlation between attributes and traces is therefore built in by construction. The benchmark shows what leaks when users run profile-relevant tasks, but it does not measure how often real tasks are profile-relevant, and there is no control condition with profile-independent tasks. The reported rates are best read as conditional quantities, close to an upper bound for this task distribution, and the abstract and conclusion do not convey this. This is consequential because the paper motivates the work as a risk to real users of MCP services, and the current design cannot say how large that risk is in practice. A control condition, or even a measured rate of how often each attribute was actually relevant to the task, would bound the gap between the benchmark and real deployments.

Questions for authors

Some of the questions I want to ask authors:

  1. Could you add a prior baseline that knows the population margins, given that these margins come from public statistics? Using the distributions in your own Figures 8 to 10 and Table 5, a majority-style guesser appears to reach roughly 24% with no trace at all. How large is the trace gain against this stronger prior, and can you report per-attribute trace-over-prior deltas?

  2. The age scoring rule accepts any interval that contains the true age. How often did each attacker answer with an interval, and how wide were those intervals on average? What does the age ASR become if interval width is capped, for example at 5 or 10 years?

  3. Metadata plus call order scores 14.53% and empty views score 13.9%, both below the 17.97% schema-only prior. What explains this drop, since these conditions observe strictly more than the prior baseline? Can you report parse failures, refusals, and null rates for each channel condition rather than only for the main conditions?

  4. The full trace scores 1.46 points below the arguments-only condition. How much of this gap comes from failed or truncated generations on long inputs, as opposed to information carried by the returned results?

  5. The linked four-domain views contain about four times as many calls as single-domain views, and Figure 4c shows accuracy rising with call count. Can you report a call-count-matched comparison, for example linked traces truncated to single-domain length, to separate the effect of linkage itself from volume?

  6. How often was each attribute actually relevant to the generated task? For instance, in what fraction of tasks did the planner draw on religion, health, or income when forming the scenario? Even an approximate audit would help readers judge how the measured rates transfer to natural usage, where many tasks may not touch the profile at all.

  7. What is the error rate of the GLM-5.2 semantic judge? Did you compare its verdicts against human judgments on a sample, and how much would the ASR values move if the judge errs by a few points in either direction?

  8. Some servers let the agent write reasoning into an argument, such as the BioMCP thought field. How much of the arguments-channel gain survives if such reasoning fields are excluded? Relatedly, where did the 34 correctly predicted names appear in the traces, given that the planner is instructed to exclude identifiers from tasks?

4 comments
AuthorsOpenReview ↗

Response to Reviewer rv4F (Part 1/2)

We thank Reviewer rv4F for the thoughtful review that recognizes the novelty of our problem formulation and the clarity of our experimental design. We agree that the trace-access comparison needs a stronger no-trace reference. All revisions and controls discussed below, including population-based guessing and reporting ASR alongside balanced accuracy, have been incorporated into the revised manuscript with highlights.

W1 & Q1: population-based guessing baseline and per-attribute results

Following your suggestion, we added population-based guessing (a fixed modal prediction per attribute from public population margins). Under 17-attribute ASR, this baseline reaches 25.62% (vs. 26.25% single-domain and 32.77% linked). Because ASR is dominated by majority classes, we also report balanced accuracy (BA) [Brodersen et al., 2010] across the 10 categorical attributes to weight common and rare values equally (Section 5.1). The results are as follows:

ConditionAverage BAGain over schema-only (95% CI)
Schema-only16.05%—
Population-based guessing16.45%—
Full single-domain trace26.75%+10.70 pp [10.11, 11.24]
Linked four-domain trace34.51%+18.46 pp [17.04, 19.81]
AttributeSchema-onlyPopulation-based guessingFull single-domainSingle-domain minus schema-only (pp, 95% CI)Single-domain minus population-based guessing (pp, 95% CI)
Citizenship4.77%4.76%54.56%+49.79 [47.73, 51.95]+49.80 [47.74, 51.96]
Ethnicity12.54%12.50%40.39%+27.85 [25.55, 30.14]+27.89 [25.59, 30.22]
Religious belief16.88%16.67%35.39%+18.51 [17.04, 20.03]+18.72 [17.28, 20.14]
Physical condition7.69%7.69%19.33%+11.64 [10.24, 13.10]+11.64 [10.24, 13.10]
Occupation0.09%4.35%3.74%+3.65 [2.74, 4.62]−0.61 [−1.51, 0.36]
Mental condition14.29%14.29%15.45%+1.17 [0.37, 2.07]+1.17 [0.37, 2.07]
Relationship status19.76%20.00%20.07%+0.31 [−0.89, 1.58]+0.07 [−1.04, 1.36]
Income level20.00%20.00%19.46%−0.54 [−1.49, 0.44]−0.54 [−1.49, 0.44]
Education level14.29%14.29%12.88%−1.41 [−2.28, −0.50]−1.41 [−2.28, −0.50]
Sex50.16%50.00%46.23%−3.93 [−5.85, −2.05]−3.77 [−5.27, −2.26]

The results show that tool traces substantially improve BA over population-based guessing, with larger gains from cross-domain linkage and the strongest attribute-level gains in citizenship, ethnicity, religious belief, and physical condition.

W2 & Q6: profile-conditioned tasks, natural usage, and an attribute-relevance audit

We agree that profile-conditioned tasks may make profile information especially relevant to the interaction. To examine this dependence, we fixed 400 tasks (100 per domain) using public datasets and resources (e.g., QASPER, PubMedQA, DailyMed, IANA) before assigning them to 100 profiles. Neither selection nor construction used the target profile, removing profile conditioning from task creation. The results are as follows:

ScopeSchema-only BAOne-shot attacker BAGain (95% CI)
Local-server22.77%24.43%+1.66 pp [0.66, 2.73]
Single-domain22.82%24.55%+1.72 pp [0.89, 2.60]
Linked four-domain22.79%25.28%+2.49 pp [1.18, 3.89]

Although generic tasks naturally yield smaller gains than profile-conditioned tasks, all confidence intervals remain strictly above zero, confirming that tool traces leak user attributes even when tasks are created independently of user profiles. Meanwhile, our audit of 400 original tasks (Q6) found explicit cues in 51.5% of tasks for health, 21.8% for income, and 1.3% for religion (Appendix C.8).

AuthorsOpenReview ↗

Response to Reviewer rv4F (Part 2/2)

Q2: age accuracy under an interval-width cap

We agree that accepting arbitrarily wide intervals can reward broad predictions. Interval predictions account for 34.03% of single-domain age predictions (mean width 12.12 years) and 22.50% of linked four-domain predictions (mean width 11.62 years), whereas schema-only guessing always outputs point estimates. We rescored predictions under 10-year and 5-year width caps (Appendix A.3). The results are as follows:

SettingOriginal rule ASR10-year cap ASR5-year cap ASR
Schema-only2.83%2.83%2.83%
Full single-domain trace15.50%10.60%4.95%
Linked four-domain trace20.30%17.90%13.00%

Q3 & Q4: channel-level drops and failure accounting

  • Metadata + sequence vs. schema-only (Q3): Neither condition suffers from generation, parsing, refusal, or truncation failures. On the 55 empty views with no visible calls, the trace-formatted prompt scores 4.74 pp lower in BA than the paired schema-only prompt (22.67% vs. 27.41%), showing that prompt framing shifts predictions even without trace content and accounts for part of the drop.
  • Full trace vs. arguments-only (Q4): Input-length failures do not explain this drop; excluding them still leaves a 2.87 pp BA gap. Rather, the effect of returned results is context-dependent: adding them to metadata + sequence raises BA by +11.20 pp, whereas adding them when arguments are already visible lowers BA by 2.93 pp. Thus, returned results carry useful signal alone, but bulky, task-generic retrieval payloads can dilute higher-signal cues already present in call arguments (Appendix C.9).

Q5: linked four-domain comparison at matched call volume

To separate cross-domain diversity from call volume, we compared paired 8-call views across 100 profiles (8 calls from one domain vs. 2 calls per domain across four domains). Linked views reach 51.75% BA versus 39.46% for single-domain views, confirming that cross-domain diversity rather than call volume alone drives the linkage advantage (Appendix C.9).

Q7: reliability of the GLM-5.2 semantic judge

In a manual audit of 300 stratified judge decisions, GLM-5.2 agreed with human labels on 292 cases (Cohen's κ=0.94\kappa = 0.94). Re-estimating ASR from human labels leaves the single-domain gain over schema-only virtually unchanged (+8.33 pp vs. +8.28 pp). Even under a conservative 3% worst-case opposing error budget on valid semantic rulings, a gain of at least +5.38 pp remains (Appendix C.11).

Q8: leakage remaining after removing the thought field

Stripping only the BioMCP.think reasoning field from the 100 traces where it is non-empty lowers BA from 41.07% to 31.73%, which still exceeds metadata + sequence alone by +12.05 pp (retaining ~56% of the argument gain; Appendix C.9). Separately, of the 35 correct strict-identifier predictions, only 2 names appear in plaintext (in BioMCP.think), whereas the rest have no literal match and reflect common-name collisions (Appendix C.10).

Reference

Brodersen, K. H., Ong, C. S., Stephan, K. E., & Buhmann, J. M. (2010). The balanced accuracy and its posterior distribution. In Proceedings of the 20th International Conference on Pattern Recognition (pp. 3121–3124). IEEE.

We wanted to kindly follow up on our response before the discussion period closes. Per your suggestions, we added a population-based guessing baseline with per-attribute deltas and CIs (BA gain of +10.70 pp single-domain, +18.46 pp linked), a profile-independent task control (gains remain above zero), and analyses for Q2–Q8. We would appreciate knowing whether these address your concerns, and are happy to clarify further.

Reviewer_rv4FOpenReview ↗

Thanks for the update

Thanks for running all of these extra checks and audits. I really appreciate the amount of work you put into the revision, and pieces like the 8-call linkage check, the judge audit, and the failure breakdown clear up several of my smaller questions.

That said, looking at the new numbers alongside the responses to the other reviewers, my two main concerns still stand, and the new results mostly confirm what I was worried about:

  1. Baseline and what is actually being inferred: Guessing the most common values from public stats already gets 25.62% ASR, basically matching the 26.25% single-domain headline number (though the abstract still leads with 26.25% vs. 17.97% before switching to 10-attribute BA). Switching to BA helps with class imbalance, but in the per-attribute table, 6 of the 10 attributes (sex, education, occupation, income, relationship status, and mostly mental health) show little to no gain, or even drop, compared to public stats. Nearly all of the gain comes from just citizenship, physical health, ethnicity, and religion. When I pair that with your literal-string audit for Reviewer 9TNr and the task audit, physical health (100%) and citizenship (93.5%) are almost always written straight into the tool calls as plain text, while religion cues show up in only 1.3% of tasks. That suggests the model is mainly reading explicit location or medical terms needed for the query and then guessing the common religion and ethnicity for that country, which is much simpler than the subtle side-channel story framed in the intro and Figure 1. Capping age intervals to 5 years also brings single-domain age accuracy down from 15.50% to 4.95%.

  2. Dependence on how tasks are generated: On the new profile-independent tasks, the single-domain BA gain drops from +10.70 points to +1.72 points (and even with the user simulator still knowing its profile during the chat). As Reviewer peuj also pointed out, this shows that most of the headline leakage comes from the planner building tasks around the profile rather than everyday tool use, and the main text still doesn't really reflect how small the effect gets outside profile-tailored tasks.

Because of this, I think the benchmark and main claims still overstate the practical risk, so I will keep my current score of 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.