acceptodds
Under review as a conference paper at ICLR 2027

AgentDoxx: Agentic Re-identification of Anonymized Text with Web Search

Abstract

As Large Language Models (LLMs) gain tool use capabilities such as web search, they can retrieve and cross-reference public information, creating privacy risks beyond memorization. One manifestation is re-identification: linking an anonymized interview transcript to a named individual. Yet without ground-truth identities, the coverage of such attacks and the protection offered by a defense cannot be reliably measured. We introduce AgentDOXX, an evaluation suite of 822 synthetic interview transcripts grounded in public information about real individuals with known identities. We evaluate fifteen configurations of open-weight and proprietary models, isolating the effect of web search, and analyze their search trajectories to distinguish retrieval-driven from parametric identifications. Ground-truth identities reveal that re-identification risk is distributed across an agent's execution: retrieval and parametric recall both contribute, with open-weight models identifying 15-28% of transcripts without search; identification succeeds in over 88% of cases once the target appears in a retrieved result; entity masking leaves at least one attacker successful on 85.3% of a stratified sample; and privacy instructions suppress naming but not retrieval, with configurations scoring 0% accuracy yet retrieving the subject in up to 62% of transcripts. We further show that observed attack trajectories can provide supervision for localizing identifying spans, offering a path toward attack-informed anonymization.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Re-identification, Privacy, Agentic AI, Anonymization, Large Language Models, Web Search

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LjHXOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes the task of re-identification from anonymized text using AI agents with access to web search.

Specifically, they construct a synthetic interview dataset based on real world identifiers from public identities. First, they source public individuals from Reddit and use LLMs to extract personal attributes. Next, they derive questions from Anthropic's workforce interviews and generate synthetic interviews while incorporating certain attributes and redacting direct identifiers.

They then evaluate 15 model configurations in re-identifying individuals from the interviews. For 4 models, they explicitly compare the success with and without web search enabled.

They find that enabling web search increases precision and recall across setups, and that confidence reported by the model (e.g., "very high") can serve as an effective calibration for high precision and scalable re-identification;

They then analyse the range of strategies in search and retrieval by model, and evaluate the effectiveness of a range of mitigation strategies. Importantly, they find that providing privacy instructions in the system prompt as a mitigation technique does not suffice, as agents may re-identify people in their traces/retrievals while complying in their final answer.

Critical strengths
  • The paper studies an important problem. Re-identification with auxiliary data used to be a very tedious and hard-to-evaluate engineering task that may or may not be possible, while AI agents enabled with web search redefine this landscape. This prompts a more thorough evaluation of agent's capabilities in this domain and of the risk and potential mitigations.
  • The paper is well written.
  • The paper is well positioned in related work.
  • The quality checks and validation of the data generation is quite rigorous and compelling.
  • The analysis of the search and retrieval strategies is rigorous and interesting.
  • Authors discuss mitigations strategies at length, and the insight that mitigation "requires auditing what the agent retrieves and assembles, not only what it states" is useful for future work.
Critical weaknesses
  • In a way I like the point about parametric knowledge being a re-identification risk, but as far as I understand it, this may be an artifact of how the data has been constructed, i.e., based on public identities. While the point remains valid for the data at hand, I wonder how generalizable these findings would be for individuals not exposing their identity on Reddit. The paper would benefit from making related claims a bit more nuanced.
  • Generally, the data considered in this work remains synthetic and based on a biased set of individuals, so while I believe it would guide progress in this field, their results might not be very generalizable. Authors acknowledge this in the limitations, but it remains a weakness that (currently) refrains me from voting for clear accept.
  • Small point, but I feel like Section 4.1 could start with an overview of the main results rather than diving directly into the self-reported confidence.
  • Maybe in general, the results section would read more easily if the authors would refer to the results in the tables or figures. This made Section 4.3 tough to digest.
  • Maybe the following pieces of related work might be relevant to cite:

Annamalai, M. S. M. S., De Cristofaro, E., & Kairouz, P. (2026). CLIOPATRA: Extracting Private Information from LLM Insights.

Krčo, N., Yao, Z., Meeus, M., & de Montjoye, Y. A. (2026). Rat-bench: A comprehensive benchmark for text anonymization.

Questions for authors
  • Authors mention that different agents might use different strategies for re-identification, and have a different calibration of confidence. How could an attacker combine different model's re-identification attempts to come to a better score? This seems the most realistic threat model to me.

  • Could authors provide a more detailed comparison between their contributions and the ones from Lermen et al. (2026)? I don't think there is a novelty issue, but I'd like to make sure I properly understand the differences.

4 comments
AuthorsOpenReview ↗

Part A: W1 (Parametric knowledge and public identities), W2 (Synthetic data bias and generalizability), W3, W4, W5 (Clarity and organization)

We thank the reviewer for the detailed feedback. We address each point below:

W1 (Parametric knowledge and public identities)

AGENTDOXX uses individuals with publicly verifiable identities to enable reliable measurement of re-identification outcomes. We exclude r/IAmA search results from the agents' retrieved evidence, so successful identification does not depend on directly retrieving the original identity-disclosing Reddit posts. Our evaluation allows us to investigate how model knowledge and web retrieval contribute to identification.

To further characterize the role of public identity information, we added occupation-level analyses across 822 individuals spanning 183 occupation categories, revealing substantial variation in re-identification risk within this population. Professors have a mean high-confidence recall of 0.51, compared with 0.06 for actors. This finding highlights how distinctive combinations of publicly searchable attributes, such as institutional affiliation, research area, and publications, shape re-identification risk beyond public prominence alone.

Relevant sections: Agentic Search and Retrieval Strategies (Section 4.2); Occupation-Level Analyses (Appendix C.3, C.4).

W2 (Synthetic data bias and generalizability)

We examined all 1250 real Anthropic Interviewer transcripts and identified 43 verified matches (3.4%), providing evidence of re-identification risks in authentic anonymized conversations. To evaluate these risks systematically, we constructed AGENTDOXX with known ground-truth identities, enabling reliable measurement of re-identification outcomes. We also assessed the realism of our synthetic interviews, our generation procedure incorporates identifying attributes naturally across conversational turns rather than forcing all source attributes into the transcripts. In our human study, 30 evaluators made 300 judgments and achieved 49.3% overall accuracy in distinguishing synthetic from real interviews, close to the 50% chance baseline.

AGENTDOXX is designed to evaluate re-identification mechanisms rather than estimate population-level risk. Its controlled setting allows us to compare 15 model configurations, isolate the contributions of web search and reasoning effort, analyze retrieval trajectories, and evaluate mitigation effectiveness. While the synthetic data and selected population limit the generalizability of measured success rates, the findings establish the practical relevance of the threat, and our controlled evaluation provides empirical insights into the mechanisms underlying re-identification.

Relevant sections: Motivating Study (Appendix A); Synthetic Data Validation (Section 3.2); Re-identification Experiments (Section 4).

W3, W4, W5 (Clarity and organization)

We revised Section 4.1 to improve readability and make the supporting evidence easier to locate. Throughout the results section, we also strengthened references to figures and tables, and we expanded Appendix C with supporting analyses and visualizations.

We appreciate the suggestions of CLIOPATRA and Rat-bench and will review these works and cite them where appropriate in a subsequent revision.

AuthorsOpenReview ↗

Part B: Q1 (Combining multiple models for re-identification), Q2 (Comparison with Lermen et al., 2026)

Q1 (Combining multiple models for re-identification)

We analyzed the union of correct identifications across all 15 model configurations, which covers 62.4% of interviews, compared with 48.3% for the strongest individual configuration. This shows substantial complementary identification capabilities across models and highlights how re-identification risks could be further amplified when multiple LLM agents leverage tools such as web search.

Q2 (Comparison with Lermen et al., 2026)

Lermen et al. (2026) demonstrate the feasibility and scalability of LLM-based deanonymization through their Extract-Search-Reason-Calibrate pipeline, including cross-platform identity matching and re-identification of 9 out of 33 scientists from the Anthropic Interviewer dataset.

AGENTDOXX addresses the complementary research question: how web search amplifies re-identification risk, what drives re-identification, and how effectively different defenses mitigate these risks. Our contributions differ in three key aspects:

  1. Controlled evaluation: We construct a known-identity evaluation suite of multi-round anonymized interview transcripts, enabling systematic analysis of re-identification rate, search behavior, and defenses across model configurations.
  2. Mechanism analysis: Through search/no-search and reasoning-effort ablations and retrieval-trajectory analysis, we examine the roles of parametric knowledge and web retrieval, including whether search discovers an identity or corroborates an existing candidate.
  3. Mitigation evaluation: We evaluate text-level anonymization and agent-level privacy safeguards, revealing how identifying information can remain exposed during retrieval even when final responses withhold identities. We further explore using attack trajectories to localize identifying spans for more targeted anonymization.

While Lermen et al. establish the practical feasibility and scalability of LLM-based deanonymization, AGENTDOXX systematically evaluates the mechanisms driving re-identification and the effectiveness of privacy protections across different stages of agent execution.


We hope these clarifications address the reviewer's concerns and thank the reviewer for the valuable feedback. We are more than happy to provide additional details if any further questions remain.

Reviewer_LjHXOpenReview ↗

Response to rebuttal

Many thanks for the detailed response, which answers most of my concerns. I keep my current (positive) score.

Thank you for your reply! We're happy that our responses have been helpful in addressing your concerns.

Reviewer_fgxsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper introduces AgentDoxx - a collection of 822 interview transcripts generated by GPT-5.4-mini from attributes about real people, who were found through Reddit posts and the personal pages linked in them, designed to address a major gap in evaluation of re-identification performance for anonymized interview transcripts by LLM agents that can search the web. The benchmark is proven to show high level of authenticity, supported by IRB-approved human study, where real people struggle to differentiate synthetic interviews from real ones. The work evaluates 15 major LLM releases and provides deep dive into feature-level impact on results, like reasoning and search levels. Using the order in which a name first appears in queries and in search results, they distinguish identifications that come from the model’s own knowledge from those found through search.

The evaluation provides several major insights. Open-weight models identify around 15-30% of transcripts without any search; with search, most failures come from never retrieving the right person: once the target appears in a result, the model names them correctly in about 90% of cases. The authors then test a few defenses on a subset of 143 transcripts. NER masking, paraphrasing and replacing attributes with false values all leave a large share of transcripts identifiable. A privacy instruction in the system prompt cuts the number of correct answers the most, but in many cases the model had still retrieved the person’s name during search or described them in its answer. Lastly, Qwen-2.5-7B was finetuned on spans labeled from attack trajectories to predict what to redact, and report that redacting these spans works better than NER.

Critical strengths

S1. The motivation of the work is clear and justified with the well-written presentation of AgentDoxx workflow. I really enjoyed the visual aspect of result/findings presentation. The idea is timely and has a degree of novelty, targeting a gap in re-identification task evaluation, presenting a benchmarks with available ground truth personas.

S2. The level of evaluation scope is impressive - all major strong models (both open and closed) are covered with ablations on reasoning mode and tool availability (search effort) impact. The submission quality is also supported by IRB-approved human study, where human signal is vital to validate authenticity, as the users reported guessing accuracy around 50% (random guess) in human vs AI classification task.

S3. The work conducts a thorough study into search trajectories and various defensive method impact, including privacy-preserving instructions, reducing the privacy violation behavior of models - there are genuinely interesting insights into how models can achieve better re-identification capabilities.

Critical weaknesses

W1. No control over attribute leakage in synthetic interviews - compared to related work (SynthPAI), AgentDoxx prioritizes human conversational authenticity over (non-)explicit attribute disclosure or hinting. These two hurt each other, but without such ground truth labels about level of "guessing" difficulty the evaluation lacks the coverage of studying the relationship between reasoning/search effort and verified effort required to infer something about the person.

W2. Memorization and population diversity gap - Reddit threads and information about notable people are publicly available are likely to get into pretraining data of all major LLMs. The model can be good at this task because they recognized the information from already seen data, even if presented in another synthetic setting. Notable people are introduce occupation distribution shift and potentially for some features as well, the data is likely to be easily searched for them, which makes it easy for a model with web search tool to do.

W3. The work covers weak baselines, while missing out on much stronger anonymization methods. Due to W1, the generator was told not to output institution names, and person names were already redacted, so NER impact is much smaller than usual with edits impacting only <1% of characters. The related strong standard practices in the field like FgAA (LLM-based adversarial anonymization) and IncogniText are mentioned, but not actually used. It would be very valuable to have those included, ideally with a search-enabled adversary in the loop.

W4. I observe some reproducibility gaps - the study does not report variance of results, but mentions the noisy runs. Moreover, we don't know the exact inference setup and parameters or cost estimations (API type, temperature, different reasoning levels causing diverse costs). Another question missing an answer is whether inference budget is directly impacting the performance.

Questions for authors

Q1. Why was Qwen 2.5 7B selected for fine-tuning? I can imagine that the training gains can be lower if one uses originally more capable model.

Q2. Are there any statistics available on model refusal without the specific instructions to compare with Table 3 in Appendix? The impact of having these is not fully clear to me.

Q3. I appreciate the presence of IRB-approved human study. One concern I have is that the study scope is pretty narrow, only checking distinguishability, but not the ability of humans to re-identify personas (like it was done in Beyond Memorization or SynthPAI) - did you try to compare yourself against the LLMs on this task? Human baseline would be very insightful for this task, since the authenticity of synthetic interviews has been already confirmed.

Q4. Can you try to address W3 or at least provide some inputs on those methods on smaller subsets? NER really looks uninformative here, unfortunately.

Major (addressing these would help me raise my score, higher priority): W1/W3+Q4 are main reasons behind lowering the score. I understand that these are compute-intensive tasks and not realistic to ask for in rebuttal period, however, they really introduce evaluation quality gap compared to some other existing methods; W4/Q2/Q3 can be quick wins here to get 1-point increase, but still short of confident acceptance level due to concerns mentioned above.

Minor (does not affect my score, only personal recommendations): Figure 2 does not have a title and Figure 2b is not readable; figure 1 has o4-mini model visualized, but not GPT-5.4-mini which was actually used (if I understood it correctly).

UPD minor edit: fixing grammar typos

6 comments
AuthorsOpenReview ↗

Part A: W1 (Attribute leakage control), W2 (Memorization and population diversity)

We thank the reviewer for the detailed feedback. We conducted additional experiments and analyses to address the major concerns and discussed the remaining minor issues as detailed below.

W1 (Attribute leakage control), W2 (Memorization and population diversity)

We quantified attribute exposure in our synthetic interviews and expanded our occupation-level analyses to examine how identifying information and re-identification risk vary across individuals. Our attribute coverage analysis shows that 30.8% of extracted attributes are explicitly mentioned, 21.6% indirectly conveyed, and 47.6% absent, demonstrating that the interviews do not simply reproduce all source attributes.

We further analyzed 822 individuals across 183 occupation categories. The new analyses characterize differences in attribute coverage (Figure 4) and reveal substantial variation in re-identification risk (Figure 5). For example, professors achieve a mean high-confidence recall of 0.51, compared with 0.06 for actors. This finding highlights how distinctive combinations of publicly searchable attributes, such as institutional affiliations, research areas, and publications, can influence identification beyond public prominence alone. We also conducted search/no-search and reasoning-effort ablations, complemented by retrieval-trajectory analysis, to examine how model capabilities and external retrieval contribute to identification.

Relevant sections: Synthetic Data Validation (Section 3.2); Re-identification Analysis (Section 4.1, 4.2); Occupation-Level Analyses (Appendix C.3, C.4).

AuthorsOpenReview ↗

Part B: W3 + Q4 (Additional anonymization baselines)

W3 + Q4 (Additional anonymization baselines)

To address this concern, we conducted additional experiments with FGAA and IncogniText, including search-enabled variants of both methods, and evaluated them under the same re-identification experiments setup as our existing mitigation methods.

We reproduced and adapted both methods for our conversational transcripts. For FGAA, we followed the authors' released code and verified our implementation against the original functions. For IncogniText, we followed the published algorithm and prompts from the paper, using the configuration in Table 2 (row 3) of the Incognitext paper and documenting the necessary implementation choices. We preserved their core anonymization mechanisms while adapting the input format and attribute handling to our data. Both methods use GPT-5.4 Mini with default reasoning effort, consistent with our existing anonymization settings. We also incorporated web search into the adversarial feedback loop to evaluate retrieval-informed anonymization (enabled web search for the adversary responsible for identifying potentially revealing attributes, while keeping the anonymizer and other components unchanged).

ModelBaseNERNER+paraNoiseFGAAFGAA (w/ search-enabled adversary)IncogniTextIncogniText (w/ search-enabled adversary)
DeepSeek V4 Flash90.283.965.735.746.945.549.745.5
Gemini 3.1 Pro Preview96.581.865.739.245.547.650.345.5
Gemini 3.7 Flash97.281.162.944.145.546.953.147.6
GLM 5.3 Flash97.279.064.351.736.436.440.637.1
GPT-4.1 Mini (+search)42.026.618.915.45.66.322.420.3
GPT-4.1 Mini (no search)11.25.64.95.64.22.17.04.9
GPT-5.6 Terra (none)86.069.246.934.328.727.344.137.8
GPT-5.6 Terra (xhigh)95.179.065.046.248.345.551.046.9
GPT-5 Mini (high)84.668.547.639.923.124.545.538.5
GPT-5 Mini (minimal)61.549.025.227.38.411.228.024.5
Kimi K3 Max97.285.365.056.647.646.954.546.9
Llama 4 Maverick42.735.018.214.74.94.228.022.4
Mean75.162.045.934.228.728.739.534.8
Changed chars (%)00.421.500.852.702.835.325.99

The new results show that FGAA reduces mean re-identification accuracy from 75.1% to 28.7%, and IncogniText to 39.5%, compared with 62.0% for NER and 45.9% for NER+paraphrasing. Adding search-enabled feedback further reduces IncogniText's accuracy to 34.8%, while FGAA remains at 28.7%. We also measured the extent of text modification. FGAA modifies 2.70% of characters (2.83% with search), compared with 5.32% for IncogniText (5.99% with search). These results reveal differences in anonymization effectiveness and editing costs, while showing that substantial re-identification risk remains even under stronger anonymization methods.

Additionally, following the deprecation of Gemini 3.7 Flash on October 8, 2026 (https://discuss.ai.google.dev/t/gemini-3-7-flash-requests-are-answered-as-gemini-3-8-flash/187817) after the submission deadline, we used Gemini 3.8 Flash for these additional experiments and evaluation. Results for this configuration are therefore not directly comparable to those obtained with Gemini 3.7 Flash in the original submission.

AuthorsOpenReview ↗

Part C: W4 (Reproducibility, inference settings, and computational cost), Q1 (Choice of Qwen-2.5-7B for fine-tuning)

W4 (Reproducibility, inference settings, and computational cost)

We expanded our reporting of inference configurations, reasoning effort, and web search settings, including search limits and retrieval depth. We also added a per-configuration breakdown of input, reasoning, and output tokens, interaction rounds, and cache usage to support reproducibility and computational-cost analysis (Appendix C.7).

Our reasoning-effort ablations directly examine the effect of inference budget on re-identification performance. For GPT-5 Mini, increasing reasoning effort improves high-confidence precision from 44.2% to 69.2% and recall from 11.1% to 29.6%. For GPT-5.6 Terra, precision remains approximately 94% while recall increases from 29.3% to 36.0%. Our token analysis further shows that enabling web search increases input-token consumption through retrieved content, revealing how retrieval access changes computational resource usage (between external information retrieval and internal computation).

Each AGENTDOXX configuration was evaluated once per transcript. The discussion of noisy runs in Appendix A concerns only our separate motivating study on real Anthropic interviews, where repeated verification was used because ground-truth identities were unavailable.

Relevant sections: Inference Details and Computational Cost (Appendix C.7); Re-identification Experiments and Analysis (Section 4.1, 4.2); Motivation Study (Appendix A).

Q1 (Choice of Qwen-2.5-7B for fine-tuning)

We selected Qwen-2.5-7B to explore whether a relatively small model could learn to identify privacy-revealing attributes from the search and reasoning trajectories of more capable LLM agents. These trajectories show how agents connect attributes distributed across conversational turns and use them to guide identity searches, which provides supervision beyond conventional explicit-identifier detection.

Our initial results show that the fine-tuned model reduces the proportion of characters flagged as identifying from 8.22% to 2.66%, while improving recall of the analyzer's selected spans from 40% to 52%. Redacting the predicted spans reduces mean re-identification accuracy to 35.1%, compared with 62.0% for NER, demonstrating the potential of trajectory-informed supervision for identifying privacy-sensitive content (Section 4.3.1). We present this as an initial proof of concept for transferring agent-derived behaviors to a smaller model, with systematic comparisons across model capabilities left for future work.

Reviewer_kdGAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether LLM agents with web search can re-identify people from anonymized interview transcripts. Since real released interviews have no ground-truth identities, the authors build AgentDoxx, a set of synthetic interviews based on real people with known identities. They evaluate 15 model configurations, separate identifications that come from search from those that come from the model's own knowledge, and test several defenses.

Critical strengths
  1. The paper studies an important and timely privacy problem. Building a benchmark with known identities is a reasonable way to measure both attacks and defenses, and separating identifications that come from search from those that come from model knowledge is informative.

  2. The defense results are useful. Entity masking leaves most of the risk, and privacy instructions change what the model says more than what it retrieves.

Critical weaknesses
  1. My main concern is whether this benchmark reflects the risk in real anonymized text. On the real Anthropic interviews, the authors confirm only a small fraction of matches, but on the synthetic transcripts the strongest agents find close to half. The settings differ, but the gap makes me wonder whether it comes from the people chosen or from how the interviews are generated. I think the synthetic data could be used better here, for example by controlling which attributes are shared or by adding distractor details.

  2. It is hard to tell what the accuracy measures. If most people in the benchmark are public figures, some correct answers may come from memorized training data rather than from reasoning or search. Several models already find many of them without any search. The no-search runs show how much comes from search, but not whether the remaining correct answers come from memory or reasoning. If the paper aims to study the role of search and reasoning, I would suggest focusing on people who are unlikely to appear in the training data, such as those whose posts came after the model's training cutoff.

Questions for authors
  1. Is the interview format itself the focus of this paper, or just a way to build the benchmark? If it is the focus, I think the title and conclusions should be narrowed, since all transcripts follow one topic and one set of questions. If it is just a means, why not use redacted source text instead of synthetic interviews?

  2. Figure 2 seems to be missing its main caption. Also, recall in Figure 2a is defined as the share of transcripts with a high-confidence label. Does the candidate also need to be correct?

  3. How many runs were used for each configuration? The appendix says single runs are noisy, but the main results do not report any variance.

  4. I am not sure the privacy instruction should be called a defense, since an attacker controls their own prompt. It may be clearer to present it as a safeguard for deployed agents and keep it separate from text anonymization.

5 comments
AuthorsOpenReview ↗

Part A: W1 (Gap between synthetic and real interviews), W2 (Memorization, reasoning and web search)

We thank the reviewer for the detailed feedback. We address each point below:

W1 (Gap between synthetic and real interviews)

The 43 verified matches (3.4%) from 1250 real Anthropic interviews represent cases confirmed through our multi-round verification protocol, rather than the full re-identification success rate, which cannot be measured without ground-truth identities. AGENTDOXX addresses this measurement gap by constructing 822 interviews grounded in known identities, enabling reliable evaluation of re-identification success and controlled comparisons of model capabilities, web search, and mitigations.

We further evaluated the realism and identifying information in these interviews. Our human study found that synthetic interviews were difficult to distinguish from real ones (49.3% accuracy). Rather than forcing source attributes into the transcripts, our generation procedure prioritizes natural conversational disclosure, 47.6% of extracted attributes are absent, 30.8% explicitly mentioned, and 21.6% indirectly conveyed. We also tested how misleading attributes affect identification by replacing identifying spans with plausible but incorrect values. This reduced mean re-identification accuracy from 75.1% to 34.2%, while revealing that some reasoning-enabled agents recognize contradictions and continue identifying individuals using other evidence. These analyses characterize attribute exposure and the effects of misleading information.

Relevant sections: Data Construction Pipeline (Section 3.1), Synthetic Data Validation (Section 3.2), Text Level Anonymization (Section 4.3.1); Motivation Study (Appendix A).

W2 (Memorization, reasoning and web search)

Our analysis goes beyond overall re-identification accuracy to examine how agents arrive at correct identities and how web search contributes to that process. We conducted paired no-search ablations and analyzed retrieval trajectories to distinguish candidate corroboration from retrieval-consistent discovery. For Kimi K3 and GLM-5.3 Flash, 81% and 80% of corroboration successes remained correct without search, compared with only 25% and 18% of discovery successes. These results reveal distinct identification pathways and show that agents with similar overall accuracy can rely on external retrieval in substantially different ways.

We also expanded our occupation-level analysis to examine variation across 822 individuals spanning 183 occupation categories. We found substantial differences in re-identification risk: professors achieve a mean high-confidence recall of 0.51, compared with 0.06 for actors. This shows that the distinctiveness and searchability of attribute combinations, rather than public prominence alone are important factors in re-identification risk. Evaluating individuals whose information became public after model training cutoffs would further extend this analysis by providing a more direct test of training-data memorization.

Relevant sections: Agentic Search and Retrieval Strategies (Section 4.2); Occupation-Level Analysis (Appendix C.4, Figure 5).

AuthorsOpenReview ↗

Part B: Q1 (Role of the Interview format), Q2 (Figure 2 caption and recall definition), Q3 (Number of runs and variance), Q4 (Privacy instructions as a safeguard)

Q1 (Role of the Interview format)

Interviews are our motivating privacy setting, not the sole focus of the paper. We study how LLM agents combine identifying clues distributed across conversational turns with publicly available information to re-identify individuals. We adapted eight core questions from real Anthropic workforce interviews, with context-dependent wording and follow-ups. Our analysis shows that 52.3% of surfaced attributes appear after the first round, demonstrating how identifying information accumulates throughout a conversation.

We use synthetic interviews rather than redacted source text because our goal is to evaluate conversational disclosures, not documents originally written to describe identifiable individuals. Grounding these interviews in known identities also allows us to compare re-identification mechanisms and mitigations under consistent evaluation conditions.

Relevant sections: Data Construction Pipeline (Section 3.1); Synthetic Data Validation (Section 3.2); Generation Prompt (Appendix B).

Q2 (Figure 2 caption and recall definition)

We added the main caption, “Precision–recall and confidence calibration analysis,” to Figure 2. Yes, the candidate must be correct. Recall in Figure 2a is the fraction of all 822 transcripts for which at least one candidate correctly identifies the interviewee with High or Very High confidence.

Q3 (Number of runs and variance)

Each configuration was evaluated once per transcript across all 822 AGENTDOXX interviews. The discussion of noisy runs in Appendix A concerns our separate motivating study on real Anthropic interviews, where repeated verification was necessary because ground-truth identities were unavailable. The main AGENTDOXX results report single-run performance rather than run-to-run variance.

Q4 (Privacy instructions as a safeguard)

We clarified this distinction by presenting privacy instructions as an agent-level safeguard for deployed systems (Section 4.3.2**)**, separately from text-level anonymization methods (Section 4.3.1). Our evaluation shows the key limitation of this safeguard that privacy instructions can suppress identities in final responses without preventing exposure during retrieval. Among responses that withheld the identity, 37% had already retrieved the correct name, demonstrating that final-answer compliance alone does not capture privacy exposure during agent execution.


We hope these clarifications address the reviewer's concerns and thank the reviewer for the valuable feedback. We are more than happy to provide additional details if any further questions remain.

Reviewer_kdGAOpenReview ↗

Official comment by Reviewer kdGA

I thank the authors for the detailed response. I have a few more quetions.

  1. I think whether the synthetic interviews are realistic (humans cannot distinguish them) is different from how much identifying information they contain for re-identification. So, I'm still unclear about how realistic the generated dataset is. I understand that the goal of synthetic data is to have a ground truth, and to control factors like search, but still I think if we want to inform the community about the risk, it is good to understand how close the design is to the real world.

  2. Do the authors have findings from controlling for direct and indirect exposure separately? For instance, do we observe any difference between providing more indirect clues versus providing more direct clues in the interview datasets? do we always need direct clues?

  3. The discovery vs. corroboration analysis is quite interesting. But can discovery really be independent of corroboration? There seem to be two cases. In one, the cues are rich, so the model can draw more from what it has memorized, which makes discovery easier (e.g., keyword search becomes easier). In the other, the model has still memorized the person, but the cues are few (e.g., indirect clues), so it cannot form good search queries and fails to find them. In both cases, isn't it still possible that the model already knows the person?

Comments on the paper

1 comment

We appreciate the time and effort of the AC and reviewers and the opportunity to clarify and strengthen our work. Across the reviews, we have addressed several shared concerns:

  • Evaluation validity: We expanded our analyses of attribute exposure, occupation diversity, and re-identification risk, and conducted an exploratory human re-identification study to complement our existing realism evaluation.
  • Re-identification mechanisms: We clarified how our analyses of model capabilities, reasoning effort, and web retrieval contribute to understanding the mechanisms underlying agentic re-identification.
  • Privacy mitigations: We evaluated additional anonymization baselines, including FGAA and IncogniText with search-enabled variants, and clarified the distinction between text-level anonymization and agent-level privacy safeguards.

We also expanded reporting of inference settings and computational costs and improved the presentation of results.

We have submitted a revised PDF with additional visualizations, clearer references between the main text and appendices, and some of the updates discussed in our individual responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.