- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper studies revision traces: statements in an LLM’s output that an item was removed, excluded, or replaced during private drafting, which can reveal the withdrawn item to a third party who receives the output. The motivating example is a model that removes a password from a configuration file but leaves a comment naming it. The authors analyze three public conversation corpora (WildChat, ShareChat, DevGPT) and report that 8.8% of 26,753 revision requests leave a trace. They then introduce RevLeakBench, 100 tasks across five scenarios adapted from existing benchmarks, with a conversation track and an agent track, and three revision situations (revocation, replacement, model-added).
Critical strengths
S1. The phenomenon is real and easy to grasp. The password example in Figure 1 is compelling.
Critical weaknesses
W1. Are revision traces actually harmful? The paper treats revision traces as dangerous in themselves, but most of them appear to be innocuous. Of the three disclosure levels introduced in section 5.2, only identifying traces are a real disclosure; descriptive and process-only traces reveal little or nothing. The motivating password example is not representative of the paper's own evidence. The in-the-wild public-repository example (Section 3, l. 176-181) is a model stating "I have removed the onclick attributes from your buttons", which reveals nothing secret. Most examples in Appendix A are similar. In the benchmark, security credentials are only 6 of the 200 withdrawn items, and the reader recovers none of them (Table 20). The paper should show how often a trace discloses something that actually matters, rather than how often a trace exists.
W2. The deliverable should be the body, not the full reply. This is my main methodological concern. Figure 4 and Section 4.3 make clear that the authors treat the entire model reply (preface, body, afterword) as the deliverable, and the headline numbers are computed on it. The justification (L182-184) is that users may forward the reply as-is. If a user copies the model's conversational remarks into a message to a third party, the error lies with the user, not with the model. The relevant measure is what ends up in the content meant for the recipient, and in that case the rate of leakage drops from 56.9% to 4.6%. All the main claims in the paper should be about the Body.
W3. Results by disclosure level are mostly missing from the main text. Given W1, the identifying level is the one that matters, yet the main text reports it in a single sentence (L322). The breakdown by level is only in Appendix J (Tables 12 to 14) and Table 18. The central result of the paper should arguably be the rate of identifying traces inside the body, per model.
Minor
m1. The detection pipeline is not shown where it is used. Section 3 relies on a "keyword rule" (L151) to select revision requests, which I assume is a regex, but it is never specified, not even in the appendix. The rule-based scanner (L288-290) is also described only in words (Appendix E). If these are regular expressions, they should be reported at least in the appendix. Furthermore, Section 3 applies "the same detectors that Section 5.2 defines" (L155-156), so the reader meets the in-the-wild results without knowing how traces or categories are defined. The revision categories (revocation, replacement, other) are similarly used in Section 3 before being properly introduced. The definitions should be moved before their first use.
m2. The validity of RevLeakBench depends on how realistic its tasks are, yet the main text never shows a complete task. Figure 4 shows a partial reply only. At least one full example (material, conversation, withdrawn item, deliverable) should appear in the main text.
m3. Six models from three providers is a small panel. Table 2 does not state which models it reports; I infer from n=240 that it pools all six. It is also unclear why a full identifier is given only for GPT-5.6.
m4. Presentation: (m4a) The figures are too small to be read, in particular Figures 3 and 5. (m4b) The introduction is long and repetitive: the research questions are stated, then restated with results, and the contributions largely repeat them. (m4c) The tweet anecdote (L52-56) adds nothing and is not appropriate for a research paper.
Questions for authors
All of the above, and also:
Q1. What is the rate of identifying traces inside the body, per model and per information type?
Q2. Of the 2,363 in-the-wild traces, how many disclose sensitive information (credentials, personal information) rather than a description of the edit?
6 comments
Response to Reviewer raDr (2/3)
W2: Body or Full Reply as the Deliverable
Thank you for your insightful comment. We agree that body traces should be reported on their own. Table 3 in Section 5.2 now reports body traces confirmed by two annotators for every model and track. When only the body is delivered, as in our body-only boundary, recovery falls from 13.7% to 0.6% in the conversation track and from 13.4% to 2.9% in the agent track. We keep the full reply as the deliverable of our main results for three reasons.
First, prior privacy benchmarks evaluate the entire response that the model writes for a recipient. CIMemories [1] checks the whole message, and ConfAIde [2] tells the model that its meeting summary will be automatically sent to all attendees.
Second, we respectfully disagree that the error lies only with the user, because revision traces remain when the user tells the model what the recipient will receive. On four models, under a forwarding notice such as "I'll forward your entire reply as-is", 36.4% of deliverables still contain a trace and the reader recovers the withdrawn item from 5.6% of all deliverables. Under a system message that asks for the deliverable only, with no note, preface, or explanation, 27.5% still contain a trace and recovery is 4.7%. In this setting, the trace is the model's own addition to what the recipient receives.
Third, models also write revision traces into the delivered content itself. In at least 15 of the 1,186 in-the-wild revocations, the delivered code or document names the withdrawn item, and Ruff [3] and a Claude Code issue [4] ask coding agents to keep such history out of what they deliver. Similar to Non-Adversarial Reproduction [5], which measures text that benign users may inadvertently redistribute, we therefore report the full reply as what a recipient sees when the reply is forwarded without editing, together with the body.
In the revision, the Conditions paragraph of Section 4 states that the full reply is the deliverable as in CIMemories [1], Section 5.4 reports recovery under both instructions, and Section 6 states that our main rates apply to replies forwarded without editing.
m2: Complete Task in the Main Text
Thank you for your suggestion. Figure 4 now shows a complete task: the material excerpt, all five conversation turns, the withdrawn and replacement items, the deliverable with its preface, body, and afterword marked and the trace highlighted, and the reader's answers with and without the deliverable, with the elided parts marked in grey. Appendix I keeps a second example.
m3: Model Panel and Identifiers
Thank you for your careful reading. Table 2 reports all six models pooled, and its caption now says so. We gave the full identifier only for GPT-5.6 because its API identifier, gpt-5.6-sol, differs from the model name. Appendix E now lists every model with its identifier, access route, temperature, reasoning effort, and generation dates. We also add MiniMax-M3.1-Flash from MiniMax and Qwen3.8-Flash-Next from Alibaba, which extends the panel from six models from three providers to eight models from five providers. We evaluate the two models on the revocation conditions of all 100 tasks and add them to Table 3.
Response to Reviewer raDr (3/3)
W1&Q2: Harm of Revision Traces and Sensitive Information in the Wild
Thank you for your insightful comment. You are correct that our traces rarely disclose credentials: models restate a credential in only 1.4% of the credential deliverables, and the reader recovers none of them. You are also correct that the onclick example in Section 3 reveals nothing secret, and it serves only to show that a trace can reach a public repository. However, many withdrawn items are ones that prior benchmarks already treat as sensitive, such as the must-not-share items of AgentCIBench [6] and the policy-violating propositions of RedacBench [7]. For credentials, personal information, and organizational information, the reader recovers the withdrawn item from 15.8% of deliverables on average and from 30.5% for DeepSeek-V4-Flash in the conversation track. For example, GLM-5.2 tells the recipient that it will "leave the Maya IVF retrieval out", which reveals a family member's medical appointment (Appendix I). The trace sentence carries this disclosure: identifying traces account for 74.3% of all recoveries and descriptive traces for a further 11.2%, and deleting only the trace sentence reduces recovery from 303 of 1,239 deliverables to 24. The revision now reports disclosure by information type in Section 5.2.
Regarding Q2, as suggested, we perform a human annotation study on the 2,363 in-the-wild traces. Two authors independently label a stratified random sample of 331 traces (95% confidence, ±5% margin) and agree at κ = 0.88 on whether a sentence is a trace and at κ = 0.85 on information type. With sampling weights, both confirm 79.6% of the sentences as revision traces, and 48.3% name the edited item, rising to 65.0% among revocations. Of the same sentences, 1.6% (95% CI [0.4%, 3.2%]) disclose credentials or personal information, and the single credential trace names a removed token field without its value. This rate is a lower bound, because the public corpora contain only conversations that users chose to share, and WildChat [8] and ShareChat [9] were de-identified with Microsoft Presidio before release, so a personal value that a trace restated appears only as a placeholder. This limitation of public data is why RevLeakBench annotates each withdrawn item with its information type. We have added this study to Appendix A and the confirmation rate to Section 3.
m4: Presentation
Thank you for your suggestions. (a) We moved the screening funnel from Figure 3 to Appendix A, which enlarges the remaining three panels, and replaced the per-model heatmap in Figure 5 with the per-model table, which enlarges the remaining two panels. We also redrew Figure 2 so that it gives an overview of the whole study. (b) We removed the introduction paragraph that restated the results and now state each result once in the contributions, organized by research question. (c) We removed the tweet anecdote.
We thank you again, and we hope you will reconsider your original score in light of these clarifications. We would be glad to answer any further questions.
References
[1] Niloofar Mireshghallah, Neal Mangaokar, Narine Kokhlikyan, Arman Zharmagambetov, Manzil Zaheer, Saeed Mahloujifar, and Kamalika Chaudhuri. CIMemories: A Compositional Benchmark for Contextual Integrity in LLMs. In International Conference on Learning Representations (ICLR), 2026.
[2] Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, and Yejin Choi. Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory. In International Conference on Learning Representations (ICLR), 2024.
[3] Astral. Ruff: AGENTS.md. https://github.com/astral-sh/ruff/blob/main/AGENTS.md, 2026.
[4] [FEATURE] Keep development history out of code comments/docstrings by default (put it in git, not the file). https://github.com/anthropics/claude-code/issues/85130, 2026.
[5] Michael Aerni, Javier Rando, Edoardo Debenedetti, Nicholas Carlini, Daphne Ippolito, and Florian Tramèr. Measuring Non-Adversarial Reproduction of Training Data in Large Language Models. In International Conference on Learning Representations (ICLR), 2025.
[6] Anmol Goel and Iryna Gurevych. Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity? CoRR abs/2606.23189, 2026.
[7] Hyunjun Jeon, Kyuyoung Kim, and Jinwoo Shin. RedacBench: Can AI Erase Your Secrets? In International Conference on Learning Representations (ICLR), 2026.
[8] Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. WildChat: 1M ChatGPT Interaction Logs in the Wild. In International Conference on Learning Representations (ICLR), 2024.
[9] Yueru Yan, Tuc Nguyen, Bo Su, Melissa Lieffers, and Thai Le. ShareChat: A Dataset of Chatbot Conversations in the Wild. CoRR abs/2512.17843, 2025.
Response to Reviewer raDr (1/3)
Thank you for the care in reviewing our work, in particular the line-level pointers and the time spent reading the appendix tables, and for recognizing that the phenomenon is real and easy to grasp and that the password example in Figure 1 is compelling. We note that your main concerns relate to whether revision traces disclose information that matters, whether the body or the full reply should count as the deliverable, and the results by disclosure level. To address your concerns, we have provided point-to-point responses to the weaknesses and questions you raised below. Besides, we have carefully revised our manuscript and highlighted the changes in blue. We have also polished the main text for readability.
W3&Q1: Results by Disclosure Level in the Main Text
Thank you for your suggestion. We have moved the per-model results into the main text as Table 3 in Section 5.2, which replaces the per-model heatmap and reports full-reply traces, body traces confirmed by two annotators, identifying traces, and recovery for each model and track. Every model leaves a full-reply trace in at least 29.5% of its revocation deliverables, and recovery reaches 24.5% for DeepSeek-V4-Flash in the conversation track and 29.5% for MiniMax-M3.1-Flash in the agent track.
Regarding Q1, the scanner places 43 of the 424 identifying traces of the eight models inside the body. In the conversation track they occur in 2 of 1,598 deliverables, while in the agent track, where the model writes the document itself, they occur in 2.6% of deliverables and in 9.0% for MiniMax-M3.1-Flash. By information type, personal information has the highest rate inside the body, at 2.1% over the two tracks. These rates are lower bounds, because the identifying level of the scanner has a recall of 0.45 against the human labels. The other identifying traces sit in the sentences around the body, which still reach the recipient when the reply is forwarded (see our response to W2). We have added these rates by model and by information type to Appendix J of the revision.
m1: Detection Rules and Order of Definitions
Thank you for your careful review. The keyword rule includes a regular expression that matches removal and replacement verbs, negated requests such as "don't mention", and separable phrases such as "leave ... out", which Appendix A now gives. The rule-based scanner is also a set of regular expressions, and Appendix E now lists its rule families with their core patterns and an example for each, with the complete expressions in the supplementary material. We have also moved the definitions of revision traces, the three disclosure levels, revocation and replacement, and the two checks to the beginning of Section 3, so that the in-the-wild analysis uses them after they are defined.