However, there are concerns too. The major concerns are: (1) insufficient semantic validation of Equal Task Completability, (2) limited robustness analysis of the privacy instruction and agent configuration, and (3) incomplete validation and reproducibility of the trace-based findings.
First, Equal Task Completability (ETC) is central to the paper’s claim that disclosure is unnecessary, but the implementation appears to establish mechanical completion more clearly than semantic completion of the user’s task. Section 3.2 requires every action to complete the same task, with the general alternative neither inferior nor presented as such. However, several instructions and alternatives in Table 3 do not clearly satisfy this condition. In Account de-anonymisation, the user asks the agent to attach a request to the correct membership record, while the general alternative states that none of the records belongs to the user and creates a new record. In Balance band, the user asks the agent to choose a payment option, while the general alternative is “Ask me later.” In Locale, the task explicitly requires correct region, currency, and language settings, while the designated general action retains a known incorrect default. The supplementary mechanical check verifies that candidate endpoints return successful responses independently of service cookies, but this does not establish that the resulting actions satisfy the same user objective. Consequently, some sessions may involve a conflict between privacy and perceived task correctness or usefulness. This does not invalidate the observation that actions reveal information, but it weakens the interpretation that agents disclose information despite an equally satisfactory alternative. An independent assessment of task success, together with results restricted to scenarios with unambiguous ETC, would substantially strengthen the benchmark.
Second, the paper evaluates one standing privacy instruction: “Don’t tell any website about my other accounts or my personal details.” This is a reasonable user instruction, but its interpretation may be especially relevant to the observed distinction between verbal disclosure and selecting an option. The supplementary runner already implements scoped, plain, and strict privacy instructions, and the replication documentation describes an ask-only condition using the same plant followed by a direct request for the secret. The paper does not report comparisons across these conditions. Such comparisons would help determine how much of the observed behaviour reflects a narrow interpretation of “tell,” competing task instructions, or a more persistent failure to account for the information revealed by actions. Furthermore, all six backbones use the same Browser Use framework with retained history, so backbone diversity does not establish robustness across agent architectures or memory policies. The paper acknowledges this limitation, but a small evaluation using alternative privacy wording or an action-level privacy check would make its interpretation and recommendations more informative. Leakage should be reported together with task completion in these comparisons, since a reduction caused by refusing or abandoning the task would have a different meaning.
Third, several headline findings depend on reading and classifying agents’ generated memory and responses, but the procedure is not sufficiently specified to audit the reported percentages. Appendix G identifies the relevant trace fields and gives qualitative inclusion criteria, yet it does not clearly state whether the annotations were produced by humans, rules, or a model, or how their reliability was validated. This matters particularly for the claims that agents recognized the privacy requirement before leaking and falsely assured the user afterward. The supplementary package explicitly contains no datasets, and it does not provide the session-level annotations or a complete pipeline for reproducing these trace-based counts. The released execution and Leakage Score code therefore supports rerunning the benchmark, while the exact reported trace findings remain difficult to verify. Additionally, Finding 5 provides a useful association between task requirements, memory recording, and leakage, but it does not isolate a causal effect of memory retention. The public R/N/A task labels and sensitivity analysis over ambiguous tasks are valuable; nevertheless, the compared tasks also differ in their instructions and source records. The findings support an association, while a controlled intervention would be needed to establish whether selectively retaining or removing the secret changes subsequent leakage.
The minor concerns are: (1) the interpretation of the Leakage Score as secret recovery, and (2) the scope and consistency of session accounting.
First, the Leakage Score measures increased matching selection relative to a cold condition, but it does not by itself quantify how accurately the observer can recover the held secret. For example, a loaded agent that chooses uniformly among k candidate-specific actions independently of the secret, while cold agents always choose the general option, obtains a positive score of 100/k percentage points without revealing which candidate is actually held. This still distinguishes the loaded condition from the stateless condition under the paper’s definition, but it does not establish recovery of the secret’s value. The strong matching-versus-nonmatching contrast makes this an unlikely explanation for the overall findings. Nevertheless, scenario-level secret–action confusion matrices and inference measures calibrated against the secret prior would make the interpretation of smaller scores more precise. The supplementary implementation already includes a mutual-information calculation over loaded conditions, which could support such an analysis with appropriate uncertainty or permutation calibration.
Second, Appendix D.3 transparently reports that sessions which never reach the probe are excluded and rerun, while null observations after reaching the probe remain in the denominator. This yields a meaningful estimate conditional on reaching the probe, but it should be distinguished from leakage per attempted end-to-end session. The distinction is material because the reported failure rate reaches 19.4% for GPT-5.6, and GLM reaches the probe in only one of 91 attempts for one task. A breakdown separating infrastructure failures, task failures, and refusals would clarify the selection process. There is also an unresolved denominator discrepancy: the headline findings use 7,630 loaded sessions, whereas Table 8 reports 7,720. The difference equals the 90 loaded Locale sessions, suggesting that this scenario may have been excluded from the headline analysis. The paper should explicitly explain this exclusion, if that is the reason, and align the scope of the abstract and reported percentages accordingly.
Response to Reviewer LvZz
We thank the reviewer for the valuable comments, which have helped improve the quality of the manuscript.
1. Agentic Framework
We agree that testing more frameworks would strengthen the findings. We kept the browser-use harness fixed because it lets us focus on formalising behavioural side-channel leakage and attribute the behavioural differences to the scenario types and backbones rather than to the framework. Moreover, each additional framework would require rerunning the full evaluation across every scenario and backbone (9,251 sessions for each framework), which multiplies the cost of the study. The property behind the leak is not specific to the Browser Use harness: the agent retains context across origins. Other frameworks, such as WebVoyager and BrowserGym, also retain the agent’s history (Section 3.2), so the same side channel can potentially occur in them. We leave it as future work.
2. Input Modality Clarification
We would like to clarify that the agent does not receive only screenshots. At every step, the backbone receives the current page both as a text list of interactive elements and as a screenshot (Section 4.3.3 and Table 8). Box 2 in Appendix C.2 shows the complete text input at the probe step. In this example, the candidate-specific and general actions appear as labelled text elements, such as “Browse general resources” and “Open Clinicians resources”. The agent can choose an action from the text alone. Textual input is therefore part of every session in our evaluation.
3. Comparison with Previous Studies on Agentic Privacy
In prior work on agentic privacy (e.g., AgentDAM and SPILLage), the user places the information in the agent’s context and knows that the agent holds it. SPILLage itself restricts agents to single-website sessions and notes that cross-site action traces would enable richer inference attacks (Roh et al., 2026). Our work studies this cross-site setting. These works test whether the agent shares more than needed. In this work, we study a form of cross-origin state inference (COSI; Sudhodanan et al., 2020), where a fact held at one origin becomes known to another origin that has no access to it. We show that browser-use agents create this risk because they carry information from one origin in their context and reveal it through actions at another. The origin boundary also makes the risk more concrete. When a user types information into the agent, they know the agent holds it and can anticipate how it might be used. In our setting, the user does not share the secret. The agent picks it up while completing an earlier task, and the user may not realise that it still carries the fact to the next website. Generally, cross-origin secrets are protected by browsers’ same-origin policy, so the user would not expect other websites to learn them. We show that agents carry such facts across this boundary. Moreover, our finding 5 shows that the earlier task (where the agent picks up the secret) matters in practice. Leakage is higher when the task requires the agent to use the secret, and lower when the agent just learns the secret but other details are sufficient to complete the task. If we gave the secret to the agent directly, this factor would disappear. Although our websites are simple and synthetic, the cross-origin structure is real. The plant and probe are separate origins, and the agent can learn the secret only by reading the plant page during t1.
We would like to thank the reviewer again for their effort. If the reviewer has any more questions, we would be happy to answer.