acceptodds
Under review as a conference paper at ICLR 2027

TRACE: FROM VERIFIED OUTCOMES TO REUSABLE PROBING STRATEGIES FOR MEMORY-EXPOSURE ASSESSMENT IN CLINICAL LLM AGENTS

Abstract

In clinical consultations, LLM-based clinical agents may use patient-specific records while also retaining information from earlier interactions, including prior dialogue, retrieved electronic health record context, and tool-use traces. This persistence introduces a privacy risk: retained patient information may later reappear in a response even when it is absent from the patient's current query. We refer to this response-level resurfacing of retained information as memory exposure. In this paper, we study whether an external assessor can acquire reusable probing strategies for systematically assessing such exposure without modifying the target clinical agent. The assessor probes the target through ordinary interactions. A probing strategy is a reusable pattern for determining the assessor's next query or action. Thus, we propose TRACE, a verifier-guided framework for the bounded evolution of reusable probing strategies. TRACE starts from a fixed seed repertoire and executes the available probes against the target agent. It uses a deterministic verifier to compare each response with protected information defined for the trial. Exposure is counted only when the matched information is absent from the visible query. The resulting exposure and non-exposure outcomes are stored as structured records of the probe, target response, and verification result. These records guide the acquisition of new probing strategies. A response-conditioned router then selects among the retained strategies based on the target's observed response. After the acquisition phase, both the learned repertoire and router are frozen. During downstream evaluation, no new probing strategies are generated and the router is not updated, so the measured performance reflects reuse of acquired probing knowledge rather than continued test-time adaptation. Across five independently seeded acquisition runs, TRACE increases mean Verified Success Rate (VSR) from 62.0% with the fixed seed repertoire to 93.6% on the held-out evaluation split. With acquisition disabled, the same frozen repertoire and router achieve a mean VSR of 93.3% across DeepSeek-V3.2, GPT-4o, and LLaVA-v1.5-7B. Additional analyses examine sensitivity to routing, context construction, dataset shift, and output-side controls. These results support verifier-guided acquisition and frozen reuse of probing strategies for memory-exposure assessment in clinical LLM agents.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Clinical LLM Agents, Memory Exposure, Probing Strategies, Verifier-Guided Strategy Evolution, Privacy Assessment

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Uod1OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies unwanted memory exposure in LLM-based clinical agents. The paper introduces TRACE, a verifier-guided framework for evolving, reusable probing strategies.

Critical strengths

None

Critical weaknesses
  1. The broader motivation of memory exposure in clinical agents seems artificial and unclear. If information retained from the earlier interaction for the same patient shows up later, why is this a dangerous privacy risk? Why can't the memory just be reset for every interaction if privacy is the priority? As stated now in the introduction, it's unclear and not compelling.
  2. The connection between the memory exposure and the research questions doesn't make sense. Why do you study probing strategies instead of auditing the conversational traces directly for memory exposure? Why can't stronger agentic LLMs like GPT-6 Astra directly audit the agents traces and conversational history to assess this?
Questions for authors
  1. Can you clarify the motivation for studying memory exposure in clinical agents?
  2. Can you use newer LLMs with stronger agentic capabilities (GPT-6 Astra, Claude Opus 5.5, Qwen3.8)?
Reviewer_T3XDOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces TRACE, a framework to store and enable the evolution of probing strategies (i.e, coaxing a clinical model/agent to reveal protected information from memory). The authors experiment with 3 datasets with different configurations across 3 LLMs, and find that TRACE augments probing strategies to attain significantly higher success rate.

Critical strengths
    1. The author explores an interesting and practical use case of LLMs in clinical setting. As agents are deployed in high-stakes settings, it is crucial to ensure that they adhere to sensitive information. TRACE proposes a strategy that can enhance privacy testing in such settings.
    1. The paper performs ablation testings across datasets that reveal useful insights about the performance of TRACE.
Critical weaknesses
    1. One of the major concerns is construct validity of both TRACE and related metrics. Particularly, definitions of key concepts like memory exposure (lacking clear examples), strategy of evolution, success rate, are either underdefined or do not appear to reliably capture what they intend to measure.
    1. The lack of rigorous validation raises concern about results' reliability. For example, the only human annotation appears to validate the 100 outputs, with no supporting metrics like precision, recall, or error analysis. The small number of backbones (3 models) and baselines challenge generalization. There is also no statistical testings to establish confidence. For a clinical application, this is particularly concerning.
Questions for authors
    1. The paper is dense, yet key concepts are not well explained in the main text. I would strongly suggest better explanations and examples what memory exposure, privacy, and other key concepts in the main paper (some of them are currently buried in the appendix).
    1. Please also consider using more intuitive terms instead of confusing jargons: bounded repertoire, source-verified.
    1. PCMU is a key component of the pipeline, yet it appears to be a rule-based classifier. This means that it could miss many forms of memory exposure written in subtle ways (eg, paraphrasing, partial quoting). Can the authors validate this, especially against (potentially) stronger classifiers like LLM judges ?
    1. Besides information on the ground profile of the patient, does TRACE also consider other equally important confidential information that do not appear there.
    1. Can the authors provide stronger analysis on error categories, other edge cases, especially with human experts? On this note, details on human annotators are lacking in the paper.
    1. Q-update on equation 4 appears underspecified. I would encourage more justification why this learning strategy is favored over, potentially simpler, methods.
    1. The paper would better benefit from it more ablation to disentangle the contributions of verifier-guided feedback, strategy generation, and routing.
Reviewer_hq6EOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies memory exposure in clinical LLM agents: information retained from earlier interactions (prior dialogue, retrieved EHR context, tool traces) resurfacing in a response even when it is absent from the patient's current query. The authors formalize exposure with a source-grounded criterion — a protected clinical memory unit (PCMU) counts as exposed only if it appears in the response and was not already present in the visible query — and propose TRACE, an external assessor that acquires a bounded repertoire of reusable probing strategies from verified interaction outcomes. Acquisition starts from five fixed seed strategies and runs for five generations: the repertoire is executed on an evolution split, a deterministic verifier labels each interaction, structured records drive an LLM proposer to emit candidate strategies, and a candidate is admitted only if it improves on its designated parent on a development split. A router conditioned on a deterministic response-state classifier selects strategies greedily by empirical utility. Both repertoire and router are then frozen and evaluated on held-out trajectories, three target backbones, and shifted datasets.

Critical strengths
  • The operationalization of memory exposure is the strongest part of the paper. Requiring a source-verified match against trial-protected information, and excluding anything already present in the visible query, is a clean criterion that rules out the usual false positive (a clinically plausible response that does not recover protected content). Reporting a 95/100 manual audit against the deterministic verifier is the right kind of check.

  • The bounded-online comparison is a serious baseline suite, and the freeze boundary is the right discipline. Table 7 separates static one-shot (14%), matched-budget fixed sequences (27.6%), MEXTRA-style (39.8%), iterative multi-query adaptation without a learned router (57.2%), random routing (65.4%) and domain-informed no-feedback (69.5%) from the frozen controller (92%). The gap against the 69.5% condition in particular indicates that response-conditioned selection, not domain vocabulary, carries the effect. Freezing before evaluation also removes the most common way this class of result is inflated.

Critical weaknesses

W1. The acquired object has no expressible sequential structure. The repertoire holds at most ten strategies, each of which is a one-step instruction that is "instantiated as the next query or action," so the only multi-turn structure in the system is a one-step greedy selection made by the router — no strategy encodes an ordered procedure such as "ask A first; if the target evades, ask B; if it contradicts itself, ask C." A multi-step probing plan therefore cannot be represented, let alone learned.

W2. The "evolution" is mutation-only selection with arbitrarily assigned parents. Candidates receive "one parent by deterministic cyclic traversal of the current repertoire," parentage is used only to define an admission comparison, and recombination is structurally absent ("the stored lineage is not multi-parent"). Five generations over 75 candidates yield five new retained strategies, while the other five members of the final ten are the untouched seeds.

W3. The learning signal is misaligned with the construct the paper claims to measure. VSR credits a trial only when a protected unit is matched, and the authors acknowledge that "PCMU verification may miss paraphrastic exposure." The consequence is not merely a false-negative rate in reporting: a probe that successfully elicits a paraphrased disclosure is scored as a failure and is not admitted, so acquisition is systematically blind to indirect elicitation — the strategy class that matters most for memory-extraction assessment.

Questions for authors
  1. A probing strategy is defined as "a reusable textual control object … that can be instantiated as the next query or action," and the proposer emits ten queries per batch. Does the framework have any representation of a multi-turn probing plan, or of a strategy that is only meaningful in sequence with another? If not, how much of the reported gain could be obtained by a flat library of ten queries with the same greedy router?

  2. Appendix B states that candidates are assigned "one parent by deterministic cyclic traversal of the current repertoire," and that "a composition proposal can use multiple experience records, but the stored lineage is not multi-parent." Since admission compares a candidate against this arbitrarily designated parent, what does outperforming the parent establish? Please also report how the admitted set changes if the parent is instead chosen as the closest strategy by failure profile.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.