- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces SafeDrift, a framework for evaluating the longitudinal safety of memory-augmented mental-health agents. Using three models, three memory conditions, and 48 simulated personas, the authors examine how rater disagreement, score aggregation, and memory attribution affect safety conclusions. Results highlight substantial evaluation sensitivity and limited overall effects of memory.
Critical strengths
- The paper compares LLM judges with human annotators and identifies notable disagreements across safety dimensions, including ambiguities in the evaluation rubrics.
- The paper shows that changing the judge, aggregation method, or attribution criteria can lead to different safety conclusions.
Critical weaknesses
- The equivalence results rely heavily on pooled analyses and a ±10 percentage-point margin, with only 4 of 18 per-model comparisons meeting this criterion. The evaluation also covers just three models, with a single run per configuration, limiting the strength and generalizability of the findings.
- Each counterfactual response is generated only once, without a no-intervention baseline. Since generation is stochastic, FAIL-to-PASS flips may reflect sampling variability rather than the removal of memory. This makes the reported attribution rates difficult to interpret.
Questions for authors
-
What motivates the ±10 percentage-point equivalence margin, particularly in a safety-critical setting? How should the pooled equivalence results be interpreted given the weaker per-model evidence?
-
Could the authors provide a no-intervention baseline and repeated counterfactual samples to assess how much of the observed FAIL-to-PASS rate is due to generation variability?
-
How stable are the main findings across different random seeds, given that each experimental configuration was evaluated only once?
3 comments
Thank you for the detailed response.
The authors have addressed W1/Q1 with additional analysis and clarified some limitations. They have also acknowledged the issues raised in W2/Q2 and Q3.
However, W1/Q1 is only partially resolved, as the per-model evidence remains limited. W2/Q2 and Q3 are still unresolved, since the authors have not provided the additional experimental results.
These missing experiments are important to the paper’s main claims. Without them, the current results are not sufficient to support the key conclusions.
Therefore, I am now more confident in my original decision to reject the paper. I will keep my rating unchanged and increase my confidence score.
We thank the reviewer for engaging with our response and for the clear statement of what remains. We agree that the per-model evidence is limited and that the no-intervention baseline, repeated counterfactual replays, and multi-seed runs are needed to support the main claims. We will run these experiments and report them in a revised version of the paper.
Response: equivalence margin, replay baseline, seeds
We thank the reviewer for these focused questions. We agree with both weaknesses and answer each question below, including what we cannot yet show.
Q1. The ±10 pp margin and pooled versus per-model evidence. The ±10 pp margin was not pre-specified and has no clinical derivation; the reviewer is right that it cannot carry a safety-relevant conclusion on its own. We will replace binary equivalence with the largest effect each contrast excludes (the bound of its 90% persona-cluster bootstrap CI), reported under both full-census judges so readers can apply their own margin:
Pooled across models, memory effects on recognition and referral larger than about 9 pp are excluded under both judges. A retrieval-related increase in sycophancy is not excluded, and the second judge estimates one. In a safety-critical setting even the excluded range may be too wide to call memory safe, and we will say so.
On per-model evidence, the reviewer is right that only 4 of 18 per-model contrasts meet ±10 pp. With 48–76 probes per cell, per-model CIs reach ±17 pp, so the data cannot exclude model-specific memory effects of 10–15 pp. The pooled result is an average across three models, not a statement about any one of them, and we will restate S2 accordingly. The model×memory interaction test detects no interaction, but at this sample size that is weak evidence of its absence.
Q2. No-intervention baseline and repeated counterfactuals. We agree this is the main gap in the attribution analysis: without a baseline, a FAIL→PASS flip after masking cannot be separated from sampling variability. The paper's own data show the risk is real. Flips concentrate in the dimension where the original judge is systematically stricter than both reference judges (86% of distress-assumption failures flip), which is what judge noise plus resampling would produce. We will add (a) a no-intervention baseline that replays the same turn with memory kept and (b) k = 5 repeated replays per turn, with and without masking, and report flip rates relative to the baseline. Until then, the masking rates should be read as an upper bound on direct, turn-level memory effects, and the revision will present them that way.
Q3. Stability across random seeds. We cannot answer this from the current data: each persona–model–condition configuration was run once, with a single seed. The persona-clustered bootstrap captures variation across personas but not across generations, so the CIs understate total uncertainty. We will state this explicitly and add repeated runs with independent seeds for the main contrasts. The largest finding, the recognition gap between llama-3.2-3b and the two larger models (23–46 pp across judges), is large relative to its CIs and unlikely to be overturned by seed variation. The memory contrasts are much closer to zero, and their stability is an open question.
We will also restructure the presentation around these points, as the reviewer found parts hard to follow.
We hope these analyses and corrections address the reviewer’s concerns, and we would be grateful if the reviewer would reconsider their assessment in light of them.