- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces LeakGauge, a method for detecting context-leakage attacks before an LLM generates a response. It appends a short gauge sequence to the input and feeds the sequence’s prefill token log-probabilities into a lightweight classifier. The authors compare content-specific Exact gauges with content-agnostic Behavior gauges and find that Behavior generally provides stronger generalization. Experiments cover eleven models, system-prompt and RAG leakage, and held-out attacks and protected content. Cross-lingual tests, cross-objective transfer, and activation-steering experiments further investigate the signal, while a deployment experiment reports 10.34 ms of additional latency and fewer than 500 classifier parameters. Additional stress tests demonstrate both sensitivity to hard negatives and vulnerability to detector-aware attacks, with alternative and combined gauges providing partial mitigation.
Critical strengths
-
Simple and potentially useful detection mechanism. LeakGauge extracts a useful signal through prefill probabilities without requiring access to intermediate activations or autoregressive generation, offering a practical alternative to existing model-derived detectors.
-
Broad empirical investigation. The evaluation spans eleven models, two leakage settings, multiple distribution shifts, and several controls, while activation steering provides additional evidence connecting the observable score to leakage-related model behavior.
Critical weaknesses
- The deployment reliability of the default gauge remains limited. Table 20 reports a 22.18% hard-negative false-positive rate on GLM-5.2, and Table 21 shows only 8.0% detection recall against successful leaks under single-probe-aware optimization. Alternative gauges and a two-probe detector improve these results, but their combined false-positive rates and deployment costs are not established; although the white-box test exceeds the main threat model, it reveals an important limitation of a fixed exposed detector.
- The reported operating-point performance requires clarification. Table 8 reports combinations such as F1 of 0.993 and TPR@FPR5 of 0.963 for Gemma-4-E4B on the RAG in-distribution test, which appear difficult to reconcile under standard binary metrics on the same predictions and the stated class proportions. Furthermore, best F1 over thresholds does not establish performance with a threshold selected independently of the test set, leaving uncertainty about practical deployment performance.
Questions for authors
-
How are F1 and TPR@FPR5 in Table 8 calculated and aggregated across seeds? Please clarify the example identified above and provide the corresponding per-seed operating points or confusion matrices.
-
How are thresholds selected for the deployment and stress-test evaluations? What performance is obtained when a validation-selected threshold is frozen across ordinary benign inputs, hard negatives, and held-out attacks?
-
What are the hard-negative false-positive rate and additional latency of the two-probe configuration in Table 21?
4 comments
Author Response to Reviewer H7hk (2/2)
W2 & Q1. Clarification on metric computation and frozen-threshold performance.
W2. The reported operating-point performance requires clarification. Table 8 reports combinations such as F1 of 0.993 and TPR@FPR5 of 0.963 for Gemma-4-E4B on the RAG in-distribution test, which appear difficult to reconcile under standard binary metrics on the same predictions and the stated class proportions. Furthermore, best F1 over thresholds does not establish performance with a threshold selected independently of the test set, leaving uncertainty about practical deployment performance.
Q1. How are F1 and TPR@FPR5 in Table 8 calculated and aggregated across seeds? Please clarify the example identified above and provide the corresponding per-seed operating points or confusion matrices.
Reply to W2 & Q1: Thank you for checking this carefully, and we are sorry for causing ambiguities in Appendix Table 8.
- Resolving the underspecification of "best F1". As we aim to measure a deployment-oriented detection performance, we rigorously curate the validation set, which shares no samples with the test set. Concretely, the "best-F1" threshold is tuned on the validation set, while the data in column F1 is computed on the test set at this frozen "best-F1" threshold. Thus, F1 in Table 8 already reflects performance at a threshold fixed before testing. As the test set is never used for threshold selection, the best F1 threshold on val set does not necessarily equal the best on test set.
- Double-checking the results. In contrast, TPR@FPR5 is the highest test TPR among thresholds whose test FPR is at most 5%. The two metrics are therefore reported at different thresholds: the frozen threshold can have a test FPR above or below 5%. As described in Appendix B.1, both metrics are computed per run and then averaged over five runs. Table R3 reports per-run confusion counts and F1 at the frozen threshold, showing that the gap in the Gemma-4-E4B-it comes from runs 4 and 5, whose frozen thresholds operate at a test FPR above 5% (5.6% and 5.4%).
Table R3. Per-run confusion counts and F1 at the frozen threshold, together with TPR@FPR5. Gemma-4-E4B-it, RAG, In-Dist (2,074 attack / 1,040 benign).
| Run | TP | FP | TN | FN | F1 | TPR@FPR5 |
|---|---|---|---|---|---|---|
| 1 | 2068 | 2 | 1038 | 6 | 0.998 | 0.997 |
| 2 | 2065 | 0 | 1040 | 9 | 0.998 | 0.996 |
| 3 | 2058 | 0 | 1040 | 16 | 0.996 | 0.994 |
| 4 | 2073 | 58 | 982 | 1 | 0.986 | 0.914 |
| 5 | 2074 | 56 | 984 | 0 | 0.987 | 0.913 |
| Mean | 0.993 | 0.963 |
We have corrected the caption of Table 8 and Appendix B.1 in the revision.
Thanks for your reply. My concerns have addressed. I have raised the rating to 3.
Thank You for Your Response
Thank you for your reply and for raising your rating. We are glad that our response addressed your concerns.
Author Response to Reviewer H7hk (1/2)
Dear Reviewer H7hk, thank you very much for your careful reading and constructive feedback. We appreciate your recognition of our simple and practical detection mechanism and broad empirical investigation. We hope the following responses address your concerns and clarify the evaluation protocol and deployment results.
W1 & Q2 & Q3. Request for settings and results of the stress tests in Appendix G.
Reply to W1 & Q2 & Q3: Thank you for highlighting this concern. As you noted, the white-box GCG test goes beyond our main threat model; it characterizes the failure boundary of an exposed detector. For this additional comparison, to directly calibrate the FPRs, we select a single threshold for max(p1,p2) on the validation set and freeze it before evaluation. Using the same GCG attack suffixes as in Table 21, this configuration achieves 81.5% detection recall and 4.45% post-filtered ASR.
Table R4. Llama-3.1-8B-Instruct, system-prompt leakage. One- and two-probe detectors are evaluated under single- and two-probe-aware GCG attacks, respectively. (Now included as Table 22 in Appendix G.2 of the revision.)
Table R4 reports benign and hard-negative FPRs of 3.46% and 6.73% for the two-probe configuration, together with 81.5% recall against successful two-probe-aware GCG leaks. Scoring both gauges sequentially takes 22.6 ms, compared with 10.34 ms for one probe, and remains below Attention-Tracker (46.03 ms) and I'vDtL (70.91 ms) in our deployment setup. For held-out attack evaluation, Table 8 separately reports frozen-threshold F1 scores of 0.953 on Held-Out Attacks and 0.944 on Held-Out Strict for Llama-3.1-8B-Instruct in the system-prompt setting.
Table 20 shows that gauge design can substantially improve hard-negative robustness. On GLM-5.2, a pre-specified user-intent gauge reduces FPR from 22.18% to 9.64% while increasing attack TPR from 84.93% to 95.76%, without tuning on the hard-negative set. This improvement in both metrics supports gauge selection as a practical mitigation.