- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper tests whether a fine-tuner can preserve planted false beliefs while evading model-diffing audits. Adaptive attacks often defeat their targeted auditor; an offset-depth statistic detects a recurring logit artifact, but attackers aware of its threshold can sometimes evade it too.
Critical strengths
Sealed tests, matched controls, and comparisons across auditors make the targeted-evasion result more informative than a single demonstration.
The authors attack their own depth detector and report benign-update false accusations, exposing important limits of the proposed defense
Critical weaknesses
The planted beliefs and prompts are narrow and partly correlated, so the results do not yet establish evasion for diverse hidden behaviors or independent test distributions.
The depth bound applies to a persistent logit-tail pattern rather than arbitrary attacks; a threshold-aware full-weight attack evades depth in five of six reported runs while retaining the belief.
Some strong alternative auditors are evaluated but not adaptively targeted. Depth calibration also varies across base models, leaving its false-positive rate and robustness uncertain.
Questions for authors
Does the evasion pattern persist with more independent beliefs, prompts, and fine-tuning objectives?
What happens when an adaptive attacker jointly targets depth and the strongest alternative auditors?
How stable is the depth threshold on larger, held-out benign-update populations across model families?
2 comments
Response to Reviewer Ry63 (2/2)
Q3. You asked about the stability of the depth threshold. We tested the stability of the depth threshold using 192 benign fine-tunes we held out for testing, split into six genre types not present in any earlier population, eight different random seeds, and four base models, with all thresholds sealed before we looked at the scores (Appendix D.3). Here are the counts in the table:
| Base model | Threshold fitted on | Depth accuses | Offset norm accuses |
|---|---|---|---|
| Qwen3-1.7B | as submitted | 0/48 | 0/48 |
| Phi-4-mini | as submitted | 0/48 | 0/48 |
| Qwen3-8B | 2 runs | 8/48 (all fiction) | 18/48 |
| Qwen3-8B | 12 runs (declared beside the first) | 0/48 | 0/48 |
| Qwen3.5-4B | 12 runs | 9/48 (6 fiction) | 0/48 |
We found the depth threshold was stable for the two models in the submitted paper (with an upper bound of 7.4% for both), but not for the two models for which the threshold was based on 2 and 12 runs without narrative text. Thresholds are not transferable across models: at the Qwen3.5-4B depth threshold, 21/48 of the Qwen3-8B runs would have been accused. At the 12-run Qwen3-8B thresholds, depth alone is insufficient to detect the six joint Qwen3-8B attacks (7.82-8.10 vs. 8.29), but the offset norm detects all six (332-358 vs. 215), with no accusations for benign models. The paper now explicitly states that we must calculate model-specific thresholds calibrated on the benign runs for each model across all genres, including narrative text.
We then put that claim to the test, as described in Appendix D.4. For each model, we reset both thresholds to the largest value seen over all of that model's earlier null and benign runs, including the held-out runs above, and then we fixed the thresholds and trained 288 fresh benign fine-tunes. We kept the same six genres, but used twelve new seeds, for a total of 72 fine-tunes per model. These 288 fine-tunes are fresh draws from genres the thresholds had seen. They are not new genres. Together, the depth and offset norm accusation rates are 0/72 on Qwen3-1.7B, Phi-4-mini, and Qwen3-8B (upper bound 5.0% per model), and 1/72 on Qwen3.5-4B (a fiction run at depth 5.321 against a threshold of 5.205, upper bound 7.5%). Pooled, this is 1/288 (upper bound 1.9%). At the thresholds used in the earlier experiment, the same runs replicate the failure reported above: depth accuses 12/72 (all fiction) and offset norm accuses 28/72 on Qwen3-8B; depth accuses 12/72 on Qwen3.5-4B. At Qwen3-8B, recalibration has a cost: of its 34 earlier planted or attacked runs, depth detected 18 (now 12) and offset norm detected 22 (now 18).
If this answers the questions you raised, we would be very grateful if you would be willing to consider raising your rating. We would be happy to add anything further during the discussion!
Response to Reviewer Ry63 (1/2)
Thanks for the very positive assessment of the paper, and for the clear questions! We ran a new experiment for each one, and the results are in the comments below.
Q1. You asked for more independent beliefs, prompts, and objectives. The classifier attack already replicates on a Kansas belief and on an Antarctic belief planted after we designed the attacks (Appendices B.9, B.10). We added new runs, each belief with its own question set (120 questions per belief). The joint attack on both beliefs evades all auditors except depth and the offset norm, and those flag 6/6. The belief is kept in 1/6. With the pooled-probe attack the belief is kept in 3/6, but the max-pooled probe's open-set flag catches it in 6/6.
For two beliefs no attack was ever tuned on (Roman concrete, and "ignore below" comments) the design is the same. We find that the joint attack again evades all auditors except depth and the offset norm, which both flag the attack in 6/6 runs, while keeping the belief in 0/6 (median perplexity ratio 4.9). In contrast, the pooled-probe attack is caught by the open-set flag in 6/6 and keeps the belief in 1/6. No run successfully evades all auditors and keeps the belief (Appendix B.18). This narrows the attack claim, but it is good news for the audit: on beliefs no attack was tuned on, every run is detected. We also experiment with a different fine-tuning objective: a trigger backdoor that emits medical misinformation. In this case, the attacked model keeps the backdoor in 3/3 runs. Every auditor except the offset auditors reads the model as clean, but the offset norm detects the attack in 3/3 runs (Appendix B.18).
We agree with both of the limits you list on depth. Proposition 2 requires the persistent-tail condition to hold, and a full-weight attacker who is aware of the threshold can evade depth in 5/6 runs. We say both of these in the paper already. The persistent-tail condition and Proposition 2 are in the introduction and in Section 3.2, and you can find the threshold-aware evasion in the abstract and Section 3.3.
Q2. You asked about an adaptive attacker that goes after depth and the strongest alternatives jointly. We tried that, with one loss that has seven terms in it. Depth is one. Contrastive decoding and the rank test are in there too, and so are the max-pooled probe, the output classifier and the internal probes, and the sample-only auditor. Using LoRA, with ten seeds: it fails. Depth finds 20/20, the belief is kept in 3/20, and the perplexity ratio is 2.2-9.0. Using full weights, with ten seeds: 3 of 20 runs pass all seven tests with the belief kept and perplexity ratio at most 1.32. The abstract of our paper says that we successfully mount this attack. Outside the loss, both the differential-SAE distance and the offset-subtracted restricted auditor find all 3 evaders. And both seem to be detecting the attack itself, not just the belief: neither flags the full-weight controls, which do carry the belief. So the full-weight attack beats the auditors it trains against, but not the ones it cannot see. In our runs, an audit that includes one auditor the fine-tuner cannot train against detects every evader, which is what the paper now recommends for full-weight releases. If we attack the restricted auditor directly, and jointly with depth, we can only evade it with a perplexity ratio above 1000, and in all 6 runs, the belief is lost. And depth still flags every run (12.5-14.4 against a threshold of 7.045) (Appendix E.4).