- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper claims the usual way of turning a generative guard’s output into a probability by renormalising over the safe and unsafe tokens discards how much total probability the guard puts on those two tokens. Specifically, if we rerun the guard with the safety instruction removed, the total probability varies, and the authors observe it being higher on harmful prompts and attacks, and it mostly holds up on attacks the guard misses. So it boils down to this: when mass is high, the score gets nudged toward unsafe, and when mass is low, it mostly fall back on the temperature-scaled native score. With one fitted temperature, it beats temperature scaling on NLL, Brier and ECE for Llama Guard 2, 3 and 4, and on ShieldGemma.
Critical strengths
The observation that a second, unprompted look at the guard still picks up jailbreaks is new to me, plus the method needs no retraining and very few labels.
Critical weaknesses
The mass that renormalization actually throws away is the mass under the released template, and since that’s close to one for every input, the native score loses almost nothing. The paper instead measures mass under an instruction-free prompt the guard was presumably never trained on. That quantity isn’t part of any score anyone computes. So the method is really asking the guard a second question and using the answer as a feature, and the identities in section 2 don’t explain why that answer should track harmfulness, which the authors concede in section 5. There’s no argument for why this particular question is the right one beyond finding that it correlates with harmful inputs.
The main comparison is with temperature scaling, which can’t fix the one-sided bias visible in Figure 2. The paper’s own Appendix F shows a spline on the native margin alone outperforms VMC_T on all three guards (and with no second forward pass), and adding mass to it gains very little. In table 10, Shuffling mass across inputs within margin and length bins still keeps most of the improvement on Guard 3 and almost all of it on Guard 2. So it looks like much of the gain comes from a general upward shift rather than information specific to each input. Meanwhile the method needs a second full forward pass, its gains come almost entirely from AdvBench-derived attacks in a test mix that’s 39% unsafe, and by the authors' analysis it does worse than temperature scaling once unsafe traffic drops below ~20%.
Questions for authors
How does VMC_T compare with two-parameter Platt scaling and with the contextual and batch calibration baselines from Liu et al. (2025a)? Given that a native-only spline already beats VMC_T, what’s the case for VMC beyond the regime with very few labels?
Why measure mass under an instruction-free prompt rather than another second query, such as a reworded instruction or an ensemble of prompts? Is there a reason to expect the guard’s behaviour on a prompt it wasn’t trained on to stay stable across models?
Can you report AUROC, the TP rate at a low FP rate (like 5%), and false-positive rates on XSTest-safe and OR-Bench for LlamaGuards? I want to know whether mass improves detection or just reshapes the probabilities, and how much FP it causes.
How does VMC do at realistic, low unsafe prevalence, on harmful data not derived from AdvBench, and on response moderation?
7 comments
Non-AdvBench data, low prevalence, and cost (W2, Q4)
The gains are not confined to “AdvBench-derived attacks in a test mix that’s 39% unsafe”: they hold on harmful data unrelated to AdvBench and at realistic low prevalence.
Harmful data unrelated to AdvBench. VMC_T also improves on OR-Bench-toxic, which is not derived from AdvBench, on every checkpoint (Figure 5).
| Method | NLL on OR-Bench-toxic, Guard 3 / 4 / 2 |
|---|---|
| Temperature scaling | 0.371 / 0.612 / 0.388 |
| VMC_T (ours) | 0.204 / 0.497 / 0.361 |
Low prevalence. VMC does not do “worse than temperature scaling” at low prevalence once the calibrator is adjusted to the deployment prevalence: with the standard prior shift, general VMC matches or beats both baselines at 1%, 2%, and 5% unsafe (table below; added as Table 19). The loss in Appendix E.1 came from evaluating at lower prevalence a calibrator fitted on data with 40% unsafe inputs, without this adjustment. General VMC is Eq. (A) above.
| NLL, Guard 3 / 4 / 2 | 1% unsafe | 2% unsafe | 5% unsafe |
|---|---|---|---|
| Temperature scaling | 0.065 / 0.073 / 0.092 | 0.071 / 0.079 / 0.100 | 0.090 / 0.097 / 0.123 |
| Platt | 0.028 / 0.029 / 0.048 | 0.042 / 0.045 / 0.067 | 0.071 / 0.077 / 0.105 |
| General VMC (ours) | 0.025 / 0.029 / 0.046 | 0.038 / 0.044 / 0.066 | 0.065 / 0.073 / 0.103 |
Cost. The second pass is not “full”: it drops the guard template, including its policy text, and reads only first-token logits, costing about a third of the native pass.
| Per input | Native guard | Added by VMC |
|---|---|---|
| Mean input tokens | 256–271 | +76 |
| Overhead, relative to native | — | ≈0.3× |
Response moderation. Response moderation is outside our focus: the jailbreak attacks we study act on prompts.
Detection metrics (Q3)
Mass “improves detection” rather than “just reshapes the probabilities”: adding M raises the TPR at 1% and 5% FPR on every checkpoint (table below; added as Table 15), which is what matters most for a guard, since a missed attack does more harm than an over-refused benign request. Only M can do this, since any monotone recalibration of the native margin, whatever its fitting budget, keeps the native ranking.
| Method, Guard 3 / 4 / 2 | AUROC | TPR @ 1% FPR | TPR @ 5% FPR |
|---|---|---|---|
| Native, temperature, or Platt scaling | 0.984 / 0.977 / 0.977 | 0.755 / 0.703 / 0.561 | 0.903 / 0.897 / 0.890 |
| General VMC (ours) | 0.985 / 0.981 / 0.977 | 0.776 / 0.720 / 0.575 | 0.922 / 0.909 / 0.891 |
“How much FP it causes.” Over all benign inputs, adding M lowers the false-positive rate at 90% recall on Guard 3 and Guard 4; the rates on XSTest-safe and OR-Bench that the reviewer requested are in the same table (table below; added as Table 16).
| FPR @ 90% recall, Guard 3 / 4 / 2 | All benign | XSTest-safe | OR-Bench |
|---|---|---|---|
| Native, temperature, or Platt scaling | 4.7% / 5.3% / 5.3% | 10.4% / 14.6% / 19.8% | 17.6% / 15.1% / 18.3% |
| General VMC (ours) | 3.9% / 4.4% / 5.5% | 7.3% / 14.6% / 19.8% | 14.6% / 18.8% / 18.6% |
Dear Reviewer 8Wye, As the discussion period is nearing its end, we would like to kindly follow up on our responses. We hope our clarifications have adequately addressed your concerns, and we would greatly appreciate it if they could be reflected in your final assessment, if you find them satisfactory. Thank you again for your time and consideration.
Why mass tracks harmfulness and why this prompt (W1, Q2)
The answer to the “second question” is expected to track harmfulness because of how guards are trained, and the fit confirms it. Guards are fine-tuned on red-teaming requests of both labels [1], and fine-tuned models produce their trained format on inputs resembling their training data even without an instruction [2], so harmful requests keep a high verdict mass when the instruction is removed.
To check the direction, we fit on labels the constants that VMC_0 (Eq. 7) fixes, which gives general VMC:
p=σ(TβmN+(1−β)logit(α+(1−α)ME)+c)(A)
Here mN is the native margin and ME the verdict mass computed from the user message without the safety instruction (m and M in the paper), β weights the native margin against the mass term, T is a temperature, c is an intercept, and α + (1 − α)ME is the paper’s a(M) (Eq. 6) with its constant ½ written as α; VMC_0 sets α = β = ½, T = 1, and c = 0. The fitted weight on the mass term is positive on every checkpoint (table below; added as Table 9).
“Why this particular question.” It is not a particular question we chose: it is the user message alone in a plain Llama chat turn, with the whole guard template removed (Appendix B), so it requires no prompt design. The removal is needed because with the template, as the reviewer notes, the native mass MN≈1 for every input (Figure 1b), which is what the training objective in Eq. (3) rewards. The signal also does not depend on this choice: with any of five other second prompts, including two rewordings of the safety instruction, VMC_T still has lower NLL than temperature scaling on every checkpoint (table below; added as Table 5). The direction is also stable across models: mass is higher on unsafe inputs on Llama Guard 2, 3, and 4 (Table 2), and VMC transfers to ShieldGemma (Appendix G).
Platt, contextual, batch, and spline baselines (W2, Q1)
There is a “case for VMC beyond the regime with very few labels”: the verdict mass M matches or improves every calibrator the reviewer names on every checkpoint (table below; added as Table 20), namely two-parameter Platt scaling, contextual and batch calibration [3], and the native-only spline. The right comparison is each calibrator with and without M, because our claim is that M improves any calibrator of the native score, not that one form of VMC is best; VMC is a plug-in term, orthogonal to the choice of calibrator. For Platt scaling, adding M gives general VMC, Eq. (A) above.
“Adding mass to it gains very little.” The gain from M concentrates on the inputs that matter: on attacks the native guard misses, adding M to Platt scaling lowers NLL four to six times more than on average on Guard 3 and Guard 4 (table below).
Shuffled mass (W2)
The gain from M is “information specific to each input,” not “a general upward shift”: on top of Platt scaling, which already captures any shift, M shuffled across inputs gives almost no gain while the real M does (table below; added to Table 12, Table 10 in the submission).
References
[1] Inan et al. Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations. arXiv:2312.06674, 2023.
[2] Wang et al. Two-stage LLM Fine-tuning with Less Specialization and More Generalization. ICLR, 2024.
[3] Liu et al. On Calibration of LLM-based Guard Models for Reliable Content Moderation. ICLR, 2025.