- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates whether low harmful-completion rates adequately characterize robustness to changes in model weights or hidden states. It constructs “dissociated” models that preserve clean refusal while complying under a fixed latent perturbation, using three aligned model families. The evaluation finds increased susceptibility to adaptive latent attacks and faster degradation under harmful fine-tuning, including with disjoint attack data. Activation patching and a single-architecture GCG evaluation provide complementary evidence on prompts excluded from construction and monitoring. The paper also reports layer-wise diagnostics, a Latent Vulnerability Score, and an exploratory audit of released checkpoints. These results demonstrate a controlled separation between baseline refusal behavior and intervention robustness, although the main PGD comparison reuses checkpoint-monitoring prompts and evidence outside the constructed models remains limited.
Critical strengths
The strongest empirical contribution is that the constructed vulnerability extends beyond the cached perturbation used during training. Adaptive optimization finds effective perturbations without that direction, harmful fine-tuning produces earlier failure with disjoint attack data, and held-out GCG exposes an input-space difference on Llama. These complementary results make the study more informative than simply verifying that a trained trigger works. The paper carefully distinguishes the properties measured by its evaluations. It acknowledges that response classification is different from predicting intervention susceptibility, that whole-state patching does not identify a complete mechanism, and that monitoring-set results are not independent generalization evidence. Reporting paired base-model comparisons under common intervention budgets also provides a useful basis for assessing the magnitude of the constructed differences.
Critical weaknesses
The novelty is insufficiently established relative to existing work on shallow refusal mechanisms, latent attacks, and trigger-conditioned vulnerabilities. The construction explicitly trains clean refusal alongside perturbed compliance, so the resulting separation is partly an intended consequence of the objective. The cross-intervention results add value, but the paper does not sufficiently isolate the new scientific insight beyond this construction. In particular, no construction-level ablation establishes whether representation matching or the harmful-reference direction is necessary. The proposed LVS also lacks demonstrated predictive or diagnostic value beyond intervention success rates and fine-tuning onset. Consequently, neither the mechanism analysis nor the new metric provides a sufficiently strong additional contribution. The evidence supporting the broader evaluation lesson is limited. The headline PGD comparison uses prompts that informed checkpoint selection and retains the construction layer and budget; held-out patching and GCG test different interventions rather than independently reproducing this result. Moreover, the frozen probe classifies supplied refusal and compliant responses, so its preserved performance provides limited evidence about whether static monitoring could detect vulnerability. The behavioral baseline is small, and GCG itself shows that active behavioral evaluation can expose the difference. Finally, the released-checkpoint audit finds only a modest steering signal in one of four aligned models. Although these limitations are acknowledged, they leave the practical significance of the controlled counterexample insufficiently established to compensate for the limited novelty.
Questions for authors
-
Were the construction layer, perturbation scale, and loss weights fixed before examining monitoring results, or were they selected through trials on the monitoring behaviors? Please clarify the extent of hyperparameter and checkpoint selection, as this affects the interpretation of the headline susceptibility gap.
-
Can the main paired PGD comparison be reported on prompts excluded from both construction and checkpoint monitoring, while keeping the layer, budget, and attack settings unchanged? This would directly address whether the strongest reported gap generalizes independently of model selection.