acceptodds
Withdrawn from ICLR 2027

When Behavioral Safety Evaluation Fails: A Representation-Level Perspective

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Refusal rates describe tested behavior, but do not directly measure robustness to changes in model weights or internal states. We construct dissociated models trained to retain refusal while complying under a fixed latent perturbation. Across three aligned model families, direct-harm compliance remains low, and a fixed response-level probe retains its classification performance, although fixed-jailbreak results vary. In a monitoring-set comparison, adaptive PGD elicits harmful compliance on 54–86% of prompts versus 3–48% for the bases, without the cached direction but using the construction layer and budget; these prompts informed checkpoint selection. Fine-tuning also reaches high compliance earlier with disjoint attack data. Patching and GCG on prompts excluded from training and monitoring provide complementary evidence. An exploratory audit of four released aligned checkpoints finds a modest direction-specific signal in one model. These results demonstrate a controlled gap between the tested baseline measurements and intervention susceptibility. They motivate reporting in white-box evaluation; generalization beyond selection data, the necessity of individual construction components, and prevalence under ordinary alignment remain unresolved.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM evaluation, LLM safety, interpretability, AI safety

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_gTk8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether low harmful-completion rates adequately characterize robustness to changes in model weights or hidden states. It constructs “dissociated” models that preserve clean refusal while complying under a fixed latent perturbation, using three aligned model families. The evaluation finds increased susceptibility to adaptive latent attacks and faster degradation under harmful fine-tuning, including with disjoint attack data. Activation patching and a single-architecture GCG evaluation provide complementary evidence on prompts excluded from construction and monitoring. The paper also reports layer-wise diagnostics, a Latent Vulnerability Score, and an exploratory audit of released checkpoints. These results demonstrate a controlled separation between baseline refusal behavior and intervention robustness, although the main PGD comparison reuses checkpoint-monitoring prompts and evidence outside the constructed models remains limited.

Critical strengths

The strongest empirical contribution is that the constructed vulnerability extends beyond the cached perturbation used during training. Adaptive optimization finds effective perturbations without that direction, harmful fine-tuning produces earlier failure with disjoint attack data, and held-out GCG exposes an input-space difference on Llama. These complementary results make the study more informative than simply verifying that a trained trigger works. The paper carefully distinguishes the properties measured by its evaluations. It acknowledges that response classification is different from predicting intervention susceptibility, that whole-state patching does not identify a complete mechanism, and that monitoring-set results are not independent generalization evidence. Reporting paired base-model comparisons under common intervention budgets also provides a useful basis for assessing the magnitude of the constructed differences.

Critical weaknesses

The novelty is insufficiently established relative to existing work on shallow refusal mechanisms, latent attacks, and trigger-conditioned vulnerabilities. The construction explicitly trains clean refusal alongside perturbed compliance, so the resulting separation is partly an intended consequence of the objective. The cross-intervention results add value, but the paper does not sufficiently isolate the new scientific insight beyond this construction. In particular, no construction-level ablation establishes whether representation matching or the harmful-reference direction is necessary. The proposed LVS also lacks demonstrated predictive or diagnostic value beyond intervention success rates and fine-tuning onset. Consequently, neither the mechanism analysis nor the new metric provides a sufficiently strong additional contribution. The evidence supporting the broader evaluation lesson is limited. The headline PGD comparison uses prompts that informed checkpoint selection and retains the construction layer and budget; held-out patching and GCG test different interventions rather than independently reproducing this result. Moreover, the frozen probe classifies supplied refusal and compliant responses, so its preserved performance provides limited evidence about whether static monitoring could detect vulnerability. The behavioral baseline is small, and GCG itself shows that active behavioral evaluation can expose the difference. Finally, the released-checkpoint audit finds only a modest steering signal in one of four aligned models. Although these limitations are acknowledged, they leave the practical significance of the controlled counterexample insufficiently established to compensate for the limited novelty.

Questions for authors
  • Were the construction layer, perturbation scale, and loss weights fixed before examining monitoring results, or were they selected through trials on the monitoring behaviors? Please clarify the extent of hyperparameter and checkpoint selection, as this affects the interpretation of the headline susceptibility gap.

  • Can the main paired PGD comparison be reported on prompts excluded from both construction and checkpoint monitoring, while keeping the layer, budget, and attack settings unchanged? This would directly address whether the strongest reported gap generalizes independently of model selection.

Reviewer_g6bnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors investigate how safety changes when internal states are perturbed. To answer this question, the authors compare the robustness to interventions of models that have similar measured refusal behavior. To this end they design “dissociated models”, aligned models fine-tuned to retain refusal behavior while complying under a fixed perturbation, and measure what they define as the “audit gap”, i.e. the discrepancy between some baseline measurements and the susceptibility under specified interventions. The authors study three LLMs and their dissociative models (Gemma 2-2B, Llama 3.2-3B, Qwen 2.5-3B) and report various metrics. They conclude proposing to report the sensitivity to specified modifications alongside the baseline behavioral tests.

Critical strengths

The dissociated models introduced in the paper provide a clean controlled experiment to investigate the research question raised by the authors.

Critical weaknesses
  1. Four out of the six evaluation protocols presented in the paper are performed on the same prompts monitored during fine-tuning to ensure success of the dissociated models construction, and as correctly pointed out by the authors, cannot be considered as independent evidence. Those excluded, the independent evidence for a significant difference in robustness between the base and dissociated models is not very strong, and would require additional results on held-out data to be convincing. The evidence from released checkpoints is also unclear and not discussed at length.

  2. The overall conclusions of the study are not clear in the main text. Discussion of the results and limitations are missing. The contributions listed in the introduction are not clear.

Questions for authors

Clarifications:

  • Can you comment on the results and the limitations of the research? this is missing in the main text.
  • Contributions (L 82-89) needs rewording and clarifications: (i) "A controlled construction separates" , should it be "A controlled construction THAT separates"? (ii) "beyond the cached construction direction" is unclear. (iii) what is the contribution here?
  • Related work: what the authors do to build dissociated models reminds of model organisms, the authors should position their work against it.
  • L. 119: “retain clean refusal” is used several times throughout the paper but never explicitly explained, please clarify.
  • L. 143: “Evaluation without the cached direction does not establish that construction is independent of it”. does “without” mean “not relying on”? What does this sentence mean? Does it even belong to section 3? It looks like it should be in the evaluation section.
  • L. 153: “necessity for intervention susceptibility remains untested.” Please explain what this means
  • L. 157: add citation for HarmBench (optional, at line 187 as well)
  • Figure 2: what is on the y axis in the two rows? There are six entries in the legend but I can only see five lines of different colors in the plot. Can you explain how the various lines are computed?
  • L. 185: “supplies additional context” say more clearly what the role of the harmfully fine-tuned reference is, how it should be interpreted and why
  • L. 239: why three different values of p and how were they chosen?
  • Figure 4 is never referred to in the main text.
  • L. 263: what does LVS stands for? you say it once in the introduction but it would help to say it here as well. You should also say whether you introduced this metric or found it in previous literature, and motivate why this is a good metric. Also please clarify how you compute it. y and y’ are the clean and perturbed, how do you get the perturbed from the clean? How do you compute it for the harmful reference?
  • Figure 6: the "random direction" is not visible and the dashed line in the legend is almost identical to the dashed line used to indicate the trained layer. This is confusing and it makes hard to read the figure.

Presentation:

  • The abstract is hard to understand before one has read the paper: “cached direction" is not defined; “Fine-tuning also reaches high compliance earlier with disjoint attack data.” “earlier” than what? “tested baseline measurements” are not defined
  • L. 51-80 “Scope and relation to prior work.” this paragraph seems like a list of disconnected sentences. The first one seems to be a motivation, but it does not elaborate enough; the last sentence (L 80) is stating one of the results, but what is its role in this paragraph? (“baseline measurements” are not defined yet)
  • L. 186: “ASR” should be fully written at least once. Also you say ASR and compliance are the same thing, but clearly they are not. Please reword this sentence.
  • Figure 3 caption collides with the text
Reviewer_F77oOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

From what I can tell, this paper suggests that a multifaceted approach is needed to evaluating safety in open-weight models. They create a class of models they call "dissociated" which are more easily jailbroken under some types of attacks (like fine-tuning attacks) by modifying some representations. They then show how standard refusal evaluation won't detect this without some sort of representation or adaptive attack evaluation.

Critical strengths

Being generous, there's a version of this paper that perhaps has value in laying out how safety testing should be done, looking at representations, adaptive attacks, and static refusal behavior analyses as best practices.

Critical weaknesses

Frankly, this paper is not well written and appears to be mostly AI-generated. As a result the language is often imprecise and uses very unusual terminology for the field. Many terms are used before they are defined, or never defined at all, and I had to go to the appendix to piece together what several sentences meant. In its current state it cannot and should not be accepted at this venue. Perhaps with a thorough revision for clarity it might be easier to assess this work.

Here are some examples:

  • "aligned models fine-tuned to retain clean refusal behavior while complying under a fixed mid-layer perturbation" -> What is a "clean" refusal behavior? What would be an "unclean" refusal behavior?
  • Figure 1 has an x-axis referring to deception, but as far as I can tell nowhere else in the paper is deception ever mentioned?
  • "Clusters and safety labels are schematic, not measured geometry or audit certificates." -> What is an "audit certificate"? Why is this qualification even here?
  • "Layer-wise diagnostics and state transfer characterize where the constructed models respond differently" -> What is "state transfer"? From the citations it seems to mean activation patching. If so, I would just call it that.
  • Many many many more such examples.

Even being generous and trying to move past the writing problems (which should be disqualifying on its own), there seem to be many other fundamental problems with this paper. These include:

  • The evaluations are on very small, old models.
  • The evaluations and experiments are based on quite high variance techniques with spotty accounting for this variance (see, e.g., Table 2's lack of CIs).
  • Not engaging with a wide range of other relevant literature that already examines these issues, including: Che et al. "Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities." Qi et al., "On Evaluating the Durability of Safeguards for Open-Weight LLMs"

But it is difficult to do a more rigorous assessment of the underlying experiments and content because, importantly, the paper is bordering on incomprehensible.

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.