acceptodds
Under review as a conference paper at ICLR 2027

SAFETY ALIGNMENT IS CONTEXT-BLIND: MODELS DETECT DISTRESS BUT DO NOT ACT ON IT

Abstract

Large language models can recognise emotional distress in user context, yet this information may fail to influence their safety decisions. We identify and characterise this failure mode as context-blind responding, in which models detect distress but fail to integrate it with the risk of a subsequent request. Using controlled counterfactual scenarios, we evaluate two open-weight models (Llama 3.1 8B and Qwen3 4B) and two proprietary models (GPT-5.4 and Gemini 3.6) and find that the two open-weight models answer roughly three-quarters of distress-conditioned risky requests that should be declined, while proprietary models continue to answer a substantial fraction. Mechanistic analysis reveals that distress is linearly decodable from the residual stream as early as the first transformer layer, yet remains weakly aligned with the refusal direction. Causal tracing further shows that the effect of distress on the model's internal refusal signal emerges only in a late network region at approximately 72% of network depth, indicating a gap between contextual representation and safety computation. Motivated by this finding, we introduce a conjunctive gate that combines risk detection with the refusal direction at this integration region, substantially improving selective refusal while preserving helpfulness. These results suggest that an important class of safety failures arises not from missing information, but from failures to integrate information already represented within the model.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
safety alignment, context-dependent safety, mechanistic interpretablity

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_QAV5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies a context-sensitive safety failure: models can represent a user’s distress but may still answer a request whose risk becomes apparent in that context. The authors introduce a factorial benchmark that varies distress context and request type, and report this behavior across several model families. Analyses of open-weight models suggest that distress information is available in hidden states but has limited influence on the later refusal computation. The paper then proposes a gate that strengthens refusal for distress-conditioned risky requests. The intervention substantially improves refusal on the target condition, although its selectivity across closely related conditions remains an important question.

Critical strengths
  1. An interesting and consequential problem formulation. The factorial design, particularly the comparisons among C1, C2, and C3, makes it possible to examine whether models respond to the combination of distress and request risk rather than either factor alone. The behavioral results across model families support the relevance of this failure mode.

  2. A useful connection between analysis and intervention. The paper goes beyond reporting failure rates: it examines internal representations and proposes a targeted intervention. The substantial improvement on C1, together with the comparisons to existing methods, makes the empirical investigation informative.

Critical weaknesses
  1. The mechanistic evidence needs stronger validation at the level of generated responses and a sensitivity check for the refusal direction. Figure 2 evaluates activation patching primarily through projection onto a refusal direction. This is a useful continuous measure for locating an effect, but it does not establish how often patching changes the model’s actual response from an answer to a refusal. A small response-level evaluation at representative layers would make the causal interpretation more convincing. In addition, the refusal direction is derived from an independent contrastive dataset. That independence is valuable, but the authors could test whether the conclusions are sensitive to this choice—for example, by constructing a direction from training-split C5−C3 contrasts and evaluating it on held-out examples. Such a contrast could also capture differences in request content, so I would treat it as a robustness check rather than assume it is a better refusal direction.

  2. The intervention substantially improves refusals in C1, but it also causes many unnecessary refusals in C3, where the same request appears without distress. In Table 5, the C3 refusal rate rises from 4.3% to 85.3% for Llama and from 2.7% to 59.0% for Qwen. Since distinguishing C1 from C3 is central to the paper’s goal, this limits the claim that the fix is context-selective. The appendix variant reduces the problem; I would like the authors to compare its C1/C2/C3 trade-off directly with the main method.

Questions for authors
  1. For representative layers in Figure 2, what fraction of patched examples actually change their generated response from answering to refusing? How closely does this behavioral change track the refusal-direction projection?

  2. How sensitive are the Figure 2 conclusions to the choice of refusal direction? Could you compare the independent direction with a direction constructed from training-split C5−C3 examples and evaluated on held-out data?

  3. Can you report the C1, C2, and C3 results together for both the main gate and the appendix’s distress-aware variant, and explain the trade-off that motivates your choice of main method?

3 comments
AuthorsOpenReview ↗

A gentle reminder

Dear Reviewer QAV5

As the discussion period is closing soon, we kindly request your acknowledgment of our responses. We have carefully addressed each of your concerns and conducted additional experiments where needed. If our response has addressed your concerns, we would greatly appreciate it if you could reconsider your ratings accordingly.

AuthorsOpenReview ↗

Final Reminder to Reviewer

Dear Reviewer QAV5,

As the rebuttal period is nearing its end, we wanted to follow up on our responses to your comments. Since we have not received any further questions or requests for clarification, we would appreciate it if you could consider revisiting your scores in light of the clarifications and additional experiments we provided.

We have put considerable effort into this work and into addressing the concerns raised, so your reconsideration would mean a great deal to us.

Thank you again for your time.

AuthorsOpenReview ↗

Final Reminder to Reviewer.

Dear Reviewer QAV5,

Thank you again for taking the time to review our rebuttal and consider our responses.

The rebuttal period is about to end. If there are no further concerns and our clarifications and additional experiments have satisfactorily addressed your initial comments, we would be very grateful if you could reconsider your score.

We sincerely value your feedback and would appreciate any further guidance you may have. Thank you again for your time and consideration.

Reviewer_f98LOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper use controlled counterfactual scenarios to analyze emotional distress in safety decision and find that the two open-weight models answer roughly three-quarters of distress-conditioned risky requests that should be declined, while proprietary models continue to answer a substantial fraction.

Critical strengths
  1. This paper is very well-written, with a clear line of reasoning, and the experiments correspond directly to the questions.
  2. The ConjFix Gate proposed in this paper demonstrates outstanding effectiveness compared to other methods.
Critical weaknesses
  1. The paper does not sufficiently explain the research motivation—specifically, why the authors chose to study the phenomenon of “context-blind responding” in the “Distress” context. This appears to be a general issue that can lead models to trigger various types of out-of-bound behavior.
  2. On the other hand, in the “Distress” scenarios mentioned in this paper, queries often carry emotional connotations, which may be one of the reasons for the model’s inconsistent performance. There are already several studies in the academic community that have explored the impact of different emotions on model inputs. The paper should consider incorporating these methods as background and related work, and compare them with the methods mentioned herein. If other “emotional + dangerous request” scenarios exist, could they also be incorporated into the framework of this paper?
  3. The primary concern with this paper is that the research is based solely on a relatively narrow scenario: “Distress.” However, numerous studies have analyzed the reasons behind the success of more general jailbreak attack methods, whose implications are similar to the “context-independent responses” mentioned in this paper. Similarly, when comparing ConjFix, these defense methods should also be included in the comparison.
  4. The remediation methods mentioned later in the paper are relatively simple. Could the authors explore why this inconsistency occurs from a deeper, model-internal perspective, rather than simply relying on a single module to have “distress” counteract “danger”?
Questions for authors

Questions:

  1. Why is the “context-independent response” discussed only in the “Distress” scenario? Does this phenomenon also exist in other security and risk control scenarios? Does this constitute a systemic risk of the model?
  2. The models used in the authors’ experiments were all relatively small. Do larger-scale models exhibit similar characteristics?
  3. If the combination were not a simple semantic overlay—“distress” + “dangerous request”—but a more complex form of coupling, would the same phenomenon occur? Can ConjFix detect and defend against it?
4 comments
AuthorsOpenReview ↗

A gentle reminder

Dear Reviewer f98L

As the discussion period is closing soon, we kindly request your acknowledgment of our responses. We have carefully addressed each of your concerns and conducted additional experiments where needed. If our response has addressed your concerns, we would greatly appreciate it if you could reconsider your ratings accordingly.

AuthorsOpenReview ↗

Final Reminder to Reviewer

Dear Reviewer f98L,

As the rebuttal period is nearing its end, we wanted to follow up on our responses to your comments. Since we have not received any further questions or requests for clarification, we would appreciate it if you could consider revisiting your scores in light of the clarifications and additional experiments we provided.

We have put considerable effort into this work and into addressing the concerns raised, so your reconsideration would mean a great deal to us.

Thank you again for your time.

Thank you to the author for the reply. My concerns about the experiment have been addressed, so I’ve raised my score.

AuthorsOpenReview ↗

Response to Reviewer

Dear Reviewer f98L

Thank you very much for revisiting our response and for reconsidering your score. We sincerely appreciate your thoughtful feedback and engagement throughout the discussion.

Reviewer_4iMuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper identifies context blind responding as a critical failure mode where large language models recognize emotional distress in a user prompt yet still fulfill requests that become means relevant for self harm in that context. Through a controlled factorial benchmark, mechanistic analysis reveals that while distress is linearly decodable early in the residual stream, its signal remains weakly aligned with refusal and only influences safety decisions late in the network around 72% depth, a gap the authors resolve using a conjunctive gating mechanism that scales refusal direction injection by active risk projections.

Critical strengths
  1. The factorial design is exceptionally well chosen for this question. By holding the request word for word identical between C1 and C3 and only toggling the preamble, the authors isolate context risk from request semantics.
  2. Furthermore, the request level train to test split ensures the gate is evaluated on unseen request types rather than memorized semantics. This turns a qualitative safety concern into a rigorous, measurable benchmark.
Critical weaknesses
  1. The primary gating mechanism in Section 4.2 relies exclusively on the risk projection rather than the true conjunction advertised in the paper.
  2. The mechanistic diagnostic localizes integration to late network depth, whereas the proposed gating mechanism achieves optimal performance at much earlier layers, creating a depth mismatch that weakens the causal narrative. Intervening inside the late integration region yields poor results, as shown by layer 27 in Figure 11. Despite framing the gate as an intervention at the integration region, the empirical sweep results fail to support this correspondence.
Questions for authors
  1. Can you report the refusal rate for the main gate under condition C3 in Table 2 and reframe the selectivity claims to account for all three conditions?
  2. What is your mechanistic explanation for why injecting at these earlier layers works, and is the gate compensating for the downstream failure rather than restoring the diagnosed integration circuit?
3 comments
AuthorsOpenReview ↗

A gentle reminder

Dear Reviewer 4iMu

As the discussion period is closing soon, we kindly request your acknowledgment of our responses. We have carefully addressed each of your concerns and conducted additional experiments where needed. If our response has addressed your concerns, we would greatly appreciate it if you could reconsider your ratings accordingly.

Thank you for your detailed response and the additional clarifications. I have carefully read the rebuttal and the other reviewers' comments. After consideration, I have decided to maintain my original score. I appreciate the authors' efforts and wish them the best with this work.

AuthorsOpenReview ↗

Response to Reviewer 4iMu

Dear Reviewer 4iMu,

Thank you for taking the time to consider our rebuttal. We understand your decision to maintain the score, but before the discussion period closes, we wanted to make one final appeal for reconsideration.

We have made a sincere effort to address your concerns through our responses and additional experiments, and we hope these clarifications have helped strengthen the paper. We would be very grateful if you could reconsider your assessment in light of these additions. Your support would mean a lot to us, given the effort we have put into this work.

Thank you again for your time and thoughtful feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.