acceptodds
Under review as a conference paper at ICLR 2027

The First Unsafe Token: When and Where Jailbreaks Take Over Language Models

Abstract

Aligned language models can refuse a harmful request yet comply when the same behavior is wrapped in a jailbreak. Recent work shows that refusal and jailbreak signals evolve across decoding trajectories, but a sharper causal question remains: when does a successful jailbreak cross a localized transition that controls subsequent generation? We introduce TIDE (Tokenwise Intervention for Decision Emergence), a layer by token analysis that replays an identical successful response prefix under a refused harmful prompt and its jailbreak, isolating prompt conditioned state divergence from lexical divergence. The First Unsafe Token (FUT) is localized from a development fixed, prefix aligned boundary score, and causal patching then verifies that final refusal and compliance is maximally sensitive near that boundary, including on natural free running trajectories. Our fixed protocol uses 160 behavior disjoint harmful requests, five attack families, four open weight model families, and 300 outer family test attempts per model. We observe an early boundary centered near token 4, with successful attacks crossing by token 6 far more often than failed attacks. This structure motivates TIDE-Guard, which reads the first four hidden states. Under nested held out attack family evaluation, TIDE-Guard reaches 0.91 AUROC, versus 0.80 for the strongest evaluated published baseline and 0.81 for a capacity matched four token state probe. At the endpoint, TIDE-Guard reduces jailbreak ASR from 64% to 27% at 4.2% benign refusal. Across four model families, the same early boundary structure reappears under model specific calibration.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Jailbreaks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2Rg7OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TIDE, a framework for identifying the First Unsafe Token (FUT), where successful jailbreaks exhibit persistent divergence in safety-related representations during early decoding. The authors employ prefix-aligned replay to compare hidden-state trajectories under direct harmful requests and their jailbreak variants, followed by activation patching to validate the causal influence of the identified boundary. Based on these findings, they introduce TIDE-Guard, an early jailbreak detector that uses temporal features from the first four generated hidden states. Experiments across four LLM families and five jailbreak attack families demonstrate improved detection performance and reduced jailbreak success rates compared to the evaluated baselines.

Critical strengths
  1. Interesting research question. The paper investigates when jailbreak-related state transitions occur during decoding, extending existing studies of static refusal representations toward temporal dynamics.

  2. Causal validation. The combination of prefix-aligned replay and bidirectional activation patching provides useful evidence that hidden states near the identified FUT can significantly influence subsequent model behavior. The additional experiments on natural generation trajectories strengthen this finding.

Critical weaknesses
  1. The comparision with refusal direction is potentially unfair. The defense method requires access of hidden states on multiple jailbreak inputs, while construction of refusal direction only requries raw harmful inputs. To better observe the performance gain, I recommend authors to measure multiple refusal directions, by: hattack(i)−hharmh_{attack(i)}-h_{harm}, then use those vectors to train a classifier upon it.

  2. The defense method is still using temporal safety features, this is similar with other trajetory defense methods, the novelty on defense mechnistics is limited. Also, the FUT study, which is the new thing, the counterfactual design that append harmful output on raw harmful input, is not reflected on the defense method. Therefore, the defense method simply process temporal features and safety projections without FUT, I cannot see too much novelty on the defense method over other trajectory based methods.

  3. The defense method should be further assessed, because the statistics used for detection/mitigation is huristic. Authors could use collected trajctory with fixed length of hidden states of jailbreaks and raw harmful to directly train a binary classifier/energy function for detection/mitigation.

  4. Should be compared with SALO and TrajGuard as mentioned in related work. In particular, TIDE-Guard uses temporal slope, layer variance and persistence, those are common statistics, authors should clarify the advantage of the used statistics of defense method over the TrajGuard ones.

Questions for authors

see weakness

Reviewer_nAVsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors localize the first unsafe token for LLM jailbreaking based on several models and a dev set. They observe that an early boundary is centered near token 4, with successful attacks crossing by token 6. The authors then propose TIDE-Guard, which focuses on the hidden states for the first four tokens to reduce ASR.

Critical strengths

I genuinely like how the authors reveal the issues of existing LLMs, analyze them, and explore solutions to help reduce ASR. I think relating safety to tokens is interesting (though debatable).

Critical weaknesses

I wonder whether token 4 and token 6 are artifacts of the model generations. What are the exact examples of the model generations, and what kinds of words or phrases do the models generate for the first few tokens? Why are token 4 and token 6 so important?

The authors should consider using stronger models like Qwen 3 and Qwen 3.5 in their experiments.

The authors should also polish the presentation of the paper in general.

Questions for authors

See weaknesses.

Reviewer_h2nPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Understanding how jailbreaks work and how to prevent them is now central in the field of AI safety and interpretability. The authors tackle a relevant question: when is the point at which the jailbreaks cross a transition that controls the generation of the output? The authors build an algorithm to identify the First Unsafe Token (FUT), and show that behaviour towards complying or refusing is very sensitive near that boundary. They observe an early boundary around token 4 and show that the method successfully identifies jailbreaks using that information.

Critical strengths
  1. The paper asks an important question, i.e. when a jailbreak takes over, not only whether it can be found. The analysis runs the same response text under the refused prompt and the jailbreak to single out the differences. This is a relevant question to ask and a good motivation for the paper.

  2. Results seem strong, with a clear advantage over other methods.

Critical weaknesses
  1. It is not clear to me that the strong results are correctly interpreted as coming from the jailbreak time being identified. For the detector, which reads the model’s own output, I would expect a refusal to already appear in the first tokens, which would trivially separate it from a compliant reply; the paper’s surface baseline only uses the first token. Also, from what I can say there is already some study on the first few tokens in safety alignment which might predate these results (Qi et al. ICLR 2025, proceedings.iclr.cc/paper_files/paper/2025/hash/88be023075a5a3ff3dc3b5d26623fa22-Abstract-Conference.html), which should be mentioned in the paper.

  2. The main object of the paper, FUT, is irreproducible from its definition in equation 5, because the quantities it depends on (f, safety axis, t_m, q, r) are not specified in the main text. If they were specified in appendix, this should be moved to main text as it is an essential part of what the paper is computing.

Questions for authors
  1. Which settings define FUT? I have not exactly understood how it is defined in the main text, along with things derived or deriving from it
  • In Equation 1 and 2, the states come from f, which is never defined
  • in equation 3, what is mu, what is sigma?
  • in equation 4: what values are chosen for tau_m, q, r?
  1. How is the 4 token window chosen? my concern is: if ~75% successfull attack cross by token 6, and the guard stops reading before token 5, for a portion of successfull attacks, the boundary has not yet happened when the guard makes the decision. Yet, AUROC scores 0.91. So it might be very possible that it detects something coming before the boundary, weakening the claim.

  2. Can you add a simple baseline that reads the first four generated tokens (e.g. refusal phrases) and make sure your results add something new to that baseline?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.