- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TIDE, a framework for identifying the First Unsafe Token (FUT), where successful jailbreaks exhibit persistent divergence in safety-related representations during early decoding. The authors employ prefix-aligned replay to compare hidden-state trajectories under direct harmful requests and their jailbreak variants, followed by activation patching to validate the causal influence of the identified boundary. Based on these findings, they introduce TIDE-Guard, an early jailbreak detector that uses temporal features from the first four generated hidden states. Experiments across four LLM families and five jailbreak attack families demonstrate improved detection performance and reduced jailbreak success rates compared to the evaluated baselines.
Critical strengths
-
Interesting research question. The paper investigates when jailbreak-related state transitions occur during decoding, extending existing studies of static refusal representations toward temporal dynamics.
-
Causal validation. The combination of prefix-aligned replay and bidirectional activation patching provides useful evidence that hidden states near the identified FUT can significantly influence subsequent model behavior. The additional experiments on natural generation trajectories strengthen this finding.
Critical weaknesses
-
The comparision with refusal direction is potentially unfair. The defense method requires access of hidden states on multiple jailbreak inputs, while construction of refusal direction only requries raw harmful inputs. To better observe the performance gain, I recommend authors to measure multiple refusal directions, by: , then use those vectors to train a classifier upon it.
-
The defense method is still using temporal safety features, this is similar with other trajetory defense methods, the novelty on defense mechnistics is limited. Also, the FUT study, which is the new thing, the counterfactual design that append harmful output on raw harmful input, is not reflected on the defense method. Therefore, the defense method simply process temporal features and safety projections without FUT, I cannot see too much novelty on the defense method over other trajectory based methods.
-
The defense method should be further assessed, because the statistics used for detection/mitigation is huristic. Authors could use collected trajctory with fixed length of hidden states of jailbreaks and raw harmful to directly train a binary classifier/energy function for detection/mitigation.
-
Should be compared with SALO and TrajGuard as mentioned in related work. In particular, TIDE-Guard uses temporal slope, layer variance and persistence, those are common statistics, authors should clarify the advantage of the used statistics of defense method over the TrajGuard ones.
Questions for authors
see weakness