- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies how adding noise to LLM hidden states changes safety judgments. It compares different noise distributions, directions, and token positions, and finds that noise can weaken the differences between inputs' outputs. The authors also propose a simple output model to predict safety margin changes and analyze whether decision flips improve or hurt classification accuracy.
Critical strengths
None
Critical weaknesses
- Results don't seem to be generalizable even between the 3 models tested. Qwen7B layer 14 seems to be the single setting that most of the conclusions are based on. Most conclusions and discussions seem to be limited to the specific experiments in the paper and not well connected to broader concepts that may be of interest to the community.
- Based on line 254, early means any token other than "the last prompt token after the chat template". This experiment seems unfair, in that there are many more tokens in the early part and also they can propagate forward, whereas the final part is just one last token alone. Claiming "earlier positions drive attenuation" in such a setting and only based on some of the tested models seems very broad and insufficiently supported.
- I feel that the paper needs a clearer explanation of its overall research goal, the motivation behind its specific experimental setup, and the rationale for each experiment. In particular, the authors should explain what question each experiment is intended to answer and how its findings contribute to a deeper understanding of LLM safety behavior.
- Section 6 is not a substantial contribution to be in the main text. Previous work already measures quality under perturbation.
Questions for authors
Suggestions:
- L15: "responsive Qwen7B layers"; clarify what responsive means in this context.
- Please add a qualitative example of the actual prompts and examples you used, showing which token you are using for the next token and calculating logits and margins on top of it. It can make reading the paper much less confusing.