- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work investigate whether activations before the first generated token can predict a later false refusal. The finding seems interesting - It finds strong predictive performance in Gemma and Qwen, including gains over a combined text-and-metadata baseline. However, the intervention results are more limited, and the authors explicitly separate prediction from evidence of causal control.
Critical strengths
- This is an interesting problem. The task is clearly defined. In particular, the paper distinguishes an answerable prompt from an answer-like response and avoids using generated refusal text as a feature. I appreciate how directly the authors report the unsuccessful intervention controls (this is commendable). The discussion of inactive random SAE features is useful and prevents an apparently strong intervention result from being overinterpreted.
Critical weaknesses
-
My main reservation is how much the incremental prediction result tells us beyond the particular baseline. Character n-grams and metadata are a reasonable starting point, but the activations contain a much richer semantic representation of the prompt. A gain over this baseline could reflect that difference rather than refusal-specific information. In short, I think this is interesting, but I am worried how informative is this finding.
-
The main results also reuse historical cohorts. Nested CV addresses selection within the tested grid, but it does not establish transfer to genuinely new prompt sources. The leave-one-dataset-out analysis is helpful, although it compares against text alone rather than the combined baseline used for the headline claim. I would particularly like to see that combined comparison on a held-out source.
Questions for authors
-
Do the activation gains remain against a semantic text encoder, using the same folds and metadata?
-
Can the combined-baseline result be evaluated on a new source, with an auditable account of the response labels and generation settings?