- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper investigates whether removing an activation direction associated with answer pressure can reduce unsupported answering in language models. The authors construct a direction from the difference between pressure and neutral instructions and remove its projection from model activations. They have test 12 settings on 4 distinct models, and applied their intervention on 2 Gemma models reduces unsupported answering by approximately 5 percentage points. Additional experiments examine random-direction controls, alternative interventions, and generalization across models.
Critical strengths
- Clear presentation. The paper is generally well-organized and easy to follow. I particularly appreciate the pipeline diagrams, concrete examples, and clear explanation of the experimental conditions.
- Reasonable experimental coverage and controls. The authors include supported and evidence-withheld conditions, random-direction controls, comparisons with refusal and arousal interventions, and additional experiments on different model families. The paired statistical analysis is also a strength.
Critical weaknesses
- No dataset. Concerns about experimental data provenance and reproducibility. The paper does not clearly identify the datasets used in the experiments. The primary evaluation consists of 4,800 items divided into ten source groups (S01–S10), but the underlying dataset identities and collection procedures are not specified. Reproducing the results of this paper would be extremely difficult.
- Limited novelty and modest empirical improvements. The paper applies an established activation intervention technique to unsupported answering under pressure. However, prior work has already investigated linear representations and interventions for answerability [1], confidence and abstention [2], verbal uncertainty [3], and refusal behavior [4]. The main novelty appears to be constructing the intervention direction from pressure-related instructions, but I am not convinced that this provides substantially new insights into the underlying mechanisms. Moreover, the intervention reduces unsupported answering by only around 5 percentage points in the primary experiments, with even smaller improvements in the ecological evaluation. Given the limited methodological novelty and relatively modest practical gains, I find the overall contribution incremental.
[1] Lavi, M. J., Milo, T., & Geva, M. (2026). Detecting (Un)Answerability in Large Language Models with Linear Directions. EACL 2026.
[2] Kumaran, D., et al. (2026). Causal Evidence That Language Models Use Confidence to Drive Behaviour. Nature Machine Intelligence.
[3] Ji, Z., et al. (2025). Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations. EMNLP 2025.
[4] Arditi, A., et al. (2024). Refusal in Language Models Is Mediated by a Single Direction. NeurIPS 2024.
Questions for authors
N/A