acceptodds
Withdrawn from ICLR 2027

Answer Pressure and Unsupported Answering in Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

When instructed to answer only from a supplied document, a language model should abstain if the document lacks the required evidence. We examine whether removing an activation direction associated with answer pressure reduces unsupported answering while retaining supported responses. The design crosses supported and evidence-withheld contexts with neutral and pressure instructions and three intervention arms. In a paired analysis of author-reported model-run records, pressure increases unsupported answering by 15.0 and 13.5 percentage points in Gemma-2-9B-IT and Gemma-3-12B-IT. Target removal reduces these rates by 5.50 and 5.00 points relative to no intervention, reversing approximately 37% of the increase. Its advantage over assigned random controls is larger under pressure than under neutral instructions. Most of the net change is an increase in abstention; supported-accuracy point estimates decrease by 0.40 and 0.52 points. Comparisons with refusal and arousal controls and a separate ecological panel further characterize the effect. The contribution is a pressure-conditioned evaluation of selective responding using an established intervention operator. The instruction contrast jointly varies decisiveness and evidence-rule wording, and substantial unsupported answering remains. Appendix A documents data versions and provenance.

Primary area:
interpretability and explainable AI
Keywords:
Language model reliability, abstention, unsupported commitment, answer pressure, activation steering, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Q5p4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates whether removing an activation direction associated with answer pressure can reduce unsupported answering in language models. The authors construct a direction from the difference between pressure and neutral instructions and remove its projection from model activations. They have test 12 settings on 4 distinct models, and applied their intervention on 2 Gemma models reduces unsupported answering by approximately 5 percentage points. Additional experiments examine random-direction controls, alternative interventions, and generalization across models.

Critical strengths
  1. Clear presentation. The paper is generally well-organized and easy to follow. I particularly appreciate the pipeline diagrams, concrete examples, and clear explanation of the experimental conditions.
  2. Reasonable experimental coverage and controls. The authors include supported and evidence-withheld conditions, random-direction controls, comparisons with refusal and arousal interventions, and additional experiments on different model families. The paired statistical analysis is also a strength.
Critical weaknesses
  1. No dataset. Concerns about experimental data provenance and reproducibility. The paper does not clearly identify the datasets used in the experiments. The primary evaluation consists of 4,800 items divided into ten source groups (S01–S10), but the underlying dataset identities and collection procedures are not specified. Reproducing the results of this paper would be extremely difficult.
  2. Limited novelty and modest empirical improvements. The paper applies an established activation intervention technique to unsupported answering under pressure. However, prior work has already investigated linear representations and interventions for answerability [1], confidence and abstention [2], verbal uncertainty [3], and refusal behavior [4]. The main novelty appears to be constructing the intervention direction from pressure-related instructions, but I am not convinced that this provides substantially new insights into the underlying mechanisms. Moreover, the intervention reduces unsupported answering by only around 5 percentage points in the primary experiments, with even smaller improvements in the ecological evaluation. Given the limited methodological novelty and relatively modest practical gains, I find the overall contribution incremental.

[1] Lavi, M. J., Milo, T., & Geva, M. (2026). Detecting (Un)Answerability in Large Language Models with Linear Directions. EACL 2026.

[2] Kumaran, D., et al. (2026). Causal Evidence That Language Models Use Confidence to Drive Behaviour. Nature Machine Intelligence.

[3] Ji, Z., et al. (2025). Calibrating Verbal Uncertainty as a Linear Feature to Reduce Hallucinations. EMNLP 2025.

[4] Arditi, A., et al. (2024). Refusal in Language Models Is Mediated by a Single Direction. NeurIPS 2024.

Questions for authors

N/A

Reviewer_HhaUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates LMs' abstention vs. answering capabilities in a contextual QA task, where questions are present with vs. without supporting evidence, additionally introducing conditions manipulating the prompt (neutral vs. "with pressure" where the model is urged to answer). It additionally investigates whether removing an activation direction that corresponds to the "pressure" increases abstention accuracy, while also retaining answering performance. Evaluating two Gemma models and in part a Llama model, the paper finds that the "pressure" prompt substantially increases the over-answering, but the activation editing can reverse a part of that effect through increasing the abstention rate. Overall, the paper contributes comparisons of LMs' selective question answering capabilities in particular prompting conditions.

Critical strengths
  • The study has a clear well-motivated factorial design, crossing different contexts (supporting vs. not supporting answering a question) and the prompt conditions (neutral vs. pressure), and uses a suite of clearly introduced estimands for evaluating the models.
Critical weaknesses
  • The pressure prompt introduces a joint variation of two aspects relative to the neural prompt: both phrasings of how exactly the model should rely on the evidence in the context, and how strongly the answer is needed, are different from the neutral prompt, making attribution of observed effects confounded. In general, the intervention results seem quite specific to the particular prompts, and the study would benefit from an assessment of robustness of the results to at least paraphrases of the prompt, and a discussion of the potential generalizability of the findings for LMs' abstention capabilities in other QA settings.
Questions for authors

Several aspects were not clear to me:

  • What exactly is the ecological panel? The introduction in section 3.1 was not clear to me.
  • Arousal controls controls are mentioned several times but not introduced, could the authors please clarify the controls?
  • There is a lot of literature investigating LMs' abstention capabilities, and the related work review is lacking that.
  • Maybe I missed it, but from which dataset are the items sourced?
Reviewer_Et7UOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether answer pressure increases unsupported answering in LLMs and whether removing pressure-associated activation directions can mitigate this behavior. Using a paired experimental design that varies instruction conditions, evidence availability, and activation interventions, the authors evaluate two Gemma models and conduct additional evaluations on Llama and Qwen. The results show that pressure increases unsupported answering, while directional removal partially reduces this effect, primarily by encouraging abstention with relatively small changes in supported-answer accuracy. The study highlights the potential of activation-based interventions for improving context-grounded answering under pressure.

Critical strengths

The paper investigates how answer pressure affects context-grounded answering, addressing an important issue in LLM reliability. Examining whether internal interventions can reduce unsupported answering while preserving supported responses is practically relevant. In addition, it employs a paired experimental design that considers instruction conditions, evidence availability, and intervention types. Comparisons with random, refusal, and arousal controls, together with evaluations of abstention and supported accuracy, provide useful evidence for assessing the intervention's effectiveness and potential trade-offs.

Critical weaknesses
  1. Novelty and distinction from prior work. Previous studies have shown that user pressure can influence model responses and that activation directions associated with answerability, uncertainty, or refusal can modify answering behavior. While the pressure-conditioned evaluation is interesting, the paper could better clarify its novel contributions beyond these findings. In particular, comparisons with existing answerability or uncertainty directions would help establish whether the observed effects are specific to answer pressure.

  2. Validity of the answer-pressure manipulation. The neutral and pressure instructions differ not only in their emphasis on decisiveness but also in the wording of the evidence constraint. Therefore, it remains unclear whether the increase in unsupported answering is primarily attributable to answer pressure or other differences in instruction wording. A controlled experiment that keeps the evidence rule identical while varying only answer pressure would strengthen the validity of the findings.

Questions for authors
  1. Could the authors further clarify the novel insights of the proposed pressure-associated direction compared with existing answerability or uncertainty directions? Would direct comparisons with these directions help establish its distinctive role?
  2. Could the authors conduct an additional experiment that keeps the evidence constraint unchanged while varying only the degree of answer pressure? This would help determine whether the observed behavioral changes are attributable to pressure rather than instruction wording.
  3. Could the authors clarify the sources and construction procedures of the 4,800 primary evaluation items? In particular, are these questions derived from established benchmarks, real-world documents, or synthetically generated examples? How were the supported and withheld contexts constructed and validated?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.