acceptodds
Under review as a conference paper at ICLR 2027

The Expression Gap: Probing What Language Models Know but Don't Say

Abstract

Instruction-tuned language models represent their own epistemic state—whether a claim is uncertain, self-contradictory, or unsupported by a provided document—more reliably inside the network than they reveal in generation. We call the difference between this internal representation and the model's output the expression gap, define it at a deployed threshold as probe fire-rate minus generation hedge-rate, and measure it across four open-weights families (Qwen2.5-7B, Mistral-7B-v0.3, OLMo-2-7B, Llama-3.1-8B). One recipe—a linear probe on the layer-17 last-input-token hidden state, fit per model and task—reads this state at AUC 0.97–1.00 on paired uncertainty and retrieval faithfulness across all four families, beats every input-only baseline (decisively on sycophancy prompts whose wording is identical across conditions), and transfers zero-shot across tasks within a model (off-diagonal mean AUC 0.96 on Qwen, 0.89 on Mistral). The gap it exposes is large and model-ordered (regex-scored 0.14–0.49; OLMo largest, Llama near zero under semantic judging) and instruction-gated: under a permissive prompt, retrieval unfaithfulness spans a 0.00–0.78 gap across families. Gating a sparse flag-token logit bias on the probe recovers expression with zero added false positives on the evaluation set (held-out gate exposure 1.7–6.3%); a blinded semantic audit of every stored generation (κ=0.94) confirms the lift on two families, exposes a third as surface-form conversion, and finds a null on the fourth. The obvious alternative—a one-line hedging instruction—lifts expression on every family but always at commensurate false-positive cost (+12 to +33pp), because a prompt cannot condition on what the model internally knows. A gated-actuator factorial locates the mechanism: the zero-FP property belongs to the probe gate, which every actuator behind it (logit bias, ITI, CAA) inherits. We release datasets, code, and the full judging record.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, linear probes, epistemic uncertainty, hallucination, retrieval faithfulness, sycophancy, activation steering, inference-time intervention, calibration, open-weights language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_1uCyOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the “expression gap” between uncertainty signals encoded in language models’ hidden states and those expressed in their responses. Across four model families, linear probes detect uncertainty, unsupportedness, and contradictions that are not consistently reflected in generated answers. The authors propose a probe-gated logit-bias intervention to encourage uncertainty expression and compare it with prompting and activation steering. A blinded semantic evaluation confirms improvements on two model families, while revealing that an apparent improvement on another primarily reflects wording changes. The results highlight model-dependent gaps between internal representations and output behavior, as well as the importance of semantic evaluation.

Critical strengths

The paper addresses an important and widely studied problem concerning the gap between LLMs’ internal uncertainty signals and their expressed behavior. It presents a comprehensive experimental evaluation across multiple model families and tasks, supported by control experiments, intervention comparisons, and a blinded semantic audit.

Critical weaknesses
  1. The expression gap is overinterpreted. Defining the expression gap as Pprobe−PgenP_{\mathrm{probe}} - P_{\mathrm{gen}} overlooks the joint distribution of probe predictions and generated responses. This difference equals P(A∩¬B)−P(¬A∩B)P(A \cap \neg B) - P(\neg A \cap B), rather than the proportion of inputs where the probe detects uncertainty but the response fails to express it, P(A∩¬B)P(A \cap \neg B). The two disagreement types can cancel out. Reporting their joint distribution would better support the “knows but does not say” interpretation.
  2. Practical applicability and probe validity remain insufficiently established. Experiments are restricted to 7–8B models, and probes require separate training for each model and task, leaving their applicability to frontier-scale models unclear. The evidence also does not establish whether a linear probe at layer 17’s last-input-token position adequately captures epistemic state, or how it compares with alternative layers, token positions, and probe architectures.
  3. Coverage and comparison of relevant methods are incomplete. Particularly relevant omissions include INSIDE (ICLR 2024), LLM-Check (NeurIPS 2024), Lookback Lens (EMNLP 2024), and PRISM (ACL 2025). Discussing these approaches and comparing selected methods under matched information and computational budgets would clarify the paper’s incremental contribution.
Questions for authors
  1. Figures 2 and 3 primarily emphasize differences across relatively small, older models. What broader conclusions are these comparisons intended to support, and what insights do they offer for understanding or improving frontier-scale LLMs?
  2. What do the authors consider the paper’s central novel contribution: the internal probe, the comparative analysis across models, or the intervention method? The novelty and supporting experimental evidence for each component appear limited. Could the authors clarify the principal advance over prior work and provide evidence that more directly establishes it?
Reviewer_2hQgOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

LLMs exhibit a behavior where their internal representations can encode their own epistemic state better than what they generate, referred to as the expression gap. The authors discovered that training a probe at layer 17 for any model tested can achieve a much higher AUC. They then introduce R-Restoration, which is a probe-gated logit bias intervention to close the gap, increasing the AUC across models.

Critical strengths
  1. Extensive range of evaluation models: The experiments were evaluated on a variety of model families, which tests for generalization of claims.
  2. Addressing the gap: the proposed framework seems to effectively close the expression gap and can be combined with other methods to achieve better performance.
Critical weaknesses
  1. Unclear methodology: For R-Restoration, it isn't clear on how intervention is applied. It only mentioned under a certain threshold, the prompts are untouched. Also, I'm not sure how the threshold is determined. Lastly, for the AUC lift, is it measuring the model's outputted uncertainty? There's no details explaining how the intervention was determined as successful since previous experiments looked at the internal activations and used a probe.
  2. It appears that the single line instruction achieves better results than the proposed method (Table 2). Then, what benefit does the method bring?
Questions for authors
  1. The paper mentions that a probe trained on layer 17 of any model can achieve a high AUC, but the models tested have different number of layers. This specific layer carries over across all models?
Reviewer_LZ25OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper defines an expression gap for the uncertainty of whether a question can be answered as the difference between a linear probe based on internal representations and expressed uncertainty.

Critical strengths

It would be valuable to have a probe that can tell reliably if a question can be answered.

Critical weaknesses
  • mathematical quantities are defined only in prose and not in terms of equations which makes it difficult to judge if the expression gap actually computes what the authors claims it computes.
  • It is unclear why the authors choose layer 17. Did they just brute force all possible layers? In that case they are running into a multiple hypothesis issue.
Questions for authors
  • how did you choose layer 17 for the probe?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.