acceptodds
Withdrawn from ICLR 2027

Same Tokens, Different Readings: Representational and Causal Tests of Lexical-Unit Encoding in Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Behavioral signatures, such as how a model's next-token probabilities move across a span, are routinely read as evidence about what language models represent internally. We ask what one such signature actually licenses, in a setting where the surface form can be held fixed: idiomatic and literal uses of the same expression. Across autoregressive models spanning 124M–122B, the surprisal-drop signal remains positive across model families. In the corpus-scale analysis, mean surprisal over the span does not strongly separate the two readings (Cohen's d = 0.03), whereas the first-to-last surprisal drop does (d = 0.91); residualizing frequency, collocation, span length and continuation predictability reduces this to d = 0.14, and matched controls retain positive effects (d = 0.25–0.52). We then ask whether this behavioral signal corresponds to internal structure. In three models (0.77B–14B), probes under nested expression-disjoint cross-validation over 402 expression types separate the readings at AUROC 0.981–0.991, against 0.535–0.564 for surface-form and context-free controls. Follow-up controls show that the preceding context alone reaches AUROC 0.989 in two models. Patching the final-word predictor state with a donor from the opposite reading of the same expression, matched on observed surface form and in-context token sequence, shifts final-word surprisal by +1.99 to +3.30 bits in difference-in-differences, while same-reading donors give intervals spanning zero (cross-minus-same-reading: +2.13 to +3.12 bits); the effect is near zero or negative before the expression and largest at the final-word predictor. Because random-expression donors are not reliably smaller, we report a reading-contingent causal component rather than an idiom-specific mechanism. More broadly, the study offers a template for validating and bounding behavioral signatures with representational and causal evidence.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, activation patching, probing, language models, idioms, multiword expressions, surprisal

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_9ocQOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies idiomatic and literal uses of the same expressions through surprisal dynamics, representation probes, and activation interventions. The experiments are varied, but their motivations and connections are difficult to follow. There may be useful findings here, but the current presentation reads more like an experimental report than a paper with a clear central result.

Critical strengths

Comparing different readings of the same expression is a useful setting because it holds the expression itself fixed while varying its use in context. The authors examine several kinds of evidence and openly report controls that limit their interpretation, including context-only probes and alternative patching donors.

Critical weaknesses
  1. The connection between the proposed measurement and lexical-unit encoding is not sufficiently established. The mathematical motivation explains why stronger support for a fixed continuation could lower surprisal, but it does not show that a first-to-last surprisal drop identifies whole-expression processing. General contextual predictability and familiar word combinations could produce similar results. The paper needs to clarify what this measurement can distinguish and why that distinction answers the research question.
  2. The experiments do not lead to a clear conclusion about lexical-unit encoding. Surface controls substantially change the behavioral effect, context-only probes nearly match the full representations, and patching shows context-dependent prediction effects without isolating an idiom-specific contribution. These controls are useful, but the paper does not clearly explain what we learn after accounting for them. The attention analysis adds further observations without resolving this gap. I would encourage the authors to organize the paper around the strongest supported finding and explain how each experiment tests a specific alternative explanation. In its current form, the amount of experimentation exceeds the clarity of the resulting insight.
Questions for authors
  1. What evidence distinguishes surprisal drop caused by lexical-unit encoding from ordinary contextual predictability or familiar word combinations? Equations 2–3 motivate the signal but do not establish this distinction.

  2. Given that context-only probes nearly match the full representations, and reading labels coincide with data sources in DICE, what evidence shows that the probes detect idiomaticity rather than source or style differences?

  3. After accounting for these controls, what is the main finding about lexical-unit encoding, and how do the patching and attention results establish it beyond general context-dependent prediction?

Reviewer_UWN4OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether surprisal-drop patterns in idiomatic expressions reflect lexical-unit encoding in language models. The authors combine behavioral analyses across multiple model families, expression-disjoint probing, and activation patching to study differences between idiomatic and literal readings of the same expressions. The experiments show that surprisal drop can distinguish the two readings, that contextual representations contain highly decodable reading information, and that swapping hidden states between different readings affects final-word prediction.

While the paper includes extensive experiments and robustness analyses, I have substantial concerns about the significance of the scientific contribution and whether the evidence supports a sufficiently novel understanding of language-model representations.

Critical strengths
  1. The experimental evaluation is extensive, covering multiple model families, behavioral measurements, probing, activation patching, and several control conditions.

  2. The authors acknowledge important limitations, including dataset confounding, the strong performance of context-only probes, and the limited specificity of the causal interventions.

Critical weaknesses
  1. Limited conceptual novelty and unclear scientific contribution.

  2. The experiments do not convincingly distinguish lexical-unit encoding from generic context-conditioned prediction.

  3. The proposed behavioral diagnostic has limited demonstrated utility, and its connection to the mechanistic analyses remains unclear.

Questions for authors

see limitations. And:

  1. What is the additional explanatory value of surprisal drop as a mechanistic diagnostic?
  2. Where is the evidence that the interventions manipulate idiomatic interpretation rather than merely next-token confidence?
  3. What meaningful representational conclusion remains from the near-perfect probing results when the expression itself is unnecessary?
Reviewer_4mY4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors investigate whether autoregressive LMs encode idiomatic multi-word expressions as context-dependent lexical units (“super-tokens”) using behavioural, representational and causal evidence. Regarding behavioral, first-to-last surprisal drops, while mean span surprisal does not. Regarding representational, no difference to probes on the preceding context is observed.

Critical strengths
  • experimental controls are carully chosen and evaluated
  • broad model raning for the ehavioral analysis, with models raning from 124M to 122B.
Critical weaknesses
  • the paper is very hard to read, starting with the abstract (a dozen effect sizes, AUROCs and bit values in one paragraph), without really saying what the question is, what the answer is, and why it matters.
  • section 4 has the same problem. Section 4.2. moves from layer indices, fold sizes and bootstrap seeds to estimands and tables. No experiment states up front what it is meant to show, and which outcome would support or refute the lexical-unit hypothesis, etc.
  • Contributions and take-home message are unclear, with many numbers, and a single paragraph discussion.
  • missing closely related work on implicit vocabulary and detokenisation Feucht et al. 2024, “Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs” (EMNLP 2024) Kaplan et al. 2024, “From Tokens to Words: On the Inner Lexicon of LLMs Haviv et al. 2023, “Understanding Transformer Memorization Recall Through Idioms” (EACL 2023)
  • The probing results do not isolate the expression, the probe on the preceding context reaches AUROC 0.989 in two models.
  • the causal result is not specific (random-expression donors are not reliable smaller, cross-reading donors differ from their recipients)
Questions for authors
  • What should a reader conlude about how LMs represent idioms, how would this influence a follow-up study? . How do your findings related to token erasure (Fecut et al) and inner-lexicon results (Kaplan et al)?
  • If context information is removed, does reading information remain in the expression tokens?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.