- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper studies idiomatic and literal uses of the same expressions through surprisal dynamics, representation probes, and activation interventions. The experiments are varied, but their motivations and connections are difficult to follow. There may be useful findings here, but the current presentation reads more like an experimental report than a paper with a clear central result.
Critical strengths
Comparing different readings of the same expression is a useful setting because it holds the expression itself fixed while varying its use in context. The authors examine several kinds of evidence and openly report controls that limit their interpretation, including context-only probes and alternative patching donors.
Critical weaknesses
- The connection between the proposed measurement and lexical-unit encoding is not sufficiently established. The mathematical motivation explains why stronger support for a fixed continuation could lower surprisal, but it does not show that a first-to-last surprisal drop identifies whole-expression processing. General contextual predictability and familiar word combinations could produce similar results. The paper needs to clarify what this measurement can distinguish and why that distinction answers the research question.
- The experiments do not lead to a clear conclusion about lexical-unit encoding. Surface controls substantially change the behavioral effect, context-only probes nearly match the full representations, and patching shows context-dependent prediction effects without isolating an idiom-specific contribution. These controls are useful, but the paper does not clearly explain what we learn after accounting for them. The attention analysis adds further observations without resolving this gap. I would encourage the authors to organize the paper around the strongest supported finding and explain how each experiment tests a specific alternative explanation. In its current form, the amount of experimentation exceeds the clarity of the resulting insight.
Questions for authors
-
What evidence distinguishes surprisal drop caused by lexical-unit encoding from ordinary contextual predictability or familiar word combinations? Equations 2–3 motivate the signal but do not establish this distinction.
-
Given that context-only probes nearly match the full representations, and reading labels coincide with data sources in DICE, what evidence shows that the probes detect idiomaticity rather than source or style differences?
-
After accounting for these controls, what is the main finding about lexical-unit encoding, and how do the patching and attention results establish it beyond general context-dependent prediction?