acceptodds
Withdrawn from ICLR 2027

Language Models Cannot Tell When a Definition Is Missing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Contracts, technical documentation, and online communities give familiar words meanings of their own. When the defining text is missing from a language model's input, the model answers by the word's usual meaning, and nothing in its output signals the error. We study these silent misreadings on four corpora with diagnostic questions whose answers follow from a term's local definition. Frontier models answer 13% to 30% of these questions wrongly without the definition. The errors concentrate on familiar words given a new meaning, often repeat across ten samples, and persist across model generations, as 70% to 94% of the newest model's errors are also made by an older one. Sampling agreement and stated confidence rank them with AUROC 0.46 to 0.56 on redefined common words, and the calibrated confidence of a decision model reaches only 0.64 to 0.66 until the definition is supplied. Checking a small open model against the answer implied by the source ranks frontier models' errors with AUROC 0.72 to 0.85, where term frequency and rarity reach at most 0.63, and places two thirds of a contract library's estimated errors in a quarter of its questions. The source definition repairs most errors, whereas the model's own gloss of the term does not. Models can follow a definition they are given but cannot recognize when one is missing, so local definitions have to be located and supplied from the source.

Primary area:
foundation or frontier models, including LLMs
Keywords:
silent misreading, locally defined terms, missing context, uncertainty estimation, error detection, retrieval-augmented generation, evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_EsY3OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

In some settings (i.e. contracts, technical docs, online communities), there are alternative definitions of various terms that the authors call "local meanings." A model answering questions inside this setting may be unfamiliar with these "local meanings," and thus produce plausibly correct responses that leverage the term's original meaning (i.e. the layman's definition of "trademark" vs a specific Trademark discussed in a contract). The authors call these errors "silent misreadings," and show that these silent misreadings occur across model families and corpuses (both synthetic and real). The authors perform subsequent analysis and show that models' internal entropy cannot detect these errors, but small models can detect them with privileged access to the answers. Finally, the authors show that augmenting the queries with the correct local definitions will recover the errors.

Critical strengths

The paper is written is a very clear and concise manner. All metrics, models, and datasets were clearly described, and all experiments were naturally motivated.

Critical weaknesses

Overall, the aims of this paper are quite unclear. The authors seem to be portraying the reduced performance of the model when withheld critical information as a shortcoming of the model. Moreover, many of these results are in domains where the "local definitions" are real words with similar meaning. Overall, while the experimental setups and subsequent findings of this paper are sound, I believe it contributes no real research value other than identifying a potential class of error without any means to predict it.

Major:

  • Your first finding is that models queried without the definition text perform worse than models queried with the definition text. This seems less of a finding on model shortcomings and more of a finding on the necessity of information?
  • The risk score is not just defined with respect to the query, but also the ground truth answer. I imagined the risk score as a way to diagnose whether an arbitrary question could contain missing definitions when you wrote RS(z)R_S(z). However, implicit in this definition is the log-probs of the ground truth answer text. As such, I'm confused as to the usefulness of this Risk score and the overall findings in Section 7. If the only way we can use the Risk score is when we have the gold answer, why do we need a proxy model to find the errors of the deployed model when we could simply check the gold answer? Since it needs the gold answer to compute, it cannot be used in settings where the ground truth is not available.
  • The best patch to these issues is just to naively include the text in the definition as per section 8. This does not feel like a particularly novel finding.

Minor:

  • On your second finding, there is a storied literature on how LLMs responses are rarely calibrated with respect to their logits:
    • How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering
    • Language Model Probabilities are Not Calibrated in Numeric Contexts
    • Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
  • All your experiments are constrained to multiple choice diagnostic questions written by an unknown model [160]. Would your results generalize to the full datasets with text?
Questions for authors

Please clarify if I have misunderstood anything written in the manuscript, especially w.r.t the Risk scores. Overall, I question your second implication that "Knowing that a definition is needed is a distinct capability." In settings where the relevant definition is not in context or even accessible to the model, how would you expect that the model should know there is an alternative definition it should be using. While this may make sense in settings where the terms are OOD words that the model tries to associate with some meaning, the examples that you showed in your paper seem to indicate that these terms are normal words (i.e. Trademark, Products, Term).

Reviewer_ZfgZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies errors caused by missing local definitions in document QA. Models often answer confidently using a term’s ordinary meaning, while sampling agreement and confidence provide weak error detection. The authors propose an offline, source-keyed small-model score to identify risky questions and show that supplying source definitions repairs many errors.

Critical strengths
  • The paper identifies a practical weakness of passage-based document QA and evaluates it across several domains.
  • The comparison between source definitions and the model’s own glosses helps establish what information actually repairs the errors.
  • Predicting stronger models’ errors with a small model is a useful empirical result.
Critical weaknesses
  • The main finding is difficult to separate from insufficient input. If the provided text gives no indication that a familiar word has a different local meaning, using its ordinary meaning is reasonable. An error relative to the full document does not by itself establish a failure to recognize missing information.
  • A more compelling capability failure would be ignoring visible evidence that the ordinary meaning is inappropriate, or disregarding an explicitly supplied definition. The paper discusses the latter, but does not systematically separate these cases from cases with no observable cue.
  • The experiments do not establish whether full-document access or an agent that can search for definitions would resolve the problem. This limits the broader claim that models “cannot tell” when a definition is missing.
  • The risk index requires source-derived answer keys for diagnostic questions. Its usefulness for unseen user requests, beyond these constructed questions, remains unclear.
Questions for authors
  • How do results differ between items with no visible cue of a local meaning and items containing a definition reference, unusual usage, or another identifiable cue?
  • Can models identify and retrieve the missing definitions when given source access, either autonomously or with an instruction to verify relevant terms?
  • What advantage does the offline index offer over directly attaching relevant definitions or providing the full document?
Reviewer_9gjcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

Whenever a document changes a familiar term's local meaning, models cannot recognize it unless the definition is explicitly provided in the context. When the definition is missing, models assume the term’s usual meaning and answer questions incorrectly, with confidence. The authors call this phenomenon silent misreading and test it with model-generated multiple-choice questions on four corpora. They find that frontier models get 13–30% of the questions wrong when the definition isn’t in the context. These errors are hard to catch at runtime, since the model's own confidence and agreement across samples cannot reliably distinguish them from correct answers. However, the authors show that a small, open model can predict ahead of time which questions larger models are likely to get wrong, by comparing its own answers to the correct ones from the source document. They also find that providing models the actual definition from the document fixes most errors, but letting models write their own definition of the term does not help.

Critical strengths
  • The paper names a real gap in how models are evaluated. They identify a type of error that existing evaluations miss, because a silent misreading contradicts neither world knowledge nor the input to the model. This gap matters most in practical settings like contracts and internal documents, where organizations rely on models to read their own text.
  • The experiments show that when an error depends on a definition, the fix has to come from the document. GPT models follow a stated definition almost every time, and on synthetic questions written without the definition, the source definition fixed most errors. Having models write their own definition, by contrast, doesn’t reliably help and often causes new errors.
Critical weaknesses
  • The paper’s central claim, that models can’t tell when a definition is missing, is mainly tested indirectly through confidence scores and sample agreement. The questions are forced-choice, so models have no way to say that an answer depends on a definition, and they never get the chance to look one up or ask for it. Giving them that option, and checking whether they use it on the right questions, would test the claim much more directly. A human baseline would also help show how hard these cases really are to spot.
  • The misreading effect is convincing on the synthetic data but not as much on real text. In the synthetic set, redefined common words are misread far more often than ordinary words. On Reddit, though, GPT-6-astra makes errors on ordinary control words about as often as on glossary terms, and the risk score actually works better on the controls. It would be great to see the cross-model error overlap compared against these controls. Also, since most questions and answer keys were written by a model and mostly unchecked, some of the shared errors could just be bad questions.
Questions for authors
  1. What does the cross-model error overlap look like on the control terms (the synthetic usual-meaning words and the Reddit control words)? If it’s about as high as on redefined terms, the overlap may just reflect hard questions?
  2. The main questions are forced-choice, but Appendix C mentions abstention rates in the glossary-wording experiments. How did models abstain there? It would be good to rerun the no-definition experiments with that option, or with a way to look up the definition. That seems like the most direct test of the paper’s main claim.
  3. Adding definitions barely changes scores on the human-written contract tasks, and the paper explains this by saying those benchmarks don’t test definition boundaries. But couldn’t it also mean these questions are just rare in practice? Is there any evidence on how often real questions depend on a local definition?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.