acceptodds
Under review as a conference paper at ICLR 2027

A Formal Limitation on Learning Human Language From Textual Corpora

Abstract

Can a listener recover what a speaker means from the form of an utterance alone? We answer this question information-theoretically, and for a listener given by any featurizer of text, including the hidden states of contemporary large language models. Modeling language use as a joint distribution over meanings, contexts, and utterances, we derive upper bounds on the probability that a decoder recovers a speaker's intended meaning from a representation of the utterance. The bounds are governed by the uncertainty that form leaves about meaning, which splits into an irreducible part and a part that only (extralinguistic) context, but never the utterance alone, can resolve. Because these quantities are intrinsic to language, no representation, however much text or supervision produced it, can surpass them. The bounds apply, moreover, to meaning spaces that are discrete or continuous. We provide empirical evidence in support of the theory through experiments on artificial languages, Mandarin zero-pronoun resolution, and color reference.

Primary area:
learning theory
Keywords:
linguistic theory, pragmatics, LLMs

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HNSROpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies, from an information-theoretic perspective, the following question: when a system can only observe linguistic form and cannot access communicative context, to what extent can the speaker's true intended meaning be recovered from an utterance? The authors' core theoretical contribution is deriving an information-theoretic upper bound on meaning recovery ability, and decomposing meaning uncertainty under a decoder condition into two terms: H(M∣U)=H(M∣U,C)+I(M;C∣U)H(M|U) = H(M|U,C) + I(M;C|U), where H(M∣U,C)H(M|U,C) represents irreducible ambiguity that cannot be eliminated even with context, and I(M;C∣U)I(M;C|U) represents meaning information resolved only by context. Based on this decomposition, the authors provide formal recovery upper bounds for discrete meaning spaces (Theorem 1) and continuous meaning spaces (Theorem 2), respectively, and validate them through four sets of experiments: artificial language tasks under discrete/continuous meaning spaces, a Chinese zero-code recovery task, and a color reference task. The paper's core conclusion is that regardless of model scale, amount of training data, or training method, meaning recovery ability relying only on textual form has a fundamental limit determined by the intrinsic ambiguity of language.

Critical strengths
  1. The paper's problem setting is clear and has cross-disciplinary value, connecting pragmatics, information theory, and representation probing.

  2. The authors attempt to cover discrete and continuous meaning spaces with a unified framework, and validate it on artificial language and two natural language tasks. The overall writing and experimental description are relatively complete.

Critical weaknesses

Page 5 and Appendix H claim that if the distance between predicted meaning and true meaning is less than ϵ\epsilon, then after nearest-neighbor quantization the two must correspond to the same packing point. This does not hold near Voronoi boundaries. For example, take meaning space [0,2][0,2], ϵ=1\epsilon = 1, and packing points {0,2}\{0,2\}, with true value 0.9 and predicted value 1.1. The prediction error is only 0.2, less than ϵ\epsilon, but the two points quantize to 0 and 2, respectively. Therefore, A⇒BA \Rightarrow B and P(A)≤P(B)P(A) \leq P(B) in the paper cannot be derived. This problem directly affects Theorem 2, Corollary 2.1, and the theoretical interpretation of all continuous meaning space experiments.

Some content seems to be uncompleted, e.g, the begining of line 353.

Questions for authors
  1. Can the authors repair the key reduction in Theorem 2 without assuming that true meanings lie at packing points or are far from Voronoi boundaries? If not, how should the main conclusions for continuous meaning spaces be modified?

  2. In the zero-pronoun resolution experiment, is the utterance used to estimate conditional entropy the full subtitle sentence, or the 13 classes of surface-pronoun indicator? If it is the latter, how does the upper bound constrain a decoder that uses full-sentence representations? Did they say?

Reviewer_hV1aOpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks how well a listener who sees only the words of an utterance can recover what the speaker meant. It models a dialogue turn as an intended meaning MM, a shared context CC and an utterance UU, and observes that the uncertainty left about MM after seeing UU splits into two parts: ambiguity that even the context does not resolve, and ambiguity that only the context resolves. Theorem 1 applies Fano's inequality and the data processing inequality to conclude that no function of UU, including the activations of any language model, can recover MM with probability above a ceiling set by that uncertainty; Theorem 2 extends this to continuous meaning spaces through a packing argument. After checking both bounds on synthetic languages built to satisfy their assumptions, the experiments feed natural-language utterances alone, with the context withheld, to six models between 2B and 14B parameters (three text-only, three vision-language) and train a probe to recover the meaning. On Mandarin zero-pronoun resolution all six stay at the trivial baseline of about 0.85, below the computed ceiling; on a color reference task the recovered colors stay below the ceiling at every tolerance tested. The authors conclude that text-only systems learn "provably impoverished" representations of speaker meaning.

Critical strengths

The zero-pronoun experiment is a clean negative result. Six models, including instruction-tuned vision-language models trained with supervision beyond text, none does appreciably better than a baseline that copies the pronoun when it is present and guesses the most frequent one when it is dropped. Whatever the sentence contains about a dropped pronoun, these models do not extract more of it than the trivial rule does.

Critical weaknesses
  1. The theorems bound decoding from a single utterance with no context; the title and conclusion claim a bound on learning from textual corpora. The paper's own definition of CC includes linguistic context such as the preceding conversation, which is text and can be given to a text-only model. In the color task the context is three chips whose coordinates fit in a prompt; in the subtitle task it is the preceding dialogue. The experiments withhold all of it, so the models' failure follows from the setup. Supporting the stated claim would require separating linguistic from extralinguistic context and showing that the part a text model cannot access is the part that carries the meaning. The limitations section lists this as future work. I think it is the experiment the paper needed.

  2. Theorem 1 is Fano's inequality plus the observation that a function of the utterance carries no more information about the meaning than the utterance does. Pimentel et al. (ACL 2020, arxiv.org/abs/2004.03061) proved this from the data processing inequality, showing that contextual embeddings "contain no more information about a word-level syntactic task than the original sentence itself", and built an information-theoretic account of probing on it. The paper's setup, a probe on frozen activations, is probing, yet none of that literature is cited: Hewitt and Liang (arxiv.org/abs/1909.03368), Voita and Titov (arxiv.org/abs/2003.12298), or Xu et al. (arxiv.org/abs/2002.10689) on what is decodable under computational constraints. Abdou et al. (CoNLL 2021, arxiv.org/abs/2109.06129) probed text-only models for CIELAB color structure and are not discussed.

  3. Neither natural-language experiment tests the theorem it is said to verify. In the Mandarin experiment the ceiling is computed with UU as the thirteen-valued pronoun slot, but the models read the whole subtitle line, so the representation is not a function of the UU the bound was computed on. The 0.85 baseline is within a fraction of a point of the Bayes-optimal decoder for that slot, so matching it says the sentence adds nothing beyond the slot; the 0.94 ceiling is not what is being tested. In the color experiment, H(J∣U)H(J \mid U) is estimated by the cross-entropy of a classifier on LLM features over the concatenated train and test sets. Conditioning on h(U)h(U) raises the entropy, overfitting on training data lowers the cross-entropy, and Appendix E.2 reports inflated I(J;U)I(J; U) at small ϵ\epsilon; none of these is controlled, so the plugged-in number is not shown to be an upper bound on anything.

Questions for authors
  1. Please add context to the model's input in steps. For the color task, add the three chip coordinates to the model's input; for the subtitle task, add the preceding kk lines if the subtitle source allows it. At each step report the model's accuracy and the recomputed Fano ceiling. This would separate linguistic from extralinguistic context and let the paper say which part of the gap text cannot close.

  2. In the proof of Theorem 2, how is the implication from d(M^,M)<ϵd(\hat{M}, M) < \epsilon to q(M^)=q(M)q(\hat{M}) = q(M) justified when MM is not a packing point?

  3. For the color experiment, can the ceiling be estimated in a way that is a provable upper bound, for instance from held-out cross-entropy of a classifier that does not share features with the regressor, or by a direct estimate of H(J∣U)H(J \mid U) on a subset of frequent utterances where the sample supports it?

Reviewer_MUQBOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a theoretical limitation on recovering the meaning of human language from utterances alone. The main contribution is an information theoretic quantification of the theoretical gap between the intended meaning of the speaker relative to the inferable meaning from the representation of the utterances alone. Here, the representation can be any featurized or neural embedding representation of the utterances. The paper first provides the proof to establish the upper-bound, and then backs it up via empirical results based on three settings: artificial language, chinese zero-pronoun resolution, and color resolution.

Critical strengths

The main strength of this paper is its formal, theoretical account of linguistic meaning, with an emphasis on the role of context in disambiguating the speaker's intended meaning.

The theoretical setting is intellectually interesting and the paper's approach of deriving an upper bound and then backing it up with empirical results is substantial.

It is refreshing to see a paper such as this at a time when much of the community's research focus is on scaling LLMs.

Critical weaknesses

My main concerns are twofold: conceptual novelty and empirical significance.

On novelty, the realization that context is critical in deciphering the intended meaning of utterances is well established. A substantial body of prior work has argued for and reported similar findings, both formally and empirically.

The exact technical formulation based on the information-theoretic decomposition in itself appears to be novel in the exact context of investigating the role of context for linguistic meanings. However, that exact formulation seems to appear in somewhat related setting in prior art, e.g., the importance of context in machine translation (Fernandes, Yin, Neubig & Martins (2021), “Measuring and Increasing Context Usage in Context-Aware Machine Translation,” ACL), and while not identical, a similar line of information-theoretical thoughts have been explored in other prior work such as Piantadosi, Tily & Gibson (2012), “The communicative function of ambiguity in language,” Cognition. The related work of the submitted manuscript is relatively thin with respect to relevant prior work. g via conditional entropy and mutual information

On empirical significance, I found it even thinner due to the very toy nature of the empirical designs, which ranges from artificial language to the Mandarine zero pronoun resolution to colors. The experimental setting of the Mandarin Zero-Pronoun Resolution is quite old-fashioned in that, the utterance only includes a short recent text without any of the prior conversation in text. This was a reasonable assumption perhaps a decade ago. Modern day LLMs condition on text that's far more than the most recent utterance, thus I'm not sure what novel insights this paper offers. Its empirical assumptions seem very distant from the current day practice.

Questions for authors
  • would it be reasonable to include experiments on mandarine zero pronoun resolution using stronger LLMs including all text context? or is there a solid reason not to?

  • I'd appreciate clarifications on the conceptual and technical novelty.

Comments on the paper

1 comment

Dear reviewers, please review the authors’ rebuttal and provide your responses as soon as possible. Your timely feedback is important for moving the discussion forward. Thank you!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.