- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies, from an information-theoretic perspective, the following question: when a system can only observe linguistic form and cannot access communicative context, to what extent can the speaker's true intended meaning be recovered from an utterance? The authors' core theoretical contribution is deriving an information-theoretic upper bound on meaning recovery ability, and decomposing meaning uncertainty under a decoder condition into two terms: , where represents irreducible ambiguity that cannot be eliminated even with context, and represents meaning information resolved only by context. Based on this decomposition, the authors provide formal recovery upper bounds for discrete meaning spaces (Theorem 1) and continuous meaning spaces (Theorem 2), respectively, and validate them through four sets of experiments: artificial language tasks under discrete/continuous meaning spaces, a Chinese zero-code recovery task, and a color reference task. The paper's core conclusion is that regardless of model scale, amount of training data, or training method, meaning recovery ability relying only on textual form has a fundamental limit determined by the intrinsic ambiguity of language.
Critical strengths
-
The paper's problem setting is clear and has cross-disciplinary value, connecting pragmatics, information theory, and representation probing.
-
The authors attempt to cover discrete and continuous meaning spaces with a unified framework, and validate it on artificial language and two natural language tasks. The overall writing and experimental description are relatively complete.
Critical weaknesses
Page 5 and Appendix H claim that if the distance between predicted meaning and true meaning is less than , then after nearest-neighbor quantization the two must correspond to the same packing point. This does not hold near Voronoi boundaries. For example, take meaning space , , and packing points , with true value 0.9 and predicted value 1.1. The prediction error is only 0.2, less than , but the two points quantize to 0 and 2, respectively. Therefore, and in the paper cannot be derived. This problem directly affects Theorem 2, Corollary 2.1, and the theoretical interpretation of all continuous meaning space experiments.
Some content seems to be uncompleted, e.g, the begining of line 353.
Questions for authors
-
Can the authors repair the key reduction in Theorem 2 without assuming that true meanings lie at packing points or are far from Voronoi boundaries? If not, how should the main conclusions for continuous meaning spaces be modified?
-
In the zero-pronoun resolution experiment, is the utterance used to estimate conditional entropy the full subtitle sentence, or the 13 classes of surface-pronoun indicator? If it is the latter, how does the upper bound constrain a decoder that uses full-sentence representations? Did they say?
Dear reviewers, please review the authors’ rebuttal and provide your responses as soon as possible. Your timely feedback is important for moving the discussion forward. Thank you!