- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This article investigates whether the number of visual tokens can reflect the availability of information when rendering text into images to compress context. The author breaks down the question into two parts: whether the target information is included in the model input (target inclusion), and whether the model can recover the answer (conditional recovery) if it is included. Based on three fixed VLM readers (Qwen3.5-2B, Qwen3.5-9B, GLM-4.6V-Flash), the author conducted a series of controlled experiments: changing the layout or scaling under identical native token numbers and input IDs resulted in a significant change in the recovery rate of precise identifiers; Under a fixed input budget, a fully rendered image input can contain all targets, while text retained by hash will lose some targets. GLM is therefore superior for long inputs; The experiments on RULER and LoCoMo show that in most cases, text is still superior to image input. The author advocates that the evaluation of compressed context should report inclusion rate, conditional recovery rate, and system overhead separately.
Critical strengths
- The experimental control is very careful, for example, comparing different layouts under the condition that the native token number and input ID are exactly the same, and using pairing tests and multiple comparison corrections to make the conclusion that "layouts will change the recovery rate under the same token number" more credible.
- Separating the reporting of target inclusion and conditional recovery provides a clear and actionable analytical framework for evaluating visual text compression.
Critical weaknesses
- The universality of the conclusion in this article is relatively limited. The only significant result of "image input better than text" only appeared in GLM model and synthesis task, and the compared text baseline was only a simple hash preservation strategy; The original layout used in the layout experiment is inherently narrow. Therefore, it is not clear whether these findings can be generalized to common documents and stronger text compression methods, which is the main reason why I tend to reject them.
- The readability of the paper is poor. The entire text uses a large number of custom terms and restrictive expressions, with multiple independent small experiments scattered in different chapters, making it difficult for readers to quickly grasp the main conclusions and their guiding significance for practice. It is recommended that the author streamline the expression and reorganize the entire text around one or two core findings.
Questions for authors
- If a more reasonable retention strategy is used on the text side (such as query based retrieval or simple summary compression), does the advantage of image input still exist on long inputs?
- Does the conclusion of layout and scaling still hold true for common document layouts (rather than abnormally narrow original layouts)?
Thank you for the detailed rebuttal and new experiments. However, the new QASPER results show text-based retention matches or beats optical input, and BM25 retrieval alone outperforms optical outright. This confirms the core concern that the optical-input advantage does not generalize beyond synthetic settings. I therefore maintain my original score.