acceptodds
Under review as a conference paper at ICLR 2027

Beyond Visual Token Count: Target Inclusion and Reader Recovery

Abstract

Rendering text as images can fit more source information into a model’s input budget, but token savings alone do not establish whether that information remains usable. We distinguish target inclusion—whether a queried fact reaches the reader—from conditional recovery—whether the reader can recover a fact that is present—and study both with three fixed vision-language models. On fresh held-out cases, changing image geometry improves exact identifier recovery across all three readers while preserving source content, native token counts, and input-ID sequences. In a separate finite-budget comparison on 150 fresh long-source cases, GLM answers 126 queries correctly from full-source optical input, compared with 90 from compact query-independent text. Optical input includes every target, whereas text recovers every target it retains, revealing an inclusion advantage despite imperfect optical recovery. This comparison concerns one simple static textual storage policy, not optimized textual memory. Density and dialogue-memory controls further delimit the findings. Together, these measurements show why useful compressed context depends on both inclusion and recovery, and why visual-token counts alone cannot predict task performance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
vision-language models, optical context compression, visual text rendering, target inclusion, reader recovery, long-context evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_uSnAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This article investigates whether the number of visual tokens can reflect the availability of information when rendering text into images to compress context. The author breaks down the question into two parts: whether the target information is included in the model input (target inclusion), and whether the model can recover the answer (conditional recovery) if it is included. Based on three fixed VLM readers (Qwen3.5-2B, Qwen3.5-9B, GLM-4.6V-Flash), the author conducted a series of controlled experiments: changing the layout or scaling under identical native token numbers and input IDs resulted in a significant change in the recovery rate of precise identifiers; Under a fixed input budget, a fully rendered image input can contain all targets, while text retained by hash will lose some targets. GLM is therefore superior for long inputs; The experiments on RULER and LoCoMo show that in most cases, text is still superior to image input. The author advocates that the evaluation of compressed context should report inclusion rate, conditional recovery rate, and system overhead separately.

Critical strengths
  • The experimental control is very careful, for example, comparing different layouts under the condition that the native token number and input ID are exactly the same, and using pairing tests and multiple comparison corrections to make the conclusion that "layouts will change the recovery rate under the same token number" more credible.
  • Separating the reporting of target inclusion and conditional recovery provides a clear and actionable analytical framework for evaluating visual text compression.
Critical weaknesses
  • The universality of the conclusion in this article is relatively limited. The only significant result of "image input better than text" only appeared in GLM model and synthesis task, and the compared text baseline was only a simple hash preservation strategy; The original layout used in the layout experiment is inherently narrow. Therefore, it is not clear whether these findings can be generalized to common documents and stronger text compression methods, which is the main reason why I tend to reject them.
  • The readability of the paper is poor. The entire text uses a large number of custom terms and restrictive expressions, with multiple independent small experiments scattered in different chapters, making it difficult for readers to quickly grasp the main conclusions and their guiding significance for practice. It is recommended that the author streamline the expression and reorganize the entire text around one or two core findings.
Questions for authors
  • If a more reasonable retention strategy is used on the text side (such as query based retrieval or simple summary compression), does the advantage of image input still exist on long inputs?
  • Does the conclusion of layout and scaling still hold true for common document layouts (rather than abnormally narrow original layouts)?
1 comment

Thank you for the detailed rebuttal and new experiments. However, the new QASPER results show text-based retention matches or beats optical input, and BM25 retrieval alone outperforms optical outright. This confirms the core concern that the optical-input advantage does not generalize beyond synthetic settings. I therefore maintain my original score.

Reviewer_4gqMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the usability of text rendered as images under constrained input budgets. It distinguishes target inclusion from conditional recovery and evaluates three fixed vision-language models through layout interventions, finite-budget retrieval, and supporting RULER and LoCoMo experiments. The main findings are that layout can affect exact-string recovery at identical native token allocations, and that full-source optical input can outperform specified static text-retention policies in a GLM long-source setting.

Critical strengths
  1. The paired layout experiment preserves source content, native token counts, input-ID sequences, and image-grid allocation. This provides convincing evidence that token allocation alone does not determine exact recovery.

  2. Auditing whether the requested record is included helps distinguish omission from failure to recover an included target. The fresh compact-text experiment makes this distinction particularly clear: text answers every retained target, whereas optical input retains all targets but incurs recovery errors.

Critical weaknesses
  1. The conceptual novelty is limited relative to the closest prior work. As the paper acknowledges, rendering sensitivity, the distinction between coverage and fidelity, and factual-recall advantages over content-discarding baselines have already been investigated. The clearest additions are stricter allocation matching and explicit per-case inclusion accounting. These are useful methodological refinements, but the paper provides limited evidence that they substantially change an existing understanding or lead to a broadly applicable new conclusion.

  2. The layout experiment starts from an unusually unfavorable representation. The original identifier layout occupies only approximately 12.3% of the page width and uses small effective glyphs. Moving from this narrow layout to full-width wrapping produces a credible improvement, but the result may largely reflect correction of an unsuitable rendering configuration. It remains unclear whether similarly meaningful differences persist among reasonable, well-tuned layouts at matched allocations. Comparisons with established rendering procedures would help establish the significance of this experiment.

  3. The strongest positive result has limited coverage across models, budgets, and tasks. The 150-source follow-up evaluates GLM in the previously favorable LONG setting. Fresh sources strengthen the evidence within that condition, but the experiment does not establish robustness across readers or budgets. The earlier results are less favorable for the Qwen models, and the LoCoMo controls generally favor text.

Questions for authors
  1. At identical native allocations, how large are the recovery differences among several reasonable, independently tuned layouts, including rendering procedures used in prior work?

  2. Does the GLM advantage persist against stronger query-independent text representations under the same pre-query commitment constraint, without access to discarded records?

Reviewer_Yh4sOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the usability of text rendered as images by distinguishing target inclusion—whether a queried fact reaches the reader—from conditional recovery—whether the reader can recover that fact when it is present. Through extensive empirical studies, including evaluations on newly generated held-out cases, the paper examines the importance of this distinction and argues that visual-token counts alone are insufficient to predict task performance.

Critical strengths

S1. The use of newly generated held-out cases is a strength of the experimental evaluation.

S2. The paper presents a broad set of empirical studies that illustrate the importance of distinguishing target inclusion from conditional recovery.

Critical weaknesses

W1. The problem setup and its connection to the empirical studies are difficult to follow. The presentation would benefit from a clearer explanation of the central research questions, the task setup, and how each experiment addresses those questions.

W2. Several controlled studies use relatively small evaluation sets, which limits confidence in the generalizability of the findings.

W3. Although the appendix provides construction details, the process for generating the held-out cases remains difficult to follow. A clearer description in the main text, supported by a concrete example, would make the evaluation setup easier to understand.

W4. The connection between the individual experimental findings and the paper's main conclusions is not sufficiently clear. A more explicit synthesis of which results support each conclusion, and how broadly those conclusions apply, would strengthen the paper.

Questions for authors

Q1. Could the authors provide a worked example of the SHORT, MEDIUM, and LONG versions, showing the source records, query, and target answer? Please clarify what remains fixed, what changes across the three versions, and how the input budget is applied.

Q2. Could the authors explain more explicitly how the results from the different experiments support the main conclusions, particularly the importance of distinguishing target inclusion from conditional recovery?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.