- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether rendering the interaction history of long-horizon agents into images is a good context manager. The authors propose VERA, a training-free rendering manager that 1) keeps native images from tools such as image search, crop, and zoom, 2) renders table outputs as compact tables, and 3) downsamples older history. Experiments with Gemini-3.5-flash on four text and three multimodal benchmarks show that visual rendering uses the fewest tokens and that VERA achieves the best accuracy on the multimodal benchmarks.
Critical strengths
-
The question of what to keep when the agent history itself is multimodal is interesting and well-motivated.
-
The paper is clearly written, and the controlled setting (same harness, same policy model, same trigger threshold) makes the comparison easy to follow.
-
The evaluation covers both text-only and multimodal agentic benchmarks, and the token cost is reported for every method.
Critical weaknesses
-
The two conclusions of the paper, that visual context is competitive with text context and that keeping native images helps when the history is multimodal, both rest on one specific harness implementation, VERA. Many of its design choices can change the results: older pages are downsampled to 0.75x, at most 10 images are supplied per call, an unspecified number of recent turns are kept as raw text, and the history is re-organized with role and tool labels. The paper says Visual Rendering changes only the representation (L72 to L76), but these choices also change how much information is kept. It is necessary to test other harness implementations, or at least to give a hyperparameter analysis of these key designs.
-
The results in Table 1 are counter-intuitive. Among the text-based methods, Discard-All is the best, Summarization and Sliding Window come next, and No Compression is the worst (69.0, and only 59.8 on BrowseComp-EN). With such an unusual ordering, the advantage or disadvantage of Visual Rendering does not look reliable, and it is hard to explain why text performs poorly while Visual Rendering performs better on average with the same information. The gain of Visual Rendering over Discard-All is also only 0.5 points without variance, and Visual Rendering is the worst method on XBench-DeepResearch and Seal-0.
-
The novelty is limited. Rendering history into images has been studied by many recent works, and VERA mainly adds deterministic engineering choices (keeping native images, table rendering, resolution decay). The unified definition in Section 3 is mostly notation, and the "Mechanistic Analysis" listed as a contribution is only the retry-probability formula in Appendix B.
-
The multimodal comparison is confounded. Sliding Window and Summarization drop or verbalize the images after compression, while VERA keeps them, so the gain in Table 2 cannot be attributed to rendering itself. A text-based manager that also keeps the native images is needed. It is also unclear how images are converted to text in the Text-only Rendering ablation (who writes the captions and with which prompt).
-
The experiments use only one policy model, Gemini-3.5-flash. Reading rendered text depends heavily on the visual ability of the model, so the conclusions should be checked on at least one open-source VLM and one stronger model. Learned optical-context methods such as AgentOCR are discussed but not compared; training-free ones such as OCR-Memory could at least be included.
-
The RQ3 analysis has low statistical power. With 100 questions and avg@2, one point has a standard error of about 4 to 5 points, so the non-monotonic trend in Table 5 may be noise. Finding 3 should be stated more carefully.
-
The cost analysis is incomplete. Only non-cache tokens are reported, but rendering breaks the prefix cache while No Compression can reuse it heavily. Wall-clock time and rendering overhead should also be reported.
Minor: "Eq. equation 3" appears several times (around L200, L205, L221). Equation 7 is used in Section 4.2 but only defined in Appendix B. Table 5 uses "1/Vis. Ratio" while the text uses "Compression Factor". The caption of Figure 2 says 0x is excluded, but it is plotted.
Questions for authors
Please refer to Weakness
3 comments
Thanks for the detailed comments! However, the contribution and novelty of this paper may not convince me. Currently, I will maintain my score.
Thank you for engaging with our rebuttal and for acknowledging the additional experiments. We fully respect your assessment. Following your earlier comments, we have substantially narrowed and restructured our contributions. Since we no longer claim rendering itself as novel, we would like to briefly clarify what the revised paper contributes.
C1. A formulation of context management.
We define context management as a budget-constrained history transformation. A passive manager deterministically maps the history at each trigger to a nonempty, budgeted representation. This definition does two things:
- it separates passive managers from active or memory-routed managers (e.g., OCR-Memory, AgentOCR) and from full-retention and reset controls;
- it determines which methods can be fairly compared under one budget.
This directly addresses your concern that Section 3 was mostly notation: the formulation now fixes the scope and the comparison set.
C2. A unified protocol for fairly comparing passive managers.
Text and visual managers are compared under the same policy, trigger and budget, with cache-aware cost. The comparison now covers two policy models and three agent harnesses (Hermes, DSH, Apodex). Training-free Visual Rendering is competitive with existing compression strategies on text-centric tasks while being the cheapest:
- −51% to −82% total tokens relative to No Compression;
- a prefix-cache hit rate of 70–73%, on par with Sliding Window and Discard-All.
Hence, with capable VLMs, the visual–text gap that motivates trained optical memories is not inherent to the visual carrier.
C3. Preserving native visual evidence in a single visual carrier.
The unified protocol reveals that, when the history is multimodal, preserving native visual evidence (searched images, crops, zoomed views, tables) matters more than the choice of compression strategy. This is the confound you identified, and your suggested control confirmed it.
VERA follows directly from this finding. It keeps the evidence on the rendered canvas together with the history:
- It clearly outperforms the variant without native evidence (Vanilla Visual: +6.7 pp on Apodex, +7.3 pp on Hermes).
- It is comparable to text managers that are explicitly given the same native images.
- It needs no summarization call and no separate image channel. Within VERA's budget, native-image baselines sometimes could not fit at all, or exceeded the image cap during the run.
In short, the paper's message is: define the problem, compare fairly, find that native evidence is what matters, and preserve it in the cheapest carrier. We believe these empirical findings are useful to practitioners designing agent memory, beyond the rendering technique itself.
We are grateful for your review: it identified the key confound and an aggregation error, and made the paper more rigorous and more honest. We hope the restructured contributions better convey the value of this work. If they do, we would be grateful if you would consider revisiting your score.
Response to Reviewer 9JDq
We thank the reviewer for an exceptionally detailed review, which led us to find and correct an error. Full tables and references are in the General Response (GR).
W1. One harness and specific design choices.
We added DeepSeek-V4.1-Flash under Hermes and DeepSeekHarness (DSH) (GR-R1). Visual rendering stays competitive with the strongest text manager in every setting and is the cheapest under both harnesses, so we claim competitiveness, not superiority. Design choices: the original task and the latest tool-call round stay verbatim; in the compressed history, the 5 most recent rounds are rendered at 1.0× and older pages at 0.75×; at most 10 images are supplied per call, dropping the oldest pages first. Role/tool labels only mark provenance; text managers get the same information via message roles and tool names. All values are in Appendix D.2; the resolution sweep is in W6.
W2. Counter-intuitive Table 1.
Timed-out No Compression runs on BrowseComp-EN had been scored as incorrect instead of being re-run under our retry protocol (Appendix D.4), which caused the anomalous 59.8. We apologize for the error. After re-running them, No Compression is 68.0, and Table 1 is corrected. The remaining gap is consistent with long-horizon degradation: over 14,920 trajectories, No Compression reaches 94.1% on trajectories of 31–60 turns but only 31.7% beyond 100 turns, in line with reports that discard-style management is strong on deep search (DeepSeek-V3.2 [5]). The gain over Discard-All is small, and VERA is weakest on XBench-DeepResearch (69.0 vs. 71.7) and Seal-0 (54.1 vs. 56.5); we now state this explicitly. All new results use avg@3 and report mean ± std over repeats (GR-R1).
W3. Novelty.
Contributions (GR): (C1) the formulation, which separates managers from boundary controls and fixed from active or memory-routed managers, and thus decides what can be compared under one budget; (C2) a unified protocol, which shows that with capable VLMs training-free rendering is already competitive with text managers, so the visual–text gap reported by AgentOCR [4] and CAPS [6] is not inherent to the carrier; (C3) VERA, whose core is preserving native visual evidence. Rendering and resolution decay are credited to prior work [1, 2, 3], and "Mechanistic Analysis" is removed from the contributions.
W4. Confound in the multimodal comparison.
Summary+Native and Window+Native keep exactly VERA's native images at a matched budget, and all three are on par (GR-R3). The Table 2 gain therefore comes mainly from preserving native evidence (C3). This also explains MM-BrowseComp in GR-R1: Sliding Window keeps its recent native images and is strong, while Discard-All drops them all and is far weaker (29.7 vs. 46.3–52.7). In Text-only Rendering,
image_searchimages are replaced by captions from the Hermes vision tool (Qwen3.7-Plus) with its default prompt "Fully describe and explain everything about this image, then answer the following question: {question}"; crops and zooms are omitted, keeping their coordinates and source; tables keep their original text form (Appendix D.2).W5. One policy model; AgentOCR / OCR-Memory.
We added DeepSeek-V4.1-Flash, an open-source model from a second family (GR-R1). Other open-source VLMs need local deployment; with limited compute and slow local inference, we could not finish all controls during the rebuttal, and we list the dependence on dense-text reading as a limitation. OCR-Memory [3] is not training-free: it LoRA-fine-tunes a DeepSeek-OCR [2] retriever that returns verbatim text, so the agent never consumes images. AgentOCR [4] is learned. Both fall outside our same-policy, training-free class and are compared conceptually in Appendix J.
W6. Statistical power of RQ3.
We re-ran the sweep with DeepSeek-V4.1-Flash, avg@3, scoring timeouts as incorrect (hence 82.3 at 0.75× vs. 84.0 in GR-R1, same runs). Mean ± std at 0 / 0.25 / 0.5 / 0.7 / 0.75 / 0.8 / 1.0×: 77.7±1.5 / 77.0±3.1 / 81.7±3.4 / 84.3±2.7 / 82.3±1.6 / 80.3±2.5 / 78.0±2.2. As on Gemini, intermediate scales are best: 0.7× and 0.75× beat 1.0× and no older history by 4–7 pp, above the std. We keep 0.75× (within one std of the best, chosen beforehand on Gemini) and call the exact optimum exploratory in Finding 3.
W7. Cost and wall-clock.
Now reported (GR-R2). VERA costs 58% / 30% less than No Compression: its cache-hit rate is lower (70% vs. 88% on Hermes), but its input is 2–5× smaller. It is not slower end to end, and rendering takes 3.7–17.4 s per question.
Minor. All fixed: equation references, the reset-retry equation (old Eq. 7) now defined in Sec. 4.2, "compression factor" used throughout (now Table 7), and the resolution-figure caption (now Fig. 3).
We are grateful for this review: it caught a real error and made the paper clearer. We hope the revised claims and new evidence address your concerns, and we would appreciate it if you would consider revisiting your score.