- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes a training-free framework for multi-step visual RAG, addressing the challenge of the ineffective use of relevant evidence over long reasoning trajectories. It introduces three components: evidence selection that balances relevance, requirement coverage, and redundancy; adaptive compression and retention of historical memory; and relevance-based allocation of visual resolution to retained images. Experiments across three document question-answering benchmarks and four acting models show that it achieves the highest performance among the compared methods and reduces image transmissions. Analyses show larger gains on questions associated with longer ReAct trajectories.
Critical strengths
- The paper targets a practical limitation of multi-step visual RAG, which is that retrieving evidence doesn’t guarantee that it can be effectively used in subsequent reasoning.
Critical weaknesses
- The paper emphasizes its evidence coordination based on unresolved answer requirements. However, in the evaluation setup, the variant in which the acting model reports resolution status at each summarise call is disabled, and slot scores are not propagated to visual allocation (Appendix A.2–A.5). Moreover, the released evidence admission implementation uses precomputed slot scores without accounting for requirements already covered by selected pages (admission.py). This makes the contribution of the proposed requirement-aware coordination unclear.
- The provided codebase records several deviations from the stated evaluation protocol in the paper: a five-call limit on ReAct's main interaction loop in 11 of 12 runs and a different first-query procedure for the Gemini/ViDoSeek TAEC result (manifest.md). Moreover, TAEC selects five pages from a top-25 candidate pool, while DAG uses the retriever's top-5 (Appendix A.3/A.8). These differences raise concerns about baseline comparability and make it difficult to isolate the contribution of requirement-aware selection. The authors should clarify the impact of these protocol deviations and provide a controlled comparison using the same candidate pool.
Questions for authors
-
On ViDoSeek, Figure 5(b) and Appendix M show that most of TAEC's gain over the DAG agent comes from questions where only TAEC retrieves the gold page, and there is little difference when both methods retrieve it. Can the authors provide a controlled comparison that holds the retrieved pages fixed and isolates the effects of memory exposure and visual allocation on answer accuracy?
-
Table 6 reports model calls and image transmission for one model and benchmark. Could the authors report end-to-end latency or a comparable measure of total model input/output usage for TAEC and the compared baselines?
4 comments
Response (2/4): W1/W2
W1 (resolution status, score propagation, and marginal coverage):
We clarify that requirement-aware coordination is active in the evaluated configuration: resolution is updated at every step, and requirement scores reach visual allocation (Appendix A.2–A.5 of the revised manuscript).
- Resolution status. After each summary, memory exposure reads the confidence and failure cues of the earlier summaries and re-classifies their nodes as resolved, failed or active (Appendix A.4, L756–760). Memory exposure then condenses resolved observations to their conclusion and renders failed branches as short traces. Requirements are extracted once per question, with uniform weights (Appendix A.2, L725–727), and are shared by admission and allocation. Resolution is read from the summaries the acting model already writes at every step, so the acting model's tool interface stays identical to the DAG agent's.
- Requirement-score propagation. Propagation is enabled in the evaluated runs. The run records of all twelve TAEC cells show it: admitted pages carry their requirement scores into memory, and allocation mixes this score into the detail term d_i (Appendix A.2, A.3 and A.5 of the revised manuscript). This is also why c_i = 1: requirement support already enters allocation through d_i, so the product d_i·c_i in Eq. (4) is realized as one score that combines detail demand and requirement support (Appendix A.5 of the revised manuscript).
- Marginal coverage. Admission computes marginal coverage as in Eq. (2) (L207–215): during the greedy selection, the coverage of each requirement is updated as pages covering it are admitted (Appendix A.2, L727–728), so a candidate's requirement gain counts only the requirements not yet covered. The released code implements this computation.
W2 (first query and candidate pool):
- First query. We clarify that at the first round, TAEC retrieves with the original question. The question itself is sent to the retriever, and its top five pages are placed in the retriever's order (Appendix A.1 and A.3 of the revised manuscript). The first evidence is thus retrieved for the complete information need of the question, from which the requirement set is also extracted (Appendix A.2, L725–727), in line with the failure TAEC targets, where "query reformulation and context updates may exclude evidence still needed to answer the original question" (Sec. 1, L79–80). From the second round on, admission, exposure and allocation act on the accumulated evidence. The released code implements the same first round.
- Candidate pool. The 25 candidates form a reranking pool that no model reads; the acting model sees only the five pages placed per search, the same context budget as every system (Appendix B, L831–833). Admission scores each candidate by its retriever score, its requirement support and its redundancy with the pages already selected, and chooses the five pages greedily; it is a deterministic, training-free reranker that adds no model calls (Appendix A.3, L736–746; A.6, L783–785). Retrieving a larger pool and reranking it to the few pages that are read is the standard two-stage setup of document RAG. For example, DocReRank (EMNLP 2025) reranks the top-20 pages returned by ColPali or ColQwen and evaluates the top five, and SetR (ACL 2025) selects from the top-20 retrieved passages the set that covers the information requirements of the query, while the rerankers it compares with place their top five from the same candidates. Admission's rule is the relevance-with-redundancy criterion of Maximal Marginal Relevance (Carbonell & Goldstein, SIGIR 1998), extended with requirement coverage. Ranking the same pool by retriever score alone returns the retriever's top five, which is exactly what the baselines receive, so the requested controlled comparison coincides with the baseline setting. On the first search, admission places the first five pages in retriever order.
Response (3/4): W2
W2 (ReAct budget):
We clarify that the protocol sets twenty model calls as the upper bound for every system (Appendix B, L861), and ReAct ran with a five-call budget in eleven of the twelve cells. The manifest in the earlier release recorded these budgets; through version management of the run configurations it no longer matched the released code, and the launch script of the released code now specifies the configuration of each system. The twenty-call cell, ViDoSeek with Gemini-3.5-Flash, is the run analysed in Sec. 1 and Sec. 4.2: to measure how accuracy changes as a trajectory lengthens, ReAct is left unconstrained there (Table 2, L973), and Table 1 reports the same run for that cell. Fixed budgets of a few rounds are common for multi-turn retrieval baselines. For example, FlashRAG (WWW 2025) sets "the maximum interaction rounds to five" for its Self-Ask-style iterative baseline. Practical cost also favoured the smaller budget. Without context management, every further call of ReAct re-sends its whole history. In our twenty-call runs, the longest trajectories are those whose context kept growing: Kimi-K3 questions on MMLongBench-Doc that accumulated 34 to 54 pages ran for over an hour, and a GPT-5.6 question that used all twenty calls over 48 pages ran for 57 minutes. With Gemini-3.5-Flash, once a question has accumulated twenty or more pages, which happens on 17.1% of SlideVQA and 29.4% of MMLongBench-Doc questions, about half and four fifths of these questions end without an answer.
Our paper shows why more searches do not by themselves help ReAct. On ViDoSeek with twenty calls, accuracy on trajectories that retrieve the evidence falls from 91.7% at two searches to 52.0% at nine or more (Sec. 1, L73–77). Each additional call enlarges a context that ReAct re-sends in full, reaching 92.8 images per question against 14.7 for TAEC (Sec. 4.6, L454–455); in the cells where ReAct falls below single-pass retrieval, the loss comes from the questions on which it searched again, 8.2 points on SlideVQA with Gemini-3.5-Flash and 8.6 on ViDoSeek with GPT-4o-mini (Appendix H, L1277–1284). Other work reports the same pattern. Lost in the Maze (COLM 2026) finds that "popular agentic search frameworks struggle to scale to long trajectories primarily due to context limitations", and that ReAct "often hits the context window limit", so that at ten turns it "already falls back to not using any tools for most queries"; s3 (EMNLP 2025) reaches its peak at three turns, "and adding more turns or broader retrieval brings limited improvement".
Nevertheless, to further address this concern, we are re-running ReAct with twenty calls in every cell; owing to the limited time of the discussion period and the large cost of twenty-call trajectories, these runs are not complete at the time of this response. On ViDoSeek, where four cells are complete, twenty calls do not give ReAct a consistent gain over five: ReAct reaches 67.0, 70.5 and 55.6 with twenty calls for Kimi-K3, GPT-5.6 and GPT-4o-mini, against 82.1, 82.1 and 56.1 with five calls (Table 1), and 79.5 with twenty calls for Gemini-3.5-Flash (Table 1), against 77.6 with five calls in a run of that cell under the same five-call budget. TAEC remains ahead in each of these cells (88.1, 87.0, 71.0 and 87.5). The table gives ReAct with five calls in all twelve cells, including the run above: TAEC remains ahead in every cell, and it does so as well when it is held to five searches, counting a TAEC question as correct only when TAEC answered it within five searches.
| Benchmark | Acting model | ReAct, five calls | TAEC (Table 1) | TAEC within five searches |
|---|---|---|---|---|
| ViDoSeek | Gemini-3.5-Flash | 77.6 | 87.5 | 87.3 |
| Kimi-K3 | 82.1 | 88.1 | 87.7 | |
| GPT-5.6 | 82.1 | 87.0 | 86.5 | |
| GPT-4o-mini | 56.1 | 71.0 | 70.8 | |
| SlideVQA | Gemini-3.5-Flash | 77.2 | 83.6 | 83.1 |
| Kimi-K3 | 82.0 | 83.8 | 82.5 | |
| GPT-5.6 | 81.0 | 82.2 | 81.1 | |
| GPT-4o-mini | 59.3 | 62.0 | 61.1 | |
| MMLongBench-Doc | Gemini-3.5-Flash | 42.4 | 50.1 | 49.4 |
| Kimi-K3 | 36.5 | 45.8 | 45.0 | |
| GPT-5.6 | 39.1 | 42.6 | 40.6 | |
| GPT-4o-mini | 15.2 | 17.7 | 17.0 |
Table 1 keeps the runs that were judged together in its single judging batch (Sec. 4.1, L298), so the five-call run of the ViDoSeek / Gemini-3.5-Flash cell is reported here rather than in the table. We commit to reporting the complete results of ReAct under both the five-call and the twenty-call budget in every cell in the final version.
Response (4/4): Q1/Q2
Q1 (holding retrieved pages fixed):
We clarify that in a multi-step agent, the retrieved pages are themselves an outcome of exposure and allocation, and the grouping of Figure 5(b) and Appendix M is the comparison on identical questions. Memory exposure and visual allocation change the context the acting model sees at the next step, and with it the query it writes and the pages retrieved afterwards. Different configurations therefore retrieve different pages in normal operation. Appendix M groups questions by what both systems retrieved, so that within each group the two systems answer identical questions, and the four groups sum exactly to the accuracy difference (L1558–1561).
Within this comparison, TAEC gains in both coverage and use:
- Coverage. On ViDoSeek, 115 questions are retrieved only by TAEC, and TAEC answers 86% of them against 22% for the DAG agent (Sec. 4.5, L405–407). Coverage is close to nested: the two systems retrieve differently on 118 ViDoSeek questions, 115 of which are TAEC's additions (Appendix M, L1564–1565). Because memory exposure and visual allocation shape the follow-up queries, part of this coverage gain is itself their effect.
- Use, with the gold page available to both. On the questions for which both systems retrieve a gold page, TAEC is 8.8 points higher on SlideVQA and 5.0 on MMLongBench-Doc, and level on ViDoSeek (Sec. 4.5, L407–410).
- Each component on its own. The component ablation adds each component individually to the DAG agent (Sec. 4.4, L376–377; Figure 4). Memory exposure alone raises accuracy by 4.0, 6.1 and 2.7 points on ViDoSeek, SlideVQA and MMLongBench-Doc, and allocation alone by 3.4, 5.7 and 2.3; the full system reaches 87.5, 83.6 and 50.1.
This grouping also explains why post-hit accuracy differs little across systems. TAEC's additional coverage brings harder questions into its retrieved set, so many of its retrieved-but-unresolved cases come from this newly added coverage. Post-hit accuracy computed per system therefore compares different sets of questions: on ViDoSeek, the DAG agent scores 90.1 and TAEC 90.6 on their own hits, while their accuracies differ by 7.4 (Appendix M, L1551–1556).
Nevertheless, to further address this concern, we examined the questions on which the two systems retrieve exactly the same pages. On ViDoSeek this holds for 150 of the 1,142 questions, and on all but one of them both systems answer from the pages of the first search, which is the question itself; TAEC and the DAG agent answer 88.0% and 87.3% of them correctly. Identical pages thus arise when the question itself already retrieves the evidence, before any history accumulates for memory exposure and visual allocation to act on. Holding the pages fixed therefore selects the questions on which these two components have nothing to change; their effect on answer accuracy is measured instead by adding each of them alone to the DAG agent (Figure 4) and by the groups in which both systems retrieve a gold page, where TAEC is 8.8 and 5.0 points higher on SlideVQA and MMLongBench-Doc (Sec. 4.5, L407–410).
Q2 (latency and total usage):
We report end-to-end latency per question together with total input usage on the runs behind Table 1. The efficiency claims of our paper concern transmitted context, model calls and image transmissions (Sec. 4.6, L429–456), and latency is reported here as an additional measure; total input usage is the context transmitted over all model calls of a question.
| ViDoSeek, Gemini-3.5-Flash | Latency, mean / p95 (s) | Model calls | Context (MB) | Images |
|---|---|---|---|---|
| Vanilla | 9.6 / 13.0 | 1.00 | 0.22 | 4.9 |
| ReAct (20 calls) | 78.3 / 286.3 | 5.34 | 3.73 | 92.8 |
| DAG agent | 32.7 / 136.4 | 4.94 | 0.87 | 20.0 |
| TAEC | 29.9 / 125.7 | 2.91 | 0.61 | 14.7 |
On ViDoSeek, TAEC has the lowest mean and p95 latency among the multi-step systems. On SlideVQA, its mean latency is also below the DAG agent's (33.9 s vs. 39.4 s), with a higher p95 (143.6 s vs. 129.5 s). On MMLongBench-Doc, where TAEC issues about a third more searches, its latency is above the DAG agent's (77.5 s vs. 71.1 s). Requirement extraction depends only on the question, so its single call per question (Appendix A.6, L783–784) can be issued before the trajectory starts or computed offline. The latencies above include this call, so moving it offline would lower TAEC's latency further.
Response (1/4): Overview
We apologize for the timing of our response. We used the discussion period to re-run experiments, including ReAct with a twenty-call budget, and to align the released code and the appendix with the evaluated configuration, so that every answer below rests on checked results. We thank the reviewer for the close reading of our code and manifest, and for recognizing that retrieving evidence does not guarantee its use in later reasoning. Owing to version management and to the code cleanup for the release, which anonymized the code and aligned its names with the terminology of the paper, parts of the submitted appendix and the repository at the anonymous link reflected earlier iterations. Both are corrected, and the code for the main table is released with run scripts and configurations at a new anonymous link, https://anonymous.4open.science/r/taec-artifact-667E/, which replaces the link given in the submission. These corrections concern how the appendix and the repository describe the evaluated configuration; they do not change any result in the main text. Every number in Sec. 1 and Sec. 4 comes from the runs behind Table 1, and the main claims rest on these runs: TAEC is the strongest system in all twelve benchmark × backbone cells of Table 1 (L357–362); it improves over the DAG agent, which shares its trajectory graph and memory, by 4.5 points on average (66.8 vs. 62.3), and each component improves the DAG agent on its own on all three benchmarks (Figure 4). The one point that bears on the comparisons of the main text is the interaction budget of ReAct, which we address in our response to W2. We remain confident in these results and in the claims they support.