- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GATOR, a post-retrieval router that selects among caption, full-text, and image representations of each page under a token budget, which is an important dimension for balancing performance and cost but neglected in previous methods. The selection mechanism combines a training strategy to train an MLP-based scorer with a cost-aware selection mechanism. Experiments on UniDoc and MMDocRAG show that GATOR substantially reduces evidence cost while maintaining or even improving answer quality.
Critical strengths
- This paper is based on a strong motivation, identifying an important and underexplored problem: selecting the appropriate representation granularity of retrieved multimodal evidence.
- The proposed GATOR mechanism considers multiple factors and can be utilized in a standalone manner or integrated with existing retrieval and memory systems.
- Experiments on 2 benchmarks show that GATOR can reach a better performance-cost tradeoff
Critical weaknesses
Presentation issue
- Why are most of the loss definitions deferred to the appendix? Moreover, the value of the margin parameter (\mu) in the margin loss appears to be unspecified.
- Given the large number of components in GATOR, Section 2 would be easier to follow if the method were explained alongside Figure 2. Currently, Figure 2 is referenced only once at the beginning of the section and is not used to guide the subsequent technical discussion.
Efficiency and Cost Accounting
-
Although GATOR reduces the reported inference-time evidence cost, it requires additional preprocessing of the original data to construct multiple representations and token-number measurement. The paper should discuss this preprocessing overhead and how it trades off against—or is amortized by—the inference-time savings.
-
The token-equivalent cost does not fully capture modality-dependent computational overhead. Images require additional processing by a vision encoder, whereas text is processed through token embeddings, resulting in different latency and resource costs. Moreover, output tokens also contribute to actual end-to-end efficiency. Evaluations based on wall-clock latency, throughput, memory consumption, or actual API cost would make the efficiency claims more convincing.
Concerns on Generalization
-
It is unclear whether evaluation on only two benchmarks is sufficient to support the generalization claims.
-
Unlike a general-purpose embedding retriever, GATOR’s learned scorer must be trained on dataset-specific supervision. It is therefore unclear how the method would perform in a new domain or on a dataset without annotations. Figure 3(b) already suggests that cross-dataset transfer can substantially alter the quality–cost trade-off.
-
In addition to its trainable parameters, GATOR employs a relatively complex selection mechanism with multiple components and hyperparameters. This raises concerns about its robustness across datasets. How sensitive is the method to these hyperparameters? Do they need to be retuned for each dataset, and what is the computational or annotation cost of such tuning?
-
Can GATOR be integrated with existing general-purpose agent harness frameworks? The paper would benefit from discussing the required interface and engineering modifications.
Limited Baselines
-
The baseline comparison is relatively weak. The paper does not compare against existing adaptive granularity-selection or routing methods, such as OMGM (ACL 2025), Mix-of-Granularity (COLING 2025), and UniversalRAG (ACL 2026).
-
I also suggest including an agentic baseline. For example, the model could initially receive only low-cost page summaries and then autonomously decide which pages should be expanded into full text or images.
Questions for authors
Please refer to the weakness.