- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces INDEXACT, an interface for agentic search that separates candidate-set exploration from source-text inspection. Motivated by an analysis of context composition and the timing of source-text exposure, the method enables agents to construct, retain, and combine candidate sets through index-based operations. These operations return reusable state references and statistical feedback, while source text is accessed through explicit reading requests.
Critical strengths
- The progression from empirical observations to problem formulation and method design is clear and well motivated.
The paper begins by examining what enters an agent’s context and when source text is encountered during evidence discovery. It then asks a concrete question: does the agent’s next search decision require reading source text, or would feedback about the candidate set suffice? This question naturally motivates the proposed combination of persistent candidate states, statistical feedback, and explicit reading. The resulting argument is coherent, with the method’s components closely connected to the identified problem. The authors also appropriately acknowledge that text from unannotated documents may still provide useful clues.
- The main benchmark results are promising, with reasonably broad comparisons and component-level analyses.
On the full 830-question BrowseComp-Plus benchmark, INDEXACT improves answer accuracy and evidence coverage while reducing average live context relative to the terminal-based alternatives. The evaluation includes conventional retrieval agents, direct corpus interaction methods, and selective-reading interfaces. The additional multi-hop QA experiments broaden the evaluation, while the ablations provide initial evidence that exact counts, cross-call state reuse, and set composition contribute to performance.
Critical weaknesses
-
Although the paper includes several relevant methods, important details remain unclear, including the evaluated RARG variant, PI-SERINI’s retrieval depth and BM25 configuration, and the adaptation of SIEVE’s structured-document interface to the evaluation corpora. These choices could materially affect performance. Without clearer documentation, it is difficult to determine how much of the observed improvement can be attributed specifically to the proposed interaction mechanism.
-
Some improvements on multi-hop QA are very small. The gains of 0.2 percentage points on 2Wiki and 0.8 percentage points on Bamboogle each correspond to only one additional correctly answered question. The main ablations use a 100-question subset, and the paper does not report repeated-run variability, confidence intervals, or paired significance tests. Consequently, the stability of the smaller improvements and some ablation differences is difficult to assess. Furthermore, all experiments use a single backbone model. While the BrowseComp-Plus result is encouraging, additional statistical analysis and evaluation with another backbone would strengthen the broader claim that the proposed interface consistently improves agentic search.
Questions for authors
see above