- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents AREX, a deep research agent that gathers evidence, assesses a provisional answer, and decides whether to accept it, conduct further research, or restart. The method combines recursive refinement with autonomous context updates and training that emphasizes important research decisions. Experiments on six benchmarks show strong performance at two model sizes, with further improvements from additional research rounds.
Critical strengths
-
The approach addresses a practical problem in long research tasks: preserving useful findings while identifying what still needs to be investigated. The proposed research loop and context management are well aligned with this goal.
-
The evaluation covers several types of search and reasoning tasks. The results at both model sizes provide evidence that the overall approach is useful.
-
The scaling experiment supports the claim that the system benefits from additional research rounds, with accuracy increasing from 71.4% to 82.5%.
Critical weaknesses
-
The comparisons with other systems need clearer reporting. The paper does not sufficiently explain where the baseline results come from or how their evaluation settings differ from AREX. This makes it difficult to determine how much of the reported advantage comes from the proposed approach versus differences in tools or inference resources. The internal comparisons provide stronger evidence for the method’s benefits than the cross-system comparisons.
-
The key-step selection procedure needs a clearer description. Key-step supervision produces a substantial gain and is an important part of the training contribution. The paper describes the kinds of steps it selects, but how these steps are identified in practice remains unclear. This limits the reproducibility of an otherwise useful finding.
Questions for authors
-
Which baseline results were reproduced, which were taken from published reports, and what important evaluation settings differ?
-
How are key steps identified and labeled in the training trajectories?
We thank the reviewer for recognizing the practical importance of our framework, the promising empirical results, and the effectiveness of our ablations. We address the concerns regarding evaluation transparency and reproducibility below.
W1/Q1. Baseline sources and evaluation settings.
All external baseline results in Table 1 are taken from publicly available technical reports and benchmark results, rather than reproduced in our evaluation environment. Only AREX-Turbo and AREX-Base are evaluated using our unified agent interface.
We have made efforts to align inference budgets with existing research agents. In particular, following MiroThinker MiroMindTeametal,˙2026, AREX adopts a maximum budget of 300 tool calls per round × 5 rounds = 1,500 tool calls Sec\. 5\.1; Appendix B.
While exact tool environments and computational costs cannot always be matched, particularly when external reports do not fully disclose their evaluation configurations, we align key inference-budget settings wherever possible. Table 1 therefore provides a broad performance reference rather than a strictly controlled comparison. The table also distinguishes full-HLE results from text-only results using the * notation.
Importantly, our conclusions regarding the effectiveness of AREX's components are primarily supported by controlled within-system ablations Figure2(b and Table 2), where the relevant model and training or inference budgets are matched.
We have updated the Table 1 caption to explicitly clarify that all external baseline results are taken from publicly available technical reports or benchmark releases, rather than reproduced in our evaluation environment.
W2/Q2. Key-step identification and labeling.
As described in Appendix E, we use offline rule-based detectors to identify three types of decision-critical steps. The operational heuristics are summarized below:
Evidence discovery: We select the first tool-calling step whose returned observation contains the reference answer after exploratory search. Matching uses normalized strings, ignoring capitalization and punctuation differences.
Path rejection and redirection: We detect explicit rejection expressions eg˙,˙"ruledout,""doesnotfit" together with correction or alternative-candidate markers eg˙,˙"instead,""reconsider,""alternative". We additionally require a search/visit action or adoption of the reference answer, with answer-bearing evidence already present in preceding tool observations.
Key context updates: We select
update_contextcalls that preserve a previously observed answer-related entity and at least one source URL, together with markers such as "verified," "evidence," "constraint," or "next step."To reduce spurious annotations, these rules are applied only to trajectories with verified final answers and valid supporting tool observations Sec\. 3\.2; Appendix E. During training, the complete preceding context is retained, while the loss is applied only to the selected assistant step.
Our motivation for rule-based detection is practical: long research trajectories contain many routine interactions, making exhaustive manual step-level annotation prohibitively expensive. We therefore design conservative heuristics targeting observable events associated with evidence acquisition, search redirection, and context consolidation. These rules aim to identify a high-precision subset of informative decisions, rather than exhaustively determine the causal importance of every step.
Their training utility is supported by Table 2: replacing key-step supervision with equal-budget random-step replay reduces BrowseComp accuracy from 82.5% to 74.1% −84˙points.
We will release the extraction code containing the complete matching patterns, filtering thresholds, and preprocessing details. As noted in our conclusion, developing more general and autonomous mechanisms for estimating step utility remains an important direction for future work.
We hope these clarifications address the reviewer's concerns.