- Rating
- 3
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces AI Research Preference Models (RPMs) to select promising candidate solutions before committing to expensive training and evaluation. The authors propose two variants: an inference-only RPM that judges candidates using their plans, code, and search history, and an agentic RPM that additionally conducts small-scale pilot experiments. Both are integrated into the child-creation stage of AIRA-dojo and evaluated on AIRS-BENCH using four LLM backbones. The paper reports improved final performance and reduced execution time to reach the unguided baseline's performance.
Critical strengths
The motivation is clear. AI research agents can generate candidate solutions much faster than they can fully execute and evaluate them. Selecting promising candidates under a limited execution budget therefore addresses a meaningful bottleneck in automated research.
Critical weaknesses
-
The novelty of preference-based candidate selection appears limited. The general idea of using a judge or preference model to select promising candidates before expensive execution already has precedents. Section 6 explicitly acknowledges Zheng et al. (2026), who predict pairwise preferences between unexecuted ML solutions, and Goldie et al. (2026), who propose using a judge or reward model to select promising leaves in research-agent tree search. The distinctive contributions here appear to be conditioning on the live search tree and incorporating pilot experiments. However, the paper does not sufficiently establish the incremental benefit over these closely related approaches through direct comparisons.
-
The final performance gains are relatively modest, particularly for the additional agentic component. In Figure 2, Agentic RPM's additional gains over Inference-only RPM are only 0.013, 0.005, 0.018, and 0.026, respectively. Inference-only RPM also slightly underperforms No RPM on Muse Glimmer, scoring 0.694 versus 0.696. These results demonstrate improvements, but their magnitude is limited. Although Figure 2 includes 95% confidence intervals, the paper does not report paired statistical tests establishing the significance of the differences between methods.
-
It remains unclear how expensive agentic verification is relative to fully executing a candidate. Section 3.2 gives the Agentic RPM access to a single H200 GPU and an environment equivalent to that of the main research agent. Appendix B.1 specifies a nominal budget of 300 seconds per agentic workflow, but also states that the budget is checked only after the current turn's tool calls complete, so actual execution can exceed it. Section 5.2.2 states that pilot experiments share the research agent's time budget, meaning that their execution cost is not simply ignored. Nevertheless, the paper does not provide a clear breakdown of the total pilot cost per candidate-selection step relative to the cost of fully executing the selected candidate. Section 7 also assumes negligible LLM inference cost and provides a latency adjustment only for the inference-only Qwen configuration. Consequently, it is difficult to determine when agentic verification remains substantially cheaper than direct execution after accounting for all comparisons, pilot experiments, and model calls.
Questions for authors
-
Section 3.3 describes candidate selection as pairwise comparisons in a knockout tournament, while Section 5.1 uses 15 candidates per operator step. Does Agentic RPM follow this same tournament structure, or are all 15 candidates passed to a single agentic workflow? Appendix B contains both two-candidate and multiple-candidate wording, so the exact end-to-end procedure would benefit from clarification.
-
Is the 300-second budget allocated to each pairwise comparison or to the entire selection among 15 candidates? A standard knockout tournament over 15 candidates requires 14 comparisons. If each comparison receives a separate 300-second budget, this would amount to 70 minutes of nominal aggregate comparison time per selection step before accounting for early termination, overruns, or parallelism. Please clarify the actual allocation and scheduling.
9 comments
W1: Novelty over Goldie et al. (2026) and Zheng et al. (2026)
A1: We thank the reviewer for highlighting these related works. We believe our work makes several contributions beyond those introduced by Zheng et al. (2026) and Goldie et al. (2026).
Comparison with Goldie et al. (2026)
Goldie et al. suggest using an LLM judge or reward model to select promising leaves within an AI research agent's search tree to reduce the cost of search. However, they present this as a future research direction, without implementing or evaluating the proposed approach. Our work implements and evaluates this direction, demonstrating improvements through extensive offline and end-to-end evaluations, while further extending it through the introduction of Agentic RPMs.
Comparison with Zheng et al. (2026)
Unlike Zheng et al., our Inference-only RPMs leverage previously executed solutions from the search tree and their validation scores to inform the RPM's decision.
Beyond this, we contribute:
- Agentic RPMs, which conduct small-scale pilot experiments to inform candidate selection, substantially improving offline selection accuracy and end-to-end performance.
- A scaling analysis investigating the effects of historical context, reasoning effort, and suggestion count on the Inference-only RPMs selection accuracy. Our original offline evaluation shows that increasing context from 0 to 100 nodes improves accuracy from 60.7% to 65.0%.
- A more extensive end-to-end evaluation across four LLM backbones, 18 AIRS-Bench tasks, and 10 independent seeds per task and method. We also conduct oracle selection evaluations and analyze the correlation between selection quality and final scores. In comparison, Zheng et al. evaluate FOREAGENT with a single LLM backbone on only 5 MLE-bench tasks with 3 seeds each. Our evaluation therefore provides broader empirical evidence for the effectiveness of preference-guided candidate selection.
As part of the rebuttal period, we have conducted additional experiments to further evaluate the benefit of historical search-tree context.
First, we extend our original offline evaluation to two additional backbones:
| Backbone | 0 Context (%) | 10 Context (%) |
|---|---|---|
| Qwen3.6-27B | 56.88 | 57.19 |
| Muse Glimmer | 59.50 | 64.12 |
We further construct an offline evaluation dataset from the end-to-end AIRS-Bench runs presented in Section 5.1, obtaining the following selection accuracy results:
| Context Nodes | Low Effort (%) | High Effort (%) |
|---|---|---|
| 0 | 55.07 | 56.10 |
| 1 | 59.13 | 59.73 |
| 10 | 60.70 | 63.17 |
| 100 | 62.30 | 63.43 |
These results further demonstrate the benefit of incorporating historical search-tree context, improving selection accuracy by up to 7.33 percentage points. On the new evaluation dataset, our Agentic RPM outperforms all inference-only configuration with a 74.5% selection accuracy, demonstrating the effectiveness of our novel pilot experimentation method.
W2: Although Figure 2 includes 95% confidence intervals, the paper does not report paired statistical tests establishing the significance of the differences between methods.
A2: To address the reviewer's concerns we conduct paired task-level t-tests across all four LLM backbones and the 18 benchmark tasks shared across methods. For each task, we first compute the difference in mean normalized scores between methods, averaging over seeds, and then average these differences across the four backbones. We perform a two-sided paired t-test on the resulting 18 task-level differences, avoiding treating model–task combinations as independent observations.
We find that the Agentic RPM achieves a statistically significant improvement over No RPM (mean difference +0.034, 95% CI [+0.005, +0.062], p=0.023), improving performance on 13 of the 18 tasks when averaged across models. The Inference-only RPM also improves over No RPM (+0.018, p=0.160) and the Agentic RPM improves over Inference-only RPM (+0.016, p=0.083), although neither differences are statistically significant. All p-values are unadjusted.
These findings provide further evidence that the Agentic RPM improves average E2E performance over the unguided baseline across the evaluated backbones, while accounting for task-level pairing.
TLDR:
We thank the reviewer for their comments and for recognizing our paper's strong motivation and clear presentation.
Below we address questions and comments raised by the review: