acceptodds
Under review as a conference paper at ICLR 2027

Agents Are Systems, Not Models: Rethinking Agentic Evaluation

Abstract

Agent evaluations increasingly go beyond a single success rate, reporting metrics such as cost, consistency, and robustness. Yet they typically treat the agent itself as fixed. In practice, an agent is a configurable system: users decide what to tell it, how long to let it run, and which model to use, and each of these choices can change how well and how consistently it performs. We study these choices on a new benchmark of four scientific tasks, where a coding agent must find and correctly operate a published specialist model. We investigate five parts of the agent's configuration: task information, reasoning, self-verification, time budget, and backbone model. We find substantial run-to-run variability, with approximately 54% of the outcome variance coming from repeating the same configuration rather than changing it. Across configurations, the information provided to the agent has the largest effect, exceeding both time budget and model size, while also reducing cost and improving calibration. Configuration choices also interact: additional time helps only when the agent has sufficient information or a capable enough model to use it. Finally, a trajectory-based taxonomy of agent behavior reveals that prompting an agent to verify its answer has little effect on its verification behavior, whereas providing a dedicated verification tool changes that behavior substantially. These results suggest that agents should be evaluated as configurable systems themselves, and that some desired behaviors are more effectively implemented in the system than requested through prompting. We will release the benchmark and more than 18,000 agent trajectories.

Primary area:
foundation or frontier models, including LLMs
Keywords:
agents, evaluations, reliability, ai-for-science

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_vYbaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates how different system configurations in LLM-based agents affect capability, reliability and behavior. It highlights that most existing evaluations focus on evaluating different models under fixed scaffolds, problem statements, and configuration settings. The investigation is conducted on a benchmark of four AI for Science tasks from astrophysiscs and genomics, running all possible configurations across five configuration axes (task information, reasoning mode, self-verification mode, wall-clock budget, model). The study finds that the majority (54%) of score variance comes from run-to-run noise, and that changes in task information lead to bigger changes than changes in model size or runtime budget.

Critical strengths
  • The problem of evaluating agentic systems thoroughly in light of all their knobs is a pertinent problem which is often not fully considered.
  • The study is considerably large with 432 configurations in total repeated 5 times each per task, taking into account in their evaluation parts of agentic systems that are not considered for ablations by related work.
  • The authors intend to release all trajectories and code used to perform this study.
Critical weaknesses
  • The evaluation is limited to four tasks in two domains, which is too little to make claims about the generalized impact of each "knob".
  • It is not clear that the evaluated scenarios (i.e., the different configurations) represent realistic scenarios for example for how two different users would setup/use an agent for such tasks.
  • The abstract claims that the Information axis has a larger effect than Budget. However, this is not the case for all models since for Claude Sonnet 5 Budget has a larger effect.
  • The methodology does not consider unfinished runs, but these unfinished runs are not unfinished at random. Several of the considered knobs (Budget, Information, Model) have a direct influence on the lenght/duration/cost of runs and on whether they finish.
  • Wall-clock is a potentially noisy metric. It would be better if budget is considered in terms of tokens or token cost.

I think the paper's goal is worthy but doing the study on such a small number of tasks reduces the relevance of results since it's quite unclear whether the results generalize. Without doing it on more tasks, the main takeaway is that there are many knobs and we should consider them when evaluating agentic systems (which is not necessarily novel).

Questions for authors

see weaknesses

Reviewer_XH1WOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2(+1)
Presentation
3
Soundness
2
Read review
Paper summary

The authors argue that agents should be evaluated as configurable systems rather than as traditional models. As evidence, they evaluate agents under different settings and show that performance varies wildly when you change things like the information given to the agent, its time budget, and its backbone model, and that roughly half of the remaining variance is run-to-run noise.

Critical strengths
  • The eval they perform is exceptionally robust and is significantly more holistic than a lot of the work going on nowadays. I cannot find anything obviously wrong with it.
  • They evaluate the ability of agents to actually identify how reliable the tools they are using are, which is still not particularly common in the literature.
Critical weaknesses
  • I am somewhat confused as to exactly what is new here.
  • I think framing this benchmark as a pure AI4Science is strange given that it only includes two domains of science.
  • The authors acknowledge that the AIs never really decline to use the AI specialist, but this seems like the obvious course of action for the agents to follow when they are told that a model is a specialist. It would not be reasonable to expect a generalist model to recognize the specialist’s deficiencies without a very long sequence of interactions bringing them to light. So it is unclear what this result really adds.
  • There exists other work focused on specifically evaluating agents that is not mentioned here. The earliest one I know is the Agent-as-a-Judge work (though I am sure there are similar works that pre-date this). Some comparisons with that work would be valuable.
Questions for authors
  • Can the authors clarify the novelty of this work? I generally favour more papers that just reproduce results, but ICLR's editorial standards disagree with me. Is the principal contribution the dataset? If so, the paper needs to be framed like that.
  • I do not quite buy the premise of this work. Previous work has always evaluated systems in isolation, even when they maintained configurable parameters; evaluating ML models has always been about putting their best foot forward. Why should these be different?

Please also respond to the points raised in the weaknesses section.

1 comment
Reviewer_XH1WOpenReview ↗

Response to authors

I'll admit I'm still not totally sold on this paper. I think my biggest sticking point is still Q2. But I accept that the authors have done a thorough job of addressing my other concerns so I'll raise my score to a weak accept.

Reviewer_hBs5OpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper argues that agents should be evaluated in each of its submodules, not only in its output. Authors evaluate how an agent’s configuration affects how well it uses specialist models to solve four scientific tasks from astrophysics and genomics. They vary this configuration along five axes: (1) what information the agent is given; (2) allowed reasoning; (3) how strongly the agent is asked to verify its own answer; (4) runtime budget to solve the task; and (5) agent’s backbone model. Axes are varied in a full factorial design that result in 432 configurations. Every configuration is repeated 5 times, which gives 2,160 runs per task; with 4 tasks × 2,160 = 8,640 runs in total. Notably, authors find that i) 54% of the score variance is run-to-run noise rather than an effect of the configuration, ii) the information given to the agent has the largest effect, and the effect of some axis depend on others.

Critical strengths
  1. Very detailed and comprehensive study of how an agent’s configuration itself drive capability and reliability.
  2. Useful decomposition of an agent's system for evaluation.
  3. Accessible and understandable for any type of audience.
Critical weaknesses
  1. The usefulness of the benchmark itself will depend on the field of application.

Minor

  • It is not clear to me how many specialists each agent has at their disposal.
Questions for authors

No questions.

1 comment
AuthorsOpenReview ↗

Response to Reviewer hBs5

We thank the reviewer for the clear review and the positive evaluation of our work, and would like to highlight the assessment of our study as “very detailed and comprehensive” as well as “accessible and understandable for any type of audience”. We address the specific points below:

(W1) The usefulness of the benchmark itself will depend on the field of application.

While our tasks are domain-specific, they are selected to test different parts of working with published scientific models (pre-processing, fine-tuning, post-processing, and declining a weaker specialist), and our main findings hold when we evaluate them in additional domains (see "Response to Reviewer XH1W - Part 4" and "Response to Reviewer vYba - Part 3"). Our conclusions therefore concern the behavior and evaluation of agentic systems in general, with the chosen domains as a testbed.

(W2) It is not clear to me how many specialists each agent has at their disposal.*

In the main experiments, the agent has all specialist models at its disposal that were part of the framework when the task's grid was run: two for the astronomy tasks (AstroCLIP and AstroSage), four for the genomics tasks (additionally DNABERT-2 and RiNALMo), and six for the additional medical tasks (additionally ECGFounder and BioMistral). In all ablations (other model families, oracle verification, and prompt variants), every task has the four models from the main four tasks available. Across these settings, the agent essentially always uses the task's specialist, also at the lowest information level, where the task does not name it, and also in the genomics tasks, which include more distractors than the astronomy tasks (Appendix A.2, Table 7). We have explicitly clarified this in L. 760 ff.

We hope these clarifications and the revisions address the reviewer’s concerns, and are happy to provide any further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.