acceptodds
Under review as a conference paper at ICLR 2027

Experimental Experience Models for Autonomous Research

16 views

Abstract

Autonomous research agents can generate hypotheses and conduct experiments, but experimentation remains a major source of computational cost. A fundamental challenge is deciding which experiments are worth running, particularly when prior evidence is insufficient to resolve uncertainty. Yet current research agents lack a systematic way to leverage experimental experience when making such decisions. We introduce Experimental Experience Modeling (EEM), a framework for making informed experimental decisions by acquiring, reusing, and accumulating experimental experience. EEM extracts decision-relevant records from earlier experimental trajectories, distills them into reusable experience, and organizes them in an experience library. For a new experimental decision, EEM retrieves relevant historical experience and assesses whether it provides sufficient support for deciding whether a candidate direction warrants further investment. When historical experience is insufficient, EEM conducts a targeted, low-cost pilot experiment to acquire the missing decision-relevant experience on demand. It then combines this newly acquired experience with retrieved historical experience to determine whether the direction warrants full-scale evaluation, which requires substantial resources. The resulting experimental outcomes are further distilled into reusable experience, allowing the library to continually grow through iterative accumulation. Experiments on autonomous research benchmarks show that EEM improves research performance while reducing model interaction overhead, demonstrating the value of reusing accumulated experience and acquiring additional experience only when needed.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Autonomous Research, Experience Reuse

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_LC22OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper describes Experimental Experience Modeling (EEM), a framework for autonomous research to make efficient use of previous experiments to decide next course of action. Other than a high-level overview, there are no details about how the different components are implemented, so there is no chance of understanding what was actually done or to consider replicating the results. Evaluation on the recent ARC-Bench shows gains on overall quality and token efficiency compared to the AutoResearchClaw framework.

Critical strengths

None

Critical weaknesses

The various component of the setup do not have any details about how they're implemented, like Record_theta, Distill_theta, Assess_theta, U (for updating the experience library). There are no examples of the lessons l_i (or even what format they typically take) or any qualitative analysis for why this approach might be effective. In particular, there would be no way of trying to replicate these findings given the lack of information.

Questions for authors

The presentation would need way more details to allow proper evaluation.

Reviewer_vS2XOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Experimental Experience Modeling (EEM), a framework for guiding experimental investment decisions in autonomous research. EEM distills execution records into reusable lessons specifying their applicability conditions, empirical findings, and implications for subsequent decisions. For each candidate direction, it retrieves relevant experience and assesses whether the available evidence is sufficient to decide whether to pursue, revise, or reject the direction. When the evidence is insufficient, it conducts a targeted pilot experiment before making the decision. Experience from both pilot and full-scale experiments is accumulated within the same research run. Evaluation on 25 ARC-Bench topics reports improved research scores and reduced LLM token consumption relative to the evaluated baselines.

Critical strengths

The paper addresses a practically important problem: using previous experimental evidence to inform subsequent resource allocation in autonomous research. The record–lesson representation is interpretable, and explicitly retaining applicability conditions and supporting records provides a useful basis for checking whether experience is appropriately reused. The overall workflow is clearly organized, with distinct roles for retrieval, sufficiency assessment, pilot acquisition, and accumulation. The empirical results are encouraging: EEM improves the overall ARC-Bench score from 0.596 to 0.674 relative to AutoResearchClaw while reducing average token consumption by 11.4% (Tables 1–2). The inclusion of relevant research-agent baselines and three component ablations provides useful initial evidence for the framework.

Critical weaknesses
  1. The current ablations do not fully isolate the contribution of selective experience acquisition. The no-pilot ablation in Table 3 demonstrates the usefulness of pilot-derived evidence, but does not establish the benefit of assessing when a pilot is needed. In addition, the distillation ablation removes both the lessons and their associated retrieval metadata, so its performance difference reflects the combined intervention rather than the lesson representation alone. These distinctions are important for identifying EEM’s increment over closely related approaches involving experimental history and pilot-based selection. Routing statistics and a controlled comparison with routine pilot acquisition would strengthen the evidence for the sufficiency gate, while a clearer description of the retrieval changes would improve the interpretation of the distillation ablation.
  2. The efficiency evidence covers model-interaction overhead but leaves experimental execution savings unquantified. Table 2 supports the reported reduction in LLM token consumption. However, the broader motivation concerns allocating resources to costly experiments, and the paper does not report the cumulative execution time or number of pilot and full-scale experiments. The 120-second pilot limit alone does not establish their cost relative to full evaluations or the net execution savings. Reporting available execution-cost statistics, or more explicitly limiting the demonstrated efficiency benefit to model-interaction overhead, would clarify the connection between the motivation and the empirical results.
Questions for authors
  1. How is experience sufficiency operationalized in Equation (3)? Please specify the assessment prompt or decision criteria. When a pilot produces inconclusive or contradictory evidence, is sufficiency reassessed before the final investment decision?
  2. What fraction of candidate directions receive a pilot, and how does this change as experience accumulates? Could a comparison with an always-pilot variant under comparable total resource budgets clarify the contribution of the selective gate?
  3. In the distillation ablation, why are retrieval metadata removed together with the distilled lessons? Can the contribution of the lesson representation be assessed while holding retrieved records and the context allowance fixed?
Reviewer_qzmcOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Experimental Experience Modeling (EEM), a framework that distills experimental trajectories into source-record/lesson pairs, retrieves applicable experience for each candidate direction, assesses whether the evidence is sufficient, and runs a bounded pilot when a decision-relevant gap remains.

Critical strengths
  • EEM gives experimental experience an interpretable representation containing applicability conditions, empirical findings, decision implications, and supporting records.
  • The explicit sufficiency check followed by a targeted pilot is a sensible mechanism for trading off evidence acquisition against expensive full-scale experimentation.
  • The ablations isolate experience distillation, on-demand acquisition, and historical reuse, while the evaluation reports both research quality and interaction overhead.
Critical weaknesses
  • I don't really understand comparing to using methods like RAG for an experience library, what's the unique novelty here? Btw, this paper is also not compare with the traditional retrieval method.
  • Retrieval, sufficiency assessment, pilot design, and final decision making are described mainly as LLM-prompted operations, with limited algorithmic detail or calibration analysis.
  • The main evaluation covers only 25 tasks, relies on an LLM-based evaluator, and reports repeated-run analysis for only one task, leaving aggregate uncertainty unclear.
  • The library is initialized from scratch for each task, so the experiments do not test cross-task transfer or negative transfer under changed conditions.
  • One obvious limitation of using an experience library for this is the generalization issue on out-of-domain tasks and hacking on in-domain tasks, but the paper doesn't mention how the authors solved this.
Questions for authors

see above

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.