acceptodds
Under review as a conference paper at ICLR 2027

Bridging Scientific Goals and AI Execution: A Multi-Agent Framework with Hierarchical Self-Evolution

Abstract

Scientific discovery underpins progress in chemistry, biomedicine, and materials research. AI increasingly supports this work through scientific modeling, candidate design, and experiment selection. However, applying AI to a scientific task still requires domain scientists and AI specialists to jointly design the investigation, choose suitable methods, and revise them as results emerge. This repeated coordination is demanding, and experience from one investigation is not readily available to the next. We propose a multi-agent framework that addresses these difficulties through hierarchical self-evolution. The framework first generates a research workflow from the scientist's task description, then formulates an AI problem and execution plan for each scientific subproblem. Scientific and AI-method reviews guide intra-task evolution of the workflow, methods, and execution strategies. After the investigation, the framework extracts workflow playbooks, policy priors, executable skills, and scientific intuitions for reuse in subsequent tasks through inter-task evolution. Experiments across chemistry, biomedicine, and materials science show that our hierarchical self-evolving multi-agent framework supports scientific discovery by translating scientific goals into executable AI plans and improving task outcomes through intra-task and inter-task evolution. In comparisons with expert-team baselines on the evaluated tasks, the framework also achieves outcomes comparable to or better than those of collaborating domain scientists and AI experts.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI-assisted scientific discovery, multi-agent systems, hierarchical self-evolution, experience reuse, human–AI collaboration

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_peMYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies a hierarchical self‑evolving multi‑agent system for AI‑driven science. It breaks high‑level scientific goals into DAG‑structured workflows via specialized agents. Two‑tier self‑evolution supports in‑task revision and cross‑task experience reuse as library assets. Tests over chemistry, biomedicine and materials show its performance rivals human expert teams. Limitations include short‑workflow bias and LLM hallucination risks.

Critical strengths

Existing scientific LLM‑agents only support single‑run execution without experience accumulation. This work integrates in‑flight task revision and cross‑task reusable human‑curated assets. Ablations verify both components are essential, delivering meaningful novelty to cut repeated human expert efforts, favouring acceptance.

Evaluations span chemistry, biomedicine and materials science, featuring expert correctness auditing, ablations, cross‑model transfer and controlled comparison against human teams. Unlike many papers relying on weak metrics, human validation confirms real scientific utility and strengthens acceptance arguments.

Critical weaknesses

Weakness 1

The evaluation only uses old, pre‑collected datasets instead of forward‑looking real‑world discovery tests. All feedback comes from published known measurements. There is no closed‑loop test where the system suggests brand‑new, untested candidates with unknown true results. Retrospective benchmarks work for early checks but cannot prove generalization to open‑ended discovery. The agent may fit statistical patterns in existing data rather than truly discovering new science. This brings uncertainty for real‑world use and hurts its case for ICLR acceptance.

Weakness 2

Testing is mostly limited to short research workflows. Few experiments target work with more than four scientific steps or frequent AI‑plan updates. Human final approval is required in this framework, yet the paper gives no numbers on actual human workload: review rounds, expert time cost or scaling bottlenecks with growing asset libraries. Real‑world science often uses long, branching multi‑stage pipelines. Unresolved scalability concerns lower confidence for complex real‑world research campaigns and harm acceptance prospects.

Questions for authors
  1. Retrospective dataset evaluation a. What concrete technical safeguards stop agents from seeing hidden ground‑truth values (yields, energies) before they “request a measurement” in simulation? Please explain how leakage is prevented. b. If we add noise or out‑of‑distribution data to simulated results, how often will intra‑task evolution make wrong revision choices? Can you provide quantitative numbers?

  2. Human‑in‑the‑loop oversight a. Across your benchmark tasks, what is the average number of human‑review actions (workflow / asset approvals) per single task in your existing experiments? b. As the asset library grows bigger, what filtering stops too many candidate assets going to human reviewers? Do you have measured data showing how review workload scales?

  3. Intra‑task evolution ablation interpretation a. How much performance gain of full intra‑task evolution comes from the Review‑Chair LLM versus the two reviewer agents? Did you test an oracle ground‑truth Review‑Chair to spot LLM judgement errors?

  4. Asset transfer and asset quality a. Before humans review generated assets: are there automatic rules to throw out low‑quality assets, or is human check the only filter? Please state the existing automated rejection criteria. b. In RQ5 cross‑model asset transfer: are assets fully model‑independent, or do they carry quirks from the original strong LLM? What asset parts stay intact across different LLMs?

  5. Workflow DAG complexity limits What real‑world failures happen when building or updating multi‑stage, complex‑dependency DAG workflows? Does your execution harness have unreported hard‑coded caps for DAG depth or branching?

Reviewer_PGvWOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a multi-agent framework that translates scientific goals into executable AI investigations. Different agents handle scientific subgoal decomposition, AI-method selection, execution, and review. The framework supports two forms of evolution: intra-task evolution revises workflows and AI methods based on feedback, while inter-task evolution organizes previous experience into workflows, policies, skills, and scientific intuitions for reuse. Using published measurements as feedback, the paper evaluates the framework across chemistry, biomedicine, and materials science with six LLMs, including experiments on evolution, asset reuse, and comparisons with expert teams.

Critical strengths

This work propose a practically relevant problem and an executable framework. Translating scientific goals into concrete AI problems and execution plans is a meaningful practical challenge. The framework goes beyond generating research suggestions: it uses registered tools for model training, candidate selection, and result validation. Cross-domain experiments, component ablations, and detailed execution cases demonstrate the feasibility of connecting scientific planning to actual computation.

Critical weaknesses
  1. Limited methodological contribution. The paper combines multi-agent orchestration, feedback-driven revision, and experience reuse for scientific research, with adaptations such as evidence dependencies, separate scientific and AI-method reviews, and asset applicability checks. However, the core mechanisms remain primarily LLM-driven plan revision and the extraction and reuse of past experience. Relative to existing scientific agents, the paper does not sufficiently establish a substantive technical advance from its role decomposition and asset organization, or explain why the particular challenges of scientific research require this hierarchical structure. The ablations support the usefulness of different capabilities on some tasks, but do not adequately distinguish the benefits of those capabilities from the benefits of the specific multi-agent and hierarchical organization. The methodological contribution therefore remains limited.

  2. Poor overall presentation quality. Figures 1 and 2 are embedded as raster images, with small internal text and insufficiently clear layouts that do not effectively communicate the core method. The tables are also poorly organized and formatted: some task identifiers break awkwardly across lines, Table 2 uses inconsistent decimal precision within the same metric, and overly abbreviated model names add unnecessary reading effort. These issues extend across both the method and experimental presentation. They are not merely aesthetic concerns, as they hinder understanding of the method and comparison of the results. The overall presentation quality fall short of what I expect from a ICLR submission.

Questions for authors
  1. What is the main technical advance over existing scientific agents? The paper explains the distinction between assessing scientific evidence and assessing AI methods. Compared with STELLA, which also incorporates planning, critical feedback, and experience accumulation, what previously unresolved technical challenge does this work address, and which specific mechanism addresses it?

  2. Do the gains depend on the multi-agent review structure? Section 6.4 demonstrates the effects of the two types of revision. Could a single controller with the same information, tools, and permission to perform both types of revision achieve similar gains? What evidence supports the additional value of separate reviewers and a Review Chair?

  3. How many runs support the main results, and how variable are they?

Reviewer_sDCpOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a framework to address the gap between scientific goals and the execution of AI agents in such domain-specific problems. A ``hierarchical self-evolution'' method is introduced to process the input materials given by human scientist, and the system will design a workflow with solution, acquire observations and test these plans. In my understanding, the term evolution here means iterative revising of research workflow or AI method, so that the system may behave like improving. Authors conduct experiments on three scenarios to validate their methods.

Critical strengths

Well motivated, and the targeted problem in this paper is impactful, worth to discuss in today's frontier of agentic AI for auto-research.

Critical weaknesses
  • Experiment results in this paper are insufficient to demonstrate that the proposed method is more effective and efficient than existing SOTA methods.
  • The presentation in experiment sections is hard to follow, and the analysis regarding results lacks depth.
Questions for authors
  • Can the authors provide clear and precise definitions of terms like workflow, self-evolution and each evaluation tasks mentioned in this paper?
  • Is the proposed method empirically superior to existing SOTA approaches (like AI Scientists and AI Researcher) for autonomous scientific research? Can its effectiveness be demonstrated across 3 to 5 domain specific tasks?
  • How does this system yields improvement, any inner status analysis? If you say self-improve, how does self-improve affect its final outcome?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.