acceptodds
Under review as a conference paper at ICLR 2027

An Agentic AI Scientific Community for Automated Neural Operator Discovery

Abstract

We present an agentic approach to autonomous neural operator discovery based on an AI scientific community, which consists of a swarm of virtual laboratories that interact under a citation-based economy of influence. Highly-cited labs found new labs that follow their research direction and replace non-performing labs. Each virtual lab contains three agents: an LLM planner that writes a complete PyTorch program for a neural operator, a numerical worker that checks, repairs and trains it, and an LLM reviewer that participates in cross-lab peer review. The labs either compose neural operator building blocks (DeepONet/branch-trunk, Fourier, Transformer/attention, wavelet, and residual convolutional) in code, or write every layer from primitives. Simulation campaigns on the Darcy flow, Navier-Stokes and Allen-Cahn 2D problems show that the AI scientific community discovers operators that are more accurate than strong baselines on some problems, while no operator is the best on every problem. We also conduct comprehensive ablation studies to isolate the effects of using a community of labs versus a single lab, of allowing coding versus using a fixed vocabulary of blocks, and of using LLM agents versus rule-based alternatives, the latter on piecewise regression, the linear advection and Burgers 1D PDEs, and the Navier-Stokes and Darcy flow 2D PDEs. The code and full results are available at https://anonymous.4open.science/r/neural-operator-community.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural operators, scientific machine learning, LLM agents, multi-agent systems, automated architecture discovery, code generation, PDE surrogates, AI for science

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_d5E4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a multi-agent system for autonomous neural operator discovery. Concretely, it instantiates 16 "labs", each consisting of a LLM planner for code generation, a numerical worker for code verification and training, and a reviewer for peer review. The communication between labs is based on citation, and for each iteration there exists an eviction rule based on whether the lab's work is "cited" recently. The experiments cover three problems: Darcy, Navier-Stokes and Allen-Cahn; and through ablations on whether community is needed, whether eviction is needed and whether coding is needed, it shows improvements on these components. However, the coding models are restricted to Qwen2.5-Coder 7B, Qwen3-Coder-30B, and gemma3:12b, which are far behind current model capabilities, hindering the effectiveness of their results. Overall, I think it is a reasonable and experimentally-justified framework for neural operator discovery, although having limited novelty and hindered by the authors' selection of models.

Critical strengths
  1. The contribution of this paper is a framework. And the authors did a great job on ablating each component in the system, and offering insights into why each component helps with the overall performance. In particular, I find the analysis connecting why community is better with recovery from invalid programs insightful.
Critical weaknesses
  1. Using open-source models with less than 30B parameters, especially not the latest generation of models hinders the applicability of the findings. Especially considering that from the author's analysis, a major failure case for single-thread is failure to execute the written program, which with more advanced models the probability of this happening is much smaller.

  2. The gain from citation-based community versus a simple failure-recovery baseline is incremental. These experiment results makes the claim of citation-based systems not well-justified, and I would like to see further proof on why this citation-based rule is necessary.

Questions for authors
  1. The coding versus no-coding comparison does not seem fair. Can the authors provide additional experiment results with matched planner model, objectives, budgets and review policy? I believe a controlled experiment that only varies the architectural representation (i.e. fixed genomes versus code programs) can clear up this confusion by a lot.

  2. Could the authors provide further explanation on why the gain from citation-based community over a simpler failure-recovery strategy seems incremental?

  3. How does the findings hold for more up-to-date models?

2 comments
AuthorsOpenReview ↗

Response to Reviewer d5E4

We thank the reviewer for the review and for the precise experimental request in Q1, which we ran. Below, we address each point.

1. Controlled comparison of code and genomes (Q1)

Reviewer Comment: "Can the authors provide additional experiment results with matched planner model, objectives, budgets and review policy? [...] only varies the architectural representation (i.e. fixed genomes versus code programs)"

What we did. A new genome arm, exactly as requested: same planner (Qwen2.5-Coder 7B), reviewer, review policy, harness, fitness and budget (16 labs x 16 iterations) as the blocks arm. The only change is that the planner chooses a genome over the block vocabulary and a fixed compiler writes the program; invalid genomes go through the same repair loop.

Results. Held-out relative L2L_2 error, mean over search seeds (Tables 2 and 13):

SettingCodeGenomeCode wins
Darcy, 5 seeds0.05250.13925 of 5
Darcy, 3 seeds, accuracy-first reviewer0.03700.16423 of 3
Navier-Stokes, 3 seeds0.02870.03963 of 3
Allen-Cahn, 3 seeds0.05330.06062 of 3

Code wins 8 of 9 pairs on the three problems (geometric mean ratio 0.53, p=0.008p = 0.008). The genome community is even worse than random search over the same vocabulary (0.0640 on Darcy): the planner keeps proposing conservative, repeated genomes, whereas in code it rewires the same blocks into graphs with gates and parallel paths. This controlled comparison now carries the claim about coding (Sec. 5.2, "Code against genomes"); the earlier comparison with the campaign without coding is kept only as the ablation of LLM agency.

2. Is the citation rule necessary? (W2, Q2)

Reviewer Comment: "I would like to see further proof on why this citation-based rule is necessary."

We absolutely agree that the reported gains are small. Replacing the LLM's citations by the fitness ranking, by random citations, or by the fitness ranking without any reviewer gives marginal gains. We expect that further work, using more sophisticated LLMs, will make these gains larger. Our original intent in using open-weight models was to make the approach accessible and reproducible by the broader community using consumer GPUs. Over ten search seeds, with the decision rule fixed before the runs: citations 0.0538, fitness 0.0435, random 0.0478; citations win 3 and 5 of 10 pairs (p=0.50p = 0.50 and 0.640.64); no reviewer 0.0525 on five seeds (Table 12). The gain over a single lab comes from refounding failed labs from working programs in a population of 16: reset recovers 80% of it on its own, and the remaining 20% is the breadth of the population (every community variant is below reset in mean error). We have removed the claim that the citation rule is necessary, and we state the size of the remaining gain.

3. More recent models (W1, Q3)

Reviewer Comment: "How does the findings hold for more up-to-date models?"

What we did. As said above, our intent in this first communication was to make the approach accessible and reproducible on consumer GPUs. We fully agree with the criticism that these models may not be sophisticated enough. To investigate the effect of a larger model, we extended the 30B comparison to the three problems and six search seeds (18 pairs), with Qwen3-Coder-30B as planner (Table 13):

Problem (6 seeds)7B planner30B planner30B wins
Darcy0.03010.04072 of 6
Navier-Stokes0.02950.03852 of 6
Allen-Cahn0.05240.05303 of 6

As anticipated, the 30B planner produces fewer programs that fail to run, but its winning operators are not more accurate on held-out seeds: at this budget the bottleneck is not program validity. We have no API access to frontier models and state this as a limitation. The framework is planner-agnostic: the frozen harness and the held-out retraining, which make the recorded numbers trustworthy (the search's own error understates the held-out error by 1.1x in the median and up to 2.0x), do not depend on the model.

We hope that our responses have adequately addressed your concerns. We respectfully ask you to consider whether they warrant a revision of your score. We welcome any follow-up questions that you might have.

Thank you for your efforts in addressing my concerns. My concern in the fairness in evaluation is largely resolved, however I still find the citation rule's necessity as well as the lack of results demonstrated on frontier models (which can be called through openrouter APIs therefore absolutely runnable on consumer hardware) critical. I will maintain my score.

Reviewer_X4wnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The manuscript proposes a multi-agent AI framework that designs neural network architectures for learning partial differential equation solution operators. Rather than asking one LLM to design a neural network, the authors create a scientific community consisting of research laboratories. The central mechanism is a citation-based evolutionary search process where successful laboratories influence future designs. The authors evaluate this framework on multiple equation families and compare the discovered architectures against existing methods.

Critical strengths
  1. The manuscript is very systematic in its approach, in particular, of the contributions of using agent collaboration and LLM-driven decision making. The authors conduct several ablation studies to examine how different components of the framework affect performance. These experiments help explain when having multiple agents is useful and when simpler approaches can achieve similar results.
  2. The approach shows promising results in automatically designing neural operators. The AI agents can write code and explore different architectures autonomously. The method achieves better results than the tested baselines on a subset of the problems. This suggests that AI agents could help reduce the manual effort involved in designing neural operators.
Critical weaknesses
  1. The paper does not clearly demonstrate whether the LLM agents are actually using domain-specific knowledge to design better neural operators. For example, it is unclear whether these choices are guided by an understanding of the underlying physics. An ablation experiment that explicitly examines the role of domain knowledge would help strengthen the paper's claims about autonomous scientific discovery.
  2. The citation mechanism is an interesting idea but the authors do not establish that this is actually necessary. Could a simpler rule-based evolutionary mechanism work equally well or even better? Since the citation mechanism is a defining contribution, I believe it is important to compare it to other selection strategies.
Questions for authors

Following the critical weaknesses above:

  1. Is there any evidence that the LLMs are using domain knowledge of the underlying PDEs? For example, have the authors considered comparing agents with and without PDE-specific information in their prompts?
  2. How would the proposed citation-based mechanism compare against simpler selection strategies, such as fitness-based selection or random citations?
  3. The ablation studies show that allowing a single agent to reset to its last working program substantially closes the performance gap with the multi-agent community. Could the authors clarify what additional benefits the multi-agent framework provides beyond avoiding unsuccessful search trajectories?
Reviewer_Gms9OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper focuses on leveraging AI agents to perform autonomous neural operator discovery for a series of scientific problems, such as Navier-Stokes. Specifically, in this paper, an agentic pipeline, which composes three agents: LLM planner, numerical worker, and LLM reviewer, is proposed to perform such a task. Empirically, this pipeline demonstrates its capability in discovering neural operators compared to those strong baselines.

Generally, I think the problem studied in this paper is important, and building agentic systems for such scientific tasks is also a popular topic in the community. However, the quality of the paper is quite poor due to the messed-up management of notation, limited novelty, and so on.

Critical strengths
  1. The research problem studied in this paper is important.

  2. Building agentic systems to solve these scientific tasks is quite popular in the current community.

Critical weaknesses
  1. The research problem is not well summarized in the paper. Specifically, in this paper, the core problem is to explore whether AI-based scientific community can help address problems in practical science, such as engineering math. However, the statement in the paper is so general that many corresponding questions are raised. How do you define the capability in a black-box pipeline? What is the importance of the research problem of your work? Thus, it would be better to carefully formulate your insights in this domain in the paper.

  2. The main contribution in this paper is building an agentic framework. This is also claimed in the intro section. However, according to the paper, the agentic system is merely composed of three LLMs without any further important technical points. In addition to this, I do not think the other two contributions mentioned in the paper are non-trivial. Thus, the novelty of this paper is limited.

  3. The notations in this paper are not well managed. For example, in Section 3, there are many notations that are not well defined, such as A\mathcal{A} and U\mathcal{U}. This makes me feel confused. As a paper that focuses on scientific domains, a strict rule for notation use is required. This significantly affects my evaluation of this paper.

  4. In the evaluation of coding, the effect of coding genreation is not isolated. The coding and non-coding systems differ significantly in many aspects, including LLMs, review policy, etc. Moreover, coding also allows parallel paths and arbitrary graphs.

Questions for authors
  1. I am curious about the practical value of the proposed framework. With the powerful models in the current community, such as GPT and Opus, a single LLM can handle planning, coding, checking simultaneously by itself. However, for those open-source weak models, such as Llama, Qwen with fewer parameters (e.g., 32B or smaller), even invoking multiple LLMs may also fail in these tasks. So, what is the core value of your framework?

  2. In Eq. (1), the left part is Lθ\mathcal{L}_{\theta}. However, the right hand aims to find an optimal aa from an undefined μ\mu. I cannot understand this function. Similar problems are also in Eq. (3). What is θ\theta? What is L^train\hat{\mathcal{L}}_{\rm train}? Why θ\theta can be the parameter and the function of pp simultaneously? What is FF?

  3. Since this paper mentions that the agentic system is driven by citation. What is the definition or concrete description of the citation? The paper does not mention this.

  4. Could you please add more details regarding the training in our proposed agentic system? How do you perform harness optimization in detail in practice?

  5. What is the insight into Eq. (4)? It is really hard to understand the content.

2 comments

Sorry for the late reply, and thanks for the authors' responses.

I have read the responses from the authors and checked a part of the aspects I am concerned about in the paper. Generally, my concerns remain in several aspects. For example, for the problem formulation, I do not know why the authors connect the neural operator discovery with AI scientific community, which refers to citation. This makes the paper weird and over-decorated. Moreover, the training here is really confusing, especially a training on just 256 data samples.

Just a reminder, although the paper is modified (according to authors), Eq. (3) still has some problems.

Thus, with all these aspects taken into consideration, I consider maintaining my score.

The reason why we connect the AI Science Community with neural operator discovery is that it is a closed-loop autonomous discovery agentic mechanism, which can propose changes to a baseline and modify them based on performance and communal criticism. In opposition to all agentic AI approaches for autonomous discovery proposed so far, which consist of a single virtual lab, the AI Science Community leverages the emergent collective intelligence arising from a swarm of virtual labs.

We would like to point out that we have revised our statement in the top global reply to all reviewers, as we had inadvertently stated that the citation mechanism is secondary. In fact, even though it did not show a decisive influence on accuracy in the reported experiments, it remains an essential logging mechanism in our approach to allow explainability of the discovery process.

We would be happy to clarify further any replies that the reviewer think are unclear. Please give us a chance to do it by pointing out the replies that are unsatisfactory and we will try to make it more clear.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.