acceptodds
Under review as a conference paper at ICLR 2027

A Kinetic Theory of the Gated Self-Evolving LLM Agent

Abstract

We find traces of fluid dynamics in the self-evolution of an LLM agent, and give the kinetic theory that predicts them. Gated self-evolution is the loop in which an agent rewrites its own skills under a validation gate. Self-evolution research has treated the agent as the unit; we study instead the individual instances inside it. Here the agent is DSH-plugin-based: it runs in production on DeepSeek Harness (DSH), and its plugins satisfy four architectural properties (permutation symmetry, reversibility, acyclicity, typed contracts), which license treating these instances as identical hard spheres; the theory is accordingly scoped to DSH-class plugin populations. On this scope the paper builds three theory layers. The rigorous layer, independent of any analogy, comprises an any-time hitting-time certificate bounding the expected rounds to any prescribed improvement, a resolution law that prices held-out validation budgets, and a separation theorem: the daemon must stay outside the population, because merging evaluator with evaluated voids the certificate. The kinetic layer is a master equation over the plugin × version × task grid with four operators (collision, reaction, external field, gate), where collision is co-activation. Its moment hierarchy, the step that turns a gas into fluid equations, generates the falsifiable statistical signatures. Throughout, the fluid reading is a bounded analogy: momentum is not conserved, so no Navier–Stokes limit exists. The measured layer runs on a faithful minimal instance, a large library of four-parameter skill plugins retrieved one per episode with a co-activation probe, in a one-model, one-task-family WebShop environment; every element maps to the DSH loop by architectural role. Population fluctuation scaling is density-gated: invisible at sparse edit density, it emerges at the predicted rate under tripled density, as directional evidence.

Primary area:
reinforcement learning
Keywords:
A kinetic theory of gated self-evolving plugin populations: certificates, fluctuation laws, and a bounded fluid analogy, verified on a faithful device.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8MkWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This research studies the self-evolution of LLM agents from a population-level perspective, shifting the focus from the entire agent to the collective dynamics of plugin-version instances. Inspired by kinetic theory, the authors draw on four architectural properties of DeepSeek Harness (permutation symmetry, reversibility, acyclicity, and typed contracts) to motivate modeling plugin-version instances as identical hard spheres. Based on this analogy, they formulate a kinetic equation with four operators: collision, reaction, external field, and a validation gate that selectively accepts updates. The authors claim three theoretical results: a hitting-time certificate for performance improvement, a resolution law relating validation budgets to detectable improvements, and a separation theorem concerning the independence of the evaluator from the evolving population. Experiments on a simplified WebShop-based system examine the predicted fluctuation scaling and non-equilibrium steady-state behavior.

Critical strengths
  1. This paper takes an unconventional yet interesting perspective on self-evolving LLM agents, drawing on kinetic theory to study how a population of plugin instances evolves through repeated updates and validation. I appreciate the authors’ effort to develop a theoretical account of this process, as I believe a solid theoretical foundation could help the community better understand the mechanisms underlying agent self-evolution.

  2. This paper acknowledges the limitations of its physical analogy and reports experimental findings even when some of them do not support the proposed predictions. The authors also provide experimental records, document corrections to earlier experiments, and describe their efforts to formally verify parts of the theoretical analysis.

Critical weaknesses
  1. This manuscript suffers from considerable writing and presentation issues. It is often difficult to follow due to awkward phrasing, abrupt transitions, and insufficiently explained connections between key arguments. The authors should carefully revise the manuscript to improve its clarity and readability.

  2. The applicability of the proposed kinetic model to plugin population dynamics remains insufficiently justified. The authors do discuss the similarities between plugin properties and hard spheres in Appendix D and acknowledge the dependencies introduced by shared experience in Appendix F.4. However, these structural similarities alone do not fully explain why the proposed kinetic model would apply to the evolution of plugin populations. In particular, it remains unclear whether the assumptions behind the mean-field approximation hold when plugin instances interact through a shared experience database. I regard this as a fundamental concern, since the paper's central kinetic-theory claims depend on the validity of this modeling assumption.

  3. The experiments do not sufficiently validate the interaction mechanisms proposed by the kinetic model. Although the authors acknowledge the limitations of their minimal WebShop-based system, representing plugins with only four tunable parameters leaves a considerable gap between the experimental setting and realistic self-evolving agents. More importantly, the main fluctuation-scaling and NESS experiments use only one plugin per episode, while the proposed theory defines collision through the co-activation of plugin instances. The authors include a separate fan-3 experiment, but it remains unclear whether the observed improvement comes from plugin interactions or simply from having more candidates to choose from. I would like to see stronger evidence connecting the predicted statistical patterns to the interaction mechanisms described by the kinetic model.

  4. The reported statistical patterns do not provide sufficiently convincing evidence for the proposed kinetic theory. The fluctuation scaling observed under higher edit density β^=−0.58\hat{β}=-0.58 is based on only three population sizes with two seeds each. More Importantly, the predicted N^(-1/2) scaling is also a common statistical property under suitable averaging assumptions, making it unclear what this result specifically tells us about the proposed kinetic model. Similarly, the observed performance plateau is interpreted as relaxation toward a non-equilibrium steady state, but it is confused whether this alone establishes stationarity of the plugin population. I would appreciate stronger evidence that these observations are specifically explained by the proposed kinetic theory, perhaps through comparisons with simpler statistical models.

Questions for authors

My main concerns are discussed in the Weaknesses section. I have a few additional questions below:

  1. The hitting-time analysis appears to assume that improving edits remain available until the target is reached. However, an agent may reach a capability ceiling under the bounded-edit setting, where no further improving edit is available. Could the authors clarify the practical significance of this guarantee in such cases?

  2. Regarding the collision experiment (Figure 8): The performance gap between fan-3 and fan-1 appears to be almost identical before and after evolution, with both settings achieving similar improvements during training. How do the authors distinguish the benefit of plugin co-activation during execution from its actual contribution to the self-evolution process?

  3. Regarding the density-dependent fluctuation scaling (Section 5.2): The sparse and dense settings differ not only in edit budget, but also in training duration and the population sizes used for fitting. How can the authors determine whether the observed scaling difference is driven by edit density rather than these other factors?

Reviewer_udRBOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how an LLM agent improves itself by repeatedly editing its skills and keeping only the edits that perform better on a validation set. The authors view the different versions of these skills as a population and use ideas from gas and fluid dynamics to describe how they interact, change, and get accepted or rejected.

They also give a simple bound on how many rounds improvement may take and argue that the evaluator should remain separate from the agent being improved. The experiments use a small WebShop setup with simple skill plugins. The authors vary the population size, edit frequency, retrieval strategy, and validation procedure, and report several results that appear consistent with their theory.

However, the experiments use very few random seeds, and the evidence is not yet strong enough to show that the system truly follows a kinetic or fluid-dynamical model.

Critical strengths

The paper has an interesting idea: instead of looking only at the whole self-improving agent, it studies how individual skills interact and change over time. The authors are also clear about which parts are real results and which parts are only analogies to physics.

The discussion of validation is practically useful. It explains why repeatedly testing an evolving agent on the same validation set can cause overfitting, and why the evaluator should be kept separate from the system being improved.

Critical weaknesses

The main theory is not defined clearly enough, and the connection to physics is too weak. The four DSH properties do not show that skill instances behave like hard spheres. For example, typed interfaces do not imply local interactions, reversible mounting is not the same as physical reversibility, and an acyclic dependency graph does not prevent global or higher-order interactions. The paper also does not clearly define the collision, reaction, and gate operators in Eq. (1). Since the system uses a shared database, top-k retrieval, an LLM, and a validation gate, its behavior depends on global history. The paper therefore does not show that the assumptions behind its mean-field claim actually hold. The hitting-time result is mostly a standard bound that follows from the strong assumption H1', and the claimed validation-budget law is not stated clearly enough to evaluate.

The experiments are too small and do not cleanly test the claims. The main scaling experiment uses only two random seeds for each population size and only three population sizes. The initial validation scores also differ substantially, so the reported variation may come from different starting libraries rather than population effects. The paper sets N_eff equal to the population size even though its appendix shows that different measures of effective population size disagree at large N. Reusing the same 20 validation episodes during training also creates a serious risk of overfitting. The fan-3 versus fan-1 experiment may simply measure the benefit of trying more candidates, rather than a true “collision” effect. Finally, the long-run curves do not provide enough evidence for a genuine steady state. Overall, the experiments do not yet support the paper’s kinetic or fluid-dynamics claims.

Questions for authors
  1. What is the exact finite-N stochastic process underlying Eq. (1)? Please provide explicit definitions of Q, R, and G, clarify whether the shared experience database is part of the Markov state, and explain how the top-k retrieval, LLM reflector, gate, and population-changing reactions satisfy the assumptions required for Theorem 4.2. Also, where is the precise statement and proof of the claimed validation-resolution law?

  2. Is the same 20-episode validation set reused across all adaptive rounds and candidate proposals? Are candidate and incumbent evaluations paired using the same task IDs and random seeds? How are adaptive selection bias, evaluation noise, and multiple testing handled in the reported gate-versus-independent comparison? Would the N^{-1/2} result persist with substantially more seeds, baseline-normalized scores, and compute-matched fan-1/fan-3 controls?

Reviewer_Mbx8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies gated self-evolution from the perspective of skill-plugin instances within a single LLM agent. It proposes a kinetic description with co-activation, mutation, task-dependent transport, and validation-based selection. Validation uses WebShop environment with four-parameter plugins, testing population-size scaling, co-activation effects, saturation, and validation–independent performance gaps.

Critical strengths
  1. The paper offers a useful population-level perspective on gated skill evolution. Modeling retrieval, edits, and validation jointly shifts attention from aggregate task scores to the mechanisms governing a skill registry’s evolution. The separation between validation-set improvement and independent-set improvement is particularly consequential for diagnosing apparent progress that does not generalize.
  2. The paper explicitly documents the limits and negative results of its physical interpretation. The conservation ledger, unsupported signatures, and corrected experimental records help distinguish the proposed kinetic description from stronger claims about classical fluid mechanisms. This transparency makes the empirical evidence easier to assess.
Critical weaknesses
  1. The connection between the plugin architecture and the kinetic model remains incomplete. A1–A4 do not establish exchangeability or justify the mean-field approximation, especially with dependence through the shared experience database.
  2. The experiments use a substantially simplified system. Four-parameter plugins in a single synthetic WebShop setting provide limited evidence that the proposed dynamics describe richer skill libraries or the production runtime.
  3. Please consider testing the predicted signatures on richer skill representations and additional task families, together with simpler gated-search controls. This would help determine which observations specifically support the kinetic description and which also arise from generic selection and averaging.
Questions for authors
  1. Does the density-dependent scaling persist when both settings use the same population-size grid and training duration? The current comparison changes these alongside the edit budget.
  2. How does Proposition 4.3 classify runs where edits continue to be accepted but their gains approach zero?
  3. What justifies the claim that δ_ind ≈ 0 implies I_G → 0? A vanishing mean gain does not by itself imply independence between gate decisions and gains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.