acceptodds
Under review as a conference paper at ICLR 2027

Predict Before You Modify: Learning Improvement Dynamics for Self-Evolving Agents

Abstract

Self-evolving agents improve through repeated cycles of modification and evaluation. However, existing methods receive feedback on modifications only after costly evaluations, and aggregate scores may obscure distinctions between failures that require different repairs. We introduce predictive self-evolution through an editable Improvement Dynamics Model (IDM). Before evaluation, the IDM predicts task-level outcomes of a proposed modification and guides an optional revision. After evaluation, past modifications and their observed outcomes are used to improve the IDM, allowing predictive knowledge to accumulate across generations. The system can thus improve how it solves tasks, proposes modifications, and predicts their effects. Building on HyperAgents, we evaluate this approach across multilingual coding, paper review, robotics reward design, and Olympiad-level mathematical grading. Our approach outperforms HyperAgents across all four domains, with up to 46% higher average performance over generations. Gains also persist under matched token budgets that include both agent evolution and task evaluation, and extend to held-out tasks. Forecast accuracy improves as the IDM evolves, and the learned predictive knowledge transfers to improve evolution in a new domain.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Self-evolving agents, Large language model agents, Automated agent design, Predictive modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TJqSOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper builds on HyperAgents and studies the automatic improvement of agents whose behavior is determined by editable programs. Their proposed addition is an improvement dynamics model or IDM. This is an editable program that invokes a frozen LLM to predict what will happen if a proposed modification is applied. The predictions of this IDM distinguish task-level outcomes that can have the same reward but imply different repairs. Findings show improvements over HyperAgents on coding, paper reviewing, robotics, etc.

Critical strengths
  • Sensible Design: The proposed loop design is sensible. HyperAgents is expensive since it has to evaluate an entire modified agent. Historical edits could be useful to avoid predictable outcomes that aren’t useful.
  • Predictor Updates: I think the best methodological choice is constructing an IDM before its validation records are collected. This avoids observation bias and I appreciate this.
  • Relevant Controls: The paper includes several ablations and controls like reflective HyperAgents, IDM reflection, etc.
Critical weaknesses
  • Matched Compute: Although the paper’s main result is strong, I think the real result with compute matched tokens is much weaker unfortunately, leading me to vote for rejection. Looking at figures 7 and 8 as well as table 7, the AUBC values are much smaller when compared to a matched-token baseline and are within error bars IIUC.
    • One more concerning case is the comparison of IDM reflection vs. Reflective HyperAgents. Both methods always allow revision. But the comparison under matched token counts, they are almost always tied. So predictive feedback doesn’t entirely improve efficiency beyond ordinary reflection?
    • The method could still be genuinely useful. But claims probably need to be adjusted about what is found here or what the improvement actually is.
  • Prediction vs Informed Critique: One confusion I have is about the task-level probabilistic forecasting. I don’t think the ablations establish this as necessary though. IDM feedback can add several things like a separate diagnosis of failures, task-level probabilities, etc. A benefit of same session reconsideration could come from several of these mechanisms. In particular, a separate critic with the same setting may identify the same problems without predicting numerical outcomes right? I think two controls would be helpful. One would include an equally resourced qualitative critic that does not provide task-level probabilities and an aggregate-only forecast feedback rather than the outcome representation.
  • Forecasting vs generalization: I have a few concerns here.
    • For the fixed-case experiment, doesn’t this measure improvement on previously seen records? I’m referring to Figure 5(b). This evaluates later IDMs on modifications from generations 1-5 of selected runs but the IDM uses accumulated historical records.
    • Absolute outcome prediction can improve without accurately predicting the effect of the edits. The IDM already knows parent outcomes. Many modifications may leave task outcomes unchanged. I think this is shown in Figure 15 where an evolved IDM adding historical task frequencies to its input. I think a parent persistence of historical frequency baseline is important.
Questions for authors

See weaknesses

Reviewer_1SZEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how to predict the effects of agent modifications before actual testing. The authors incorporate an editable Improvement Dynamics Model (IDM) into HyperAgents, using historical modifications and their results to predict the performance of new modifications on various tasks and guide further revisions. The IDM itself is also updated as it evolves.

Critical strengths

The research question is significant. The system not only accumulates experience in solving tasks but also learns to predict which modifications might be effective. This can improve efficiency.

The experiments include ample controls. The authors conducted a thorough ablation study, comparing settings such as additional reflection, freezing the IDM, and unconditionally using IDM revisions, and incorporated prediction and update costs into the budget.

Critical weaknesses

The incremental gains from relatively strong comparisons still require more direct statistical support. The 46% highlighted in the abstract comes from AUBC comparisons under a fixed number of algebras; in the equal-token budget comparisons in Figure 8, Selective IDM is 0.3211, while other revised methods are approximately 0.3090–0.3119. With only five runs per setting, the uncertainty of inter-method differences needs to be reported to more clearly determine the stable value of the new mechanism.

Evidence for cross-domain generalization remains limited. The lack of task validation primarily comes from code and mathematical scoring. This domain is a narrow inference domain; furthermore, the five runs of the cross-domain migration reused the same evolved IDM, therefore it is not yet possible to determine whether the gains are sensitive to the choice of donor IDM.

Questions for authors

see weaknesses

Reviewer_zRcpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a “Predictive Self-Evolution” framework that builds upon HyperAgents by introducing an editable improved dynamics model (IDM) to enhance the efficiency and effectiveness of self-evolving agents. The motivation is that existing methods rely on costly post-modification evaluation; however, if the consequences of modifications can be predicted in advance, more informed corrections can be made prior to evaluation.

The framework comprises two mechanisms:

  1. Prediction-Guided Selective Revision: Prior to evaluation, the IDM predicts task-level outcomes based on the current agent, proposed code changes, and historical evaluation records; a bias-corrected aggregated prediction determines whether further revision is needed. When triggered, the meta-agent receives the task-level prediction and adjusts the proposal accordingly.
  2. Self-Evolution of the Prediction Model: IDM is an editable program built around a frozen LLM; the meta-agent periodically modifies it based on historical “prediction–result” deviations. Candidate updates must be validated against subsequent evaluation records and meet prediction quality standards to be adopted.

Experiments were conducted across four domains: multilingual coding, paper review, robot reward design, and math grading. Comparisons were made against HyperAgents and various ablation methods related to revisions, with each setup run independently 5 times, each run spanning 40 generations.

The results show a sustained improvement in evolutionary efficiency: In the Polyglot coding task, the median AUBC score rose from 0.210 to 0.306 over 40 generations, maintaining a lead under the same token budget and across 166 held-out tasks. Furthermore, IDM’s prediction quality improved with evolution, and IDMs evolved through paper review and robotics tasks were able to transfer beneficial predictive knowledge to the math scoring task.

Translated with DeepL.com (free version)

Critical strengths

S1. A meaningful integration of predictive modeling into agent self-evolution The paper's strongest contribution is the introduction of a predictive feedback loop that operates before expensive agent evaluation. Unlike approaches that use historical outcomes primarily to inform future proposal generation or rank candidates, the proposed IDM explicitly forecasts task-level consequences of a draft modification and uses those forecasts to guide a targeted revision. This distinction is consequential because task-level predictions can expose qualitatively different failure modes that would otherwise be obscured by aggregate scores. The separation between an evolving forecasting program and the agents being optimized is also a useful architectural choice.

S2. Broad empirical evaluation with budget-aware comparisons and transfer experiments The evaluation covers four substantially different domains, including software engineering, paper review, robotics, and mathematical grading. The authors appropriately recognize that predictive guidance introduces additional computational costs and therefore report matched-token comparisons that account for both evolution and task evaluation. The coding experiments also include controls for unconditional revision, frozen prediction models, and unconditional IDM-guided revision. The transfer experiment is particularly interesting: transferring only an IDM evolved on paper review and robotics improves mathematical-grading evolution relative to a freshly initialized IDM, while keeping the initial task and meta agents identical. Although the magnitude and reliability of these transfer improvements warrant further examination, this result provides preliminary evidence that the IDM can acquire predictive knowledge useful beyond its original optimization domain.

Critical weaknesses

W1. Insufficient evidence isolating the causal contribution of the learned IDM and its selective revision policy

The experiments do not fully separate the effects of forecast accuracy, revision selection, and extra meta-agent computation, and the gains over the frozen-IDM and unconditional-revision variants are modest (Figure 7: 40-generation AUBC of about 0.3058 vs. 0.2792 and 0.2888), with substantial uncertainty shown in Table 7. The paper also lacks a control that keeps the same revision allocation and compute budget but replaces IDM feedback with shuffled or otherwise uninformative forecasts, so the gains cannot be attributed specifically to learned predictive knowledge.

W2. Limited statistical robustness and insufficient evidence of predictive generalization

Conclusions rely on only five runs per setting, and several improvements are small relative to run-level variability. The 2.3% matched-token AUBC gain from Transferred IDM is obtained by reusing one transferred IDM across all five target runs, so it is unclear whether the learning procedure transfers or just that particular IDM. Figure 5(a)’s 19-of-28 Brier-loss wins come from five runs and are not necessarily independent, and Figure 5(b)’s fixed-case evaluation covers only historical modifications from selected runs.

Questions for authors

Q1. Can the authors more directly isolate the causal contribution of informative IDM predictions?

Could the authors compare Selective IDM against a control that uses the same revision decisions and approximately the same computational budget, but replaces the IDM's task-level feedback with shuffled or otherwise uninformative predictions?

Q2. How reliable are the improvements attributed specifically to IDM evolution?

Table 7 reports confidence intervals containing zero for the comparisons against Frozen Selective IDM and IDM Reflection. Could the authors provide additional independent runs, paired-seed comparisons where feasible, or another appropriate uncertainty analysis to establish whether these differences are robust?

Q3. Does improved prediction accuracy translate into better revision decisions?

Could the authors report the relationship between IDM prediction quality and actual revision outcomes? Relevant measurements would include the frequency with which IDM-triggered revisions improve performance, the frequency with which they degrade performance, and the calibration of predicted score differences relative to observed differences.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.