acceptodds
Under review as a conference paper at ICLR 2027

Heterogeneous Agentic Coevolution

Abstract

Recent advances in self-improving LLM agents have demonstrated the potential of AI systems that drive their own intelligence. However, many domain-specific applications rely on specialized foundation models that possess strong domain expertise but remain dependent on human-designed improvements. Extending recursive self-improvement to these models presents a key challenge: as a specialist improves, the strategies used to advance it must also adapt. To address this challenge, we propose Heterogeneous Agentic Coevolution (HACE), a framework that jointly evolves specialized foundation models and LLM agents through reciprocal learning. HACE improves the specialist through supervised learning while training the agent with on-policy reinforcement learning, guided by verified performance gains on the current specialist. The agent's exploration, in turn, informs subsequent specialist training, enabling both models to adapt to each other's progress. To sustain this reciprocal learning, HACE further employs evolutionary search to optimize their interaction harness, adapting how the models exchange information and select training experiences as their capabilities evolve. We evaluate HACE across three challenging domains: recommendation systems, protein function prediction in biology, and crystal stability prediction in material discovery. We show that HACE outperforms the evaluated self-improving methods on the primary task metrics across all three domains. These results shed light on heterogeneous coevolution as a universal approach to improving specialized foundation models through reciprocal interaction with a learning agent.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Self-improving AI, foundation models, AI for science

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2RVpOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HACE, a heterogeneous coevolution framework that jointly improves a specialized foundation model, an LLM agent, and their interaction harness. The agent is trained from verified improvements over the current specialist, while its exploration guides which examples are used to further train the specialist. An evolutionary search procedure additionally adapts the harness. Experiments on recommendation, protein function prediction, and crystal stability prediction show consistent improvements over several self-improving baselines.

Critical strengths
  1. The reciprocal-learning formulation is interesting and timely. Unlike fixed teacher-student approaches, both the specialist and agent adapt to each other, while the interaction harness also evolves.

  2. The empirical evaluation is broad, covering three domains, multiple specialist architectures, ablations, and scaling experiments. The gains on protein function prediction are particularly strong.

Critical weaknesses
  1. The main results evaluate the fused specialist+agent system rather than clearly showing that the specialist itself improves. Since the paper’s central claim concerns improving specialized foundation models beyond convergence, specialist-only results after coevolution are important.

  2. The fairness of the baseline comparison is unclear. HACE uses LLM sampling, specialist training, agent training, and repeated harness search, but the paper does not clearly describe whether ReST-EM, Meta-Harness, OpenEvolve, DGM, and TTT-Discover receive comparable generation, training, validation, and compute budgets.

Questions for authors
  1. Can the authors report the final performance of the HACE-trained specialist alone, without agent fusion, on all three domains?

  2. How are the computational and search budgets matched across HACE and all baselines, including LLM generations, gradient updates, validation evaluations, and total compute?

  3. Since the same validation set is repeatedly used for selection over many rounds, have the authors evaluated possible adaptive overfitting to the validation set?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 2RVp (Part 1/2)

We sincerely thank Reviewer 2RVp for recognizing the interesting and timely reciprocal-learning formulation, the broad evaluation across three domains with multiple specialist architectures, ablations and scaling experiments, and the particularly strong gains on protein function prediction.

W1/Q1. Specialist-Only Performance after Coevolution

The specialist itself improves after coevolution in all three domains, without agent fusion. As the reviewer noted, our central claim is that a specialized foundation model can improve beyond convergence, so the specialist alone is its most direct test. Base is already trained to convergence, in the sense that three more epochs of its original supervised schedule change the reported metric by less than 0.001 in every domain (App. C.3). To further address the reviewer's concern, we show the HACE-trained specialist alone (fusion weight λ=0\lambda = 0) next to the initial specialist in each domain, averaged over three seeds.

Domain (metric)Initial specialistHACE-trained specialist
Recommendation, Toys / SASRec (HR@10)0.05520.0605
Protein function, CAFA / ESM-2 650M (NDCG)0.62700.6726
Protein function, CAFA / ProtT5-XL (NDCG)0.62320.6734
Crystal stability, WBM / CHGNet (F1F_1)0.51590.5513

The recommendation values come from our 20-round Toys runs, which also score the specialist alone in every round, on 2,000 held-out test users that differ from the users in Table 1. On these users, the specialist improves in every seed, and the fused system reaches 0.0692, so in recommendation the agent's ranking adds further gains on top of the improved specialist. The protein values come from new runs of the Table 2 configurations that also score the specialist alone, evaluated on the same test blocks and seeds in the same way as Table 2. With 95% confidence intervals over the paired seeds, the specialist gains +0.0456 [+0.0431, +0.0480] NDCG with ESM-2 and +0.0501 [+0.0433, +0.0570] with ProtT5-XL.

On CAFA, the specialist alone comes close to the fused HACE system with both backbones, for example 0.6726 against 0.6745 NDCG with ESM-2. It also scores above every baseline in Table 2, all of which fuse their agent's ranking with the specialist and reach at most 0.6325 with ESM-2 and 0.6273 with ProtT5-XL. The improved specialist can therefore be deployed on its own there, without the agent at inference time.

For crystal stability, Table 6 already reports the specialist alone for every method, because F1F_1, the discovery acceleration factor (DAF), MAE and R2R^2 are computed from the energy predictions of each method's specialist, and the agent enters only the NDCG ranking. In our WBM setup, every method refits the same residual energy head, so these metrics compare how each method trains the specialist. With the same refit, the baselines reach a DAF of at most 2.203 and an MAE of at least 0.0748, and the specialist trained inside HACE has the best value on all four metrics. Compared with Base, it raises DAF from 1.644 to 2.242. Its MAE also falls from 0.1049 to 0.0729, so the energy predictions themselves become more accurate.

The revised Sec. 4.1 also states the specialist-alone gain in the main text, and the revised PDF adds this table as App. C.4.

AuthorsOpenReview ↗

Response to Reviewer 2RVp (Part 2/2)

W2/Q2. Computational and Search Budgets

On CAFA, HACE keeps its advantage when all methods are compared at the same wall-clock time, and we report each method's budget per run. In the CAFA runs of Table 2, all methods run the same number of rounds on the same rollout, validation and test blocks, and every run uses one NVIDIA B200 GPU, so its wall-clock time is also its GPU time. For gradient updates, HACE, ReST-EM and TTT-Discover train the agent on a comparable number of examples, while Meta-Harness, OpenEvolve and DGM search over harnesses without agent updates. HACE also trains the specialist and tests each specialist update against the incumbent, so it runs more validation scorings in each round than the baselines. The wall-clock time counts these scorings together with every LLM generation and gradient update of each method. The tables below give the NDCG each method reaches within a fixed wall-clock time, together with its budget per run, for both protein backbones and averaged over three seeds.

CAFA / ESM-2 650M, 50 roundsHACEReST-EMTTT-DiscoverMeta-HarnessOpenEvolveDGM
NDCG within 4 h0.67450.62860.62870.63090.63130.6324
NDCG, full run0.67450.62860.62870.63090.63200.6325
Agent training examples w/ replay12,19610,50611,841---
Wall-clock per run (h)10.74.34.35.04.44.4
CAFA / ProtT5-XL, 30 roundsHACEReST-EMTTT-DiscoverMeta-HarnessOpenEvolveDGM
NDCG within 2.5 h0.67330.62420.62410.62610.62540.6252
NDCG, full run0.67390.62420.62410.62610.62540.6273
Agent training examples w/ replay6,8685,2636,729---
Wall-clock per run (h)6.42.62.62.82.82.9

With both backbones, HACE exceeds the highest full-run score of any baseline within a time that is shorter than every baseline's full run. Between four hours and the end of their runs, the ESM-2 baselines gain at most 0.0007 NDCG, while HACE leads the strongest of them by 0.0420. The revised PDF adds these tables as App. C.6.

Q3. Possible Adaptive Overfitting to the Validation Set

We check for adaptive overfitting to the validation set with a held-out test block. In all three domains, this block is disjoint from the rollout and validation blocks and never enters an accept or reject decision. In the CAFA runs of Table 2, HACE stays above every baseline on this block at the final round, in every seed and with both backbones, after 30 to 50 rounds of selection on the validation pool. Within that pool, each proposal is compared with the incumbent on a fresh random draw in every round, and the incumbent is re-scored on that draw. Both states in each decision therefore see the same examples, and no decision reuses a score from an earlier draw. The revised App. C.3 describes the paired acceptance test.

We hope these results address your concerns, and we would be happy to provide any further clarification.

Dear Reviewer 2RVp,

Thank you again for your constructive comments. As the discussion phase ends in one day (November 18), we would like to ask whether our responses have addressed your concerns. If there are any remaining questions, we would be happy to respond.

If you find our responses satisfactory, we would appreciate your consideration in updating the evaluation.

Sincerely,
Authors

Thank you for the detailed rebuttal and additional experiments. The specialist-only results and computational comparisons have adequately addressed my original concerns.

I still have some reservations about the unique contribution of reciprocal coevolution. On CAFA, SFT with an acceptance test nearly matches full HACE (0.6737 vs. 0.6745), while decoupled training achieves comparable performance on Toys (0.0707 vs. 0.0692). These results suggest that the benefits of reciprocal coupling remain somewhat unclear.

Nevertheless, I appreciate the authors' thorough responses, the broad experimental evaluation, and the demonstrated improvements of the specialist models. Considering the additional evidence, I am updating my rating from 2 to 3.

Reviewer_jdLsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HACE, an iterative framework that jointly updates a specialized foundation model, a small LoRA-tuned LLM agent, and a discrete harness controlling context and data selection. In each round, the agent reranks the specialist’s shortlist to pick instances for SFT on the specialist, while the agent is updated via ReST on rollouts that beat the specialist. Final predictions ensemble the domain model's score and agent rank. The method is evaluated across recommendation, protein function prediction, and materials discovery, outperforming several recent baselines.

Critical strengths
  1. Defining the agent’s reward relative to the current specialist is well-motivated. Together with the shortlist cap observation, it nicely explains why the agent must co-evolve with the specialist and how a weaker LLM can still provide useful per-instance signal.
  2. The empirical scope is broad and the evaluation setup is clean, featuring a temporal split on CAFA, a chemical-system split on WBM, cross-fitted λ, and leave-one-out ablations across all three domains.
Critical weaknesses
  1. I am not convinced that the gains come from co-evolution rather than simply from extra SFT on the specialist combined with test-time LLM ensembling. According to Appendix A, none of the baselines are allowed to update the specialist, and a straightforward SFT baseline on D_train is missing. In fact, the ablations suggest that fine-tuning with validation gating is doing most of the heavy lifting: even without agent-guided selection, CAFA reaches 0.6575, which already captures 63% of the total gain and beats every baseline, and Appendix C.3 admits that the validation gate is what makes continued training work. On top of that, standalone specialist performance at λ=0 is never reported, so I cannot tell whether the specialist itself actually improves without test-time reranking.
  2. Several results across the tables are also difficult to reconcile. On CAFA, Table 3 shows that dropping specialist updates lowers performance to 0.6310 while a frozen E4B still reaches 0.6520, which again suggests that specialist SFT does the heavy lifting. It is then unclear why swapping the agent to E2B or 26B-A4B in Table 4 nearly wipes out the gains to +0.0012 and +0.0054. Moreover, the 50-round main run, the specialist sweep, and the 30-round agent sweep all report the exact same 0.6745 score and +0.0460±0.0006 margin for ESM-2 650M with E4B. The baseline numbers in the other two domains look questionable as well. On WBM, MAE and R² improve sharply for baselines that never touch CHGNet's energy predictions, suggesting that post-hoc calibration was applied to HACE and the baselines but not to Base. This would also inflate thresholded F1, and even then, HACE's F1 of 0.5513 is within one standard deviation of Meta-Harness at 0.5510. Similarly, on ML-1M, every baseline underperforms Base on SASRec and HSTU, even though λ=0 is in the search grid. Finally, Theorem 1 is a generic supermartingale bound minimized by uniform exposure, so the theory actually favors uniform sampling over an agent-guided curriculum.
Questions for authors
  1. What are the standalone test scores for the final specialist S_T at λ=0 on Toys, CAFA, and WBM? How does the "w/o agent-guided experience" ablation select training instances?

  2. Did Base ever train on the second-to-last interactions in RecSys, or fine-tune on WBM systems for CHGNet before HACE?

  3. How do you reconcile the E2B and 26B results in Table 4 with the frozen E4B result in Table 3, and why are the ESM-2 + E4B numbers identical across three different configs? What are the gate acceptance rates per agent?

  4. On WBM, was calibration applied to Base? If not, why do MAE and R² change for baselines that never modify CHGNet's predictions?

  5. On CAFA, is the 50-item pool C_i fixed from Base or recomputed each round? How are global ontology metrics like Fmax, AUPR, and Smin computed from a 50-item shortlist, and why is ProtT5's Smin unchanged at 13.127?

5 comments
AuthorsOpenReview ↗

Response to Reviewer jdLs (Part 1/3)

We sincerely thank Reviewer jdLs for recognizing the well-motivated reward defined relative to the current specialist, which together with the shortlist cap explains why the agent must co-evolve with the specialist, and the broad empirical scope with a clean evaluation setup.

W1/Q1. Co-Evolution versus Extra Specialist SFT and Test-Time Ensembling

The specialist itself improves inside HACE without test-time reranking, and on CAFA it keeps nearly all of the gain on its own. Test-time ensembling is shared by HACE and every baseline. Each baseline also fuses its agent's ranking with the specialist's scores, with λ\lambda cross-fitted in the same way, and these baselines improve over Base by at most 0.0040 NDCG on CAFA (Table 2).

Following the reviewer's suggestion, we report the standalone specialist at λ=0\lambda = 0 for both protein backbones, the SFT baseline on Dtrain\mathcal D_\mathrm{train} with ESM-2, and continued SFT on the data that trained Base. All values are read as in Table 2, the new rows on the same three seeds, and the λ=0\lambda = 0 values of HACE come from new runs that also score the specialist alone in every round.

CAFA backboneMethodNDCG, full systemNDCG, specialist alone
ESM-2 650MBase, trained to convergence0.62850.6270
ESM-2 650MContinued SFT on Base's original training data0.63220.6322
ESM-2 650MSFT on Dtrain\mathcal D_\mathrm{train} with the acceptance test, 30 rounds0.67370.6737
ESM-2 650MStrongest baseline, DGM (specialist fixed)0.63250.6270
ESM-2 650MHACE0.67450.6726
ProtT5-XLBase, trained to convergence0.62410.6232
ProtT5-XLContinued SFT on Base's original training data0.62900.6290
ProtT5-XLStrongest baseline, DGM (specialist fixed)0.62730.6232
ProtT5-XLHACE0.67390.6734

The specialist trained inside HACE reaches 0.6726 on its own with ESM-2, close to its fused score of 0.6745. HACE refits the specialist on the full training pool Dtrain\mathcal D_\mathrm{train} in every round, which is the uniform sampling option of Sec. 3.1, and running the same refit with the acceptance test but without the agent reaches 0.6737. On CAFA, most of the gain of HACE therefore comes from its specialist update, which refits the specialist on Dtrain\mathcal D_\mathrm{train} under the acceptance test. Every baseline in Table 2 keeps the specialist fixed on CAFA, and continued SFT on the data that trained Base, with the same update schedule and acceptance test, reaches 0.6322 with ESM-2 and 0.6290 with ProtT5-XL.

The agent contributes most in recommendation, where its ranking adds information beyond the specialist. On Toys, removing agent optimization lowers HR@10 from 0.0706 to 0.0622 (Table 3). In our 20-round Toys runs, scored on 2,000 held-out test users that differ from those of Table 1, the specialist alone improves from 0.0552 to 0.0605 HR@10. The fused system reaches 0.0692 on the same users, so the agent's ranking adds 0.0087 beyond the trained specialist. On WBM, Table 6 already reports the specialist alone, because F1F_1, DAF, MAE and R2R^2 use the specialist's energy predictions without fusion. Every method refits the same energy head there, and the HACE-trained specialist still has the best value on all four metrics, for example an MAE of 0.0729 against at best 0.0748.

The 0.6575 row. In "w/o agent-guided experience", each specialist refit trains on a random draw from Dtrain\mathcal D_\mathrm{train}, while agent optimization, harness search, the acceptance test and fusion all remain active. On Toys, where HACE trains the specialist on the users the agent verified, the draw has the same size as that set. On CAFA, where HACE refits on the full training pool, the draw is a smaller subset of that pool.

The revised PDF adds the table to App. C.5 and the standalone specialist results to App. C.4.

AuthorsOpenReview ↗

Response to Reviewer jdLs (Part 2/3)

W2a/W2b/Q3. Table 4 and the Agent Sweep

The 650M row of the specialist sweep is our main ESM-2 650M configuration, so it coincides with Table 2 by design. We thank the reviewer for spotting that the E4B row of the agent sweep repeated this row, which was an error in assembling Table 4. The agent sweep holds the specialist fixed, so its E4B row comes from the same specialist-fixed run as Table 3's "w/o specialist optimization" row. To further address the reviewer's concern, we reran E2B and 26B-A4B with exactly the settings of that E4B run, which keeps the ESM-2 650M specialist fixed for 30 rounds. The gains below are measured from the initial state of each run (0.6279 on average for the E4B run, App. C.3), with standard deviations over three seeds.

AgentNDCG gainHarness changes accepted
Gemma-4 E2B+0.0032±0.0006+0.0032 \pm 0.000653%
Gemma-4 E4B+0.0032±0.0020+0.0032 \pm 0.002048%
Gemma-4 26B-A4B+0.0069±0.0013+0.0069 \pm 0.001342%

The 26B-A4B agent more than doubles the gain of the two smaller agents, and the revised Sec. 4.3 replaces the earlier statement on agent size. Table 3's 0.6520 row and the specialist sweep both keep specialist optimization, while the agent sweep holds the specialist fixed and corresponds to the 0.6310 row, so the two tables measure different configurations and are consistent. The revised Table 4 reports all three agent rows from this configuration and its caption identifies the E4B row as the run of Table 3.

W2c/Q4. WBM Calibration, MAE and R2R^2

We would like to clarify that no post-hoc calibration of CHGNet's energy predictions is applied to Base or to any method, and that MAE and R2R^2 move because the specialist itself is updated during the run. Every WBM score converts the specialist's predicted formation energy into a distance from the convex hull as in the Matbench Discovery protocol, and the same code scores all methods. In our WBM setup, every method refits the specialist's residual energy head, a small network on frozen descriptors. With every method updating the specialist, HACE still has the best mean on NDCG and on the four energy-based metrics of Table 6, for example an R2R^2 of −0.203-0.203 against at most −0.251-0.251. We also ran the baselines with the energy head held fixed.

WBM / CHGNetF1F_1DAFMAE ↓\downarrowR2R^2
Base0.51591.6440.1049−1.289-1.289
Every baseline, energy head held fixed0.51591.6440.1049−1.289-1.289
Best baseline per metric, energy head refit0.55102.2030.0748−0.251-0.251
HACE0.55132.2420.0729−0.203\mathbf{-0.203}

With the head fixed, every baseline keeps the scores of the initial specialist on the four energy-based metrics in every round, so the refit head, not calibration, moves MAE and R2R^2. The revised App. A and Table 7 correct the submitted description of this setup. On F1F_1, HACE's 0.5513 is close to the 0.5510 of Meta-Harness, and the revised Sec. 4.1 states this.

W2d. ML-1M Baselines below Base

This pattern comes from the evaluation blocks rather than from the methods. Table 1 scores Base on all 6,040 ML-1M users. Each baseline is scored on its own held-out block of 2,762 or 3,222 users, where the specialist alone scores lower on average. The table below shows the specialist alone on all users and its mean over the baseline blocks, together with each method's gain over it on the same users. The scores on all users and the gain of HACE are those of Table 1.

ML-1M, HR@10Specialist, all usersSpecialist, mean over baseline blocksLargest baseline gainHACE gain
SASRec0.25570.2454+0.0031+0.0075
HSTU0.26040.2562+0.0022+0.0081

On their own users, every baseline scores above the specialist alone on average, consistent with λ=0\lambda = 0 in the search grid, and the gain of HACE is more than twice the largest baseline gain with both specialists. The revised Table 1 adds each method's gain over the specialist alone on the same users.

AuthorsOpenReview ↗

Response to Reviewer jdLs (Part 3/3)

Q2. Training Data of Base

Recommendation. Base never trained on second-to-last interactions, which served only as the early-stopping signal on 4,000 users. During the loop, the specialist update trains on the second-to-last interactions of rollout users, who are disjoint from the validation and test users.

WBM. CHGNet's weights are never trained. Before the loop, we fit 81 per-element reference energies on 30,000 WBM structures, and Base and every method use these same references. They come from chemical systems with fewer than ten structures, which are excluded from every rollout, validation and test instance. The residual head starts with zero output, so before any update the specialist reproduces CHGNet's own ranking.

Q5. CAFA Candidate Pool and Field Metrics

Pool. The 50-term shortlist is recomputed from the current specialist after each accepted specialist update. App. B.3 and C.1 now state this.

Field metrics. Fmax⁡F_{\max}, AUPR and Smin⁡S_{\min} are computed on the specialist's full protein-by-term score matrix, pooled over the three ontologies, for 2,058 to 2,096 proteins per seed. Where an agent is present, its votes are added at a fixed scale to the scores of the 50 shortlisted terms.

ProtT5-XL Smin⁡S_{\min}. The means are 13.1269 for HACE and 13.1274 for Base, and both round to 13.127. The revised Table 5 caption gives both means.

W2e. Theorem 1 and Uniform Exposure

Theorem 1 is a guarantee for every curriculum, and its sharper form below does not favor uniform sampling. Its contribution is that the finite-class rate remains valid when the selected instances depend on the specialist, the agent, the harness and the acceptance test, and when the final specialist is chosen from the observed history. Analyses based on uniform sampling require training draws independent of the learner and do not cover this loop.

The first term of Eq. 7 charges loss one to every under-exposed instance, which is why broad coverage looks favorable. The same proof (App. B.1, the first inequality of Eq. 17, before the losses there are bounded by one) gives a sharper bound on the same event

Rμ(ST)≤∑i:CT(i)<vμiμiℓi(ST)+L^T(ST)+log⁡(∣F∣/δ)(1−e−1)v.R_\mu(S_T) \le \sum_{i: C_T(i) < v\mu_i} \mu_i \ell_i(S_T) + \frac{\widehat L_T(S_T)+\log(|\mathcal F|/\delta)}{(1-e^{-1})v}.

An instance left out of training therefore contributes only μiℓi(ST)\mu_i\ell_i(S_T), its mass times the final loss on it. As an example of the regime that an agent-guided curriculum targets, suppose 80% of the source mass consists of instances on which the final specialist keeps a loss of at most ε\varepsilon. Let 50 rounds be accepted, with log⁡(∣F∣/δ)=10\log(|\mathcal F|/\delta)=10 and a specialist that fits its training experience. Exposure in proportion to the source distribution gives a bound of 0.316. The same total exposure placed on the remaining 20% gives 0.8ε+0.0630.8\varepsilon+0.063, which is smaller whenever ε<0.316\varepsilon < 0.316. The revised PDF adds this form as a remark after Theorem 1 and the example to App. B.1.

We hope these answers address your concerns, and we would be happy to provide any further clarification.

Reviewer_cY2bOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper focused on a critical issue: specialized foundation models still rely on human intervention, thus blocking recursive self-improvement. To fix this, the authors proposed a new method HACE. It co-evolves specialists and LLM agents via reciprocal learning, blending supervised and reinforced self-training. Moreover, evolutionary search dynamically optimizes their interaction. The method is tested in three scientific domains, and the results show that HACE beats existing baselines.

Critical strengths

S1. The paper focuses on a new and significant issue that how to improve specialized models in terms of stragegies.

S2. The proposed method takes advantage of LLM agents and specialized models coevolution, which is helpful for handling cases with limited data.

S3. Experimetal analyses are relatively comprehensive.

Critical weaknesses

W1. Insufficient isolation of the reciprocal coevolution mechanism.

Although Tables 1–2 demonstrate HACE's effectiveness and Table 3 confirms the contributions of individual components, it remains unclear whether the improvements arise specifically from dynamic reciprocal adaptation or from combining effective optimization procedures. Could the authors clarify the additional benefits of reciprocal coupling, potentially through a decoupled or independently alternating optimization baseline in which both models remain trainable under comparable budgets?

W2. Limited analysis of computational cost and budget comparability.

HACE jointly optimizes the specialist, agent, and harness, whereas the evaluated baselines differ in their optimization capabilities, making it difficult to assess whether the reported advantages persist under comparable computational resources. Although Appendix C.3 provides implementation details, a systematic cross-method cost comparison is missing. Could the authors clarify the computational budgets of the compared methods and, if feasible, provide representative performance-versus-compute results?

W3. Limited statistical support for marginal improvements.

The results are reported over only three random seeds, without paired comparisons or confidence intervals, leaving the reliability of some small improvements unclear. For example, Table 6 reports WBM/CHGNet F1 scores of 0.5513 ± 0.0004 for HACE and 0.5510 ± 0.0004 for Meta-Harness. While this does not undermine the larger gains observed elsewhere, could the authors provide further statistical evidence or qualify conclusions drawn from marginal differences?

W4. Limited analysis of the agent self-training objective.

Equation (5) optimizes next-token prediction over both prompt and response tokens, although the intended learning signal comes from verified agent responses. As noted in Appendix B.2, reducing the full-sequence loss does not necessarily improve the likelihood of selected responses. Could the authors clarify the motivation for including prompt tokens and discuss whether response-only training would affect agent adaptation or downstream performance?

W5. Insufficient characterization of candidate pool construction and coverage.

The experiments employ fixed reference candidate pools and specialist-dependent candidate views, while Appendix B.3 acknowledges that candidate coverage constrains attainable ranking performance (Eq. 40). However, the construction of the underlying reference pools and the coverage of relevant targets remain insufficiently characterized. Could the authors clarify these evaluation details and discuss how candidate availability affects the interpretation and generalizability of the reported results?

Questions for authors

See Weaknesses. Two further questions are listed as follows: (1) The paper could further clarify the alignment between NDCG-based verification and HR@10 evaluation in recommendation; (2) Appendix C.3 (p. 22, line 1140) should refer to Equation (4), rather than Equation (3), for the acceptance criterion;

2 comments
AuthorsOpenReview ↗

Response to Reviewer cY2b (Part 1/2)

We sincerely thank Reviewer cY2b for recognizing the new and significant problem of improving specialized models, the coevolution of LLM agents and specialized models for settings with limited data, and the breadth of our experimental analyses.

W1. Isolation of the Reciprocal Coevolution Mechanism

We thank the reviewer for suggesting the decoupled baseline, and we ran it on Toys / SASRec with the blocks and three seeds of our 20-round Toys runs in App. C.4. Both models remain trainable with the same budget, harness search and acceptance test as HACE. Each specialist refit trains on randomly drawn rollout users instead of the users the agent verifies, and the agent's reward is measured against the initial specialist instead of the current one, as follows:

Toys / SASRec, HR@10FusedSpecialist alone
Decoupled, both models trainable0.07070.0613
HACE0.06920.0605

The decoupled run is on par with HACE, and HACE has the higher fused HR@10 in two of the three seeds. In Table 3, where HACE reaches 0.0706 on the Toys users of Table 1, training the specialist on random users while the reward stays coupled lowers HR@10 to 0.0662. Removing either optimizer lowers it further, to 0.0622 without agent optimization and 0.0641 without specialist optimization. Together, these results show that on Toys the gain comes mainly from training the specialist and the agent in one loop, and that neither optimizer reaches it alone. The revised App. C.5 reports the decoupled run together with the one-sided ablations of Table 3.

W2. Computational Cost and Budget Comparability

HACE keeps its advantage on CAFA when all methods are compared at the same wall-clock time. In the CAFA runs of Table 2, all methods run the same number of rounds on the same rollout, validation and test blocks, and the three methods that train an agent receive a comparable number of agent training examples. Each HACE round also trains the specialist, and the wall-clock times below count this work together with everything else that each method does. Every run uses one NVIDIA B200 GPU, so these times are also GPU hours. To further address the reviewer's concern, we show the NDCG that each method reaches within a fixed wall-clock time, together with its budget per run, for both protein backbones and averaged over three seeds:

CAFA / ESM-2 650M, 50 roundsHACEReST-EMTTT-DiscoverMeta-HarnessOpenEvolveDGM
NDCG within 4 h0.67450.62860.62870.63090.63130.6324
NDCG, full run0.67450.62860.62870.63090.63200.6325
Agent training examples w/ replay12,19610,50611,841---
Wall-clock per run (h)10.74.34.35.04.44.4
CAFA / ProtT5-XL, 30 roundsHACEReST-EMTTT-DiscoverMeta-HarnessOpenEvolveDGM
NDCG within 2.5 h0.67330.62420.62410.62610.62540.6252
NDCG, full run0.67390.62420.62410.62610.62540.6273
Agent training examples w/ replay6,8685,2636,729---
Wall-clock per run (h)6.42.62.62.82.82.9

With both backbones, HACE exceeds the highest full-run score of any baseline within a time that is shorter than every baseline's full run. The revised PDF adds these tables as App. C.6.

W3. Statistical Support for Small Improvements

On CAFA, HACE has the higher NDCG in all 30 seed-level comparisons with the five baselines across both backbones. These comparisons are paired by design, because for each seed all methods share the same initial specialist and the same test block. The table below gives the mean paired difference in NDCG on CAFA with ESM-2 650M, with its 95% confidence interval over seeds.

HACE minusNDCG difference [95% CI], ESM-2 650M
ReST-EM+0.0458 [+0.0434, +0.0482]
TTT-Discover+0.0457 [+0.0432, +0.0482]
Meta-Harness+0.0436 [+0.0423, +0.0448]
OpenEvolve+0.0425 [+0.0369, +0.0481]
DGM+0.0420 [+0.0380, +0.0460]

Every interval excludes zero. With ProtT5-XL, the mean paired difference ranges from +0.0466 to +0.0498. On Recall@10, HACE is also higher in all 30 seed-level comparisons with both backbones. For the WBM example the reviewer gives, HACE leads on NDCG (Table 2), and the revised Sec. 4.1 states that its F1F_1 of 0.5513 is close to the 0.5510 of Meta-Harness. On the other energy metrics of Table 6, HACE leads every baseline by a wider margin, for example with a DAF of 2.242 against at most 2.203. The revised App. C.7 reports these paired comparisons.

AuthorsOpenReview ↗

Response to Reviewer cY2b (Part 2/2)

W4. The Agent Self-Training Objective

In our experiments, the agent update of HACE computes the next-token loss on the selected response tokens and masks the prompt positions, which is the response-only form the reviewer suggests. The revised Eq. 5 states this form. We train on the response alone because the prompt and the response hold different information (App. D). The prompt lists the task input, the specialist's leading predictions and a shortlist of up to 50 candidates with their metadata or definitions, while the requested response is at most ten candidate IDs. All responses to one instance in a round share this prompt, so its tokens are identical for verified and unverified responses (App. B.2). Only the response tokens differ between them, and only these tokens carry the signal that verification provides (App. B.3).

For agent adaptation, the response-only objective makes a lower training loss raise the likelihood of the verified responses. Eq. 37 shows that a full-sequence loss can decrease through its prompt term alone, which is the case the reviewer raises about App. B.2, and the prompt makes up most of the tokens of each record. Under Eq. 5, the prompt term is absent, so a lower training loss gives a positive weighted average of the token likelihood gains on the verified responses, which Eq. 38 relates to the gain gtg_t. Proposition 1 shows that, under the same specialist and harness, the probability of a verified response rises whenever gt>ktg_t > k_t, where ktk_t is zero when the selected responses follow the agent's own distribution over verified responses. When the specialist or harness changes, Eq. 31 carries this gain into the next round, minus a term ηt\eta_t that measures how much the response distribution and the verified set shift. Eq. 33 then gives a lower bound on the chance that at least one of mm responses passes verification. For downstream performance, the results we report are obtained with this objective. In our Toys runs that also score the specialist alone, the agent trained this way lifts HR@10 from 0.0605 for the trained specialist to 0.0692 for the fused system. The revised App. B.2 states this link.

W5. Candidate Pool Construction and Coverage

Each pool is built as follows, and the revised App. C.1 adds these definitions with the coverage values.

  • Recommendation. The pool is the current specialist's top 50 items for each user, recomputed after each accepted specialist update. With the specialist fixed, the pool stays the initial one, and HR@10 of any reranker cannot exceed the fraction of users whose target is in it. Updating the specialist, as HACE does, can bring a missing target into the pool, which reranking alone cannot do. The table below gives this fraction for the initial specialist in every setting of Table 1.
  • Protein function prediction. The pool is the specialist's top 50 GO terms per protein and ontology, recomputed after each accepted specialist update. Benchmark protein-ontology pairs without a true term in the initial pool, 6.7% of the benchmark, are left out of the rollout, validation and test blocks for all methods.
  • Crystal stability. An instance is a chemical system with at least ten WBM structures and at least eight deduplicated candidates, which must include both stable and unstable ones. These conditions give 3,745 instances. Up to 40 deduplicated candidates ranked by CHGNet are kept, and the first 20 form the pool that is scored and shown to the agent.
Fraction of users with the target in the initial top-50 poolSASRecHSTUWukong
ML-1M (6,040 users)0.4980.5020.484
Beauty (20,000 users)0.1280.1250.150
CDs (20,000 users)0.1260.1380.155
Sports (20,000 users)0.0760.0770.091
Toys (19,412 users)0.1180.1180.143

Q1. NDCG-Based Verification and HR@10 Evaluation

The selection score rewards the same event as HR@10, with an additional discount for rank. Verification and selection score the target's rank with NDCG, and the selection score averages NDCG@5 and NDCG@10. With one held-out target per user, NDCG@kk is positive exactly when the target is in the top kk, so HR@10 is the indicator that NDCG@10 is positive. The revised App. C.1 states these cutoffs.

Q2. The Equation Reference in App. C.3

Thank you for the careful reading. The revised App. C.3 refers to Eq. 4 for the acceptance criterion.

We hope these results and clarifications address your concerns, and we would be happy to provide any further clarification.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

Dear Reviewers and Area Chair,

Thank you for your time and efforts in evaluating our paper. We are encouraged that the reviewers recognize the new and significant problem of improving specialized models (cY2b), the well-motivated reward defined relative to the current specialist (jdLs), and the interesting and timely reciprocal-learning formulation (2RVp). All three reviewers also recognize the broad evaluation across three domains (cY2b, jdLs, 2RVp). We summarize below the new results and revisions that address the initial concerns, and we answer every point of each review in the individual responses.

Key Contributions

  • The new and significant problem of improving a specialized foundation model after its supervised training, without human-designed updates (cY2b).
  • HACE, which brings the specialist itself into the self-improvement loop. The agent is rewarded for verified gains over the current specialist, and the specialist is updated through the loop in turn (jdLs, 2RVp).
  • An evaluation across recommendation, protein function prediction and crystal stability prediction, with multiple specialist architectures, ablations and scaling experiments (cY2b, jdLs, 2RVp).

Key Revisions

The revised PDF marks all changes in red.

  • Specialist-only results. The HACE-trained specialist alone improves over the initial specialist in all three domains and with both protein backbones, App. C.4. On WBM, where every method refits the same energy head, the HACE-trained specialist has the best mean F1F_1, DAF, MAE and R2R^2 of all methods (2RVp, jdLs).
  • Compute. We report each method's budget and the score it reaches within a wall-clock budget. With both protein backbones, HACE exceeds the highest full-run score of any baseline within a time shorter than every baseline's full run, App. C.6 (cY2b, 2RVp).
  • Statistics. We include paired per-seed comparisons with 95% confidence intervals. On protein function prediction, HACE is higher in all 30 seed-level comparisons with the baselines, and the interval of the specialist's own gain excludes zero with both backbones, App. C.4 and C.7 (cY2b, 2RVp).
  • Controls. The revision adds an SFT baseline on Dtrain\mathcal D_\mathrm{train}, continued SFT on the original training data, and a decoupled run on Toys with both models trainable, App. C.5. Together with the specialist-alone results, they show that the CAFA gain is an improvement of the specialist itself, made by the specialist update of HACE rather than by test-time reranking. On Toys, the gain comes mainly from training both models in one loop, and removing the update of either model lowers HR@10 from 0.0706 to at most 0.0641 (cY2b, jdLs).
  • Candidate pools. App. C.1 describes how the candidate pool is built in each domain and reports its coverage in every recommendation setting (cY2b, jdLs).
  • Theory. A loss-weighted form of Theorem 1, as a remark after Theorem 1 and in App. B.1 (jdLs).
  • Clarifications. Table 4 reports the E4B agent row from the same specialist-fixed run as Table 3, with all agent rows in one configuration (jdLs). App. A and Table 7 describe the WBM energy head (jdLs). Eq. 5 states the response-only loss (cY2b). App. B.3 and C.1 state that the CAFA shortlist is recomputed after each accepted specialist update, and Table 1 adds each method's gain over the specialist alone on the same ML-1M users (jdLs). App. C.3 describes the paired acceptance test (2RVp) and cites Eq. 4 (cY2b).

We hope these results and revisions further strengthen your confidence in our work, and we would be happy to provide any further clarification. Thank you again for your time and consideration.

Sincerely,

Authors of Submission 28845

Dear Reviewers,

The authors have submitted their responses to the questions and concerns raised in the reviews. Please review their responses and provide any further comments or clarifications, if applicable.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.