acceptodds
Withdrawn from ICLR 2027

Cross-Model Reproductive Fitness: Evolving Transferable Skills across Model Hosts

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language model agents increasingly use reusable skills to perform long-horizon tasks involving planning, tool use, and artifact manipulation. Existing evolution protocols typically optimize skills on a primary host and evaluate transfer after optimization, even though skills can transfer across execution models and agent harnesses. This selection procedure can favor host-specific instructions and invocation patterns, and pooled performance can conceal degradation on weaker or incompatible hosts. We introduce Cross-Model Reproductive Fitness (CMRF), a transfer-aware evolutionary framework that models model-harness configurations as heterogeneous host environments and selects skills directly for cross-host utility. CMRF organizes hosts into evolution islands and evaluates candidate skills based on aggregate task gain, cross-host variability, weak-model gain, and worst-host performance. Skills migrate between islands and receive reproductive credit when they provide measurable utility in the destination host, aligning evolutionary survival with transferability. CMRF decomposes each skill into a platform-agnostic core, a model adapter, and a harness invocation layer, preserving shared procedures while supporting host-specific adaptation. Under matched execution budgets, we evaluate CMRF on seen and held-out host configurations against single-host evolution, pooled-average selection, worst-host-aware selection, and island migration. Ablation studies and structural analyses isolate the contributions of migration, transfer-aware fitness, and layered skill design to cross-host robustness.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
skill evolution, cross-model transfer, LLM agents

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FAbCOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes CMRF, an evolutionary framework for reusable agent skills across heterogeneous model–harness configurations. Source hosts maintain separate populations, and selection combines mean skill gain, cross-host variance, weak-host gain, worst-host gain, and probe-host transferability. Migrants receive reproductive credit for improving their destination relative to no-skill and local-population references. Skills are separated into a platform-independent core, a model adapter, and a harness invocation layer. Evaluation uses a 48-task verifier-backed suite with a 24/8/16 evolution/development/test split, four Qwen2.5 source configurations, and four additional model–harness configurations. Under a 768 candidate-host evaluation budget and five seeds, the paper reports higher source and transfer pass rates than single-host, pooled-average, and worst-case selection methods, together with ablations of migration, fitness terms, and layered representation.

Critical strengths
  • The motivation is consequential and receives experimental support. Table 1 shows that single-host optimization reaches a 43.7% source mean but only 30.9% on the worst host; pooled-average selection improves the mean to 45.8% while the worst host remains at 35.8%. These comparisons make the heterogeneity concern concrete. CMRF's explicit weak/worst-host objectives and separation of core procedures from invocation details respond directly to the problem, with the full system improving the reported worst-source score to 40.7%.
  • The experiment suite includes meaningful baselines, outcomes, and feasibility controls. Single-host, pooled-average, worst-case, and Pareto selection are relevant alternatives to the proposed objective. The four-domain task suite uses functional verifiers, so success reflects completed requirements rather than stylistic preference. Migration, fitness-term, representation, frequency, and budget studies test several parts of the idea beyond the main result. These provide useful evidence for robustness across the evaluated configurations, although they do not establish every claimed mechanism or unseen-host generalization.
Critical weaknesses
  • The transfer evaluation does not isolate the generalization problem stated in the motivation. The paper seeks skills that remain useful beyond the hosts under which they are optimized. However, Equations 6–7 put probe-host performance U(s) into reproductive fitness, with coefficient 0.25, and Algorithm 1 evaluates candidates on those hosts before selection. Appendix B.1 describes the four transfer configurations as those probe configurations. Their feedback therefore influences which cores survive even though they generate no mutations or parents. The task split and frozen-core transfer are useful controls, but cannot establish generalization to a host absent from the selection objective. Functional verifiers make the reported performance credible on the measured configurations; the methodological issue is that the experiment addresses new tasks on selection-exposed hosts rather than the claimed host-independent transfer. Code inspection confirms this interpretation: src/core_method.py uses probe gain in both fitness and nondominated archive construction, and src/pipeline.py includes these same probe hosts in final evaluation. The code therefore clarifies the protocol rather than resolving host-independent generalization.
  • The necessary ablation for the named reproductive-credit idea is absent from the otherwise substantial experiment suite. Table 3 tests migration, weak/worst-host terms, variance, and layered skills, but no-migration removes both exchange and destination credit. Removing the weak-host term does not isolate the destination reproductive credit R_h defined in Equations 8–9. Thus, these controls cannot determine whether reproduction history improves over ordinary island migration using the same multi-host fitness. This is consequential because the monolithic ablation alone lowers the reported transfer worst-host score from 38.4% to 33.8%, showing that representation is a major source of benefit. The relevant baseline is a layered, multi-host island optimizer with identical host feedback, migration schedule, and evaluation budget, but zero R_h. Without that comparison, the system-level gains do not establish the distinct mechanism emphasized in the title and motivation. The supplied demo explicitly computes destination credit and feeds accumulated credit back into later parent selection, so the mechanism is implemented. However, simulator success probabilities already encode benefits of core quality, adapters, and invocation compatibility. Its outputs cannot replace a matched zero-credit control on the paper's agent tasks, and its pooled-baseline search budget is also different from CMRF's multi-island budget. These observations concern what the demo establishes; they do not invalidate the separately reported benchmark numbers.
  • Writing quality: repeated defensive self-validation and excessive emphasis on protocol boundaries. The introduction asserts that the experiments separate adaptation from sampling and post hoc transfer; Section 4.1 claims the protocol “isolates zero-shot portability” and excludes “hidden evaluation feedback.” Sections 3.4 and the appendices repeatedly emphasize that probe hosts do not generate mutations or parents and that transfer involves no task-specific refinement. The conclusion again claims the ablations “verified the contributions” of the proposed components. These repeated assurances and exclusions give the manuscript an overly defensive, preemptive-rebuttal tone and can create the impression of protesting too much. They also obscure the distinction between the intended controls and the fact that probe scores influence selection. This is the consequence of the author's excessive reliance on GPT writing, and the author should optimize these writing sections to make their paper statements more persuasive, thereby improving the quality of their writing.
Questions for authors
  • The demo confirms that probe gain enters selection and archive construction. In the reported benchmark, which final hosts are entirely excluded from these feedback routes? Does the advantage persist on such hosts when every baseline receives the same selection-host access? A task split on selection-exposed hosts does not answer this question.
  • With layered skills, multi-host fitness, probe feedback, migration, and evaluation budget fixed, how does CMRF compare with setting destination credit R_h to zero on the actual agent tasks? Does cumulative credit improve over using only current destination gain? The demo implements credit, but does not isolate its contribution under the benchmark conditions.
Reviewer_ZEZCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper considers that skills evolved on one model-harness host often fail to transfer, and current pipelines require to optimize on a single host and only check transfer afterwards. They propose CMRF, which runs one evolution island per host, scores skills with a five-term fitness over mean gain, variance etc, and gives reproductive credit. The main novelty seems to be selecting for cross-host utility during evolution itself, with a adapter that keeps shared logic separate from host-specific interfaces.

Critical strengths

Rewarding a migrant only when it beats the no-skill baseline and the destination population, sounds a sensible operational definition of real transfer

Critical weaknesses

Only 16 held-out test tasks, and no table reports p-values or CIs even though Appendix B.2 promises bootstrap tests at p << 0.05. Some margins, e.g. 1.8 points in Table 4, are close to seed noise. tbh the main superiority claim is not statistically backed

The budget comparison is hard to audit. Section 4.1 gives every method 768 evaluations, but step 17 of Algorithm 1 scores candidates on all 8 hosts and CMRF costs 2.71$\times$. What counts against the shared ledger is never stated in this submission, so fairness is unclear. please take it into acount in the revision

The fitness is redundant and under-tuned. With H=4H = 4 and ρ=0.5\rho = 0.5, W(s)W(s) and M(s)M(s) in Eq. (5) seem to overlap heavily? The five weights are fixed once on 8 dev tasks with no sensitivity analysis. Also, your Eq. (8) compares a migrant scored on a held-out minibatch against g(t)g(t) computed on a different batch?

i'd be willing to modify the scores if the above issues are addressed. please correct me if i was wrong

Questions for authors

see the weakness

Reviewer_jBELOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This submission introduces a framework named CMRF (Cross-Model Reproductive Fitness) that develop transferable skills across different model hosts. CMRF considers several factors such as aggregate task gain and weak-model gain. CMRF decomposes the skills into: a shared procedural core, a model adapter and a harness invocation layer. Experimental results across multiple settings validate the effectiveness of the proposed CMRF.

Critical strengths

This work has these critical strengths:

  • The studied problem is important and timely. Evolving reusable skills is of value.
  • The proposed design is easy to understand.
Critical weaknesses

This work has these critical weaknesses:

  • Some claims might not be very accurate. The unseen hosts appear to contribute to fitness as shown in Eq (6). It is unclear whether final evaluation uses separate hosts excluded from the selection.
  • The visualization quality could be improved. For example, the title for Figure 1 seems incorrect. It is more like a motivation example figure rather than framework figure. The Figure 2 is difficult to read, especially the mathematical labels.
  • There are some inconsistencies in this work: For example, in methods' performance in Table 1 and Table 9 report the same mean values but different Std values. For example, Pooled Average has a std of 0.9 in table 1 but a std of 0.7 in Table 9.
Questions for authors

I have the these questions or suggestions:

  • It would be better to make some claims and settings more clear.
  • It would be better to replace the figures with higher resolution.
  • It is appreciated if the authors could double check the results as mentioned in Weakness 3.
  • There is one typo: Line 84, 'As illustrated in 1,' -> 'As illustrated in Figure 1'.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.