acceptodds
Under review as a conference paper at ICLR 2027

DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolution

Abstract

Large language models (LLMs) cannot retain post-deployment experience without parameter updates. We introduce DIVE, a diversity-driven framework that enables frozen LLMs to improve by evolving persistent natural-language skills from task experience and verifier feedback. These skills encode reusable reasoning procedures, verification strategies, common failure modes, and output constraints and are both executed and revised by the same underlying model without access to a teacher model. Since natural-language skill evolution is a stochastic, non-convex search process, optimizing a single skill trajectory can overfit to sampled experience or converge to a suboptimal solution. DIVE mitigates this optimization variance by independently evolving multiple skill populations from bootstrapped experience, adaptively refining them through diverse transformations, and jointly selecting a complementary set of skills. Across six mathematical and logical reasoning tasks and multiple model families, DIVE consistently outperforms existing reasoning methods, prompt-optimization approaches, skill-development frameworks, and memory-based baselines. It achieves rapid self-improvement from accumulated experience, obtaining substantially larger performance gains with fewer rollouts than parameter-based methods such as SFT and GRPO, and prompt optimization with GEPA. Further, the resulting skills transfer across model scales and families, enabling smaller models such as GPT-5-nano to match or outperform larger counterparts, i.e., GPT-5, under conventional prompting. These results establish diversity-driven skill evolution as an effective, interpretable, and parameter-free approach to LLM self-improvement.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Self-Improvement, Skill Evolution, Frozen Language Models, Natural-Language Skills

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_W4vUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DIVE, a framework for improving frozen language models by evolving persistent natural-language skills from verifier-labeled task experience, without parameter updates or a stronger teacher model. DIVE initializes multiple independently bootstrapped skill populations and evolves them using heterogeneous operators (reflective repair, exploratory revision, compression, and recombination). It allocates evolution steps across operators with UCB and can generate additional operators from the evolution history. After evolution, it greedily selects a complementary set of skills based on the performance of the full candidate-aggregation pipeline. Experiments on six mathematical and logical reasoning benchmarks and several model families show improvements over prompting, memory-based methods, and GEPA, including strong results with multiple inference candidates and cross-model skill transfer.

Critical strengths

1. Strong and reasonably broad empirical results. DIVE demonstrates consistent improvements across six mathematical and logical reasoning benchmarks and multiple model families. In the 10-candidate setting, Multi-DIVE improves average accuracy over Multi-GEPA from 75.9% to 81.4% on GPT-5-nano, from 92.1% to 96.4% on DeepSeek-v4-flash, and from 81.0% to 86.0% on Qwen3.5-27B (Table 2). The paper also reports promising cross-model skill transfer. These results make the proposed framework practically interesting even if some individual mechanisms are incremental.

2. A useful, systematic approach to skill evolution. Separating different skill-transformation operators and adaptively allocating the evolution budget using UCB is a reasonable and potentially robust improvement over relying on a single reflection/update strategy. The independent populations provide a straightforward way to explore different evolution trajectories. The resulting framework is modular and applicable to frozen models, and the operator-allocation comparisons in Figure 4 provide some empirical support for this design.

Critical weaknesses

1. The improvement over GEPA is not sufficiently disentangled from the final skill-set selection procedure. GEPA already exploits diversity through Pareto-based candidate selection and can merge complementary prompt improvements. DIVE's use of diversity at inference is different, but conceptually modest: the selected skills independently generate answers, which are then aggregated, rather than being routed based on the input or task context. More importantly, the Multi-GEPA baseline retains the best prompt from each of ten independent GEPA runs, whereas Multi-DIVE selects its final set jointly for complementary performance. Thus, the reported comparison changes both the evolution procedure and the final selection procedure. A critical missing baseline is GEPA + the same joint skill/prompt-set selection, using a comparably sized candidate pool, the same aggregation procedure, and a matched optimization budget. Without this control, it is difficult to determine how much of DIVE's advantage comes from its new evolution strategy rather than from the final set-selection mechanism. This matters directly to the paper's claimed methodological contribution beyond GEPA.

2. Figure 4 does not sufficiently isolate or explain the contribution of adaptive operator generation. The caption suggests that the difference between M-UCB and Full is operator generation, but the exact configurations and controlled differences between these variants are not fully specified. Moreover, the default setting introduces only one new operator per population at evolution step 8 out of 10, leaving very limited opportunity to evaluate and exploit it. The paper does not clearly report how often the generated operators are subsequently selected, which transformations they perform, or how parent selection is handled for these newly generated operators. Consequently, the observed Full–M-UCB gap is hard to attribute specifically to history-conditioned operator generation, rather than to simply introducing an additional operator or additional exploration. A control that adds a fixed or randomly generated operator at the same step, along with operator-use statistics and an explicit mapping from evolution steps to the rollout curves, would substantially strengthen this claim.

Questions for authors

Q1. Could the authors evaluate Multi-GEPA + Joint Set Selection, applying the same greedy complementarity-aware selection and inference-time aggregation used by DIVE to a comparable pool of GEPA candidates? In particular, could the candidate-pool sizes, validation protocol, and optimization rollout budgets be matched? This would help separate the contributions of the evolution algorithm from those of final ensemble construction.

Q2. In Figure 4, is Full strictly identical to M-UCB except for adaptive operator generation? Please specify the precise configuration of each variant, indicate when generated operators become available and are actually selected on the rollout curves, and clarify their parent-selection rules. Could the authors also report examples and usage frequencies of generated operators and compare against adding a fixed or random extra operator at the same step? These details would clarify whether the reported gain is attributable to adaptive generation itself.

Q3 (minor clarification). GPT-5-nano Multi-DIVE is reported as 81.4% in Table 2 and 81.5% in Table 3, with a small difference in the corresponding HMMT results (82.7% in Table 6 vs. 82.9% in Table 7). Are these from separate evaluation runs, and if so, why were separate runs needed? Also, Table 2 describes frozen-model ranking, whereas Appendix A.6 specifies majority voting for HMMT. Could the authors clarify which aggregation procedure was used for HMMT in these results, and whether there is a reason for any difference?

4 comments
AuthorsOpenReview ↗

Response to Reviewer W4vU (Part 1)

We sincerely thank the reviewer for their careful review and valuable feedback.

Response to W1

We conducted the requested controlled comparison: applying the same joint selector to GEPA. We applied DIVE’s selector to the original Multi-GEPA candidate pools across five runs.

MethodGPT-5-nanoDeepSeek-v4-flashQwen3.5-27B
Multi-GEPA + Ranker75.992.181.0
Multi-GEPA + Joint Selection + Ranker77.092.682.1
Multi-DIVE + Joint Selection + Ranker81.296.486.0

We can see that the final selector contributes to performance but does not account for most of DIVE’s advantage. Qualitative inspection also shows that prompts from independent GEPA runs are often highly similar, providing relatively limited complementary coverage for the joint selector to exploit.

The complete task-level results are:

ModelMethodHMMTEq. TheoriesSudokuCrypt.Calc.Futo.Avg.
GPT-5-nanoMulti-GEPA + Ranker81.062.893.155.769.293.475.9
GPT-5-nanoMulti-GEPA + Joint Selection + Ranker81.763.493.856.872.294.477.0
DeepSeek-v4-flashMulti-GEPA + Ranker90.377.296.495.796.196.792.1
DeepSeek-v4-flashMulti-GEPA + Joint Selection + Ranker91.078.296.896.296.597.192.6
Qwen3.5-27BMulti-GEPA + Ranker94.787.678.279.166.480.381.0
Qwen3.5-27BMulti-GEPA + Joint Selection + Ranker95.388.379.380.068.081.782.1

Response to W2

Figure 4 configurations. All variants use K=10K=10 independently populations, B=10B=10 local evolution steps per population, ∣Devo(k)∣=50|D_{\mathrm{evo}}^{(k)}|=50, ∣Dref(k)∣=50|D_{\mathrm{ref}}^{(k)}|=50, and ∣Dval(k)∣=50|D_{\mathrm{val}}^{(k)}|=50.

  • Single uses only Reflective Repair throughout evolution.
  • M-Unif uses the four initial operators in Appendix A.2, Eq. (19), Reflective Repair, Exploratory Revision, Compression, and Recombination, with uniform allocation (uniform random selection from the set of eligible operators) and no operator generation.
  • M-UCB uses the same four operators with UCB allocation (β=0.3\beta=0.3) and no operator generation.
  • Full is identical to M-UCB except that each population generates one history-conditioned operator after its local step tnew=8t_{new}=8.

Observations. Because UCB rewards the novelty of the adaptive operators, every population selects them at both remaining local steps: 10/10 selections at step 9 and 10/10 at step 10. Thus, 20% of the evolution budget is allocated to the newly generated operators, which we find sufficient to produce a measurable effect.

The 20 generated operators, 10 for each dataset, primarily perform five types of transformations:

  • Structural reorganization (6): Preserve the content while revising its ordering, grouping, and transitions.
  • Modularization (6): Divide a skill into independent, conditionally triggered modules that can be selected as appropriate. These operators are used primarily for Sudoku.
  • Failure-targeted revision (5): Identify the stages or error types most implicated by failed trajectories and revise only the relevant sections.
  • Problem-type specialization (2): Provide different strategies for different problem types.
  • Conflict resolution between two parents (1): Reconcile contradictory strategies inherited from two parent skills.

Seven of the operators use two parents. Their transformations vary: some retain the structure of one parent, some organize the two parents as parallel modules, some reconcile conflicting strategies, some assign different parents to different problem types, and others synthesize the failure patterns of both parents.

Parent selection depends only on the number of parents declared by the operator:

  • Single-parent operators: Sample a parent from the candidate pool using a softmax over accuracy with temperature τ=0.6\tau=0.6, favoring higher-performing skills.
  • Two-parent operators: Select the first parent using the same procedure. Select the second using a softmax over 0.5×accuracy+0.5×complementarity0.5\times\text{accuracy}+0.5\times\text{complementarity}, where complementarity is the proportion of examples solved by the second parent but not by the first. This encourages pairing a strong parent with a complementary one.

We present two examples below. All generated operators are available at: https://anonymous.4open.science/r/DIVE_experiments-8BC4/qwen_3_8b_ablation/new_operators.

AuthorsOpenReview ↗

Response to Reviewer W4vU (Part 2)

HMMT population 1:

---
name: Phase-Coherent_Reorganization
parent_arity: 1
parent_selection: score_proportional
---

Analyze the parent skill’s section structure and trajectory feedback to identify disjointed or redundant reasoning phases. Reorder and regroup existing content into a sequence aligned with problem-solving stages (e.g., problem characterization, hypothesis generation, validation), preserving all original guidance while improving logical progression.

Sudoku population 3:

---
name: Conflict_Resolution_Restructuring
parent_arity: 2
parent_selection: strong_then_complementary
---

Analyze contradictions between the two parents' reasoning approaches using their success/failure summaries. Identify conflicting assumptions, verification methods, or decomposition strategies. Restructure the skill to create a coherent framework that resolves these conflicts through principled prioritization or integration of competing approaches, preserving essential knowledge from both parents while eliminating contradictory guidance.

We conducted the requested controlled comparison:

  • M-UCB + Fixed replaces the adaptively generated operator with the same fixed operator (shown below) in every population. It is made available at exactly the same local evolution step (tnew=8t_{new}=8), with all other settings held constant.

The fixed operator:

---
name: Dynamic_Strategy_Adjustment
parent_arity: 1
parent_selection: score_proportional
---

Analyze the parent skill and its failed trajectory summaries to identify recurring contexts where persistent strategies led to failure. Revise the skill to include generalizable cues for recognizing such contexts and guidance to prompt the solver to reassess assumptions, pivot reasoning approaches, or explore alternative decompositions at those points. Avoid specifying fixed alternatives; instead, emphasize principles for evaluating strategy effectiveness and triggering adaptive responses.

The results are avalable at: https://anonymous.4open.science/r/DIVE_experiments-8BC4/qwen_3_8b_ablation/fixed_operator_experiments.

Adaptive operators outperform the fixed operator because they are generated in response to the evolution history produced by the initial operators, allowing them to target the observed limitations and failure patterns.

Rollout-local evolution steps mapping. Each population tracks its task-solving rollouts and records the count at the end of every evolution step. A monitor sums these counts across populations. When the total first reaches a multiple of the checkpoint interval (500 in Figure 4), the monitor records a global checkpoint. At each checkpoint, we collect all skills whose evaluation has completed, retain the top (L=3) candidates from each population, jointly select up to (M=10) skills on the validation set, and evaluate the resulting set. Because populations evolve asynchronously, local evolution steps do not correspond one-to-one with global rollout checkpoints.

We will add the implementation details and results to the paper.

Response to Q1

See Response to W1.

Response to Q2

See Response to W2.

Response to Q3

Tables 2/6 and Tables 3/7 were produced by separate evaluation jobs with the same skills, model, and protocol; the latter additionally collected token and cost statistics. Stochastic decoding resulted in minor differences. We will use the canonical accuracy results from Tables 2/6 consistently in the revision.

Appendix A.6 inadvertently retained the aggregation rule used in an earlier implementation. All reported multi-candidate results, including the HMMT results in Tables 2 and 6, use the same ranker. We will correct the description in Appendix A.6 accordingly.

AuthorsOpenReview ↗

Follow-up on Author Response

Dear Reviewer,

Thank you again for your constructive feedback. We have posted our responses to your comments and would greatly appreciate any further thoughts you may have before the discussion period ends.

We would be happy to clarify any remaining concerns.

Best regards,
Authors

Thank you to the authors for the detailed responses and additional experiments.

The new GEPA + Joint Selection baseline and the clarification of the Figure 4 configurations, together with the additional operator ablation, have adequately addressed my main concerns. The minor reporting inconsistencies have also been clarified.

Overall, I am satisfied with the rebuttal and will maintain my rating.

Reviewer_SdKKOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DIVE improves frozen language models without updating their weights by evolving diverse populations of reusable natural-language skills from task experience and verifier feedback, then selecting complementary skills to generate candidate solutions and choose a final answer. This is done across mathematical and logical reasoning tasks, the authors report improved accuracy, efficient use of task-solving attempts, and transfer of learned skills across model sizes and families.

Critical strengths

DIVE addresses a practical problem: how to improve a model when updating its weights is not an option. The same model solves problems and updates its skills based on verifier feedback, so the approach does not require a stronger model to guide it. The skills are also readable and reusable, and the transfer experiments suggest that they can remain useful when moved to a different model.

Another strength is that the paper checks whether DIVE’s gains come simply from generating more answers. It reports improvements with one answer per question and compares the multi-skill version against baselines that also get ten attempts. The skill-selection experiment is helpful too: choosing skills that work well together gives better results than simply picking the strongest individual skills, providing direct support for this part of the method.

Critical weaknesses

It is hard to tell how much of DIVE’s improvement comes from its evolution strategy. DIVE jointly selects complementary skills, while Multi-GEPA keeps the best prompt from each run, so the comparison also changes how the ensemble is built. SkillOpt is missing from the multi-candidate experiments, and the paper does not directly test whether independent populations help when the total budget is held fixed.

I also have some concerns about the efficiency results. Based on the implementation details, the reported setup seems to require around ~40000 rollouts, but Figure 1 only shows about 7,000. It is not clear whether the authors used different settings for these experiments or excluded certain rollouts from the count. Different settings or excluded costs could explain this, but the paper needs to clarify that. Cost comparisons should account for validation, revision, and ranking, and clearly state inference budgets; the small-model comparison also leaves out skill-development costs.

Finally, some reporting details make the results harder to assess. The main tables give five-run averages without uncertainty estimates, despite small gains over SkillOpt. Equational Theories checks verdicts without describing proof verification, and the descriptions of HMMT’s aggregation procedure disagree. Since the theory mainly motivates the approach, stronger empirical controls and clearer reporting are needed to support the conclusions

Questions for authors

see weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer SdKK (Part 1)

We sincerely thank the reviewer for their careful review and valuable feedback.

Response to W1 (Comparison to baselines)

We conducted the two requested controlled comparisons.

1. Applying the same joint selector to GEPA. We applied DIVE’s selector to the original Multi-GEPA candidate pools across five runs.

2. Including SkillOpt in the multi-candidate experiments. We include single-prompt SkillOpt and multi-prompt SkillOpt with the same ranker and selector.

MethodGPT-5-nanoDeepSeek-v4-flashQwen3.5-27B
Single-GEPA + Ranker73.189.877.8
Multi-GEPA + Ranker75.992.181.0
Multi-GEPA + Joint Selection + Ranker77.092.682.1
Single-SkillOpt + Ranker75.790.979.7
Multi-SkillOpt + Joint Selection + Ranker77.292.381.5
Multi-DIVE + Joint Selection + Ranker81.296.486.0

We can see that the final selector contributes to performance but does not account for most of DIVE’s advantage. Qualitative inspection also shows that prompts from independent GEPA runs are often highly similar, providing relatively limited complementary coverage for the joint selector to exploit. In addition, we observe that SkillOpt makes incremental refinements to the current skill at each step, resulting in highly similar skills across independent runs and limited gains from complementarity.

The complete task-level results are:

ModelMethodHMMTEq. TheoriesSudokuCrypt.Calc.Futo.Avg.
GPT-5-nanoMulti-GEPA + Ranker81.062.893.155.769.293.475.9
GPT-5-nanoMulti-GEPA + Joint Selection + Ranker81.763.493.856.872.294.477.0
GPT-5-nanoSingle-SkillOpt + Ranker81.362.093.054.568.794.775.7
GPT-5-nanoMulti-SkillOpt + Joint Selection + Ranker82.063.093.756.572.595.577.2
DeepSeek-v4-flashMulti-GEPA + Ranker90.377.296.495.796.196.792.1
DeepSeek-v4-flashMulti-GEPA + Joint Selection + Ranker91.078.296.896.296.597.192.6
DeepSeek-v4-flashSingle-SkillOpt + Ranker90.072.095.894.596.296.990.9
DeepSeek-v4-flashMulti-SkillOpt + Joint Selection + Ranker90.777.096.395.296.797.992.3
Qwen3.5-27BMulti-GEPA + Ranker94.787.678.279.166.480.381.0
Qwen3.5-27BMulti-GEPA + Joint Selection + Ranker95.388.379.380.068.081.782.1
Qwen3.5-27BSingle-SkillOpt + Ranker94.787.577.577.864.576.279.7
Qwen3.5-27BMulti-SkillOpt + Joint Selection + Ranker95.088.579.579.768.078.381.5

We will add these results to the paper.

Response to W2 (Population ablation)

We additionally compared K=1K=1 and K=10K=10 on HMMT and Sudoku using Qwen3-8B. We matched the DevoD_{\mathrm{evo}} and DvalD_{\mathrm{val}} sizes, total evolution rollouts, initialization procedure, operators, UCB strategy and hyperparameters, adaptive operator generation time (tnew=8t_{\mathrm{new}}=8), and inference budget. For the K=1K=1 condition, the same selector selected at most ten skills from the top 30 candidates.

Results are available at https://anonymous.4open.science/r/DIVE_experiments-8BC4/DIVE_single_population.

These results support the contribution of multiple populations. Multiple populations act as structured restarts: each population receives independently bootstrapped experience and develops its own evolution history. This reduces dependence on an unfavorable seed or early revision and explores distinct reasoning, representation, and verification strategies.

A single population can search more deeply along one trajectory, but it remains centered on the same early history and reflection signals, making saturation or overfitting more likely. Multiple populations also produce candidates with more complementary error patterns, which is important because joint selection optimizes marginal coverage rather than standalone accuracy. Finally, populations can be evolved concurrently, providing an additional wall-clock advantage.

We will add the results to the paper.

AuthorsOpenReview ↗

Response to Reviewer SdKK (Part 2)

Response to W3 (Rollout count & cost)

For the Qwen3-8B experiments in Figure 1, we use a small split with ∣Dexp∣=50|D_{\mathrm{exp}}|=50, ∣Dref∣=50|D_{\mathrm{ref}}|=50, and ∣Dval∣=50|D_{\mathrm{val}}|=50.

The figure counts task-solving rollouts on DexpD_{\mathrm{exp}} and DrefD_{\mathrm{ref}}, which constitute the training rollouts. For DIVE, the total is (∣Dexp∣+(Nseed+B)∣Dref∣)K,\left(|D_{\mathrm{exp}}|+(N_{\mathrm{seed}}+B)|D_{\mathrm{ref}}|\right)K, where Nseed=3N_{\mathrm{seed}}=3, BB is the number of evolution steps per population, and K=10K=10. Therefore:

  • HMMT: (50+(3+10)×50)×10=7000(50+(3+10)\times50)\times10=7000 rollouts.
  • Sudoku: (50+(3+6)×50)×10=5000(50+(3+6)\times50)\times10=5000 rollouts. We show the Sudoku curve only up to B=6B=6, as its performance had already saturated.

Some additional model calls: BK=100BK=100 calls for skill revision, 3K=303K=30 calls for initial skill generation, and K=10K=10 calls for adaptive operator generation.

Evaluation on DvalD_{\mathrm{val}} is excluded consistently for all methods: DIVE does not count validation rollouts used to evaluate and select skills, while SFT and GRPO do not count the evaluations used for checkpoint selection. For DIVE’s validation, we only evaluate the top-LL candidates from each population. Thus, the total validation cost is LK∣Dval∣=3×10×50=1500LK|D_{\mathrm{val}}| = 3 \times 10 \times 50 = 1500 rollouts. For skill-set construction, our implementation uses lazy greedy selection with early pruning, using approximately 1000 ranking calls per dataset.

For inference, we run 1 generation call per problem for single-DIVE and 10 generation calls and 1 ranking call per problem for multi-DIVE.

We will add these details to the paper.

Skill-development cost. We agree that it should not be omitted. However, it is a one-time adaptation cost rather than a recurring per-query cost. Moreover, the same skills can be transferred across model scales and families, allowing this upfront cost to be shared across multiple target models. We will mention this in the paper accordingly.

Response to W4 (Uncertainty estimates)

We will replace the main tables with the versions below:

GPT-5-nano

MethodHMMTEq. TheoriesSudokuCrypt.Calc.Futo.Avg.
Zero-shot64.3 ± 3.756.5 ± 2.671.7 ± 2.420.1 ± 2.330.4 ± 2.670.8 ± 2.852.3 ± 1.0
Few-shot63.3 ± 4.459.5 ± 2.477.0 ± 1.725.5 ± 2.426.2 ± 2.369.1 ± 2.553.4 ± 1.1
Experience RAG55.0 ± 5.464.0 ± 2.874.1 ± 2.623.8 ± 2.828.5 ± 3.167.5 ± 2.952.2 ± 1.5
ExpeL60.0 ± 4.461.6 ± 2.277.2 ± 2.026.7 ± 2.437.0 ± 2.275.4 ± 1.756.3 ± 1.0
Direct Skill57.7 ± 4.955.3 ± 2.370.3 ± 2.524.7 ± 2.338.7 ± 3.271.7 ± 2.253.1 ± 1.3
SkillOpt66.3 ± 3.659.1 ± 2.082.6 ± 1.829.9 ± 2.039.8 ± 2.480.4 ± 1.659.7 ± 0.9
MIPROv259.0 ± 4.755.2 ± 2.575.1 ± 2.124.4 ± 2.334.7 ± 3.174.5 ± 2.253.8 ± 1.3
GEPA61.3 ± 4.556.0 ± 2.478.6 ± 2.125.6 ± 2.731.8 ± 2.377.2 ± 2.055.1 ± 1.2
DIVE71.0 ± 3.760.1 ± 1.985.8 ± 1.428.2 ± 2.636.8 ± 2.478.7 ± 1.760.1 ± 1.0

DeepSeek-v4-flash

MethodHMMTEq. TheoriesSudokuCrypt.Calc.Futo.Avg.
Zero-shot75.7 ± 4.043.7 ± 2.277.0 ± 2.670.8 ± 2.584.6 ± 2.091.2 ± 1.673.8 ± 1.0
Few-shot75.0 ± 3.942.2 ± 2.171.0 ± 2.466.1 ± 2.186.4 ± 1.990.3 ± 1.271.8 ± 1.0
Experience RAG72.7 ± 4.851.4 ± 2.474.1 ± 2.668.4 ± 2.982.7 ± 2.784.6 ± 2.172.3 ± 1.4
ExpeL79.7 ± 3.454.2 ± 2.180.4 ± 2.174.1 ± 2.287.6 ± 1.395.0 ± 0.978.5 ± 0.8
Direct Skill80.0 ± 4.153.0 ± 2.277.9 ± 2.371.3 ± 2.485.5 ± 2.096.7 ± 1.177.4 ± 1.1
SkillOpt82.3 ± 3.255.3 ± 1.982.0 ± 1.879.6 ± 1.992.7 ± 1.095.1 ± 0.881.2 ± 0.7
MIPROv275.3 ± 4.348.3 ± 2.378.1 ± 2.274.0 ± 2.688.5 ± 2.093.2 ± 1.076.2 ± 1.1
GEPA74.0 ± 3.850.8 ± 2.278.9 ± 2.277.2 ± 1.891.4 ± 1.693.1 ± 1.177.6 ± 1.0
DIVE90.3 ± 2.249.4 ± 2.184.3 ± 1.777.0 ± 2.193.0 ± 1.397.2 ± 0.781.9 ± 0.7
AuthorsOpenReview ↗

Response to Reviewer SdKK (Part 3)

Qwen3.5-27B

MethodHMMTEq. TheoriesSudokuCrypt.Calc.Futo.Avg.
Zero-shot82.7 ± 4.765.6 ± 1.851.5 ± 2.850.5 ± 2.848.3 ± 3.259.4 ± 3.059.7 ± 1.2
Few-shot81.7 ± 3.766.4 ± 2.251.2 ± 2.750.8 ± 3.347.0 ± 3.158.1 ± 2.859.2 ± 1.2
Experience RAG76.3 ± 5.173.2 ± 2.051.3 ± 3.051.2 ± 3.546.4 ± 3.654.5 ± 3.458.8 ± 1.6
ExpeL86.7 ± 3.375.2 ± 1.655.1 ± 2.554.4 ± 2.953.8 ± 3.064.0 ± 2.164.9 ± 1.0
Direct Skill81.3 ± 4.669.7 ± 2.151.3 ± 2.653.1 ± 3.252.9 ± 3.562.6 ± 2.761.8 ± 1.4
SkillOpt93.0 ± 2.284.2 ± 1.359.0 ± 2.457.4 ± 2.555.6 ± 2.966.9 ± 2.369.4 ± 0.9
MIPROv285.0 ± 3.575.2 ± 1.852.8 ± 2.652.4 ± 3.450.3 ± 3.161.7 ± 2.862.9 ± 1.2
GEPA92.7 ± 2.582.5 ± 1.554.4 ± 2.752.1 ± 3.050.0 ± 3.262.4 ± 2.665.7 ± 1.2
DIVE89.0 ± 3.086.1 ± 1.361.2 ± 2.462.0 ± 2.860.3 ± 2.771.5 ± 2.271.7 ± 1.0

Although SkillOpt performs comparably to DIVE in the single-skill setting, DIVE outperforms it in the multi-skill setting (see Response to W1). We observe that SkillOpt incrementally refines the current skill, producing highly similar skills across independent runs and therefore limited complementarity. This distinction is especially important for test-time scaling: repeatedly sampling from similar prompts can yield highly correlated errors and reproduce the same failure modes. By evolving skills with distinct solution procedures, DIVE increases both the number of candidates and the diversity of reasoning strategies.

Response to W5 (Clarification)

For Equational Theories, the reported metric evaluates only whether the final binary verdict is correct. We will therefore rename this metric verdict accuracy throughout the paper.

Appendix A.6 inadvertently retained the aggregation rule used in an earlier implementation. All reported multi-candidate results, including the HMMT results in Tables 2 and 6, use the same ranker. We will correct the description in Appendix A.6 accordingly.

Reviewer_2EqzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DIVE adapts a frozen language model by turning its task trajectories and verifier feedback into persistent natural-language skills. It evolves multiple skill populations with several editing operators, allocates operator use through UCB, and can generate an additional operator during search. A validation procedure then selects a complementary skill set according to the utility of the complete inference pipeline, which generates and aggregates skill-conditioned answers. Experiments on six mathematical and logical tasks report improvements in single-candidate and ten-candidate settings, together with skill transfer across models, but the evidence leaves important questions about data provenance and which components produce the gains.

Critical strengths
  1. The ten-candidate comparisons provide evidence beyond simply generating more answers. Multi-DIVE improves over Single-DIVE and Multi-GEPA when each method uses ten candidates; the reported average gains over Multi-GEPA are 5.5, 4.3, and 5.0 percentage points across the three main models. This is a meaningful positive result even though the selection procedures still differ.
  2. Joint selection is aligned with the deployed pipeline and has a useful ablation. Selecting skills by their combined answer-generation and aggregation utility is more directly relevant than ranking skills individually. On Qwen3.5-9B, joint selection achieves 56.6% average accuracy versus 54.1% for individual top-M selection.
  3. The method is described through inspectable artifacts. Operator prompts, initialization procedures, algorithm descriptions, and extensive learned skill text make the intervention more concrete than an opaque optimization procedure.
  4. The experiments explore practical deployment settings. The paper includes model-to-model skill transfer alongside prompt optimization, skill optimization, and memory baselines. These results motivate further study of reusable textual adaptation.
Critical weaknesses

W1. The provenance of the HMMT skills is unclear.** Several displayed skills closely resemble specific HMMT February 2025 problems. For example, the instructions on nested tangent semicircles with a tilted inner diameter resemble Geometry problem 4; those on a six-edged rectangular-prism section and a distance calculation resemble Geometry problem 10; and those on counting distinct maxima of cyclic sliding windows resemble Combinatorics problem 8. Please trace these instructions to their training examples and edit histories, identify the experiment that produced them, and clarify whether evaluation examples entered reflection or selection.

W2. The contribution of population-based evolution could be separated more clearly from the final selector. Multi-DIVE uses joint selection, while Multi-GEPA [1] selects individually strong prompts, and the experiments omit a fixed-budget comparison of single- and multi-population search. Please apply the same selector to both candidate pools and hold total search and inference budgets fixed when testing the population structure.

W3. The comparison with closely related methods could be strengthened. GEPA [1] already maintains a branching candidate pool, and Promptbreeder [2] evolves mutation prompts. Please clarify DIVE's contribution relative to these methods and its relationship to the persistent-context approach in ACE [3].

W4. The cross-model transfer gain is modest in one reported setting. Comparing the reported transfer result for DeepSeek-v4-flash with the ten-candidate Zero-shot+R baseline gives an average gain of only 1.8 percentage points. Please present these results together and clarify the task-specific aggregation rules to make the contribution of skill transfer clear.

W5. Aggregation and candidate-selection rules need clarification. HMMT is described as using both majority voting and model ranking, the procedure can stop with fewer than ten skills despite a ten-candidate evaluation, and the single-candidate selection rule is unspecified. Please clarify the rules used in the experiments and reconcile duplicated results using experiment identifiers.

W6. The scope of the evaluation and theoretical claims could be stated more precisely. Equational Theories scores final verdicts without checking proofs or counterexamples, and the population analysis relies on idealized assumptions acknowledged in the manuscript. Please label the metric as verdict accuracy and keep the theoretical claims tied to those assumptions.

Questions for authors

Where did the HMMT-specific instructions originate, and does the advantage persist when DIVE and the comparison method use the same selector? These are the two questions that matter most to my decision (W1–W2): a satisfactory provenance account and a comparison supporting DIVE's contribution will move me toward acceptance (3/4).

  1. Does HMMT joint selection optimize majority-vote utility or ranker utility, and how are ten candidates allocated when selection returns fewer than ten skills?
  2. Which data split is used for the hyperparameter analyses?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 2Eqz (Part 1)

We sincerely thank the reviewer for their careful review and valuable feedback.

Response to W1

Thank you for identifying this issue. Our audit revealed an implementation error specific to the GPT-5-nano HMMT experiment: the command-line argument for DrefD_{\mathrm{ref}} is inadvertently set as the test split. Consequently, the HMMT test examples entered skill reflection. We apologize for this error.

We reran the entire GPT-5-nano HMMT experiment from scratch. The corrected results are:

  • Table 1, HMMT: 71.0
  • Table 2, average: 76.0 for Single-DIVE + Ranker and 81.2 for Multi-DIVE + Ranker
  • Table 6, HMMT: 80.7 for Single-DIVE + Ranker and 81.7 for Multi-DIVE + Ranker

We also release the regenerated per-population skill evoluation sequence for further review: https://anonymous.4open.science/r/DIVE_experiments-8BC4/HMMT_gpt5nano_rerun.

We will replace the affected values and skill examples throughout the paper.

Response to W2

We conducted the two requested controlled comparisons.

1. Applying the same joint selector to GEPA. We applied DIVE’s selector to the original Multi-GEPA candidate pools across five runs.

MethodGPT-5-nanoDeepSeek-v4-flashQwen3.5-27B
Multi-GEPA + Ranker75.992.181.0
Multi-GEPA + Joint Selection + Ranker77.092.682.1
Multi-DIVE + Joint Selection + Ranker81.296.486.0

We can see that the final selector contributes to performance but does not account for most of DIVE’s advantage. Qualitative inspection also shows that prompts from independent GEPA runs are often highly similar, providing relatively limited complementary coverage for the joint selector to exploit.

The complete task-level results are:

ModelMethodHMMTEq. TheoriesSudokuCrypt.Calc.Futo.Avg.
GPT-5-nanoMulti-GEPA + Ranker81.062.893.155.769.293.475.9
GPT-5-nanoMulti-GEPA + Joint Selection + Ranker81.763.493.856.872.294.477.0
DeepSeek-v4-flashMulti-GEPA + Ranker90.377.296.495.796.196.792.1
DeepSeek-v4-flashMulti-GEPA + Joint Selection + Ranker91.078.296.896.296.597.192.6
Qwen3.5-27BMulti-GEPA + Ranker94.787.678.279.166.480.381.0
Qwen3.5-27BMulti-GEPA + Joint Selection + Ranker95.388.379.380.068.081.782.1

2. Fixed-budget comparison of single- and multi-population evolution. We additionally compared K=1K=1 and K=10K=10 on HMMT and Sudoku using Qwen3-8B. We matched the DevoD_{\mathrm{evo}} and DvalD_{\mathrm{val}} sizes, total evolution rollouts, initialization procedure, operators, UCB strategy and hyperparameters, adaptive operator generation time (tnew=8t_{\mathrm{new}}=8), and inference budget. For the K=1K=1 condition, the same selector selected at most ten skills from the top 30 candidates.

Results are available at https://anonymous.4open.science/r/DIVE_experiments-8BC4/DIVE_single_population.

These results support the contribution of multiple populations. Multiple populations act as structured restarts: each population receives independently bootstrapped experience and develops its own evolution history. This reduces dependence on an unfavorable seed or early revision and explores distinct reasoning, representation, and verification strategies.

A single population can search more deeply along one trajectory, but it remains centered on the same early history and reflection signals, making saturation or overfitting more likely. Multiple populations also produce candidates with more complementary error patterns, which is important because joint selection optimizes marginal coverage rather than standalone accuracy. Finally, populations can be evolved concurrently, providing an additional wall-clock advantage.

We will add these results to the paper.

Response to W3

The key distinction is that DIVE preserves diversity from the search process through test-time inference and jointly designs its search and selection procedures for self-improvementment.

GEPA maintains a branching candidate pool. However, its standard procedure ultimately selects a single prompt with the best validation performance. In contrast, DIVE evolves multiple independent populations and jointly selects a set of skills from the candidate pools of different populations. This distinction is particularly important for test-time scaling. The resulting errors sampling from the same prompt may remain highly correlated: when every rollout follows the same reasoning strategy, additional samples can reproduce the same failure mode. By evolving skills with distinct solution procedures, DIVE uses additional rollouts to increase not only the number of samples but also the diversity of strategies. Joint selection further retains skills that correct errors made by the current skill set.

AuthorsOpenReview ↗

Response to Reviewer 2Eqz (Part 2)

Promptbreeder evolves mutation prompts. However, it still aims to identify a single best task prompt. DIVE instead focuses on constructing a test-time skill set. In addition to evolving skills, it adaptively allocates the search budget across operators using UCB, and generates new operators conditioned on the evolution history. We observe that smaller models are particularly sensitive to the choice of evolution operators. DIVE therefore serve not only to expand the search space, but also to concentrate a limited search budget on mutation directions that the underlying model can effectively exploit.

ACE maintains a persistent playbook and uses incremental grow-and-refine updates to accumulate, organize, and compress experience. We view ACE and DIVE as complementary. ACE primarily studies how experience can be retained and organized within a context (in-context knowledge management). DIVE studies how multiple distinct contexts can be evolved and how a complementary set can be selected at inference time (search and selection across contexts). In principle, ACE’s curator could manage the internal content of each DIVE skill, while DIVE could evolve and select among multiple ACE-style persistent contexts.

We will modify the related work accordingly.

Response to W4

We agree that transfer should be compared directly with multi-candidate baselines.

Target modelZero-shot@1SC@10Zero-shot+R@10Multi-GEPA+R@10Multi-DIVE+R@10 (from Qwen3.5-9B)Multi-DIVE+R@10 (self)
Qwen3.5-27B59.769.071.981.081.286.0
DeepSeek-v4-flash73.886.888.792.190.596.4

For transfer within the Qwen family, the transferred skills outperform Zero-shot+R on all six tasks, yielding an average improvement of 9.3 points. Without any optimization on the target model, transferred DIVE also performs comparably to Multi-GEPA optimized directly on Qwen3.5-27B (81.2% versus 81.0%).

The cross-family transfer to DeepSeek-v4-flash produces a more modest average improvement of 1.8 points over Zero-shot+R. The benefit is concentrated on Equational Theories, where transfer improves accuracy by 17.8 points. On tasks for which DeepSeek-v4-flash already exceeds 92% after ranking, skills distilled from a substantially weaker source model provide little additional benefit and can be less well aligned with the target model. In this saturated regime, evolving skills directly with the target model remains more effective.

Aggregation rules. SC@10 applies majority voting. For Zero-shot+R, Multi-GEPA+R, and both Multi-DIVE+R variants, we use the same ranker selects the final answer (including HMMT).

We will revise the paper to characterize this result as strong within-family transfer but task-dependent cross-family transfer.

Response to W5

Appendix A.6 inadvertently retained the aggregation rule used in an earlier implementation. All reported multi-candidate results, including the HMMT results in Tables 2 and 6, use the same ranker. We will correct the description in Appendix A.6 accordingly.

In all reported multi-candidate experiments, DIVE selected exactly M=K=10M=K=10 skills; therefore, each candidate solution corresponds to a single skill.

For the single-skill setting, when K=1K=1, we select the skill with the highest validation accuracy. When K>1K>1, joint selection begins with the candidate having the highest standalone validation accuracy across all populations.

We will include these details in the paper.

Response to W6

We agree with both clarifications. For Equational Theories, the reported metric evaluates only whether the final binary verdict is correct. We will therefore rename this metric verdict accuracy throughout the paper. We will also revise the theoretical discussion to make its conditional nature explicit.

Response to Q1

We use ranker utility for HMMT, and in all reported multi-candidate experiments, DIVE selected exactly 10 skills; therefore, each candidate solution corresponds to a single skill.

Response Q2

We use DvalD_{val} for all the hyperparameter analyses.

AuthorsOpenReview ↗

Follow-up on Author Response

Dear Reviewer,

Thank you again for your constructive feedback. We have posted our responses to your comments and would greatly appreciate any further thoughts you may have before the discussion period ends.

We would be happy to clarify any remaining concerns.

Best regards,
Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.