acceptodds
Under review as a conference paper at ICLR 2027

From a Prompt to Repertoires: Evolving Functional REpertoires Enable LLM Continual Learning

Abstract

Continual learning remains challenging for large language models, which must enable models to acquire new skills and knowledge without degrading existing capabilities. Existing approaches typically address this challenge by carefully designing how model parameters are updated. In contrast, prompt optimization avoids costly parameter updates while achieving competitive or even superior performance to reinforcement learning methods such as GRPO on individual knowledge-intensive and reasoning tasks. This raises a natural question: Our analysis shows that, under sequential task adaptation, it suffers from catastrophic forgetting, while optimized prompts accumulate rules that overfit to local task distributions. To address these limitations, we propose (EFRE), which replaces a single prompt with a repertoire of functions that evolves as new tasks arrive: compatible updates refine existing functions, while conflicting updates trigger the emergence of new ones. On a three-task continual-learning stream, EFRE achieves a final average performance 7.50 percentage points higher than GRPO. Moreover, after adaptation to the Bio task, its performance on FinQA decreases by only 1.56 percentage points, compared with 25.10 percentage points for the base prompt optimization method. We further instantiate EFRE in a minimal agent system and observe consistent improvements across different backbone models. Overall, these results demonstrate EFRE's strong performance in continual learning for large language models and highlight its substantial potential for continual learning in advanced agent systems.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Continual learning, Large language models, Prompt optimization, Agentic continual learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4oo7OpenReview ↗
Rating
3(+1)
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies continual learning through prompt optimization with frozen language models. It proposes Evolving Functional Repertoires (EFRE), which replaces a single prompt with an evolving repertoire of prompt-based functions. Historical consistency checks guide functional refinement, reuse, and emergence, while a selector routes inputs to suitable functions. Experiments on the ToolUse → FinQA → SciKE-Bio sequence report improved adaptation and retention over the evaluated prompt-optimization baselines, with additional gains in single-task optimization and minimal-agent settings.

Critical strengths
  1. By combining historical consistency checks with the maintenance of prompt-based functions, the paper provides a concrete approach to continual adaptation without updating model parameters.
  2. EFRE† largely retains its FinQA performance after SciKE-Bio adaptation, with substantially less forgetting than GEPA; default EFRE also improves final performance over the prompt-optimization baselines. Table 2 compares history retention settings, historical consistency tolerances, and function-selection strategies, while Figure 2 and Table 1 show cross-task reuse patterns, helping characterize the system’s behavior.
Critical weaknesses
  1. Under Equation 7 and Appendix Algorithm 2, ε=1 disables effective historical constraints and, with single-function initialization, dynamic expansion. Always selecting the first function likewise does not enforce single-function training. Neither the main text nor the appendix compares against a single function with matched historical checks, prompt snapshots using the same selector, or a fixed-size repertoire. The incremental benefit of conflict-driven expansion therefore remains unclear.
  2. The main cross-task experiment uses only ToolUse → FinQA → SciKE-Bio, without alternative orders, longer task streams, or task revisits. Appendix D.2 acknowledges incomplete historical coverage and potentially accumulating performance losses, but these limitations lack corresponding long-term evaluation. Aggregate rewards in the agent experiments also do not directly establish retention of earlier capabilities.
Questions for authors

Please refer to the two points in Critical Weaknesses. I will raise my score if the authors adequately address these concerns through additional experiments and clarification.

Reviewer_THS9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes EFRE, a non-parametric continual learning framework that maintains an evolving repertoire of prompt-based functions. Through historical consistency checks, EFRE determines whether to refine existing functions, reuse other functions, or create new ones to reduce interference with previously learned capabilities. Experiments on sequential learning tasks, single-task benchmarks, and interactive agent environments demonstrate improved performance and reduced catastrophic forgetting compared to several baselines.

Critical strengths
  • The topic is meaningful, as it explores how LLMs can keep acquiring new capabilities without parameter updates while avoiding catastrophic forgetting.
  • The proposed framework is intuitive and well-structured. Historical consistency checks provide a reasonable mechanism for balancing new-task adaptation and retention of existing capabilities.
  • The experiments cover multiple backbone models, various baselines, ablation studies, and interactive agent environments, demonstrating promising continual learning performance.
Critical weaknesses
  • The core ideas of maintaining reusable functions, refining them based on feedback, and dynamically expanding a function library have been explored in prior work. For example, L2P [1] investigates prompt pools for continual learning, while GSE [2] studies skill compatibility, consolidation, and replay-driven verification. The paper should better clarify what fundamentally distinguishes EFRE's consistency-guided function evolution from these existing approaches.

  • Missing baselines to isolate the core contributions. Comparisons with GEPA, ACE, and GRPO do not fully establish the necessity of dynamic function evolution. For example, maintaining separate prompts for different tasks may already reduce forgetting through task isolation. Comparisons with task-specific prompts, a multi-prompt system with routing, and a single prompt with historical consistency checks would help clarify whether the improvements come from dynamic refinement and reuse or simply from increased prompt capacity.

  • Unclear scalability of function selection in long-term continual learning. As learning progresses, the function repertoire may grow substantially, potentially introducing overlapping or semantically similar functions. Can the LLM selector consistently identify the appropriate function as the repertoire expands? Although the paper compares several function selection strategies, the main experiments involve only three sequential tasks, providing limited evidence of routing reliability at scale. Further analysis of how repertoire size affects selection accuracy, overall performance, and inference cost would strengthen the evaluation.

[1] Learning to Prompt for Continual Learning. [2] Learning Globally Reusable Skills for Coding Agents.

Questions for authors

See weaknesses.

Reviewer_LqhAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies continual learning for a frozen LLM through prompt optimization. It argues that sequential prompt optimization forgets because its acceptance rule checks only the current task, and proposes EFRE, a growing repertoire of prompt functions: a revision replaces a function only if it keeps most of that function's score on its recent examples, otherwise EFRE revises another function or creates a new one, and an LLM selector routes each test input. Experiments on a three-task stream, single-task benchmarks and agent environments report a final average above GRPO and much less forgetting than GEPA.

Critical strengths
  • Relevant problem. The paper documents severe forgetting when prompt optimization is applied task after task, and its update rules are specified precisely enough to reproduce.
  • Low adaptation cost. With Qwen3-8B alone, EFRE matches the strongest parameter-updating baseline at roughly one seventh of its GPU time.
Critical weaknesses

1. Unclear source of the stability gain. Because scores are nonnegative, Eq. 7 always holds at ϵ=1\epsilon = 1, so the repertoire should stay a single prompt; yet this setting already closes about 80% of the stability gap to GEPA (Table 2, after Bio):

SettingRepertoirePlasticityStability
GEPAsingle prompt+4.59−11.82
EFRE, ϵ=1\epsilon = 1single function+13.04−2.57
EFRE, ϵ=0.1\epsilon = 0.1 (default)growing+19.52−0.31

Since the ablation refines with GPT-5.6-Luna while the GEPA refinement model is not specified (Appendix C.3), the gain over GEPA is not yet attributed to the repertoire.

2. Unmatched headline comparison. The 7.50-point margin over GRPO comes from EFRE†, where GPT-5.6-Luna both refines the prompts and selects a function for every test input, a cost not counted in Table 10; with Qwen3-8B alone, EFRE is on par with SDPO (58.75 vs. 58.74, Table 7). The per-task prompt baseline with the same selector is not run, and methods that grow a library of reusable workflows or reasoning strategies [1, 2] are not discussed.

Typos. L296 "defualt" → "default"; L336 "+20.58=" → "+20.58"; L408 "Equation. 2 and 3" → "Equations 2 and 3"; L653 "provided by (Harrington et al., 2026)" → "provided by Harrington et al. (2026)".

Questions for authors
  1. Please confirm that the ϵ=1\epsilon = 1 setting keeps a single function, and report GEPA rerun with the refinement model and prompt of EFRE†, together with per-stage counts of refinement, reuse and emergence.
  2. The per-task prompt baseline, EFRE† with Qwen3-8B as the selector, and standard deviations in Table 7 should be reported, and the relation to expansion-based continual learning with routed modules [3] should be discussed.

References

[1] Agent Workflow Memory. ICML 2025.
[2] ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. ICLR 2026.
[3] Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning. CVPR 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.