acceptodds
Under review as a conference paper at ICLR 2027

Semantic Projection for Continual Self-Evolution of Language Agents

Abstract

Language-model agents increasingly rely on persistent natural-language skills to adapt beyond their frozen model parameters. When a shared skill is repeatedly revised from a non-stationary, heterogeneous task stream, however, improvements for new tasks can overwrite procedures needed for earlier ones. In continual learning, Orthogonal Gradient Descent (OGD) addresses analogous interference by projecting a new-task gradient onto a subspace that locally preserves prior predictions. Natural-language skill revisions, however, have neither gradients nor a canonical vector space in which such a projection can be performed. We introduce *Semantic-Scope Projected Evolution* (SSPE), which transfers the functional principle of gradient projection from parameter space to behavior space. SSPE treats an unconstrained skill revision as a proposed update, identifies acquired capabilities with which it may interfere, and uses the observed gains and regressions to construct a compatible revision rather than merely rejecting the update. This enables one shared skill to evolve across latent and recurring task contexts without exposing semantic domain identities to the evolution model. Across controlled synthetic streams and heterogeneous real-agent benchmarks, SSPE improves final cross-domain competence and mitigates forgetting relative to strong skill-evolution baselines. The evolved skill also retains the strongest average performance after transfer to a different executor model. These results establish semantic projection as a promising principle for stable and adaptive self-evolution of language agents.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
self-evolving agent, continual learning, test-time learning, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2MbxOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Semantic-Scope Projected Evolution (SSPE) to alleviate "procedural catastrophic forgetting" in language agents during continuous skill evolution. The core idea is to transfer the "projection" concept from OGD from parameter space to behavior space: first generate a free skill revision for the current task, then predict its potential impact on historical capabilities, and use selective historical auditing to detect actual performance degradation. If conflicts are detected, the skill is rewritten using feedback combining "current gain + historical loss," iteratively until a compatible update is achieved or the update is abandoned. The experiments included controlled synthetic streams and real heterogeneous task flows (SpreadsheetBench, SearchQA, BFCL, DocVQA). The results showed that SSPE outperformed SkillGrad and SkillOpt in terms of overall cross-domain capabilities and cross-model transfer, and historical capability evidence was crucial for the performance improvement.

Critical strengths
  1. The problem is well motivated and practically relevant. The paper studies procedural forgetting when a shared natural-language skill is repeatedly revised across heterogeneous task streams, which is more realistic than single-domain prompt optimization.

  2. Transferring the operational principle of OGD into behavior space is conceptually interesting. SSPE preserves the functional idea of protecting prior behavior while retaining useful current-task progress.

  3. SSPE repairs harmful updates. Historical regressions become constructive feedback for a new skill revision, which is a meaningful step beyond validation-only gating.

  4. The experiment covered truly heterogeneous Agent capabilities. Spreadsheet, search, function calls, and document question-answering have different interfaces and output constraints.

Critical weaknesses
  1. The framework in Figure 1 feels like AI-style. The authors are suggested to manually improve the quality of the figures.

  2. “Semantic projection” is closer to LLM-based iterative repair than to a formal projection operator. The core operator ∏sem\prod_{sem} is an LLM rewrite conditioned on conflict evidence, without a well-defined geometry or verifiable projection property. Thus, the OGD connection is mainly conceptual.

  3. Fairness concern regarding information access. SkillGrad receives no verification bank, SkillOpt accesses only the current bank, while SSPE can inspect selected historical banks. Part of the gain may therefore come from additional historical information and execution budget rather than semantic projection alone.

  4. The same powerful model simultaneously assumes the roles of executor and evolution, which may enhance the effectiveness of the method. GPT-5.4 not only performs tasks but also participates in diagnosis, risk prediction, proposal generation, and semantic repair. Therefore, SSPE actually highly relies on the model's own semantic reasoning and self-repair capabilities. It is currently unclear whether the method remains effective when the evolution model is weaker or decoupled from the executor.

  5. The authors did not examine whether the skill itself undergoes continuous expansion or generates internal conflicts. The shared natural-language skill is constantly rewritten and new programs are accumulated. In the long-term self-evolution process, this might be a very important issue.

Questions for authors
  1. SSPE is not consistently best on SpreadsheetBench and SearchQA. Could the authors analyze which types of historical conflicts cannot be repaired and how false negatives in risk routing affect final forgetting?

  2. Could the authors add a history-aware reject-only baseline that uses exactly the same historical audits as SSPE but simply rejects an update when interference is detected, without semantic repair? This would directly isolate the contribution of semantic projection itself.

Reviewer_cgD3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the challenge that updating shared skills across heterogeneous tasks leads to procedural forgetting. To solve that, it proposes Semantic-Scope Projected Evolution (SSPE) inspired by Orthogonal Gradient Descent (OGD). It checks and revises skill updates to support new tasks while retaining earlier capabilities, rather than simply accepting or rejecting them. Experiments show higher average accuracy and a lower harmful-update rate than baselines.

Critical strengths

The method proposes a practical way to learn new skills while preserving earlier capabilities.

The experiments cover four heterogeneous benchmarks, historical-evidence ablations, and cross-model skill transfer, providing useful evidence beyond the main performance comparisons.

Critical weaknesses
  1. Unexpected performance on SpreadsheetBench.

All skill-based methods perform worse than the no-skill executor on this benchmark. Since all evolving methods start from the same Static skill, this also raises questions about the suitability of the shared initialization.

  1. Limited gains on SearchQA.

In Sections 5.2 and 5.3, SSPE does not achieve the best SearchQA performance among the skill-based methods. It even has the lowest SearchQA accuracy in the cross-model transfer experiment in Section 5.3. Does this suggest that SSPE may struggle to preserve previously acquired capabilities?

  1. Limited evaluation of forgetting and task-order effects.

The real-task experiments do not track performance on each domain after learning subsequent domains. This makes it difficult to distinguish reduced forgetting from better acquisition of new capabilities. The experiments also use a fixed task order, as acknowledged in the Limitations section, leaving sensitivity to task order unclear. I recommend reporting performance after each domain and testing alternative task orders.

  1. Missing ablation of semantic repair.

The ablations do not isolate the contribution of repair from that of historical evidence. Section 5.4 shows the importance of historical evidence, but does not establish how much repair itself contributes. A comparison using the same historical audits but rejecting conflicting proposals without repair might help clarify this.

Questions for authors

The mathematical formulation is somewhat lengthy and complex, which makes it harder to follow. I suggest shortening and simplifying this part to improve readability.

Reviewer_U6oBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SSPE to mitigate cross-task interference generated when frozen language models continuously update shared natural language skills. The method first generates skill modification proposals based on the current task, then selectively audits historical capabilities, using observed gains and setbacks to resolve conflicts; if a candidate fails to meet acceptance criteria, the original skill is retained. The authors evaluate the method through synthetic simulations, real-world task streams across four domains, and cross-actor transfer. The results show that SSPE reduces forgetting in simulations and achieves higher final cross-domain average accuracy in the tested real-world task streams and transfer settings.

Critical strengths
  1. This paper focuses on cross-task interference caused by shared skill updates under a frozen model, which has practical implications for the continuous deployment of agents.
  2. SSPE combines selective historical auditing, conflict feedback remediation, and failure rollback to determine whether candidate skills are acceptable based on actual execution results; the process is clear and highly readable.
Critical weaknesses
  1. SSPE simultaneously incorporates historical information, additional auditing, and remediation steps; existing ablation studies cannot distinguish the roles of these factors. Without a control group that “retains the same historical audits but only rejects without remediation,” it is not yet possible to attribute the performance improvement to semantic remediation alone.
  2. The experiments primarily report final accuracy but lack historical task performance data for each stage, making it impossible to distinguish between the acquisition of stronger new capabilities and reduced forgetting. Furthermore, the real-world task stream follows a fixed order and provides anonymous domain boundaries; validation for scenarios involving unbounded, interleaved, or recurring old capabilities remains lacking.
  3. Since the experiment lacked independent replicates and an analysis of experimental stability, it is difficult to determine whether the observed advantage is stable.
Questions for authors
  1. How did the authors evaluate the impact of task-specific gains and random execution noise on acceptance reliability? Were there any independent reviews or repeated executions of the results?
  2. Could you please indicate the specific locations in the supplementary materials for the confirmation and merging rules for capability capsules, the risk assessment format, the prompts for each role, and the handling of SearchQA secondary scoring?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.