- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Semantic-Scope Projected Evolution (SSPE) to alleviate "procedural catastrophic forgetting" in language agents during continuous skill evolution. The core idea is to transfer the "projection" concept from OGD from parameter space to behavior space: first generate a free skill revision for the current task, then predict its potential impact on historical capabilities, and use selective historical auditing to detect actual performance degradation. If conflicts are detected, the skill is rewritten using feedback combining "current gain + historical loss," iteratively until a compatible update is achieved or the update is abandoned. The experiments included controlled synthetic streams and real heterogeneous task flows (SpreadsheetBench, SearchQA, BFCL, DocVQA). The results showed that SSPE outperformed SkillGrad and SkillOpt in terms of overall cross-domain capabilities and cross-model transfer, and historical capability evidence was crucial for the performance improvement.
Critical strengths
-
The problem is well motivated and practically relevant. The paper studies procedural forgetting when a shared natural-language skill is repeatedly revised across heterogeneous task streams, which is more realistic than single-domain prompt optimization.
-
Transferring the operational principle of OGD into behavior space is conceptually interesting. SSPE preserves the functional idea of protecting prior behavior while retaining useful current-task progress.
-
SSPE repairs harmful updates. Historical regressions become constructive feedback for a new skill revision, which is a meaningful step beyond validation-only gating.
-
The experiment covered truly heterogeneous Agent capabilities. Spreadsheet, search, function calls, and document question-answering have different interfaces and output constraints.
Critical weaknesses
-
The framework in Figure 1 feels like AI-style. The authors are suggested to manually improve the quality of the figures.
-
“Semantic projection” is closer to LLM-based iterative repair than to a formal projection operator. The core operator is an LLM rewrite conditioned on conflict evidence, without a well-defined geometry or verifiable projection property. Thus, the OGD connection is mainly conceptual.
-
Fairness concern regarding information access. SkillGrad receives no verification bank, SkillOpt accesses only the current bank, while SSPE can inspect selected historical banks. Part of the gain may therefore come from additional historical information and execution budget rather than semantic projection alone.
-
The same powerful model simultaneously assumes the roles of executor and evolution, which may enhance the effectiveness of the method. GPT-5.4 not only performs tasks but also participates in diagnosis, risk prediction, proposal generation, and semantic repair. Therefore, SSPE actually highly relies on the model's own semantic reasoning and self-repair capabilities. It is currently unclear whether the method remains effective when the evolution model is weaker or decoupled from the executor.
-
The authors did not examine whether the skill itself undergoes continuous expansion or generates internal conflicts. The shared natural-language skill is constantly rewritten and new programs are accumulated. In the long-term self-evolution process, this might be a very important issue.
Questions for authors
-
SSPE is not consistently best on SpreadsheetBench and SearchQA. Could the authors analyze which types of historical conflicts cannot be repaired and how false negatives in risk routing affect final forgetting?
-
Could the authors add a history-aware reject-only baseline that uses exactly the same historical audits as SSPE but simply rejects an update when interference is detected, without semantic repair? This would directly isolate the contribution of semantic projection itself.