acceptodds
Withdrawn from ICLR 2027

Counterfactual Evolution Credit: Causal Selection for Self-Evolving Agent Skills

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Self-evolving language-model agents convert execution experience into persistent updates to reusable skills. A central challenge is determining which candidate skills to retain. Existing selection strategies typically assign credit based on episode-level outcomes or invocation audits, conflating skill availability, invocation, behavioral influence, and causal contribution to task success. As a result, a candidate may be retained because it co-occurs with easy instances or discarded because unrelated execution noise masks its utility. We introduce Counterfactual Evolution Credit (CEC), a causal selection framework that estimates the incremental contribution of structured skill artifacts through matched execution interventions. CEC compares executions using a candidate skill against counterfactual runs in which the skill is removed, replaced by its parent, blocked from invocation, or modified at a specific functional module. These comparisons control for the task instance, environment state, execution budget, and, when possible, randomness. Repeated paired evaluations with fresh agents quantify residual uncertainty, while sequential allocation assigns additional evaluations to promising candidates whose effects remain unresolved. CEC also supports hierarchical attribution at the levels of whole skills, triggers, procedures, and recovery logic, allowing beneficial revisions to be retained and harmful components to be localized. We evaluate CEC under controlled failure modes involving unused, behaviorally inert, and spuriously correlated skills. The evaluation includes matched-budget comparisons, cross-agent reproduction, and ablations of replay control, confidence estimation, attribution granularity, and budget allocation. CEC supports skill selection under stochastic execution trajectories and limited evaluation budgets by separating causal contribution from invocation and episode-level correlation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Counterfactual Skill Attribution, Causal Skill Selection, Self-Evolving Agents

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_NvU2OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Counterfactual Evolution Credit (CEC), a framework for selecting skills in self-evolving LLM agents. It uses counterfactual comparisons to estimate each skill’s contribution to task performance and guide skill selection. Experiments on three benchmarks show improvements over existing baselines.

Critical strengths
  • The paper addresses an important problem in skill selection: whether a skill actually contributes to task success, rather than merely being invoked during a successful execution.
  • The reported results show modest improvements over the baselines.
Critical weaknesses
  • The method compares executions after removing a skill, replacing it with its parent, blocking its invocation, or modifying an individual module. The technical novelty appears limited, as the method mainly combines several straightforward interventions.
  • TI and FS are defined as the proportions of selected skills with positive and non-positive test effects, respectively. By these definitions, they should sum to 100%. However, the values reported for CEC in Table 1 are 74.8% + 8.9% = 83.7%. The authors should clarify this difference.
  • The three-round evolution setting provides limited evidence for the claimed evolutionary stability. Longer-term evaluations tracking performance improvements and regressions across rounds would strengthen this claim.
  • The experimental evaluation is limited to three benchmarks. Additional evaluations on established agent benchmarks, such as WebArena and AppWorld, would strengthen the evidence for the generalizability of CEC.
  • Figure 2 is very difficult to read. The text is too small to be clearly legible, which makes the framework difficult to follow.
Questions for authors

See weakness 1-4.

Reviewer_qsJpOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Counterfactual Evolution Credit (CEC) for selecting reusable agent skills. It compares candidate executions with no-skill, parent-skill, blocked-invocation, and module-replacement conditions, then uses confidence bounds and sequential evaluation to guide selection. Experiments cover three task suites, deceptive skills, and cross-agent transfer.

Critical strengths

The distinction between skill invocation and actual benefit is relevant, and the controlled deceptive-skill experiments address a practical selection problem.

Critical weaknesses
  1. Limited insight. The method combines matched comparisons, module ablations, confidence bounds, and sequential allocation. I do not find a sufficiently compelling new insight beyond applying these familiar ideas to skill selection.

  2. Insufficient overall quality. Important experimental details are internally inconsistent. Under the definitions in Appendix B.2, true improvement and false selection should sum to 100%, yet Table 1 reports 74.8% and 8.9% for CEC. Appendix D also documents different execution prompts across methods, while D.9.5 states that uniform allocation leaves prompts unchanged. These discrepancies make the results difficult to interpret and leave unclear whether the comparisons isolate the selection strategy. The paper needs substantial clarification and correction.

Questions for authors

see above

Reviewer_c4R8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Counterfactual Evolution Credit to assess which skill updates a self-evolving agent should keep. Under matched tasks and execution conditions, it compares candidate skills. The comparisons include no skill, the parent skill, a version with the invoked content blocked, and versions with one module replaced. These comparisons estimate the incremental contributions of the skill and its modules. The selection process combines sequential evaluation with confidence bounds on performance differences. It gives more budget to candidates that still look promising but whose value remains uncertain. The paper evaluates CEC on SkillsBench-CEC, SpreadsheetBench-CEC, and SearchQA-CEC, reporting improvements in test success rates and selection reliability. It also tests the method through experiments on deceptive skills, module localization, and different executors.

Critical strengths

What I find most interesting is the distinction between a useful skill and an update worth keeping. Comparing a candidate with no skill addresses the first question. Comparing it with the parent skill addresses the second. An update can outperform a weak parent while still doing worse than using no skill at all. CEC considers both comparisons, which makes its selection criteria better suited to continual evolution. It also avoids crediting the current update for abilities inherited from the parent. I also like how the deceptive-skill experiments are designed in the main paper. They cover skills that are never invoked, skills that are invoked without changing behavior, and skills that merely coincide with success on easy tasks. Each case reflects a different attribution error. The case where a skill is invoked but has no actual effect is especially useful. It directly tests the limits of invocation auditing. These experiments help readers understand why selection methods make mistakes.

Critical weaknesses

My main concern is that CEC may favor updates with broad benefits, while being less favorable to those that help only in specific situations. Reusable skills often have clear conditions for when they apply. A skill may add no benefit on most tasks and still be worth keeping. For example, consider a recovery update that only helps with rare spreadsheet refresh failures. It should leave execution on normal tasks unchanged. But when the relevant failure occurs, it can rescue a task that would otherwise fail. If validation tasks rarely cover this case, its overall gain will be diluted. Within a limited budget, it may then struggle to pass the lower confidence bound acceptance threshold. The paper already defines skill triggers and task-conditional effects. However final selection uses aggregated gains, while sequential allocation mainly decides which candidate to evaluate further. For these specialized skills, the issue is not just getting more evaluations. It also matters whether they are tested in situations where they can actually help. The synthetic uncalled skills are designed to contribute nothing. Rejecting them does not show that CEC can distinguish truly useless skills from those that have had too few relevant opportunities. This makes me worry that CEC may reduce false acceptances while also missing narrow but useful updates that address specific failures. Overall success rates may improve, and a larger fraction of selected skills may be useful. Neither result alone shows whether these additional capabilities are retained. I would like to see tests with candidates that trigger rarely but offer clear benefits on relevant tasks.

Questions for authors
  1. Are validation tasks chosen based on when each skill applies? For updates that only help with rare failures, how does CEC check whether those cases have been evaluated enough?

  2. Could you also test useful candidates that trigger rarely, and see whether CEC retains them? This would go beyond testing rarely invoked skills that were designed to be useless.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.