- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes Skill-V, a skill library that evolves by both adding new skills and revising existing ones. Each skill is a versioned contract with a semantic intent and an executable rubric (applicability conditions plus weighted criteria). Failures add skills; disagreements between the rubric verdict and the environment outcome trigger boundary revisions, which are accepted only if they preserve the protected semantic constraints and do not regress on replay evidence. An applicability filter drops confidently inapplicable skills at inference. Results: 95.3% success on ALFWorld, 85.9% on WebShop, with a smaller library than growth-only baselines.
Critical strengths
- The problem is well-motivated. Existing skill libraries grow by adding skills from failures; checking and revising stored skills is a real gap.
- Separating semantic identity from the operational boundary is the key idea. It lets the boundary be revised without changing what the skill means, and it gives a traceable audit record.
- The evidence gate and the applicability filter target two distinct failure modes: keeping wrong knowledge and invoking wrong knowledge.
Critical weaknesses
-
The terms reliable / testable / verifiable / falsifiable are used interchangeably and never defined. The abstract alone refers to the same skill representation as "a verifiable self-evolving skill library," "testable," and "falsifiable contracts" within three sentences. These are not equivalent: "reliable" is a system-level property; "testable" means a claim can be checked against observations; "falsifiable" (Popper) means a counterexample can refute it; "verifiable" is the strongest and implies a decision procedure or proof. The method only checks criteria against an environment outcome, which is "testable", not "verifiable". The title uses "verifiable" without a definition.
-
The environment outcome is the only independent check. The environment outcome is the only ground-truth anchor. The rubric criteria are generated by an LLM, the hard/soft labels are assigned by an LLM, the applicability filter is an LLM judge, and semantic preservation is derived by an LLM. The paper gives no guarantee and runs no sensitivity or adversarial tests for: validity (does the rubric measure the intent), accuracy (does pass/fail generalize), completeness (do finite matchers cover all failure modes), or reliability (stability under model or distribution shift). Semantic preservation is implemented as verbatim copying of the protected constraints, not as a semantic-equivalence check, so the procedure and criteria can drift semantically while the intent text is unchanged.
-
The scope is narrow, and generality is not shown. The method assumes: (1) a binary outcome y ∈ {0,1} to compare the rubric against; (2) domain-specific matchers (ALFWorld's event-sequence/count monitors do not transfer). It is evaluated only on ALFWorld and WebShop. The "general interactive agents" claim is not supported. The paper should state a clear scope and define it explicitly in the contributions (e.g., text, discrete, binary-reward, stable-rule tasks).
-
The method cannot localize a rubric error to a specific skill or rule. A task uses multiple skills, but the outcome is a single success/failure, so when the rubric disagrees with the outcome, the error cannot be attributed to one skill. The two error cases are not equally fixable. In the "too restrictive" case (rubric fail, outcome success), a specific criterion failed, so there is a concrete rule to relax. In the "too permissive" case (rubric pass, outcome failure), the rubric passed, meaning it thinks every rule was satisfied; there is no clue about which rule is wrong, so the system can only guess what check is missing from "semantic overlap with the task". This is the main case where boundaries need tightening. The paper reports no attribution-error rate and no single-skill-update control.
-
The ablation shows that adding skills hurts. Table 3: Static Library 73.4%, Acquisition Only 68.8%, Failure-only 67.2%, Full Skill-V 95.3%. Adding skills alone, and failure-only revision, are both worse than a static library. The paper does not explain this, and does not state whether the variants share the same policy checkpoint and training budget. This raises questions about whether the acquisition half of the method helps at all, and suggests possible training instability or hyperparameter mismatch.
Questions for authors
- Give operational definitions of reliable, testable, verifiable, and falsifiable, and state how they differ in this framework.
- How are validity, accuracy, completeness, and reliability guaranteed? What evidence supports calling the method "verifiable"?
- Where does the mechanism's generality end? Can it be validated on a third environment with a different action space or feedback type (e.g., visual or continuous control)?
- When the rubric disagrees with the outcome, how does the method decide which skill or rule to revise? In the "too permissive" case (rubric pass, outcome failure), where does the signal for which check to add come from?
- Why does Acquisition Only underperform Static Library in Table 3? Do the variants use the same policy checkpoint and training budget as Full Skill-V?
2 comments
Response to Reviewer 8cy6 [Part 2]
W5 & Q5: Experimental Setup and the "Acquisition Only" Drop
Yes, all variants in Table 3 share the exact same initial policy checkpoint (SFT initialization), training budget, and hyperparameters. They are all trained jointly from scratch.
The fact that "Acquisition Only" (68.8%) underperforms the "Static Library" (73.4%) is not a result of training instability or hyperparameter mismatch; rather, it empirically highlights the central motivation of our paper.
Blindly accumulating new skills from failures without ever revising them quickly leads to a bloated library filled with unverified, overly permissive, or conflicting boundaries (i.e., knowledge poisoning). When the agent retrieves these flawed skills during joint RL training, they severely misguide the policy updates. Consequently, the policy converges to a worse state than if it had simply learned with a small, clean static library.
This result proves that the acquisition of new skills is actually detrimental to long-term autonomy unless it is paired with rigorous maintenance. Skill-V explicitly cures this knowledge poisoning through bidirectional revision and strict validation at the Evidence Gate. By ensuring stored knowledge remains reliable, Skill-V successfully unlocks the benefits of continuous acquisition, ultimately driving the success rate to 95.3%.
Response to Reviewer 8cy6 [Part 1]
We thank you for your careful review and constructive feedback. Your comments on terminology, soundness guarantees, and the scope of our claims have highlighted several areas where greater clarity and precision are needed. We address these points below.
In Skill-V, “verifiable” refers to executable checking of explicitly represented contract criteria and revision acceptance rules. It does not refer to formal verification of the natural-language skill or a proof of future task success.
The related terms have distinct roles:
We agree that LLMs hallucinate and cannot provide formal guarantees for validity, accuracy, or completeness. This is precisely why the Environment Outcome (y∈{0,1}) serves as our sole ground-truth anchor.
The LLM acts strictly as a generator of hypotheses. To be accepted, every LLM-generated rubric or revision must pass the deterministic Evidence Gate against hard environment outcomes on historical replays. If the LLM generates an inaccurate or incomplete rule, it will fail the metric checks (e.g., FNR, BAcc) and be safely rejected. In short, we rely on data-driven evidence to filter out LLM unreliability.
Regarding semantic drift, verbatim copying of the
when_to_applyandprinciplefields is a deliberate structural defense. By strictly locking the skill's core textual identity, we ensure that while the operational boundary (the executable rubric) adapts to new evidence, the fundamental semantic intent of the skill remains constant and cannot drift.We appreciate your point regarding the scope of our current evaluation. In this work, we focus on rigorously verifying the self-evolving skill mechanism on text-based interactive agents.
However, our use of the term "interactive agents" reflects the broad applicability of the underlying methodology. The core conceptual framework, which represents skills as falsifiable contracts and drives their evolution via outcome-grounded validation, is fundamentally modality-agnostic. We firmly believe this approach can be successfully transferred to vision-related tasks, for instance, by using Vision-Language Models to evaluate visual state changes as rubric criteria. Extending this self-evolving mechanism to visual domains is a key focus of our ongoing and future research.
You have pinpointed a fundamental challenge in trajectory-level RL: without step-by-step causal attribution, it is mathematically impossible to perfectly isolate a single faulty rule, especially in the "too permissive" case where no explicit criteria failed.
To address this, we rely on a Contextual Analysis Heuristic (detailed in Appendix B.5). When a "too permissive" error occurs, the LLM is provided with the full trajectory alongside all activated contracts and is prompted to hypothesize which skill is missing a check.
Crucially, we do not assume the LLM's credit assignment is always correct. The LLM will inevitably make attribution errors. However, this is precisely why our architecture depends on the Evidence Gate. If the LLM blames an innocent skill or adds an invalid check, that faulty revision will immediately fail the deterministic replay check (because it will regress on other historical trajectories that successfully used the innocent skill). Consequently, the incorrect attribution is safely rejected without damaging the library. By filtering out attribution errors at the evidence gate, Skill-V maintains system-level reliability without requiring a perfect single-skill attribution rate from the LLM.