- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SE-GoS uses execution traces to update skill-graph relations, weights, and descriptions while keeping models and skills fixed. Benchmark evaluations, ablations, and held-out tests support improved retrieval utility.
Critical strengths
Encoding execution experience into inspectable and shareable retrieval state is practically useful. Keeping models and skill implementations fixed also helps isolate the effects of graph maintenance.
Held-out evaluation and complete component ablations provide encouraging evidence.
Critical weaknesses
The paper is motivated by missing prerequisite skills, but no dependency edges are generated in the reported trace analysis. The added navigation links mainly capture workflow. The results shows that execution feedback can improve retrieval, but the method’s ability to recover actual prerequisite relationships is untested.
It is unclear how a newly added skill would become connected to existing skills without execution history. The experiments keep the skill library fixed. Semantic initialization provides a starting point, but this scenario is not evaluated. In the experimental result, reward drops to 54.0% in the third round, raising concerns about the stability of repeated updates.
Questions for authors
How does ranking by skill usage or reward without graph diffusion perform using the same traces?
How would a newly added skill enter the existing graph without execution history, and how much usage evidence is needed to establish useful workflow connections?
Response to Reviewer S7DT: Graph-Free Ranking and New-Skill Integration
Thank you for asking how graph-free ranking compares and how new skills enter the graph. Below, we compare reward-based ranking with SE-GoS and explain skill insertion and the execution evidence supporting workflow edges.
With the same traces, SE-GoS exceeds graph-free reward ranking by 6.1–9.8% across the three SkillsBench backbones and by 9.3% on held-out tasks. Our MemRL* baseline adapts episodic-memory retrieval to rank skills by semantic relevance and reward utility, keeping source skills and model weights fixed. Tables 2 and 3(a) report:
deepseek-v4-flash-0731minimax-m2.7gpt-5.2-codexdeepseek-v4-flash-0731For this control, we considered four related approaches reviewed in Section 2: MemRL, SELM, TrackRL, and SkillBrew. Among them, we found public implementations for MemRL and SkillBrew. MemRL directly matches the requested graph-free ranking control with fixed skills and model weights, while SkillBrew curates the library. Appendix B.2 details the baseline choice and adaptation.
A new skill can enter retrieval without execution history. Appendix C.2 specifies three steps:
Add its node, retaining the existing graph and learned state.
Connect it to relevant skills using deterministic text similarity.
Include it in lexical seeding, making it retrievable immediately, even before it has graph neighbors.
Initialization needs no embedding model or LLM calls. Subsequent traces support the three updates, with only description rewriting calling the backbone.
One qualifying observation creates a workflow edge. Under Section 3.3's rule, a query seed A connects to a distinct skill B used on a task with reward ≥0.9. Further observations strengthen the edge. Table 5 reports query observations, which count qualifying retrieval queries and determine induced workflow weights, and distinct tasks, which count how many tasks supplied that evidence. Two qualifying queries from one task count as two observations but one supporting task. The table therefore shows both observation frequency and task coverage:
Table 4 shows that topology-only updating improves reward from 50.8% to 54.1%, even though most induced workflow edges have only one supporting observation.