- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SKL gives a frozen LLM agent environment knowledge learned from training interactions. It learns object-location weights by combining observed co-occurrences with commonsense priors, and procedural rules by comparing trajectories on the same task and keeping edits that improve training return. On ALFWorld and ScienceWorld, it improves average reward with GPT-4o-mini and Gemini-2.0-flash. On ALFWorld, the scores reach 81.31 and 88.03, compared with 62.63 and 61.17 for the strongest controlled baselines. Ablations support using both components. Passing the learned knowledge to Gemini-2.5-flash raises its ALFWorld score from 74.60 to 85.26 or 86.79, without target-side learning. The tests cover held-out instances of known task families. ScienceWorld scores measure partial completion, not full-task success.
Critical strengths
-
The split between where to search and how to act is useful and easy to inspect. Location hints come from pooled observations, while rules come from comparing trajectories. On ALFWorld, the two components reach 55.84 and 62.04 on their own, and 81.31 together. That gives a concrete reason to use both in this setting.
-
The gains show up on both environments with both main models. I also find the transfer result useful: Gemini-2.5-flash goes from 74.60 to 85.26 or 86.79 when given knowledge learned by the other models, with no target-side learning. This shows that another frozen model can use the same knowledge within these task families.
Critical weaknesses
-
The main comparison doesn't tell me how much SKL adds over existing ways to learn knowledge offline. Base, ReAct, and the single-reflection version of Reflexion don't build reusable knowledge from the training set. AutoGuide and ExpeL already learn guidance from past trajectories. The paper discusses them and clearly labels the cross-paper results as reference only, but there is no controlled comparison with a close offline learner. With about 90M tokens spent on the reported Cool learning run, that comparison matters: some of the gain may come from doing offline adaptation at all. I'd like to see what the proposed representation and learning procedure add under a comparable budget.
-
I can't tell how stable the results are when the knowledge is learned again from scratch. The five test runs reuse one learned knowledge object, and the paper says this clearly. They don't cover changes in exploratory rollouts, rule edits, or training order. This matters because a candidate rule is checked with one rollout per training instance against a cached score. The added sampling settings also give quite different results. The authors rightly note that rollout count and temperatures change together, so that experiment doesn't isolate either factor. Repeating the full learning process would show whether the gains hold up across the knowledge objects it produces.
Questions for authors
-
Evidence needed: Have you compared SKL with a close offline learner, such as AutoGuide or ExpeL, using the same backbone and training tasks, a similar offline interaction/token budget, and knowledge frozen at test time? I'd need this to separate the method's benefit from the benefit of offline adaptation itself.
-
Evidence needed: What happens when you repeat the whole knowledge-learning process with independent exploratory rollouts and different training orders? Please report how held-out results vary across newly learned knowledge objects, separately from repeated execution with one fixed object. This is the missing check on the learning procedure's reliability.