- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposed the idea of distilling the knowledge skill from teacher model to student models. The key motivation is to show that prompting the skill extractor to compare the difference between teacher trajectories and student trajectories can have more performance gain compared to distill knowledge between success and failure trajectories. The D-Skill has three stages: 1/ rollout collection for teacher and student agent 2/ conduct difference analysis to extract divergence lesson and agreement lesson 3/ update skill by skill extractor like Trace2Skill. The experiment results show the performance gain.
Critical strengths
1/ The idea of D-Skill is clear, reasonable and easy to follow. 2/ The experiment is comprehensive and clear. For example, Table 2 conducts the ablation study to compare skills extracted from student trajectories, teachher strrajectories and D-Skill. It shows the the D-skill can improve the performance significantly.
Critical weaknesses
1/ Not a weakness, but interesting to have the generalization analysis. As in the line 52 mentioned "...and sufficient training data is needed for the student to generalize". It would be great to include the out of domain evaluation. I think SFT would hurt the generalizability a lot. I am not sure to what degree D-Skill would hurt generalizability.
Questions for authors
See weakness
Out-of-domain generalization.
We follow Trace2Skill’s OOD setup (Ni et al., 2026), which tests SpreadsheetBench skills on WikiTQ and HiTab. Each table becomes a spreadsheet task, scored by the benchmark’s official evaluator on 300 random test questions per benchmark. We use the SpreadsheetBench documents of all four Table 1 pairs unchanged. Values are mean ± SD over three documents, each with 2 passes.
D-Skill does not hurt generalization. The documents improve Qwen by 38-54 points and Luna by 6-9 points on HiTab, and leave Luna on WikiTQ unchanged.