- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
MDSKILL aims to move an agent's self-written skills into its weights during RL, instead of only retrieving them into the prompt. It starts from the view that self-generated skills are noisy and mix different functions, which makes distillation from a skill-conditioned teacher unreliable. Its contributions are threefold. First, retrieval uses Thompson sampling over Beta-distributed "Reliability" and "Advantage" scores. Second, skills are kept in separate banks for grounding, planning and control (the multi-view part). Third, distillation is gated and runs in forward, reverse and cross-view directions. MDSKILL reports the highest success rates on ALFWorld and WebShop at 1.7B, 3B and 7B, and is roughly on par on Search-QA.
Critical strengths
S1. The small-scale gains are large. With Qwen3-1.7B, MDSKILL beats UCOB by +7.1 on ALFWorld and +8.1 on WebShop (Table 1), against a broad pool of 13 baselines. If this holds up, it is a useful recipe for small agents.
S2. Section 3 documents a real failure mode. Skill-conditioned OPSD is sensitive to distillation direction, and reverse beats forward (Figs. 2–3). The skill-level diagnostics in Fig. 6 are a reasonable way to ask whether memories are being internalized.
Critical weaknesses
W1. Nothing isolates the multi-view decomposition as the cause. Swap in relevance-only retrieval and ALFWorld drops from 93.8 to 73.4 (Table 3), below skill-free GiGPO (80.5), even though the multi-view machinery is still there. With the result is basically GiGPO (81.3 vs. 80.5). The skill-view advantage stays at or below zero for the whole run (Fig. 6a). The key ablation, "w/o multi-view rollout", is shown only as a single noisy training curve. So I suspect the gain comes from the retrieval scoring plus extra distillation signal, not from the grounding/planning/control split. I'd want final numbers for that ablation, UCOB plus your retrieval metrics, and a non-functional three-way split as a control.
W2. The margins are within noise, and some claims are overstated. The ALFWorld rates fit a 128-episode set (e.g. 120/128). That would make the 3B and 7B margins over UCOB 3–4 episodes, from single runs with no seeds or variance. The gate type and also look tuned per scale. Search-QA is not best at any scale, yet the conclusion claims SOTA on three benchmarks. "No additional computational overhead" ignores the extra reflection, verification, retrieval and teacher re-scoring. Three seeds for MDSKILL, GiGPO and UCOB, plus a cost table, would settle much of this.
Questions for authors
- [Clarification] Is Table 1 evaluated with or without skill retrieval at test time? This decides whether it measures internalization or augmentation.
- [Clarification] What is the "w/o multi-view rollout" configuration, and what are its final success rates? If it is within 2 points of the full model, W1 gets much stronger.
- [Clarification] Does UCOB's retrieval use relevance only? Have you tried your Thompson-sampled retrieval on UCOB without multi-view banks?
- [Clarification] How many seeds sit behind Table 1, were GiGPO and UCOB re-run under your protocol, and on which split were the gate and chosen?
- [Suggestion] A control with three non-functional skill banks would directly test whether the decomposition matters.
7 comments
(2) Why does relevance-only retrieval degrade performance despite retaining the multi-view mechanism?
Multi-view organization does not replace skill-quality control. Relevance-only retrieval removes value filtering, allowing relevant but incorrect, redundant, or inapplicable skills to enter each branch. Its degradation shows that multi-view organization alone cannot ensure reliable supervision from a noisy bank; it does not show that decomposition is ineffective when skill quality is controlled.
Moreover, UCOB does not use relevance-only retrieval: it combines utility estimates with a count-based UCB bonus. The deterministic w/o Thompson Sampling variant is also not equivalent to UCOB. Consequently, neither the relevance-only degradation nor a direct comparison between 88.3% and UCOB's 86.7% isolates the multi-view contribution. The added retrieval-replacement controls address this attribution directly.
(3) Why does setting produce results close to GiGPO?
Setting removes the distillation objective, largely reducing the method to GiGPO-based RL-only training. RL-only still includes skill-conditioned rollouts; it simply does not apply distillation. The purpose of view decomposition is to improve OPD supervision for more efficient and stable knowledge internalization, rather than to obtain larger RL gains merely by reorganizing rollouts. The result is consistent with this motivation.
(4) Why can the skill-view branch still underperform the skill-free branch?
Self-generated skill banks contain substantial noise, whose negative effects can outweigh the benefits of useful guidance. The skill-view branch therefore often trails the skill-free branch during the training dynamics we observe. Skill-view advantage measures the difference between two contexts of the same policy, not improvement over the initial policy.
The relevant evidence is the gap between the branches and how they evolve together, rather than whether the skill-view branch consistently leads. In Figure 6a, full MDSKILL exhibits a smaller negative gap than the ablated variants and does not remain substantially negative. Figure 7a further shows that the functional branches improve together. These observations indicate that harmful guidance is mitigated while useful experience remains available for internalization through distillation.
A healthy agent self-evolution process is one in which the skill-view branch does not substantially lag behind, the branches improve together, and mutual learning and distillation promote efficient knowledge internalization. The value of skill guidance should therefore be assessed jointly through the branch gap, training dynamics, and final skill-free performance, rather than persistent skill-view superiority alone.
(5) Does functional decomposition outperform an arbitrary three-way partition?
Following your suggestion, we added a random-partition control with matched skill collections, bank sizes, retrieval counts, and training budgets. The mixed-bank variant does not partition skills and retains only forward and reverse distillation. The random-partition variant randomly assigns skills to three banks and retains cross-view distillation, using the same distillation procedure as the functional multi-view method.
| Skill organization | ALFWorld | WebShop |
|---|---|---|
| Mixed bank, no decomposition | 89.1 | 76.6 |
| Random three-way partition | 87.5 | 75.0 |
| Functional multi-view banks | 93.8 | 82.8 |
Random partitioning does not improve over the mixed bank, falling behind by 1.6 points on both tasks. With the same distillation procedure, functional partitioning outperforms random partitioning by 6.3/7.8 points, showing that arbitrary partitioning alone does not explain the gains.
The random three-way partition retains cross-view distillation but does not organize skills by decision function. Knowledge transfer across these branches can therefore remain affected by functional entanglement and conflicting supervision, making distillation unstable and degrading performance. This result supports our central idea: the benefits of multi-view distillation depend on meaningful functional disentanglement, rather than arbitrary partitioning of the skill bank.
(Q1) 2. Does the final evaluation measure internalization?
Thank you for identifying the need to clarify this setting. All final evaluations in Table 1 disable skill retrieval and use the skill-free policy. They therefore measure internalized policy capabilities rather than test-time retrieval augmentation. We have clarified this setting in the evaluation protocol and table caption.
Figures 6a and 7a show both branches to diagnose the role of skill guidance during training. Table 1 instead tests whether acquired capabilities remain available after external skills are removed. Temporary skill-view underperformance during training therefore does not contradict improvements in final skill-free performance.
The skill-view branch ultimately serves to improve knowledge internalization into the skill-free policy; its value should not be judged solely by whether it consistently outperforms that policy.
(W2, Q4)3. Reliability, parameter selection, and computational cost
Three-seed results and baseline reruns. We agree that a single run cannot establish the stability of smaller margins. GiGPO and UCOB were re-run under our experimental protocol, and we have added three-seed results for GiGPO, UCOB, and MDSKILL across all three backbones, reporting means and sample standard deviations:
| Backbone | Method | ALFWorld (mean ± std) | WebShop (mean ± std) |
|---|---|---|---|
| Qwen3-1.7B | GiGPO | 80.23 ± 1.22 | 45.83 ± 1.67 |
| Qwen3-1.7B | UCOB | 86.97 ± 1.22 | 74.90 ± 2.31 |
| Qwen3-1.7B | MDSKILL | 93.77 ± 0.75 | 82.30 ± 1.61 |
| Qwen2.5-3B-Instruct | GiGPO | 82.80 ± 1.39 | 70.83 ± 1.67 |
| Qwen2.5-3B-Instruct | UCOB | 91.93 ± 1.22 | 76.53 ± 1.33 |
| Qwen2.5-3B-Instruct | MDSKILL | 95.83 ± 0.92 | 80.47 ± 1.55 |
| Qwen2.5-7B-Instruct | GiGPO | 90.17 ± 0.93 | 72.43 ± 1.98 |
| Qwen2.5-7B-Instruct | UCOB | 93.77 ± 1.55 | 81.00 ± 1.61 |
| Qwen2.5-7B-Instruct | MDSKILL | 96.37 ± 0.46 | 84.90 ± 1.61 |
MDSKILL achieves higher mean performance than GiGPO and UCOB on both tasks across all three backbones. Its mean gains over UCOB are 6.80/7.40, 3.90/3.93, and 2.60/3.90 percentage points for 1.7B, 3B, and 7B, respectively. These results make the evidence for improvement more convincing than reliance on a single run. We do not equate mean ± standard deviation with a statistical significance test.
Gate configuration and distillation weight. Only Qwen3-1.7B uses the strict gate; Qwen2.5-3B and Qwen2.5-7B use the relaxed gate. The motivation and corresponding comparisons are discussed in Appendix D. The strict gate provides more conservative filtering for less reliable skill supervision, whereas the relaxed gate retains more useful supervision when skill guidance is more reliable.
The distillation coefficient is not selected separately for each backbone: all models use . The coefficient ablation is provided in Appendix C.
Scope of performance claims. The abstract explicitly limits the SOTA improvements to ALFWorld and WebShop, while Main Results describes Search-QA performance as “competitive,” not best. However, the broader wording in the conclusion could be ambiguous, and we have revised it to match the abstract and main results.
Computational cost. The multi-view design allocates branches within the existing rollout budget without increasing environment interactions. Skill generation, including reflection and category verification, accounts for approximately 2% of per-step training time, whereas policy rollout and policy update account for approximately 90%. Our design does not increase the computational budget of these two dominant stages.
Thank you again for these suggestions. The retrieval-replacement, matched-distillation, and random-partition controls provide more direct evidence for functional decomposition. The distributional analysis helps explain the benefits of multi-view distillation, while the baseline reruns and three-seed results make the performance improvements more convincing.
Thank you for taking the time to review our paper and provide detailed comments and suggestions. We appreciate the opportunity to clarify our work and improve the manuscript.
Summary. Your main concerns are whether the improvements genuinely arise from functional multi-view decomposition and whether the reported gains reliably demonstrate knowledge internalization. Following your suggestions, we have added controlled ablations separating retrieval, view decomposition, and distillation, a random-partition control, and three-seed results. We also clarify skill-free evaluation and provide distributional evidence explaining the benefits of multi-view distillation. The expanded ablations are presented in Section 5.3 of the revised main text.
(W1, Q2, Q3, Q5)1. Does the multi-view design contribute to the improvements?
(1) Does multi-view decomposition help after controlling retrieval and distillation settings?
The
w/o Multi-View Rolloutvariant retrieves from the entire mixed skill bank without functional-view-specific selection. It retains the same skill count, rollout budget, and gating rules, achieving 89.1%/76.6%, compared with 93.8%/82.8% for full MDSKILL.To further separate decomposition from additional distillation objectives, we compare
w/o Multi-View Rolloutwithw/o Cross-View Distillation: both retain only forward and reverse distillation, but the latter retains functional views. The functional multi-view variant achieves 92.2%/81.3%, improving over the mixed-bank control by 3.1/4.7 percentage points. This supports a contribution from functional decomposition itself, rather than simply adding another distillation objective.We also expanded the distillation ablations and conducted the retrieval-replacement experiments you suggested. The key results below use Qwen3-1.7B:
Within UCOB's configuration, dynamic retrieval improves success by 1.6/1.6 percentage points, whereas the multi-view design improves it by 4.7/5.5 points. Within MDSKILL, replacing dynamic retrieval with UCOB retrieval decreases success by 1.6/1.5 points, while removing multi-view rollout decreases it by 4.7/6.2 points. These within-framework comparisons support contributions from both components, with larger performance differences associated with the multi-view design in this configuration. Each framework retains its own training hyperparameters, so attribution is based on within-framework comparisons.
The additional distillation ablations support complementary contributions: removing forward or cross-view distillation decreases success by 0.8/3.9 or 1.6/1.4 points, respectively. Section 5.3 of the revised main text provides the detailed configurations and complete component ablations.
Figure 6b provides complementary distributional evidence. Compared with
w/o Multi-View Rollout, MDSKILL achieves higher token overlap and a smaller, more stable absolute entropy gap. These observations are consistent with functional disentanglement improving teacher–student distributional alignment and enabling more stable and effective knowledge transfer. The controlled ablations establish performance benefits, while Figure 6b helps explain these benefits through the distillation dynamics.