- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes SGVD (Semantically Grounded and Variable-Duration skill learning), a skill-driven method for offline multi-task multi-agent reinforcement learning (MARL). The core idea is to learn team-level coordination skills from offline source-task data and reuse them at execution time for zero-shot generalization to unseen tasks. The method has three main components: (1) an LLM-assisted pipeline that iteratively constructs and refines Label Functions (LFs), which map interpretable trajectory statistics to semantic skill labels (e.g., FOCUS FIRE, ADVANCE) and transition boundaries; (2) a boundary-guided switching mechanism that enables variable-duration skills by updating the active skill only when an observation-conditioned boundary predictor fires; and (3) two auxiliary objectives — segment-terminal prediction and posterior-prior dynamics distillation — to encode segment-level transition dynamics into skill representations. The skills are trained offline with behavior cloning plus the proposed supervision losses. The approach is validated on three SMAC task sets (Marine-Easy, Marine-Hard, Stalker-Zealot) under four data qualities (expert, medium, medium-expert, medium-replay), and compared against ODIS, HiSSD, BiKT, and behavior cloning baselines, with ablation studies and skill-representation visualizations supporting the claims.
Critical strengths
-
Clear and practically motivated problem formulation. The paper targets two real limitations of existing multi-agent skill-discovery work: (i) latent skills lack semantic grounding, making their differences hard to interpret or trust, and (ii) skills inferred at every step or at fixed intervals impose artificial temporal structure misaligned with genuine changes in team coordination intent. The proposed decomposition — semantic grounding via LLM-refined label functions, and variable-duration skills via boundary-conditioned switching — addresses both in a principled way. This framing is intuitive, the overall method is easy to follow, and the design (LLM and LFs used only offline, not at execution) is practically sensible for deployment.
-
Consistent empirical improvements with thorough ablation and visualization support. Across the three SMAC task sets and four data qualities, SGVD generally outperforms ODIS, HiSSD, and BiKT on both source and unseen tasks, with the gains largest on mixed-quality data (medium-expert, medium-replay). The ablation isolating each component — semantic grounding, variable duration, terminal prediction, dynamics distillation — is informative, and notably shows that fixed-duration skills cause the largest degradation, which directly validates the paper's central temporal-abstraction claim. The t-SNE skill-representation visualization and the boundary-probability evolution figure further support that the learned skills are semantically coherent and temporally persistent.
Critical weaknesses
-
Ambiguous and potentially under-scrutinized use of LLM-generated supervision. Although Appendix A describes the LF construction and refinement process, several important details remain unclear: How sensitive is the final performance to the initial skill vocabulary? How many refinement rounds are typically needed, and does the refinement budget materially change results? Most importantly, there is no direct ablation or quantitative analysis of the LLM-based skill-discovery pipeline itself — e.g., comparing against alternative labeling schemes (random window labels, heuristic LFs without LLM, human-specified LFs), or reporting annotation statistics (conflict/abstention rates on the full dataset, label-coverage per skill). Given that all downstream training relies on this supervision, the absence of such analysis makes it difficult to disentangle whether the gains come from semantically grounded supervision per se or simply from additional structured temporal information, and raises a legitimate concern of overfitting to LLM-derived annotations. The paper also does not discuss how the method scales to environments where trajectory statistics are less interpretable or where skill vocabularies are harder to define. Note also that the two appendix versions report different LLMs (DeepSeek-V3 in D.1 vs. DeepSeek-V4 in D.5), which adds to the confusion about the exact experimental setup.
-
Performance appears "too good to be true" in places and lacks sufficient validation of mechanism. On several tasks SGVD reaches near-100% win rates on unseen maps, and on some settings it dramatically exceeds prior methods (e.g., 10m vs. 12m: SGVD 15.4 vs. HiSSD 14.5 on expert, but on medium-replay SGVD 3.2 vs. 0.5; and several instances where baselines are near zero while SGVD is substantially higher). While such gaps are not impossible — the medium-replay baselines in Table 5 (e.g., ODIS at 0.0) suggest the baselines may be poorly tuned on mixed-quality data — the paper provides no explanation or supporting evidence (e.g., learning curves, seed-level results, baseline tuning sensitivity) to establish that these margins are robust rather than an artifact of baseline under-tuning. Combined with the missing analysis of the LLM annotation pipeline (weakness 1), it is hard to fully trust the reported improvements. Additionally, the paper would benefit from a more comprehensive literature review of offline MARL algorithms (e.g., conservative value estimation, implicit Q-learning, behavior-regularized methods), which are cited only briefly in Section 2.1 without discussion of why behavior-cloning-based skill learning is the appropriate foundation here.
Questions for authors
-
On the LLM annotation pipeline: What fraction of windows receive abstain/conflict labels from the final LF set on each task set, and how sensitive is zero-shot performance to these masked-out windows? Please provide a direct ablation comparing (a) full SGVD, (b) LFs constructed without LLM assistance (e.g., manually specified thresholds), and (c) random or heuristic window labels, to verify that the gains come from semantic grounding rather than any form of segment-level supervision.
-
On the robustness of results: Some results on unseen tasks approach 100% win rate while baselines are near zero on the same settings (e.g., medium-replay Marine-Easy). Could you provide learning curves or seed-level variance analyses, and clarify how carefully baselines were tuned on the mixed-quality data settings? This would help rule out the possibility that the improvements partly reflect baseline under-tuning.
-
On skill selection and execution: During execution, the skill is updated only when the predicted boundary probability exceeds a threshold (δ = 0.5). How sensitive is performance to this threshold and to boundary-prediction errors? In episodes where the predictor fires spuriously (or misses a true boundary), does the policy degrade gracefully? A quantitative analysis of boundary precision/recall and its impact on win rate would strengthen the claims.
-
On generality: The initial skill vocabulary and interpretable statistics (attack ratios, team spread, enemy-distance change) are well-suited to SMAC. How would the LF-based approach be instantiated for environments where interpretable trajectory statistics are not obvious, and what is the expected cost of vocabulary design and refinement in such settings?
-
Clarification: Appendix D.1 states DeepSeek-V3 was used while D.5 states DeepSeek-V4. Which model was actually used for the reported experiments, and does the choice of LLM affect the final LF quality or downstream performance?