acceptodds
Under review as a conference paper at ICLR 2027

The Last Mile of Skill Transfer: In-Sample Team-Value Alignment for Offline Multi-Agent Generalization

Abstract

Offline multi-agent reinforcement learning (MARL) from multi-task datasets aims to learn cooperative policies that generalize zero-shot to unseen tasks with varying numbers of agents, and hierarchical skill learning is the dominant approach. In these methods, however, value supervision stops at the skill: the low-level executor is trained by conditional behavior cloning, so successful and unsuccessful realizations of one skill are blended, and a correctly selected skill is still executed by low-value actions. We formalize this as the hierarchical value gap, which closes only under a signal that varies with realization quality within a skill, and that no skill-level objective supplies. We propose Skill-conditioned Team-value Execution Projection (STEP), a training-time framework that completes the value pathway of hierarchical skill policies by extending team-value supervision from skill selection to skill execution. Our approach introduces an in-sample expectile team value that distinguishes the quality of logged executions under the same skill and supplies the weighting signal for execution projection, a team-residual execution projection that uses this signal to make decentralized executors imitate high-value realizations of each skill, and execution-consistent consequence conditioning that aligns the value with the skill semantics actually being executed. On nine unseen SMAC maps, STEP improves over the strongest hierarchical skill baseline by 5.4 and 4.8 win-rate points on medium-expert and medium-replay data, respectively, while matching its performance on expert data, whose returns are saturated. These results show that STEP delivers value supervision to skill execution, strengthening zero-shot cooperation on mixed-quality data without sacrificing performance on high-quality data.

Primary area:
reinforcement learning
Keywords:
offline multi-agent reinforcement learning, multi-task skill transfer, hierarchical skill learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_scyoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies zero-shot generalization in offline multi-task cooperative MARL, where hierarchical skill-based methods are currently dominant. The authors identify a "hierarchical value gap": in existing methods (e.g., ODIS, HiSSD), value supervision stops at the skill-selection level, while the low-level decentralized executors are trained by uniform conditional behavior cloning — so successful and unsuccessful realizations of the same skill are imitated with equal weight, and selecting the right skill does not guarantee effective joint execution. To close this gap, the paper proposes STEP (Skill-conditioned Team-value Execution Projection), a training-time-only framework with three components: (1) an in-sample expectile team-value estimator (attention-mixed per-agent utilities) that produces one-step TD residuals from logged transitions; (2) a team-residual execution projection that converts standardized residuals into weights on the executors' imitation losses, favoring higher-value realizations of each skill; and (3) execution-consistent consequence conditioning, which injects the decoder-driving skill code into the backbone's consequence predictor. The method is formalized via an execution residual (Definition 1) with supporting propositions on the reweighted execution target. Validation is conducted on SMAC: the primary benchmark is Marine-Hard (3 source maps, 9 unseen maps, four data qualities), with Marine-Easy and Stalker–Zealot as secondary checks, using a reproduced HiSSD backbone as the direct paired control plus ODIS, UPDeT-m, and BC baselines. STEP improves unseen-map win rates over its backbone by +5.42 and +4.79 points on medium-expert and medium-replay data while matching it on expert data; ablations attribute the gains to residual-weighted execution, and a tested retrieval extension yields no additional benefit. The paper also includes holdout-map checks and final-checkpoint (selection-free) results.

Critical strengths
  1. Well-diagnosed and clearly formalized problem. The "hierarchical value gap" is a genuine and under-appreciated limitation of the dominant skill-learning paradigm in offline multi-task MARL: skill-level objectives (value, contrastive, or dynamics) shape which skill is selected but say nothing about the quality of execution within a skill, and uniform conditional behavior cloning then averages over good and bad realizations. The execution-residual formulation (Definition 1) precisely characterizes the residual discrepancy that skill selection alone cannot remove, and Propositions 1–2 honestly delimit what reweighting does and does not guarantee. This conceptual clarity — identifying that the "last mile" of skill transfer lies in executor training rather than skill discovery — is the paper's main intellectual contribution and is consequential because it is general: any hierarchical skill method with a behavior-cloned executor could in principle adopt this supervision.

  2. Rigorous, well-controlled empirical protocol with convincing causal evidence. The evaluation is unusually careful for this literature: paired seeds with the reproduced HiSSD backbone isolate STEP's effect; shuffled-weight controls (C-shuf) demonstrate that the association between residuals and logged actions matters rather than the weighting procedure per se; graded weighting is shown to beat hard return filtering; final-checkpoint (selection-free) re-evaluation confirms the gains are not an artifact of best-over-training reporting; and never-inspected holdout maps rule out development-loop selection pressure. The mechanism analysis (weight deciles tracking realized returns across expert/medium provenance, gains concentrated on maps where the backbone has headroom) aligns the empirical evidence with the theory. This evidentiary standard meaningfully raises confidence that the reported gains are real and attributable to the proposed mechanism.

Critical weaknesses
  1. The skill signal itself is under-examined and largely inherited without justification. STEP is deliberately agnostic to how skills are discovered — it wraps the HiSSD backbone and adds execution-time value supervision. However, the paper provides no clarification of the skill discovery process on which everything depends: how many skills the backbone learns, whether the skill space is discrete/continuous, how skill semantics relate to the reweighting, or whether the quality of skill discovery limits the ceiling of execution projection. More importantly, the residual weighting operates on the team-level TD residual shared across all agents, which is a coarse and ambiguous signal: it carries no individual credit assignment, and as the paper itself notes (Appendix G), per-step residual-return correlations are weak and mixed in sign — the diagnostic supports only episode-level selection, not identification of which joint actions within an episode were good. There is also no guarantee that a high residual corresponds to skill-consistent execution rather than lucky stochastic outcomes; Proposition 2 is explicitly conditional on realized outcomes and does not imply improved environment return. Because the entire framework rests on this single scalar signal per transition, the absence of any analysis of the skill–residual interaction (e.g., do weights separate realizations within a skill, as claimed, or merely across tasks/returns?) leaves the central mechanism only partially validated.

  2. Limited behavioral/visualization evidence and questionable value objective design. Despite the paper's interpretability-adjacent framing, there is no visualization or qualitative analysis of the learned skills or of inter-agent correspondence — e.g., which agents' actions drive high-residual episodes, whether reweighting sharpens coordination patterns (focus fire, kiting) on unseen maps, or how the consequence conditioning changes the skill representations. Given that the method's premise is that executors conflate good and bad coordinated behaviors, showing that STEP's policies actually exhibit cleaner coordinated behaviors would be direct evidence, and its absence weakens the claim. Regarding the value objective: the expectile team value (τ_e = 0.7, fixed without sensitivity analysis) is admittedly not an advantage estimate, and the design has observable costs — on expert data the weights carry no signal and reweighting becomes pure variance, producing a measurable cost at the final checkpoint (−2.64, CI excluding zero on HE), which the paper acknowledges but does not address with any mechanism to detect or suppress no-signal regimes. The consequence-conditioning component (D) also shows inconsistent, mostly non-significant effects (δ_forward: +2.46 on HE, −0.93 on HMR, CIs crossing zero), and its inclusion in the "full STEP" obscures that the method is effectively a single-idea contribution whose second and third components are not clearly pulling their weight.

  3. Attribution of component contributions is unclear, the evaluation is confined to a single benchmark family (SMACv1), and the margins over the strongest existing baseline are modest and frequently not statistically significant. First, the ablation chain (Table 12) shows that residual weighting (C) accounts for essentially all of the observed gains, while the value estimator alone (B) is neutral-to-harmful and consequence conditioning (D) is inconsistent and mostly insignificant (δ_forward: +2.46 on HE, −0.93 on HMR, all CIs crossing zero). The method is therefore effectively a single-component contribution (residual-weighted BC) wrapped in a framework whose other two components neither help nor are cleanly attributed — the "three-component" framing overstates the substance. Second, all evidence comes from SMAC/SMACv1 micromanagement maps; no experiment on any other environment family (e.g., multi-robot control, Google Research Football, Melting Pot, or non-StarCraft domains) is provided, so the generality of the "hierarchical value gap" claim — which the paper presents as a fundamental limitation of the paradigm — rests entirely on one benchmark. Third, and most consequential for my assessment: the headline gains (+5.42/+4.79 average win-rate points on HME/HMR) are relative to a self-reproduced HiSSD backbone whose training protocol deliberately differs from both the original paper and its official repository (optimizer, learning rate, training steps — Table 4), and recent stronger baselines such as BiKT are not evaluated at all. Per-map paired differences are noisy: on HM the weighting gain is +1.06 ± 5.08 and on HE the method is worse than its own backbone at the final checkpoint (−2.64, CI excluding zero); the holdout-map evaluation (Table 13), which is the least selection-biased evidence in the paper, shows differences that mostly scatter around zero with CIs crossing zero in 8 of 10 cells. Taken together — one real component, one benchmark family, no comparison against the strongest published baselines under their official protocols, and frequently insignificant margins — the empirical support is not sufficient to justify accepting a method whose theoretical guarantees are explicitly conditional and limited.

========

After the rebuttal, I maintain my score and the reasons are shown below:

  1. Insufficient experimental evidence to support the paper's broad claims.

My primary concern is the substantial gap between the paper's broad claims and its limited experimental validation. STEP is presented as a general framework, yet it is evaluated exclusively on HiSSD, making it unclear whether the improvements stem from STEP itself or the strong underlying backbone.

Furthermore, all experiments are restricted to the standard SMAC benchmark used in ODIS[1], without considering SMACv2, MAMuJoCo, or Cooperative Navigation, despite their adoption in related works such as HiSSD, BiKT[2], and STAIRS-Former[3]. Notably, even the earlier ODIS paper evaluated Cooperative Navigation. The comparisons also omit stronger recent methods, particularly BiKT and STAIRS-Former.

As skill-discovery-based approaches, ODIS and BiKT would be natural alternative backbones for evaluating STEP's generality. Without cross-backbone and cross-benchmark validation, the claimed general applicability of STEP remains insufficiently supported.

  1. Underestimated computational cost relative to existing methods.

In their response to Reviewer GtAy, the authors report only 1.16×–1.18× training overhead and approximately 7% additional parameters relative to HiSSD. However, these seemingly modest figures overlook the substantial computational cost of HiSSD itself.

According to my measurements on Marine-Easy-Expert, ODIS has approximately 138k parameters, compared with 679k for HiSSD. Thus, STEP's additional parameters alone amount to roughly 34% of the entire ODIS model. Moreover, HiSSD already requires approximately 2.1× the training time of ODIS (including ODIS pretraining), suggesting that STEP could cost approximately 2.4–2.5× as much as ODIS. Similar findings regarding the overhead of HiSSD are also reported in Appendix H of STAIRS-Former[3].

These computational costs also raise concerns about the scalability of both HiSSD and STEP. In particular, it remains unclear whether these methods can efficiently scale to larger multi-agent systems, such as scenarios involving more than 25 agents, under a reasonable GPU memory budget.

Reporting efficiency solely relative to an already expensive backbone is therefore insufficient. A fair assessment should include absolute computational costs and comparisons with other competitive methods.

  1. Unfair evaluation protocols and incomplete baseline reporting.

Despite providing additional end-of-training results for HiSSD and STEP, the authors have not resolved a fundamental inconsistency in their comparisons. The BC-best, UPDeT-m, and ODIS results in Tables 1 and 7 appear to be directly taken from the original ODIS paper, which reports end-of-training performance, while HiSSD and STEP are evaluated using best-over-training checkpoints. This protocol mismatch unfairly favors the latter methods and undermines the validity of the comparisons.

The reporting is also incomplete: Table 6 in Appendix E omits all baselines except HiSSD and STEP on Marine-Easy, while Table 7 omits the unseen-task average performance of the other baselines.

These issues substantially weaken the credibility of the experimental conclusions. The authors should reproduce existing methods under a consistent evaluation protocol and provide complete baseline comparisons across all task sets.

References

[1] Zhang, F. et al., Discovering Generalizable Multi-agent Coordination Skills from Multi-task Offline Data. ICLR 2023.

[2] Zhang, J. et al., Bi-Level Knowledge Transfer for Multi-Task Multi-Agent Reinforcement Learning. NeurIPS 2025.

[3] Jeon, J. et al., STAIRS-Former: Spatio-Temporal Attention with Interleaved Recursive Structure TransFormer for Offline Mulit-task Multi-agent Reinforcement Learning. ICLR 2026.

Questions for authors
  1. On the skill signal: Does the residual weighting actually separate realizations within a skill? Please provide an analysis conditioning on the skill code z (e.g., distribution of weights within each skill cluster, or within-skill covariance Cov(w, δ | z) per skill), rather than the current episode-level analysis. Without this, the "within-skill quality signal" claim (as opposed to a global return filter, which %BC already partially achieves) is not fully established.

  2. On the value objective: Since the paper admits the weights add pure variance on expert data (final-checkpoint HE: −2.64, 95% CI [−5.10, −0.40]), is there a principled way to gate or regularize the weighting when residual dispersion is uninformative — e.g., scaling α by measured residual variance or disabling weighting below a dispersion threshold? Also, why is τ_e = 0.7 chosen, and how sensitive are the results to τ_e and α?

  3. On component contribution: The ablation shows residual weighting (C) drives essentially all gains while consequence conditioning (D) has inconsistent, mostly insignificant effects. Would the simpler C configuration be the recommended method? If D is retained in "full STEP," what evidence justifies it beyond a small effect on HME?

  4. On skill discovery clarification: Given that STEP inherits the entire skill-discovery pipeline from the reproduced HiSSD backbone, could you clarify what happens with alternative backbones (e.g., ODIS's discrete skills or different skill dimensionalities)? Does the magnitude of the execution gap, and hence STEP's benefit, depend on skill granularity? This would clarify the generality of the "hierarchical value gap" claim beyond one backbone implementation.

  5. On visualization (suggestion for revision rather than a blocking issue): Would you consider adding qualitative analysis — e.g., replay visualizations or skill-code statistics on unseen maps — showing that STEP's executors produce visibly different (more coordinated) joint behaviors than the backbone for the same skill? This would directly substantiate the execution-quality claim that the current win-rate and weight-diagnostic analyses only support indirectly.

  6. On significance and baselines: The holdout-map evaluation (Table 13) — the only evaluation free of checkpoint selection — shows paired differences whose CIs cross zero in 8 of 10 cells, and on expert data STEP is significantly worse than the backbone at the final checkpoint. Given this, what evidence beyond average win-rate deltas supports the claim of a robust improvement? Additionally, could you report results against BiKT (the strongest recent hierarchical skill-transfer baseline) using its official implementation and protocol, and provide per-map significance for the HME/HMR gains rather than only regime-level averages?

5 comments
AuthorsOpenReview ↗

Response to Reviewer scyo (Part 1/3)

We thank the reviewer for the detailed review. We take the score as a clear signal that the submission did not give enough evidence on several points, and we have worked to close these gaps. We read the main concerns as (a) whether the weights act within skills, (b) whether the gains are robust beyond regime averages and measured against a faithful backbone, (c) component attribution, and (d) generality. For (a) to (c) we ran new analyses, namely two weight controls with five seeds, a variance decomposition of the weights and skill-code statistics, a within-skill action analysis, a sensitivity study of τe\tau_e and α\alpha with a sweep over the expert share of the data, a behavioral analysis on two unseen maps, and a per-map table. Generality (d) remains a limitation, which we address in Part 3. All results use the main-text protocol with paired seeds and are added to the revised appendix.

We begin with Q6 / Weakness 3, which you identified as most critical to your assessment. Part 2 addresses Q1 and Q2, and Part 3 addresses Q3, Q4 and Q5.

Q6 / Weakness 3: Is the improvement robust, and is the backbone comparison fair?

Beyond the regime average. The gains are consistent at the seed level. On HME and HMR all five paired seeds favor STEP (+2.36 to +7.71 on HME and +2.50 to +8.19 on HMR, Figure 2). They also hold without picking the best point of training. Besides the holdout maps, Appendix F reports every cell at the end of training (the win rate logged at 35k steps, with no maximum over training), where the gains are +4.40 [+2.22, +7.12] and +6.31 [+3.88, +8.70], again with five of five seeds positive. Per map, the mean difference is positive on 7 of 9 HME maps and 8 of 9 HMR maps. Five of the 18 map-level intervals exclude zero, and all five favor STEP (12m and 10m_vs_12m on HME, and 4m, 7m_vs_8m and 10m_vs_11m on HMR). This holds even though five seeds give wide per-map intervals and on HME the backbone is already at 99.5% to 99.9% on 4m and 5m. The full 18-cell table is added (Appendix J.8). On HM the gain is smaller (+1.96), consistent with the expert-share sweep in Part 2, where the gain is largest when the data mix expert and medium executions.

The reproduced backbone. Our backbone follows the official repository and inherits its optimizer and learning rate, on which the paper and the repository differ. The HiSSD paper trains for 30k steps and the official repository defaults to 251k, and we train all arms for 35k steps with the paper's best-over-training rule. Under this protocol the reproduced backbone is stronger than the published HiSSD results in all four regimes. Its unseen-map averages are 70.3, 51.9, 62.8 and 52.4 (HE, HM, HME and HMR), while the averages of the per-map values in Table 1 of the HiSSD paper are 61.9, 47.2, 55.8 and 48.1. STEP's gains are thus measured against a stronger backbone, not a weaker one. All arms share seeds, data, budget, optimizer and evaluation, so the paired differences isolate the effect of STEP. Section 5.1 now states this justification, with the numbers in Appendix D.

BiKT. We did not run BiKT. Reproducing it under our matched protocol was not feasible within the rebuttal period, and its reported numbers use a different protocol, so we make no claim relative to BiKT. The two methods act on different parts of the hierarchy. BiKT changes what the executor is conditioned on by reusing a learned tactic codebook, whereas STEP changes how the executor is trained, so the two could be combined. As a related check, attaching a codebook and segment library of our own design to paired STEP checkpoints adds no further gain (Table 9). Our Related Work places BiKT among other routes to skill transfer, and a matched comparison with BiKT is part of our future work.

AuthorsOpenReview ↗

Response to Reviewer scyo (Part 2/3)

Q1 / Weakness 1: What is the skill signal, and do the weights distinguish realizations within a skill?

The skill space. In the HiSSD backbone a skill is a continuous 64-dimensional code per agent and step. Vector quantization is not enabled, so there is no fixed list of skills. We measured the executed codes on HE, HME and HMR (five seeds, three source maps). Their effective dimension is moderate, with 11 to 24 principal components explaining 90% of the variance. They show no clear cluster structure, since the silhouette score is 0.12 to 0.42 and rises with K without a preferred K, whereas a synthetic set with eight clusters scores 0.60. They also drift smoothly over time, with a median cosine similarity of 0.76 to 0.95 between consecutive steps, compared with 0.20 to 0.47 for random pairs. The number of skills thus has no fixed answer, and our skill-group analyses use k-means with K = 16 as a working discretization (Appendix J.2).

Within a skill, or only across skills and returns? The weights vary almost entirely inside skill groups. Across the 45 cells, only 0.1% to 2.6% of the weight variance lies between the K = 16 groups, and at most 3.4% for K = 8 or 32. We then ran two interventions with five paired seeds.

HMEHMR
STEP − skill-group mean weights+5.04 [+3.65, +6.42]+2.82 [+1.54, +4.42]
STEP − return-to-go weights+1.81 [+0.63, +2.99]+3.46 [+2.25, +4.75]

Skill-group mean weights keep the differences between skill groups but remove those within a group. This removes most of the gain on HME, where the result returns to the backbone level (63.15 vs. 62.78), and about 60% of it on HMR, so the gain comes mainly from within-skill variation rather than from reweighting across skills. Return-to-go weights are a soft form of the global return filter you mention. They are weaker than the TD residual in both regimes, as is the hard filter %BC (Table 11), so the weights do more than rank episodes by return. Return-to-go also favors early steps of each episode, since SMAC rewards are non-negative, whereas the residual compares each step with the learned baseline.

On Cov(w,δ∣z)\mathrm{Cov}(w, \delta \mid z), ww is monotone in δ\delta, so the covariance is non-negative whenever δ\delta varies within zz, and the informative question is what drives this variation. The variance decomposition shows that it is the dominant part of the weights, and the interventions show that it is needed for the gain. We also conditioned on a context cluster in addition to the skill cluster and tested whether the recorded action of each agent explains the weights. In this finer test the individual action alone did not reliably explain the weights. We therefore describe the mechanism at the level the interventions support, namely that STEP exploits within-skill differences in realized execution quality, which can reflect both the actions taken and the states they lead to.

Q2 / Weakness 2: The value target, its sensitivity, and expert data.

Sensitivity. We varied τe\tau_e and α\alpha one at a time on HMR, with seeds 1 to 5 paired with the backbone.

SettingUnseen-map win rate (%)Gain vs. backbone
HiSSD backbone52.43(reference)
τe=0.5\tau_e = 0.557.13+4.70
τe=0.7\tau_e = 0.7, α=1\alpha = 1 (default)57.22+4.79
τe=0.9\tau_e = 0.955.88+3.45
α=0.5\alpha = 0.558.24+5.81
α=2\alpha = 255.22+2.79

Every setting has a higher mean than the backbone, and 24 of 25 seed-level differences are positive. Among the τe\tau_e values the default 0.7 has the highest best-over-training mean, with −0.10 for 0.5 and −1.35 for 0.9. For α\alpha, sharpening to 0.5 gives no reliable change (+1.01, 95% CI [−0.39, +2.42]), while flattening to 2 lowers performance (−2.00) but stays above the backbone. At the end of training α=1\alpha = 1 also has the smallest seed-to-seed spread among the three α\alpha settings, which makes the default a stable choice.

Expert data and gating. The diagnosis is accurate. On near-saturated expert data the residuals have little to rank, and because Eq. 9 standardizes residuals within each batch, the weights keep the same relative spread even when the raw residual spread is small. This is consistent with the end-of-training difference on HE (−2.64), while under the main protocol STEP is on par there (−0.56, CI includes zero). A new sweep over the expert share of the data (Appendix J.7) points the same way. Under the best-over-training rule, the gain is +1.96 at 0% (HM), +2.95 at 25%, +5.42 at 50% (HME), +1.11 at 75% and −0.56 at 100% (HE), largest at 50% and absent on pure expert data. Your suggestion of gating on the raw dispersion σB\sigma_B is a sound remedy, and we will develop and evaluate it. The revised abstract states this result under the best-over-training protocol.

AuthorsOpenReview ↗

Response to Reviewer scyo (Part 3/3)

Q3 / Weakness 3: Component attribution.

We agree that residual-weighted execution is the core of STEP, and the revision makes this explicit. To answer your question directly, residual-weighted execution (C) alone obtains the win-rate gains on mixed-quality data and can be used on its own. It improves on the backbone by +3.66 on HME and +5.72 on HMR (C − A), and on the value estimator alone by +5.73 and +4.14 (C − B). The value estimator (B) is not meant to improve performance by itself, since it supplies the residual that C uses, so B and C form one mechanism. Consequence conditioning (D) gives the executed skill code an explicit prediction signal, which lowers prediction error on HMR in all five seeds on the two unseen maps with offline data (5m and 10m), while its effect on win rate is within seed-to-seed variation. The abstract now states this attribution explicitly, as the contributions and conclusion already did. Residual-weighted execution changes only how the executor is trained, and the deployed architecture, inputs and inference cost are unchanged.

Q4: Other backbones and skill granularity.

Portability to other hierarchical backbones and a controlled study of skill granularity are directions we will pursue, and the Limitations section states that our evidence is on HiSSD.

Benchmark family. Our evidence spans three SMAC task sets with different unit types and team compositions, including the heterogeneous Stalker–Zealot set. SMAC is the standard benchmark for offline multi-task MARL transfer and is used by HiSSD. The backbone's MAMuJoCo data and code have not been released, and we plan to extend STEP to continuous control such as MAMuJoCo.

Q5: Qualitative analysis.

We have added a behavioral analysis on two unseen maps (Appendix J.3). We re-ran the medium-replay checkpoints of STEP and the backbone on 7m_vs_8m and 4m in the same evaluation mode as during training (five seeds, 32 episodes per seed and map, 640 episodes in total) and recorded every action. STEP concentrates fire modestly more (the share of attacking agents that target the most-attacked enemy is 0.694 vs. 0.666 on 7m_vs_8m and 0.863 vs. 0.836 on 4m) and moves more (the fraction of movement actions is 0.312 vs. 0.293 and 0.256 vs. 0.237), and it wins more of these episodes (20.0% vs. 12.5% and 85.0% vs. 72.5%). The appendix also shows the per-agent action traces of one episode per map for both methods, chosen without regard to outcome. The skill-code statistics in Part 2 are also added to the appendix.

We hope these additions, together with the general response, address the concerns that shaped your assessment.

Reviewer_SKXvOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the hierarchical value gap in offline multi-task multi-agent reinforcement learning, where value-guided skill selection does not necessarily lead to high-quality skill execution because low-level policies are typically trained through uniform behavior cloning. To address this issue, the authors propose Skill-conditioned Team-value Execution Projection (STEP), which extends team-value supervision to the execution stage of hierarchical policies. STEP introduces three components: (1) an in-sample expectile team-value estimator that provides TD residuals for logged transitions; (2) a team-residual execution projection that reweights skill-conditioned behavior cloning according to these residuals; and (3) execution-consistent consequence conditioning that incorporates the skill codes used by the executor into the consequence predictor. The method is evaluated on zero-shot transfer tasks in SMAC, using HiSSD as its backbone. STEP improves average win rates over HiSSD by 5.42 and 4.79 percentage points on medium-expert and medium-replay data, respectively, in the Marine-Hard benchmark. Additional experiments on Marine-Easy and Stalker–Zealot tasks support the benefits of residual-weighted execution, particularly on mixed-quality offline datasets.

Critical strengths
  1. A meaningful problem formulation with a practical solution. The paper identifies an important limitation of hierarchical skill-learning methods: selecting a suitable cooperative skill does not guarantee its effective execution. The distinction between skill selection and skill execution is well motivated, and the hierarchical value gap provides a useful perspective for analyzing this limitation. The proposed training-time modification is also practical, as it uses in-sample value estimates to improve the low-level executor without introducing additional inference-time modules or requiring target-task interactions.

  2. Encouraging empirical improvements supported by controlled ablations. The experiments demonstrate consistent improvements over the HiSSD backbone on several mixed-quality offline datasets. The paired-seed evaluations, incremental component ablations, and shuffled-weight control provide useful evidence that the correspondence between TD-residual weights and logged actions contributes to performance gains. The additional evaluations on different SMAC task sets further strengthen the empirical observation that value-weighted execution can benefit zero-shot transfer.

Critical weaknesses
  1. Insufficient justification that TD-residual weighting specifically improves execution quality within a fixed skill. The central motivation of STEP is to distinguish high-quality and low-quality action realizations under the same skill. However, it remains unclear whether the proposed TD residual provides a sufficiently reliable signal for this purpose. Specifically, Eq. (7) computes a one-step TD residual using an expectile-trained team-value function. This residual is not necessarily an unbiased estimate of the action advantage, since the learned expectile value is not generally the value function of a fixed policy. Moreover, a residual computed from a single observed transition may reflect stochastic outcomes and the contributions of other agents rather than the quality of a particular action. Applying the same residual-based weight to all agents further limits the ability to distinguish individual contributions to successful coordination. More importantly, Eq. (12) indicates that improving action preferences within a fixed execution context requires the outcome-averaged weights to differ across actions. However, the empirical analysis in Figure 3 primarily demonstrates an association between residual-based weights and episode-level returns. As acknowledged in Appendix G, the per-step correlations between residuals and return-to-go are weak and mixed in sign. These observations do not directly establish that the proposed weights can reliably distinguish better and worse actions under the same skill and comparable execution contexts. Similarly, the theoretical result in Eq. (13) establishes that monotonic residual weighting increases the expected residual under the reweighted data distribution, but it does not guarantee an improvement in the actual return of the learned decentralized policy. Consequently, the reported performance gains could partly arise from emphasizing more favorable states, trajectories, or task contexts rather than specifically correcting poor action realizations within a skill. Additional evidence is needed to substantiate the claimed mechanism connecting TD-residual weighting to the hierarchical value gap.

  2. Limited methodological novelty and insufficient evidence for the necessity of the full STEP design. The main optimization mechanism combines expectile value estimation with exponentially weighted behavior cloning, which is closely related to established advantage-weighted policy extraction methods, including AWR, AWAC, and IQL. Although applying team-level residual weighting to skill-conditioned decentralized executors is a reasonable extension, the paper does not sufficiently establish why the particular choice of TD residual is preferable to existing advantage-based alternatives. In particular, the experiments lack a direct comparison with skill-conditioned advantage-weighted behavior cloning using an explicitly learned action-value function. Such a comparison would help determine whether the improvement comes from the proposed team-residual formulation or from the more general principle of assigning higher imitation weights to promising offline experiences. Furthermore, the contribution of execution-consistent consequence conditioning is not consistently supported by the ablation results. According to Table 12, adding this component improves performance on medium-expert data but slightly reduces performance on medium-replay data. Therefore, the experimental evidence more strongly supports residual-weighted behavior cloning than the necessity of the complete three-component framework. There are also limitations in the strength of the generalization claims. The main results use the best checkpoint selected separately for each evaluation task, and the nine primary unseen maps were repeatedly inspected during development. Although the authors provide final-checkpoint and additional holdout evaluations, the holdout study is mainly presented as a check against performance collapse rather than independent evidence of consistent gains. Notably, Appendix F reports a statistically significant decrease of 2.64 percentage points on expert data at the final checkpoint, which qualifies the claim that STEP does not sacrifice performance on high-quality datasets. Taken together, the results support the practical effectiveness of residual-weighted imitation on mixed-quality datasets, but they do not yet fully establish the necessity of the proposed formulation or the robustness of all claimed benefits.

Questions for authors
  1. How can the authors verify that TD-residual weighting improves action selection within the same skill, rather than primarily reweighting favorable states or trajectories? Given the condition identified in Eq. (12), it would be valuable to provide a more direct analysis of whether residual-based weights distinguish the quality of different joint actions under fixed skills and comparable execution contexts. Could the authors provide such an analysis, or clarify what evidence supports the proposed within-skill mechanism beyond the episode-level correlations reported in Figure 3 and Appendix G?

  2. Why is a one-step TD residual preferred over an explicitly estimated skill-conditioned advantage? Since the proposed weighting mechanism is closely related to advantage-weighted regression, could the authors compare STEP against a matched baseline using learned action advantages, such as a skill-conditioned Q-value and value baseline, under the same backbone and training budget? This comparison would help clarify the necessity and effectiveness of the proposed TD-residual formulation.

  3. How essential is execution-consistent consequence conditioning, and how should the performance claims be interpreted under selection-free evaluation? Table 12 shows that consequence conditioning does not consistently improve over residual-weighted execution alone. Could the authors clarify its independent contribution and the conditions under which it is beneficial? In addition, how do the authors reconcile the claim of preserving expert-data performance with the statistically significant expert-data degradation reported at the final checkpoint in Appendix F? A clearer discussion of checkpoint selection and independent holdout performance would strengthen the conclusions about zero-shot generalization.

6 comments
AuthorsOpenReview ↗

Response to Reviewer SKXv (Part 1/3)

We thank the reviewer for the constructive review. We ran new controls to answer the central question, whether residual weighting improves execution within a fixed skill or mainly reweights favorable states and trajectories. All results below use the main-text protocol, with average win rates over the nine unseen Marine-Hard maps, paired seeds and paired bootstrap 95% CIs.

Q1 / Weakness 1: Does TD-residual weighting act within a skill, or mainly on favorable states and trajectories?

Each control below changes only the weights and keeps everything else in STEP fixed, with five paired seeds.

Arm (what the weights keep)HMEHMR
STEP68.1957.22
Skill-group mean weights (across-skill differences only)63.1554.40
Return-to-go weights (favor high-return trajectories)66.3953.76
HiSSD backbone (uniform weights)62.7852.43
STEP − skill-group mean+5.04 [+3.65, +6.42]+2.82 [+1.54, +4.42]
STEP − return-to-go+1.81 [+0.63, +2.99]+3.46 [+2.25, +4.75]

The first control replaces each weight by the mean weight of its skill group in the batch (k-means on the executed skill codes, K = 16). It keeps any preference for better skills or batches but removes variation inside a skill, which is the condition-only multiplier that cancels in Eq. 12. The weights vary mostly inside skill groups, since across 45 source-task, seed and regime cells only 0.1% to 2.6% of the weight variance lies between the K = 16 groups, and at most 3.4% for K = 8 or 32. Removing this within-group variation removes most of the gain on HME, where the result falls back to the backbone level, and about 60% of it on HMR, with all five paired seeds favoring STEP in each regime.

The second control weights by return-to-go, which directly implements "imitate transitions from better trajectories". It is above the backbone on average, clearly so on HME, but the TD residual is better in both regimes. Return-to-go also decreases along an episode because SMAC rewards are non-negative, so it favors early steps regardless of execution quality, while the residual compares each step with the learned team baseline and removes this trend. As in the submission, shuffling the weights across the batch also removes the gain (C − C-shuf is +5.93 on HME and +4.18 on HMR, Table 12), so the weights must match their transitions.

Together, these controls show that the gain depends on how the weights vary within skill groups and that this variation carries information beyond trajectory return. We also ran the finer-grained test you suggested, asking whether the recorded action of each agent explains the weights within a fixed skill cluster and a comparable context. At this resolution the individual action alone does not reliably explain the weights, so we state the mechanism at the level the controls establish. STEP exploits within-skill differences in realized execution quality, which can reflect both the actions taken and the states they lead to. All analyses are added to Appendix J.2.

On the signal itself. These are deliberate design choices, explained in Section 4.3. The residual is not an unbiased advantage, and it does not need to be one, because Eq. 9 standardizes it within each batch so that it only ranks logged realizations and a uniform offset from the expectile baseline cancels. Part 2 compares it with an explicit advantage estimate. Regarding stochastic outcomes and other agents, Eq. 12 shows that the target depends on outcome-averaged weights for each action, so the noise of a single transition is averaged over the logged data rather than determining the target. The weight is shared by the team because, with a single cooperative reward, the team residual is the quantity the decentralized executors jointly optimize, and weighted imitation projects this team-level target onto the decentralized product policy (Appendix A). Per-agent credit assignment is a direction we plan to explore. Finally, Eq. 13 is a target-level statement, and its link to return is supported empirically by the controls above.

AuthorsOpenReview ↗

Response to Reviewer SKXv (Part 2/3)

Q2 / Weakness 2: Why a one-step TD residual rather than an explicit skill-conditioned advantage? Novelty relative to AWR, AWAC and IQL.

We agree that exponentiated-advantage weighting and expectile value learning are established tools, and our Related Work (Section 2) positions STEP relative to them. Our contribution has three parts. (i) We identify the hierarchical value gap. (ii) Definition 1 and Eq. 12 show why value-guided skill selection cannot close it, because any condition-only multiplier cancels within a skill. (iii) STEP is a training-time method that carries team value into skill-conditioned decentralized executors in the multi-source zero-shot setting, and our controls identify which part of the signal matters.

To test the choice of signal, we ran the matched baseline you suggested. It replaces δ\delta with an explicit team advantage A=Qtot(s,a)−Vtot(s)A = Q_{\mathrm{tot}}(s, a) - V_{\mathrm{tot}}(s) from a learned Q critic. The critic uses per-entity Q heads of the same width as our value head, an attention mixer of the same type, and the standard IQL Q loss on the same TD target. Standardization, α\alpha and clipping are unchanged. All of its settings follow our value head and were fixed in advance. Results use paired seeds 1 to 3.

HMEHMR
STEP (one-step TD residual)69.4256.16
Explicit advantage Q − V67.0656.48
STEP − explicit advantage+2.36 [+0.21, +3.47]−0.32 [−1.39, +0.77]

On HME the one-step residual outperforms the explicit advantage, with all three paired seeds positive, and on HMR the two are on par. One possible reason is that the Q critic must learn the dependence on joint actions from offline data, whereas the residual uses the observed reward and next state directly. The residual thus matches or exceeds the explicit advantage without an additional critic. Both signals also improve on the backbone on the same seeds (explicit advantage +4.51 on HME and +3.52 on HMR), which supports contribution (iii), since the benefit comes from carrying team value into the skill-conditioned executor.

The return-to-go control in Part 1 answers the related question of whether the general principle of giving promising experience more weight is enough. It helps, but less than the residual in both regimes. Both comparisons are added to Appendix J.2, and Section 5.4 summarizes the return-to-go and skill-group controls.

AuthorsOpenReview ↗

Response to Reviewer SKXv (Part 3/3)

Q3a: How critical is consequence conditioning?

We agree with the reviewer's reading of the ablations, and the revision makes it explicit. The win-rate gains are carried by residual-weighted execution, which improves on the backbone by +3.66 on HME and +5.72 on HMR (C − A) and on the value estimator alone by +5.73 and +4.14 (C − B), so residual-weighted execution alone obtains the gains. Consequence conditioning gives the executed skill code an explicit prediction signal. Our diagnostics confirm that this path is active, as the projection moves away from its zero initialization, prediction gradients reach the code, and on HMR it lowers consequence-prediction error on the unseen maps with offline data in all five seeds. Its effect on win rate is within seed-to-seed variation (D − C is +2.46, +0.63, +1.75 and −0.93 on HE, HM, HME and HMR). The abstract now states this attribution explicitly, as the contribution list and conclusion already did.

Q3b: Performance on expert data.

Under the main protocol STEP is on par with the backbone on HE (−0.56, CI includes zero), while it improves the mixed-quality regimes by +5.42 (HME) and +4.79 (HMR). The end-of-training difference on HE in Appendix F (−2.64) is consistent with return saturation, since two HE source tasks are nearly return-saturated (Table 10) and the residuals have little to rank there. A new sweep over the expert share of the data (Appendix J.7) shows the same picture. Under the best-over-training rule, the gain is +1.96 at 0% (HM), +2.95 at 25%, +5.42 at 50% (HME), +1.11 at 75% and −0.56 at 100% (HE), largest at 50% and absent on pure expert data. The revised abstract states the result precisely, with gains on mixed-quality data and parity on expert data under the best-over-training protocol. Weighting that adapts to the residual dispersion is a direction we will develop.

Q3c: Checkpoint selection and the holdout maps.

Best-over-training per task follows ODIS and HiSSD. Appendix F reports every cell at the end of training (the win rate logged at 35k steps), where the gains are +4.40 [+2.22, +7.12] on HME and +6.31 [+3.88, +8.70] on HMR, with all five paired seeds positive in both regimes. The new controls in Part 1 keep their direction at the end of training as well, so the HME and HMR gains do not depend on picking the best point of training. The holdout maps, never inspected during development, were designed as a collapse check. No collapse appears, and the two cells whose intervals exclude zero both favor STEP.

We appreciate the reviewer's questions, which led to the most informative new controls in this revision.

Reviewer_GtAyOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work discussed the setting of multi-task reinforcement learning from offline multi-task datasets in a cooperative setting. Usually, hierarchical skill learning is used to provide high-level skill selection, while skill execution is carried out by behavior cloning. The authors have highlighted this as a critical issue for learning since the value supervising signal is stop at the selection without guarantee high action selection during execution. The authors refer to this as hierarchical value gap. To account for this, this work introduced Skill-conditioned Team-value Execution Projection (STEP), a training-time framework that completes the value pathway of hierarchical skill policies by extending the team-value supervision signal from skill selection to skill execution. Empirical results show the potential of STEP on nine unseen maps from the SMAC benchmark in comparison to other methods, including the backbone method HiSSD.

Critical strengths
  • The value gap claim sounds and is supported by the explanation of the authors as well as the empirical results reported for the proposed solution. I agree with the authors that forwarding such a value signal to the execution part of the skills would ideally allow for high-performance realization for skills aligned with the high-level skill selection.
  • The experiments and ablations provided are dense, especially the ablations, which cover the importance of each component proposed in this work.
Critical weaknesses
  • In some parts, the paper was hard to follow. For instance, the method section is dense with mathematical content and has a less clear narrative. This does not mean that many mathematical quantities are not defined, but the flow of this section was a challenge to keep up with while some quantities are briefly explained. I believe part of this is my lack of knowledge of this topic, yet the manuscript should assume such a situation and provide a clear explanation with a coherent flow. In addition, a few parts in the experiment section were also challenging to follow, especially the plots and their captions.
  • The limitations of the proposed method, STEP, were not discussed by the authors. Given the complicated nature of the algorithm, the insights of the authors on the method would be appreciated.
  • For instance, the method introduces many hyperparameters; how sensitive is the method to tuning those? I understand that the tuning was done on a limited number of maps, but I am not sure if it was discussed how sensitive the method is.
  • The components introduced would increase the overhead relative to the backbone method. I believe this was not discussed in the main paper. The overhead could be in time or space complexity. Such discussion would allow a better understanding of the benefit of the method given the reported results.

A combination of the clarity of the writing and the unmentioned limitations is the reason for my recommendation. I could change my recommendation if my points are addressed and the introduced overhead by STEP is not a major bottleneck.

Questions for authors
  • What is the role of α\alpha in Equation 4?
  • What is the limitation of the proposed method?
  • What is the overhead that STEP adds on top of the backbone, HiSSD?
  • How sensitive is STEP to the introduced hyperparameters?
6 comments
AuthorsOpenReview ↗

Response to Reviewer GtAy (Part 1/2)

We thank the reviewer for the careful reading. We address each point below with new measurements and concrete revisions.

Weakness 1: "the method section is dense with mathematical content and has a less clear narrative"

Thank you for this candid feedback. The revision makes three changes to improve readability. Section 4.1 now ends with a plain-language summary of STEP and points to the notation. In short, STEP learns a team value from the offline data, scores each logged transition by how much its outcome exceeds this value's prediction, and imitates higher-scoring executions of each skill more strongly. Appendix J.1 adds a notation table that lists the main symbols with a one-line meaning. The captions of Figures 2 to 4 are rewritten to be self-contained, stating the data regime, the reporting protocol, what each panel plots and how to read it.

Q1: What is the role of α\alpha in Equation 4?

Eq. 4 and its implementation in Eq. 9 use the same α\alpha. It is a temperature that sets how strongly the execution target favors higher-value realizations of a skill, since Eq. 4 is the solution of the KL-regularized problem max⁡qEq[δ]−αKL(q∥μ)\max_q \mathbb{E}_q[\delta] - \alpha \mathrm{KL}(q \Vert \mu), where μ\mu is the logged distribution of skills and actions. As α\alpha grows, the target approaches the logged distribution and the executor loss reduces to uniform behavior cloning, as in the backbone. As α\alpha shrinks, the target concentrates on the highest-residual logged realizations and approaches hard selection, and consistent with this, keeping only high-return episodes (%BC, Table 11) is weaker than graded weighting. Intermediate values trade off imitating higher-value executions against staying close to the data, always within logged support.

In Eq. 9, residuals are standardized within each batch, so α\alpha is measured in units of the batch residual standard deviation (Appendix A.1). With α=1\alpha = 1, a transition one standard deviation above the batch mean receives about 2.7 times the weight of a transition at the mean. The ratio is about 7.4 for α=0.5\alpha = 0.5 and about 1.6 for α=2\alpha = 2. Weights are then mean-normalized and capped at wmax⁡w_{\max}. The revision states this interpretation right after Eq. 4.

Q2: What is the limitation of the proposed method? (also Weakness 2)

Your comment led us to add a dedicated Limitations section, which sets out the directions that the current work leaves open. First, STEP shares one team weight across the agents of a transition. This matches the single cooperative reward, and assigning per-agent credit within this weight remains open. Second, the benefit of reweighting depends on quality variation in the logged executions. On near-saturated expert data there is little to reweight, and adapting the weighting strength to the residual dispersion is an extension we will develop. Third, our controls locate the useful signal within skills, and separating the contribution of the action from that of the state it leads to requires finer analyses. Finally, our experiments cover three SMAC task sets (Marine-Easy, Marine-Hard and Stalker–Zealot) on the HiSSD backbone. Because STEP acts on the executor's training objective, it applies to other hierarchical skill methods with a behavior-cloned executor, and evaluating them and environments such as MAMuJoCo is future work that we plan to pursue (the backbone's MAMuJoCo data and code have not been released).

AuthorsOpenReview ↗

Response to Reviewer GtAy (Part 2/2)

Q3: What is the overhead that STEP adds on top of the backbone, HiSSD? (also Weakness 4)

The overhead should indeed be reported. The revision adds this comparison to the appendix (Appendix J.6) and states the cost in Section 5.1. Overall, STEP adds no cost at deployment and a modest cost during training.

HiSSD backboneSTEP
Computation at deploymentbackbone forward passidentical forward pass
Extra trainable parameters (training only)none+47.8k (+7.0%)
Time per training update (same GPU, same batches)1.00×1.16× to 1.18×
Training budget35k stepsunchanged, no extra stage
Online interaction or target-task datanonenone

The value head, its mixer and the conditioning projection are called only inside the training losses, so the deployed policy runs the backbone's forward pass, and decision latency and deployment memory are unchanged. STEP adds 47,794 trainable parameters, of which 31,282 belong to the value head and its mixer and 16,512 to the conditioning projection. Including the untrained Polyak target copy of the value estimator, the added parameter memory is below 0.4 MB. The submitted Appendix D attributed all of these parameters to the value estimator, and the revision corrects this. For time, we measured 2,000 updates per arm on the same GPU with identical batches, in two independent rounds. The median time per update is 1.16 and 1.18 times that of the backbone, and 1.14 times without consequence conditioning. Because the step budget is unchanged, training takes about 16% to 18% longer, with no extra training stage.

Q4: How sensitive is STEP to the introduced hyperparameters? (also Weakness 3)

STEP has two substantive hyperparameters, the expectile level τe\tau_e (Eq. 8) and the weighting temperature α\alpha (Eqs. 4 and 9), while wmax⁡w_{\max} and ϵ\epsilon are numerical safeguards. All regimes and task sets use the same defaults, with no per-regime tuning. The revision adds a sensitivity study of both on Marine-Hard medium-replay (HMR), varying one at a time around the default (τe=0.7\tau_e = 0.7, α=1\alpha = 1) with seeds 1 to 5 paired with the backbone. Entries are average win rates (%) over the nine unseen maps, reported as in the main tables.

SettingWin rateGain vs. backbone
HiSSD backbone52.43(reference)
τe=0.5\tau_e = 0.557.13+4.70
τe=0.7\tau_e = 0.7, α=1\alpha = 1 (default)57.22+4.79
τe=0.9\tau_e = 0.955.88+3.45
α=0.5\alpha = 0.558.24+5.81
α=2\alpha = 255.22+2.79

Every setting has a higher mean than the backbone, and 24 of the 25 seed-level paired differences are positive. Every setting also stays above the backbone at the end of training (Appendix J.5), so on HMR the benefit does not depend on a narrow hyperparameter choice. Among the τe\tau_e values, the default 0.7 has the highest best-over-training mean, with no change at 0.5 (−0.10) and a moderate decrease at 0.9 (−1.35). For α\alpha, sharpening to 0.5 gives no reliable change (+1.01, 95% CI [−0.39, +2.42]), while flattening to 2 lowers performance (−2.00, CI [−3.56, −0.75]) but still improves on the backbone by 2.79. At the end of training, α=1\alpha = 1 also has the smallest seed-to-seed spread among the three α\alpha settings (std 2.27, compared with 3.55 for α=0.5\alpha = 0.5 and 2.98 for α=2\alpha = 2) and improves over the backbone on all five seeds under both metrics (+4.79 best, +6.31 final), which makes the default a stable choice. The full table is in Appendix J.5, and Section 5.4 summarizes it.

These revisions cover readability, limitations, overhead and sensitivity. STEP adds no deployment cost, takes about 16% to 18% more training time, and has a higher mean than the backbone under every tested setting.

Dear Reviewer GtAy,

We hope this message finds you well. We are grateful for your careful review and have provided thorough responses to all your questions, together with a revised PDF in which the changes are marked in blue.

With the discussion deadline approaching, we would deeply appreciate the opportunity to hear your feedback on our replies. Should you have any remaining doubts, we are glad to clarify or revise further at your convenience.

Thank you again for your valuable time and guidance.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers and the area chair for their careful reading and constructive feedback. We are encouraged that the reviewers found the hierarchical value gap a genuine and well-motivated limitation of skill-based offline MARL, the training-only change practical, and the controlled evaluation with paired seeds, shuffled-weight controls and ablations careful.

In response to the common questions, we have run new experiments and revised the paper. New weight controls show that the weights vary almost entirely within skill groups and that this within-group variation carries the gain, and the TD residual outperforms return-to-go weighting and matches or exceeds a matched explicit-advantage baseline. We add end-of-training and per-map results, a sensitivity study of τeτ_e and αα, and a sweep over the expert share of the data. The abstract now attributes the gains to residual-weighted execution and states the expert-data result under the best-over-training protocol, and we add a Limitations section, a plain-language method summary, self-contained figure captions and the training overhead. We also correct the seed count of the submitted Stalker–Zealot table, which now reports all five paired seeds and shows larger gains.

The new material is located as follows: weight controls and within-skill analysis (Section 5.4, Appendix J.2), behavior on unseen maps (J.3), consequence-conditioning diagnostics (J.4), sensitivity (J.5), compute cost (Section 5.1, J.6), expert-share sweep (J.7), per-map results (J.8), end-of-training results (Appendix F) and notation (J.1). Each reviewer's questions are answered in the individual responses, and we would be glad to discuss any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.