- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces BEAD (Blind-spot Entity-aware Advantage Distillation), a centralized-teacher, decentralized-student framework for cooperative multi-agent RL. It uses gated attention to incorporate globally available entities that individual agents cannot observe into the teacher’s representations, then distills relative action advantages, rather than raw Q-values, into the decentralized student. Experiments on SMAC and SMACv2 show that BEAD outperforms strong CTDE and teacher–student baselines, particularly in challenging asymmetric-coordination scenarios. Ablation studies further demonstrate the benefits of both the gating mechanism and advantage-based distillation..
Critical strengths
- The paper is well-written, clear, and easy to follow. I enjoyed reading it.
- The use of attention-based gating in the teacher appears novel, and the intuition behind the approach is clear- not all globally-observable information is useful to a given agent.
- The paper tests on challenging settings on SMAC and SMACv2 maps.
Critical weaknesses
- Insufficient statistical reporting: The most critical weakness, in my view, is that the paper does not report the number of random seeds used, and the results are presented without standard deviations. The authors should evaluate their main experiments across at least 5–7 random seeds and report the variability of the results. Without this, it is impossible to assess whether the reported results are statistically meaningful [1].
- Analysis: The paper does not analyze what the attention and gating mechanisms actually learn to select or suppress, e.g., via attention-weight visualizations or a case study of which blind-spot entities are prioritized in a given state. Without such analysis, the mechanism remains an uninterpreted transformation: it could be performing the claimed entity-level relevance filtering, or it could be learning some other function entirely that happens to help performance. An example where gating helps would substantiate the selectivity claim made in the paper.
- Comparison to relevant methods: The authors lack any mechanistic or qualitative analysis comparing what their gated blind-spot attention actually selects or suppresses against what prior selective approaches (e.g., PTDE’s agent-specific representations) do, relying instead on aggregate downstream performance as the sole evidence of improved selectivity.
Without a more detailed analysis, the benefit of the proposed mechanism cannot be distinguished from that of a general attention/gating mechanism, preventing me from giving a higher score. I would be happy to raise my score if the authors can adequately address those issues.
[1] Agarwal, Rishabh, et al. "Deep reinforcement learning at the edge of the statistical precipice." Advances in neural information processing systems 34 (2021): 29304-29320.
Questions for authors
- Statistical significance and seed variability: The paper reports win rates (e.g., 56.25%, 65.63%) without specifying the number of random seeds or reporting variance or confidence intervals. Given the known run-to-run variability in SMAC, how many seeds were used, and are the reported improvements over the baselines statistically robust?
4 comments
Dear Reviewer,
Thank you for your further feedback. We understand your concerns regarding the rigor of the multi seed evaluation and the justification of our teacher side contribution. We address these two points below.
1. Complete five seed evaluation and reporting of variability.
We agree that reporting results from a single training seed was insufficient and that the number of seeds should have been stated clearly. Following our previous response, we have completed evaluations over five seeds for all experiments in the main text, including the complete SMAC and SMACv2 comparisons and the ablation studies.
The revised manuscript now reports means and sample standard deviations across five independent training seeds, and the training curves display the mean across seeds with shaded regions indicating ±1 standard deviation. The additional MPE and Google Research Football evaluations also report results over five seeds. Unless explicitly stated otherwise, experiments use five independent training seeds.
These revisions extend the evaluation over multiple seeds to the complete main benchmark comparison rather than only selected maps. BEAD achieves the highest mean decentralized win rate on six of the eight SMAC and SMACv2 scenarios. Its largest mean improvements over HPN are observed on 3s5z_vs_3s6z and 6h_vs_8z, reaching 24.69 and 25.63 percentage points, respectively. We believe these expanded results provide substantially stronger empirical support for BEAD’s effectiveness while explicitly documenting variability across training runs. We acknowledge that this evaluation should have been included in the original submission.
2. Teacher side contribution and its distinction from PTDE.
We would like to clarify that we do not claim the Transformer architecture or attention mechanism itself as our teacher side novelty. As summarized in the Introduction and explained in the subsection on gated blind spot compensation, our contribution is gated blind spot compensation. This design identifies globally available but locally unobserved entities, aggregates their information for each agent, and adaptively regulates its incorporation into the local decision pathway.
As discussed in our previous response, this design differs from the PTDE GIP teacher in three specific respects.
(i) Information eligibility. PTDE constructs an agent personalized latent representation from the flattened global state through a projection conditioned on the local input. BEAD explicitly applies a visibility mask to restrict privileged candidates to active, locally unobserved entities. It therefore distinguishes information that complements the local observation from information already available to the agent.
(ii) Entity weighting. PTDE personalizes global information through a learned transformation of global state coordinates. BEAD instead uses the local embedding as a query to assign explicit attention weights to eligible blind spot entities. This separates entity eligibility from weighting for each agent, rather than combining both within a transformation of the flattened global state.
(iii) Injection strength and location. PTDE concatenates its personalized global latent with the local representation after the recurrent update. BEAD applies a learned elementwise gate to the pooled blind spot context and injects it additively before the recurrent update. This introduces explicit control over how strongly privileged context influences the recurrent decision representation.
These architectural distinctions are complemented by the experimental results. Across five training seeds, BEAD achieves higher mean decentralized performance than PTDE on all eight SMAC and SMACv2 scenarios, as well as on MPE and Google Research Football. The gate ablation further shows that learned gating improves mean teacher win rate from 62.50 ± 13.62% without gating to 81.25 ± 14.32%, supporting the importance of regulating context injection beyond attention based aggregation alone.
Our matched entity interventions additionally connect entity weighting to changes in the teacher’s action values, while the controlled gate interventions illustrate how context injection affects subsequent decisions. Together, the structural comparison, component ablations, and benchmark results support gated blind spot compensation as a distinct teacher design. The contribution lies in the coordinated treatment of entity eligibility based on visibility, weighting for each agent, and gated injection, rather than the use of attention alone.
Thank you again for your careful reading and constructive feedback. We sincerely hope that our response and the revised manuscript will encourage you to reconsider your assessment of our work.
Dear authors,
Multi-seed evaluation is an integral part of an empirical algorithmic paper. It should have been in the original submission rather than added over the discussion period after repeated requests, and I do not view this favorably.
I do not dispute the motivation or the core idea behind the work. My main concern is that the algorithmic instantiations of the idea (e.g., attention and gating) are not sufficiently analyzed or shown to be connected to the underlying motivation. Since mechanisms such as gating and attention are widely used, I believe the paper needs to demonstrate that these mechanisms are specifically connected to and support the proposed novel idea.
In the current state, I do not feel comfortable assigning a score above 2. I will raise my score to 2, and given the higher scores from the other reviewers, this may be sufficient for the paper. However, I cannot argue in favor of acceptance.
Dear Reviewer,
Thank you for considering our revisions and for raising your score. We appreciate your recognition of the motivation and core idea. We also acknowledge that the evaluation across multiple seeds should have been included in the original submission.
We would like to briefly emphasize our two contributions.
1. Gated blind spot compensation. Our contribution is the coordinated design that explicitly identifies locally unobserved entities, weights their relevance to each agent, and adaptively regulates their injection into the local decision pathway. This connects privileged information to the agent’s specific information gap, beyond simply adding attention or gating.
2. Student value anchored explicit advantage distillation. We transfer the teacher’s relative action preferences while retaining the student’s own TD objective for value learning. Our theoretical analysis establishes that mean centering preserves pairwise action preferences and decomposes action value matching error into advantage disagreement and mean value disagreement, providing a principled justification for this objective.
Together, these contributions address both how to selectively exploit privileged information and how to transfer decision relevant knowledge to decentralized students. We hope this clarifies the specific novelty of our approach.
Thank you again for your careful assessment and your willingness to revise the score.
Dear authors, I would like to thank you for the effort put into your response.
Regarding the original manuscript, for a paper proposing a new algorithm and relying primarily on empirical evaluation, reporting results from only a single seed, without clearly stating the actual number of seeds used, is problematic in terms of both soundness and clarity. An approach such as BEAD cannot be properly evaluated on this basis. That said, I would be satisfied if the authors revised the manuscript to report standard deviations for all graphs and complete results over 5–7 seeds. However, from the response and revised manuscript, it appears that the graphs remain largely unchanged, with complete multi-seed results provided only for some maps.
I understand that the discussion period is limited and that you had only a week to respond. However, since this is essential to evaluating the paper's main claims, these results should ideally have been prepared before submission.
Regarding the analysis, examining only two training seeds on limited states per seed is insufficient to establish that the observed effects are systematic and consistent. A convincing mechanistic analysis should be conducted across more training seeds and report aggregate statistics and variability to rule out seed-dependent effects. As I mentioned in my original review, the paper is well written and presents a simple, clean idea, which I found compelling as a motivation for the proposed attention modification. However, since the algorithmic novelty on the teacher side ultimately amounts to adding an attention mechanism, which is already widely used, the authors need to provide a detailed analysis demonstrating why this particular modification is effective to support the novelty of their approach.
I wish you the best with the paper, but given the remaining concerns, I cannot justify a higher score in its current state.