- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SPAR, a framework for learning structured and reusable roles in cooperative MARL. Built on ACORM's role-conditioned QMIX framework, it uses recurrent semantic slots to assign each agent a primary and an auxiliary role. These assignments guide contrastive learning, preserving role distinctions while reducing repulsion between related roles. A performance-guided slot memory retrieves historical role structures based on contextual similarity and episode performance for reuse during training. Experiments on SMAC and SMACv2 report competitive performance, with ablations and qualitative analyses examining the proposed components and role dynamics.
Critical strengths
The paper is clearly written and well organized. The motivation for capturing relationships between roles and reusing role structures across episodes is clear. Throughout the manuscript, many aspects are not clearly or rigorously presented, which makes the paper confusing to read. Some details of the experimental setup are also insufficiently described. Overall, the manuscript requires another round of review.
Critical weaknesses
The method largely relies on established techniques, and the paper does not sufficiently distinguish its contribution from closely related work.
Questions for authors
The paper focuses on role learning, with primary-auxiliary role assignments and episodic slot memory as its main additions. At present, these seem mainly to combine established ideas, and the methodological advance is not sufficiently clear to me. I would appreciate clarification on the following points:
- What does the acronym “SPAR” stand for?
- The introduction identifies the challenge of connecting agents' distinct behaviors with their contributions to team coordination. What concrete problems arise when this connection is missing, and what evidence supports this motivation?
- The related-work discussion should consider more recent methods, such as GRDC (Gao et al., 2026). How does SPAR differ from its approach to prototype-based role discovery?
- What motivates the recurrent slot refinement in Eq. (7)? Figure 1 indicates that a GRU is used: are its parameters trained, and through which loss do they receive gradients, given the hard role assignments and detached weighting gates? What is the benefit of dynamically updating the slots, and could a comparison with fixed slots be provided?
- Zhang et al. (2025), in Role can be beneficial to mean field multi-agent reinforcement learning, already use contrastive learning to obtain role representations. Although their learning framework differs, what substantive advances does SPAR introduce over such approaches beyond primary-auxiliary role assignments?
- What explains SPAR’s lower mean win rates than the strongest baselines on all three SMACv2 10_vs_11 tasks?
- Please provide the SMACv2 learning curves and clarify the training budget for each scenario. Were these runs trained for two million environment steps, or a different budget, and were the budgets matched across methods?
- Beyond the single-trajectory visualization, what evidence shows that the primary and auxiliary assignments correspond to meaningful behaviors or coordination functions? In particular, how can we tell whether the auxiliary assignment represents a meaningful secondary role rather than simply the second-highest slot score?
4 comments
Response to Reviewer tiPy (2/2)
Response to Question 5
We agree that contrastive learning of role representations has been studied previously, and SPAR does not claim to introduce its first use in role discovery. We have added RMFQ (Zhang et al., 2025) and clarified its differences from SPAR in Section 2.1, lines 107–114.
RMFQ combines adaptive role assignment with a role-weighted mean-field approximation. SPAR's substantive extensions include selectively adjusting negative repulsion through primary–auxiliary associations and using performance-guided SlotBank to select and reuse historical slot states during training. Beyond storing historical representations, the latter filters candidates according to the current context and then selects content for reuse based on historical episode performance. The mechanisms are detailed in Sections 3.2 and 3.3.
We have added controls targeting these mechanisms in Section 4.3 and Figure 4 of the main text. Uniform, Shuffled, and the two memory-content controls achieve win rates of 60.38%, 55.63%, 66.88%, and 61.25%, respectively, all below the full SPAR method's 77.50%. These results support the contributions of weight allocation, performance-guided content selection, and contextual correspondence on this scenario, rather than merely supporting the addition of an auxiliary label or memory bank.
Response to Question 6
Thank you for highlighting this limitation. We believe one possible reason is that SMACv2 randomizes unit types, team compositions, and initial positions, requiring role inference to adapt to changing coordination configurations and making it harder to match historical slot structures to the current context. In the 10_vs_11 tasks, which involve larger teams and an enemy numerical advantage, these demands may further limit the effectiveness of role modeling and memory reuse.
ACORM also falls below the strongest baseline in mean win rate on all three tasks. SPAR improves over ACORM on each task, indicating that the proposed mechanisms remain useful under randomized configurations, but these gains are insufficient to close the gap to the strongest methods.
We have explicitly acknowledged this performance boundary in Section 4.1 of the main text, lines 399–402, and Appendix E.1, lines 1093–1098, and provided the corresponding learning curves in Appendix Figure 7.
Response to Question 7
We have added learning curves for seven methods across six SMACv2 scenarios in Appendix E.1, Figure 7, with a pointer in Section 4.1 of the main text, lines 401–402. Different scenarios use different cutoffs for curve display and result aggregation, while all methods within the same scenario use the same cutoff.
| Scenario | Logged-step cutoff for result aggregation |
|---|---|
| Protoss 5_vs_5 | 2.0M |
| Terran 5_vs_5 | 2.0M |
| Zerg 5_vs_5 | 2.0M |
| Protoss 10_vs_11 | 3.0M |
| Terran 10_vs_11 | 2.5M |
| Zerg 10_vs_11 | 3.0M |
To facilitate comparisons at common budgets, we have also added results for SPAR, ACORM, and QMIX at 0.5M, 1.0M, 1.5M, and 2.0M logged steps in Appendix E.2, Table 4. For each seed, we use the last recorded evaluation at or before the specified budget, then report the mean and sample standard deviation across five seeds. At 2.0M, SPAR achieves higher mean win rates than ACORM on five of the six tasks and higher means than QMIX on all six. ACORM remains stronger on Protoss 5v5, and the rankings also vary at earlier budgets.
Response to Question 8
The auxiliary role is indeed computed from the second-highest slot-matching score, and its additional function is to inform contrastive learning. When one agent's auxiliary role matches another agent's primary role, SPAR selectively weakens the negative repulsion between them. The relevant question is therefore whether this secondary association provides useful relational information.
To examine this, we have added Uniform and Shuffled controls in Section 4.3 and Figure 4 of the main text. Full SPAR achieves a win rate of 77.50%, compared with 60.38% and 55.63% for the two controls, respectively. In particular, Shuffled preserves the negative-weight multiset and only shuffles its assignment to sample pairs. Its lower mean win rate supports the practical learning value of allocating weights according to role associations.
However, this evidence does not establish that auxiliary slots correspond to distinct, nameable behavioral responsibilities, such as covering or supporting other agents. Section 4.2 of the main text, lines 465–474, reports 16 auxiliary-role switches and 12 primary-role switches in the displayed trajectory and explicitly identifies the visualization as illustrative. We therefore interpret auxiliary roles as secondary slot associations useful for cooperative representation learning. Whether they consistently correspond to distinct behavioral functions still requires validation through intervention analysis.
Thank you for your response. Your clarifications helped me better understand the paper, and I have raised my score to 3.
Thank You for Your Feedback
We sincerely thank the reviewer for the positive feedback and for reconsidering the assessment. We are glad that our clarifications and additional experiments helped address the concerns. We greatly appreciate your constructive suggestions.
Response to Reviewer tiPy(1/2)
Response to Question 1
SPAR stands for Structured Primary Auxiliary Roles. The full name is given in the Abstract, line 017, and the Introduction, lines 069–071 of the revised manuscript, together with an explanation that primary and auxiliary roles represent the strongest and second-strongest slot associations, respectively.
Response to Question 2
The specific issue we address is that agents with different behaviors may still have coordination-relevant relationships that a single role label cannot adequately express. For example, an agent drawing enemy fire and another concentrating damage may have different primary roles while acting complementarily. If contrastive pairs are constructed solely according to whether their primary roles match, such cross-role pairs are also uniformly repelled as negatives. This example illustrates the design motivation. It is not a behavioral annotation of the learned roles.
We have specified the limitations of single-role representations and cross-episode structure retention in the Introduction, lines 061–074, and explained potentially related cross-role negatives and their treatment in Section 3.2, lines 246–265.
In Section 4.3 and Figure 4 of the main text, SPAR achieves a test win rate of 77.50%, compared with 60.38% for Uniform, which matches the mean weakening strength, and 55.63% for Shuffled, which preserves the weight multiset but shuffles its assignment. These results support the usefulness of role-conditioned weight allocation, but do not directly measure the causal contributions of individual behaviors to team return. We therefore interpret them as mechanistic evidence for cooperative representation learning, rather than as an attribution of individual contributions.
Response to Question 3
Thank you for this suggestion. We have added a discussion of GRDC (Gao et al., 2026) and its differences from SPAR in Section 2.1, lines 107–114.
GRDC constructs local interaction graphs from trajectory representations and uses prototype-based role discovery to organize intra-role communication. SPAR instead uses primary–auxiliary slot associations to adjust the contrastive weights of negative pairs with different primary roles, and reuses slot states during training according to context and historical episode performance. Both methods use prototype-based role representations, but use role information differently, focusing on role-driven communication and relation-aware representation learning with cross-episode structure reuse, respectively.
Response to Question 4
Recurrent refinement adjusts role prototypes using the current agent representations and historical slot states to accommodate changing behavioral features within an episode. In the current configuration, the GRU and other slot-module parameters remain at their initialized values. The slot states are updated. The contrastive loss learns representations through the trajectory encoder, query role encoder, and similarity matrix. The hard top-k labels and detached weights do not transmit gradients to the slot module, and the TD loss does not backpropagate through it either. The non-differentiability of hard role labels does not mean that their values are fixed over time. They still change with the inputs and states.
We have clarified these distinctions in Section 3.1, lines 223–230, and Section 3.2, lines 276–279. Section 3.1 also cites the reservoir computing review by Lukoševičius and Jaeger (2009) to explain that fixed recurrent network weights can coexist with input-driven dynamic hidden states. This citation clarifies the distinction between state updates and parameter learning.
To examine the practical effect of dynamic updates, we have added a Fixed Slots control on
3s5z_vs_3s6zin Appendix D, Figure 6, showing both test win rate and evaluation return. Fixed Slots disables GRU-based state refinement while retaining slot-based role assignments, auxiliary-aware weighting, and SlotBank reuse. Role assignments remain input-dependent in both methods. The mean curves over five seeds show that the full SPAR method improves its win rate earlier and achieves higher late-training mean win rates and returns.