acceptodds
Under review as a conference paper at ICLR 2027

Beyond Fixed Coordination Graphs: Learning Interaction Topologies in Open Ad Hoc Teamwork

Abstract

In ad hoc teamwork, the goal is to enable an autonomous agent to collaborate with other unknown agents without prior coordination or joint training. We study an open-environment variant of ad hoc teamwork in which agents with fixed policies can enter and exit the system at any time. Modeling multi-agent interaction is central to this problem, and prior coordination-graph methods rely on fixed topologies — dense complete graphs or sparse structures such as star graphs — often paired with cooperative-specific regularizers. These structural priors are efficient within the setting they are designed for, but degrade sharply outside it: dense fixed topologies accumulate noise as team size grows, while cooperative regularizers break down in the presence of adversarial agents. We introduce a dynamic, attention-based approach to model multi-agent interaction that requires no structural inductive bias or domain-specific regularizer, and whose learned interaction weights yield an interpretable coordination graph. We evaluate our method on both purely cooperative and mixed cooperative-competitive environments: it matches or exceeds the baselines on different benchmark problems, remains the only method that is competitive across both and scales substantially better as team size grows in the presence of adversarial agents. Our method is also highly robust to behavioral uncertainty, achieving substantial improvements in zero-shot generalization to unseen teammate policy distributions and team sizes.

Primary area:
reinforcement learning
Keywords:
Open Ad Hoc Teamwork, Multi-Agent Reinforcement Learning, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_KRUKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes LITO for open ad hoc teamwork, where unfamiliar agents can enter or leave the environment. It combines LSTM-based agent type inference with a Transformer to produce contextual representations and symmetric interaction weights for coordination-graph value decomposition. A separate behavior model predicts other agents' actions, allowing the learner to marginalize the joint value function when selecting its own actions. Experiments on cooperative Wolfpack and mixed cooperative-competitive FortAttack report competitive performance and improved robustness to unseen agent policies and changing team sizes.

Critical strengths

The paper is generally well presented, with a clear description of the problem setting and overall architecture.

Critical weaknesses

The work offers limited methodological novelty and reads more like an engineering solution than a clearly differentiated research contribution.

Questions for authors
  1. Attention-based interaction selection and weighting have been widely studied in MARL communication. How does LITO differ from these earlier approaches beyond applying them to the OAHT setting?

  2. The ablations indicate that contextualized embeddings account for most of the performance gains. What substantive methodological contribution comes from the proposed coordination-graph weights beyond standard attention-based representation learning? At present, the approach reads more like an engineering adaptation to a particular setting than a clearly differentiated research contribution.

  3. The overall conclusion currently appears within Section 5.3, “Ablation Studies.” Please place it in a separate section and review the manuscript's organization.

2 comments
AuthorsOpenReview ↗

Response - 1

We thank reviewer KRUK for the valuable feedback:

Q1. How does LITO differ from these earlier...?

LITO differs in the following ways:

(i) Unlike prior MARL attention methods that attend over observations or timesteps to select information for a fixed policy, LITO's attention outputs a symmetric, time-varying edge weight wij=1+αijαijw_{ij} = 1 + \sqrt{\alpha_{ij}\alpha_{ij}} that re-weights the value decomposition, making the interaction graph an interpretable part of the credit-assignment objective.

(ii) It is permutation-invariant (no positional encoding) and entmax-sparse, so it is the only attention-based coordination graph that natively handles variable, stochastically entering/exiting teammates, a requirement no closed-environment MARL attention method addresses.

(iii) In MARL communication, attention weights select messages between agents: a directional quantity. In LITO, the same attention must also parameterize a credit-assignment structure (the value decomposition of Eq. 8), which is inherently symmetric. Converting directional attention into a valid symmetric coordination graph is a methodological problem that communication-topology work does not face, and LITO addresses it with $w_{ij} = 1 + \sqrt{α_{ij} * α_{ji}}, the geometric mean of reciprocal attention plus a complete-graph residual.

Q2. What substantive methodological contribution...?

The coordination graph weights make credit assignment interpretable. The only-embeddings variant's post-hoc weights are often uninterpretable and sometimes random (Fig. 5e–h), while LITO's weights track kills, spawns, and prey captures (Fig. 4). As stated in the paper's conclusion, contextual embeddings are key to performance improvement, while learned edge weights provide valuable clarity on the coordination structure, which is the paper's stated contribution 3.

Q3. Conclusion section.

The "Conclusion" paragraph will be moved out of Section 5.3 into a standalone Section 6: Conclusion in the revised version.

We hope these responses address the reviewer’s concerns and are happy to discuss any remaining questions.

Thank you for the response, and I have decided to maintain my original score.

Reviewer_KaseOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LITO, an attention-based approach for Open Ad Hoc Teamwork (OAHT). Unlike previous methods that use fixed coordination graphs, LITO uses a Multi-Agent Transformer with entmax attention to learn contextualized agent representations and dynamically adjust interaction weights. It also includes an agent behavior model to predict other agents' actions for value estimation. Experiments on FortAttack and Wolfpack show that LITO performs well in both cooperative and mixed cooperative-competitive settings, with good generalization to unseen teammate policies and team sizes.

Critical strengths
  1. Clear motivation and an interesting idea. The paper clearly explains the limitations of fixed coordination graphs in OAHT. Using attention to learn dynamic interaction topologies is a natural and interesting way to address this problem.

  2. Simple and general framework. The proposed method is relatively straightforward and does not require task-specific regularizers. It can be applied to both cooperative and mixed cooperative-competitive environments without changing the overall framework.

  3. Well-motivated design choices. The main components are carefully designed. For example, entmax is used to encourage sparse attention, bidirectional attention scores are combined to obtain symmetric edge weights, and a residual term is added to stabilize training. The authors also remove positional encodings to handle varying team sizes and agent orderings.

  4. Good experimental results and generalization. The experiments cover different environments and team sizes. I particularly appreciate the zero-shot evaluations on unseen teammate policy distributions and larger teams, where LITO shows good generalization. The ablation studies and attention visualizations are also helpful for understanding the method.

Critical weaknesses
  1. Limited analysis of sparse attention. The paper uses entmax to filter irrelevant interactions, but does not report how sparse the learned attention weights actually are. A direct comparison with softmax would also be useful to understand whether sparsity itself contributes to the performance gains.

  2. Scalability could be evaluated more thoroughly. While the results on larger teams are encouraging, the experiments only consider maximum team sizes up to 9. Since the method still computes pairwise interactions, experiments with more agents or an analysis of computational cost would help support the scalability claims.

Questions for authors
  1. How sparse are the learned attention weights? Does entmax perform better than softmax?

  2. What additional benefits do dynamic edge weights provide beyond contextualized embeddings?

3 comments
AuthorsOpenReview ↗

Response - 1

We thank the reviewer for the valuable feedback and appreciate the encouraging remarks.

Q1: We ran an ablation comparing entmax with softmax on Wolfpack-9 and FortAttack-9. The two give comparable returns in both environments. The benefit of entmax is in interpretability. As the sample edge-weight graphs show, entmax puts the weight on a few task-relevant interactions, such as guard 0 and the attackers it engages (12 and 15). Softmax spreads the weight across many more edges and gives a denser, less readable graph, as shown in Figure 10 in Appendix H. Table 5 quantifies this: entmax zeroes 44% of attention entries and cuts 66% of edges, while softmax produces no exact zeros.

Q2: The main benefit of the dynamic edge weights is interpretability. In our embedding ablation (Section 5.3, Figure 5a-b), the only-embeddings version performs close to our full method. However, its edge weights, computed post hoc with the same formula, are often uninterpretable (Figure 5e-h). They show no meaningful edges with the learner-agent and arbitrarily high weights among unrelated agents. With learned edge weights, our method gives a clear coordination graph that links the relevant guards and attackers. This structure is learned during training (Appendix E), and it shows which interactions the agent relies on, which is one of the main objectives of this work.

We hope these responses address the reviewer’s concerns and are happy to discuss any remaining questions.

Thank you for the clarification. I appreciate the authors' acknowledgment that the performance improvements primarily come from contextualized embeddings, while learned edge weights mainly contribute to interpretability.

However, I am still concerned about whether the experimental evidence fully supports the paper's claims. Given that the only-embeddings variant achieves nearly identical performance to full LITO, could the authors clarify to what extent the reported improvements in performance, scalability, and generalization can be attributed to the learned dynamic edge weights rather than contextualized embeddings alone?

We thank the reviewer for the valuable feedback.

We have revised the paper claims to match the experimental evidence. The changes are highlighted in blue colored text in the manuscript.

For performance and scalability, the ablations show that at the largest team size the only‑embeddings variant (wij=1w_{ij} = 1) tracks LITO closely on both FortAttack‑9 and Wolfpack‑9 (Fig. 5a–b). Because these two variants differ only in wijw_{ij}, this shows that edge weights do not drive the return or its scaling at the most scalability‑stressed size; the gain is carried by the contextualized embeddings.

For generalization, the clean answer would be a direct ablation: evaluating the only‑embeddings variant on the zero‑shot benchmarks in Table 1 (unseen‑policy distributions) and Table 2 (team‑size generalization). Doing so requires retraining the variant and evaluating it across unseen teammate policies, multiple team sizes, two environments, and all six seeds, a new set of training/evaluation runs, which we cannot complete within the rebuttal window. We do not want to report partial figures that fall short of our usual standard, so we will not be able to include those exact numbers in this response.

To be precise about scope: the paper's generalization claims that LITO generalizes well to unseen policies, and larger teams are directly measured in Tables 1–2 and are not in question; the pending experiment would only refine the internal attribution (edge weights vs. embeddings). We will add the only‑embeddings generalization study and extend the only‑embeddings ablation across team sizes 3, 5, and 9, and include an explicit attribution statement in the final/camera‑ready version so the attribution rests on the full set of measurements.

We hope these responses address the reviewer’s concerns.

Reviewer_gERvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the Open Ad Hoc Teamwork (OAHT) problem, where a single learner-agent must collaborate without prior coordination with unknown agents that can dynamically enter and exit the environment. To overcome the scalability limits of fixed complete graphs and the vulnerability of cooperative game-theoretic regularizers in mixed settings, the authors propose LITO (Learning Interaction Topologies in Open ad hoc teamwork), a framework that uses a permutation-invariant Multi-Agent Transformer (MAT) with sparse entmax attention to learn contextualized agent embeddings and dynamic, symmetric coordination graph edge weights (wij=1+αijαjiw_{ij} = 1 + \sqrt{\alpha_{ij}\alpha_{ji}}) for both joint action-value decomposition and teammate behavior modeling. The authors validate LITO on purely cooperative (Wolfpack) and mixed cooperative-competitive (FortAttack) benchmarks across varying team sizes (3, 5, and 9), demonstrating competitive or superior scalability, zero-shot generalization to unseen policy distributions and team sizes, and qualitative interpretability of the learned interaction weights alongside component ablations.

Critical strengths
  • Unified, Regularizer-Free Performance Across Cooperative and Mixed Environments: Unlike prior state-of-the-art OAHT methods that rely on strict structural priors or cooperative game-theoretic regularizers that fail when adversarial agents are present (such as CIAO), LITO is the only evaluated framework that remains consistently competitive across both purely cooperative (Wolfpack) and mixed cooperative-competitive (FortAttack) domains. This directly supports acceptance by demonstrating that permutation-invariant sparse attention can replace domain-specific regularizers while scaling effectively up to 9-agent teams.

  • Strong Zero-Shot Generalization and Interpretability: LITO demonstrates substantial zero-shot robustness when evaluated on out-of-distribution teammate policy ensembles (Table 1) and unseen team sizes (Table 2), which is the core practical challenge of ad hoc teamwork. Furthermore, the learned symmetric edge weights produce clear, interpretable visualizations of role differentiation and spatial relevance (Figure 4), offering diagnostic transparency into multi-agent credit assignment that fixed-topology baselines lack.

Critical weaknesses
  • Mismatch Between the Core Topology Claim and the Ablation Evidence: The paper's central thesis is that learning dynamic, sparse coordination graph topologies (wij=1+αijαjiw_{ij} = 1 + \sqrt{\alpha_{ij}\alpha_{ji}}) overcomes the scalability bottleneck of fixed complete graphs. However, the authors' own ablation study (Figure 5a–b) shows that the only-embeddings variant—which fixes wij=1w_{ij} = 1 for all edges (i.e., a static complete coordination graph)—achieves virtually identical asymptotic returns to the full LITO model on both 9-agent benchmarks. Furthermore, because wij=1+αijαji≥1w_{ij} = 1 + \sqrt{\alpha_{ij}\alpha_{ji}} \ge 1 everywhere, entmax never actually zeros out edges in the coordination graph decomposition (Eq. 8). This undermines the paper's core scientific narrative, indicating that the empirical gains stem almost entirely from replacing GPL's GNN encoder with a Multi-Agent Transformer rather than from dynamic coordination graph weighting.

  • Unfair Exploration Hyperparameter Discrepancy and Missing entmax Ablation: According to Table 3 (Appendix D), LITO uses a minimum exploration rate of ϵmin=0.01\epsilon_{\text{min}} = 0.01, whereas GPL and CIAO are evaluated with ϵmin=0.05\epsilon_{\text{min}} = 0.05. In penalty-sensitive coordination tasks like Wolfpack and FortAttack, forcing baselines to take 5×5\times more random exploratory actions late in training can substantially depress their reported returns, calling the fairness of the empirical comparisons into question. Additionally, despite claiming that entmax sparsity in the transformer encoder is critical for filtering out irrelevant agents as team size grows, the paper provides no ablation comparing entmax against standard softmax attention.

  • Limited Architectural Novelty and Omitted Literature: Nearly all of the ad-hoc teamwork scaffolding—the OSBG formulation, LSTM type inference, conditional-independence NLL teammate modeling, marginalized Q-value, and low-rank pairwise utility factorization—is inherited directly from GPL, making LITO essentially GPL with an off-the-shelf MAT + entmax encoder swapped in for the GNN. Meanwhile, Section 2 omits established MARL work that already uses self-attention and sparsity to learn dynamic coordination graphs (e.g., DICG, CASEC, SOP-CG, G2ANet).

DICG: Li, S., Gupta, J. K., Morales, P., Allen, R., & Kochenderfer, M. J. (2021). Deep Implicit Coordination Graphs for Multi-agent Reinforcement Learning. AAMAS 2021.

CASEC: Wang, T., Zeng, L., Dong, W., Yang, Q., Yu, Y., & Zhang, C. (2022). Context-Aware Sparse Deep Coordination Graphs. ICLR 2022.

SOP-CG: Yang, Q., Dong, W., Ren, Z., Wang, J., Wang, T., & Zhang, C. (2022). Self-Organized Polynomial-Time Coordination Graphs. ICML 2022, PMLR 162.

G2ANet: Liu, Y., Wang, W., Hu, Y., Hao, J., Chen, X., & Gao, Y. (2020). Multi-Agent Game Abstraction via Graph Attention Neural Network. AAAI 2020, 34(05), 7211–7218.

MAGIC: Niu, Y., Paleja, R., & Gombolay, M. (2021). Multi-Agent Graph-Attention Communication and Teaming. AAMAS 2021, 964–973.

Questions for authors

Clarification & Rebuttal Questions:

  • Exploration Rate Discrepancy (ϵmin\epsilon_{\text{min}}): Table 3 (Appendix D) states that LITO uses a minimum exploration rate of ϵmin=0.01\epsilon_{\text{min}} = 0.01, whereas GPL and CIAO use ϵmin=0.05\epsilon_{\text{min}} = 0.05. First, do the training curves in Figure 3 report online training returns (with ϵmin\epsilon_{\text{min}} active) or greedy evaluation rollouts (ϵ=0\epsilon = 0)? Second, if GPL and CIAO are evaluated or trained with ϵmin=0.01\epsilon_{\text{min}} = 0.01 to match LITO, how much of the asymptotic performance gap in Wolfpack and FortAttack remains?

  • Role of Dynamic Edge Weights (wijw_{ij}) vs. Transformer Embeddings: In Figure 5(a–b), the only-embeddings-version—which fixes wij=1w_{ij} = 1 for all edges (reducing Eq. 8 to a uniform, static complete coordination graph)—achieves virtually identical returns to the full LITO model on both 9-agent benchmarks. Why does modulating the coordination graph decomposition weights via wij=1+αijαjiw_{ij} = 1 + \sqrt{\alpha_{ij}\alpha_{ji}} not yield empirical return gains over a fixed complete graph once MAT embeddings are introduced, and how do you reconcile this with the paper's central claim that learning dynamic coordination topologies drives scalability?

  • entmax vs. softmax Attention Ablation: You argue that entmax is essential to zero out irrelevant agents in the MAT encoder and prevent complete-graph noise accumulation as team size scales. Can you provide an ablation comparing entmax15 against standard softmax attention on Wolfpack-9 and FortAttack-9 to verify that sparse attention is indeed responsible for the scaling performance?

  • "Sparse" Coordination Graph Terminology: Because the edge weight is defined as wij=1+αijαjiw_{ij} = 1 + \sqrt{\alpha_{ij}\alpha_{ji}}, setting αij=0\alpha_{ij} = 0 via entmax results in wij=1w_{ij} = 1, meaning no edge is ever pruned or zeroed out in the coordination graph decomposition (Eq. 8). Shouldn't the resulting coordination graph be characterized as attentively re-weighted complete (wij∈[1,2]w_{ij} \in [1, 2]) rather than sparse? Additionally, did you experiment with removing the +1+1 residual (wij=αijαjiw_{ij} = \sqrt{\alpha_{ij}\alpha_{ji}}) so that the coordination graph itself is genuinely sparse?

Minor Corrections & Suggestions for Revision (Non-Blocking):

  • Summation Notation & Algorithm Typos: Please clarify the summation indexing in Equations (6), (8), and (12) regarding ordered vs. unordered pairs (j,k)(j, k) so it aligns with the single-counting explanation in Appendix B. Also, in Algorithm 1 (Line 27, Appendix C), the soft target update is written for ϕtarg\phi_{\text{targ}} (the agent model parameters) rather than ξtarg\xi_{\text{targ}} (the utility network parameters).
  • Prior Work on Dynamic Coordination Graphs (Suggestion): Consider discussing prior cooperative MARL methods that learn dynamic or sparse coordination graphs via attention (e.g., DICG, CASEC) in Section 2 to better situate LITO's architectural contribution relative to the broader MARL literature.
4 comments
AuthorsOpenReview ↗

Response - 1

We thank the reviewer for the valuable feedback.

W2, Q1: Exploration Rate Discrepancy.

Yes, the training curves in Fig. 3 report online training returns. The plots of training GPL and CIAO with LITO’s final exploration rate (0.01) are shown in Figure 8 in Appendix G, along with the original plots of training GPL and CIAO with their exploration rate (0.05); LITO’s exploration rate is 0.01. As we can see, the performance difference is not there. Infact, on the contrary, for FortAttack the performance for CIAO-S degrades with 0.01 being the final exploration rate. Hence, tuning of this parameter visually gives no performance benefit for either GPL or CIAO. Please note CIAO-C was not trained with eps=0.01 for this experiment given that it is already performing worse than CIAO-S on both settings.

W1, Q2-a: In Figure 5(a–b), the only-embeddings-version….

We would like to clarify that the only-embeddings variant is not a static complete graph: fixing wij=1 removes only the scalar re-weighting of Eq. 8, while the entmax-sparse MAT encoder, the mechanism that dynamically filters irrelevant agents, remains fully active, and it is precisely this dynamic filtering that yields LITO's gap over GPL at team size 9, which persists in this variant as well.

W1, Q2-b: Why does modulating the coordination graph…and how do you reconcile this with the paper's central claim…?

We agree with the reviewer's assessment that the only-embeddings variant achieves results similar to LITO on both 9-agent benchmarks in Figures 5a & 5b. The paper already states this in the conclusion: "contextual embeddings is key in performance improvement, while learned edge weights provide valuable clarity on the coordination structure". We will revise the wording in the paper by replacing "learning dynamic coordination topologies drives scalability" with “the attention-based dynamic interaction modeling drives scalability, and the learned soft weights provide interpretability of the coordination graph".

Q3: entmax vs softmax Attention Ablation.

Our motivation for entmax over softmax was qualitative, not an empirical claim that it drives scalability. As the ablation shows in Figure 9 in Appendix H, the returns are comparable on Wolfpack-9 and FortAttack-9. The difference is in interpretability, which is one of the paper’s main objectives. As shown in Figure 10 in Appendix H, with entmax, the edge weights concentrate on a few task-relevant interactions, such as guard 0 and the attackers it engages (12, 15). Softmax spreads the weight across many more edges and gives a denser, comparatively less readable graph. Table 5 quantifies this: entmax zeroes 44% of attention entries and cuts 66% of edges, while softmax produces no exact zeros.

Q4: Sparse Coordination Graph Terminology

Sparsity in our method refers to the entmax attention distributions, not to the coordination graph. The reviewer is correct that the graph is an attentively re-weighted complete graph, with edge weights between 1 and 2, and we are happy to use this term in the text . The suggested experiment is already in Section 5.3 (edge-weight ablation, Figure 5a-b). The square-root-only version of the edge weight removes the +1 residual, which makes the graph sparse. This variant learns in FortAttack-9 but performs slightly below LITO in Wolfpack-9. In our experiments, the +1 residual therefore gives the most consistent performance across both environments. Table 5 quantifies this for LITO itself: 66% of edges already satisfy wij<10-3, i.e. they would be cut without the +1 residual.

AuthorsOpenReview ↗

Response - 2

W3-a: Limited Architectural Novelty

We agree that the paper incorporates some of the GPL's components, and we make this explicit throughout Section 4. However, we emphasize that LITO and the OAHT baselines differ precisely in how they model agent interactions: GPL uses a GNN over a complete graph, CIAO-S/C use fixed star or complete graphs with cooperative regularizers, and Open-OTAF uses a clustered complete graph. The contribution of this paper is to replace that mechanism with a single dynamic, attention-based mechanism that (i) needs no domain-specific regularizer in either environment class, (ii) handles variable team size by construction (no positional encodings), (iii) scales and zero-shot generalizes, and (iv) yields interpretable interaction weights.

W3-b: Omitted literature

We thank the reviewer for pointing us to the papers related to learning interaction/coordination graph (CG) in MARL. DICG directly uses the raw attention weights as the edge weights of the CG, encoding a one-way relationship rather than a symmetric one, unlike LITO. CASEC does not use an attention mechanism; instead, its CG is constructed based on the variance of pairwise payoff functions. SOP-CG also does not rely on attention mechanisms but uses an imaginary coordinator agent to determine the topology of the coordination graph. G2ANet uses hard attention to select interacting agents and soft attention to weight their relevance, yielding a per-agent sub-graph that a GNN encodes into a contribution vector for policy training. MAGIC uses a graph attention-based scheduler to decide when and with whom each agent communicates, and a graph attention network to process messages for MARL with communication. The main difference between LITO and these MARL papers is that LITO is designed for the OAHT setting, which differs from typical MARL scenarios in that it involves an open environment, unknown and potentially unseen teammates, and no access to joint training. We will include these papers in the related work section in the revised edition.

We will also incorporate the minor corrections and suggestions provided in the revision.

We hope these responses address the reviewer’s concerns and are happy to discuss any remaining questions.

I thank the authors for their detailed responses, the comparison with prior MARL coordination-graph literature, and the additional experiments on the exploration rate (ϵmin=0.01\epsilon_{\text{min}} = 0.01, Appendix G) and entmax vs. softmax attention (Appendix H).

  1. Exploration Rate (ϵmin\epsilon_{\text{min}}): Figure 8 satisfactorily resolves my concern regarding the exploration rate discrepancy between LITO and GPL/CIAO-S.

  2. Core Mechanism & Contribution (only-embeddings + softmax Ablations): While I appreciate the authors' candor in reporting the new softmax ablation, combining the responses to W1/Q2 and Q3 raises a more fundamental concern regarding the paper's central claim and technical contribution:

    • In Q2-a, the response states that only-embeddings (wij=1w_{ij}=1) outperforms GPL at team size 9 specifically because "the entmax-sparse MAT encoder, the mechanism that dynamically filters irrelevant agents, remains fully active, and it is precisely this dynamic filtering that yields LITO's gap over GPL."
    • Yet in Q3, Figure 9 shows that standard dense softmax achieves comparable returns to entmax on both Wolfpack-9 and FortAttack-9—noting that entmax was motivated qualitatively rather than as an empirical driver of scalability (which also runs counter to Section 4, lines 232–235, arguing that softmax recreates the complete-graph noise bottleneck).
    • Taken together, these two ablations show that neither dynamic coordination-graph edge weighting (wij=1+αijαjiw_{ij} = 1 + \sqrt{\alpha_{ij}\alpha_{ji}} vs. wij=1w_{ij}=1) nor sparse attention (entmax vs. softmax) drives LITO's empirical return or scalability gains. Rather, the performance improvement over GPL comes from replacing the GNN encoder with a standard dense softmax Transformer (MAT) encoder, while entmax and wijw_{ij} act purely as a post-hoc interpretability mechanism. (Minor note in Q4: "66% of edges satisfy wij<10−3w_{ij} < 10^{-3}" appears to be a typo for wij−1<10−3w_{ij} - 1 < 10^{-3}, since wij≥1w_{ij} \ge 1 by construction.)
  3. Remaining Reproducibility & Control Clarifications: Could the authors briefly clarify two remaining points from my review:

    • (a) What exact train vs. unseen test policy split and evaluation episode count were used in Table 1?
    • (b) What are the total trainable parameter counts of LITO relative to GPL and CIAO?

I look forward to the authors' clarification on (3) and will factor both the resolved exploration-rate check and the clarified scope of the technical contribution into my final assessment.

We thank the reviewer for the valuable feedback.

  • We agree that our Q2-a response overstated the role of entmax. In light of the softmax ablation (Figure 9), LITO’s gap over GPL should not be attributed to entmax sparsity specifically.
  • We have removed the claim that softmax recreates the complete-graph noise bottleneck.
  • We would like to clarify that the MAT encoder is a well‑established sequence‑modeling architecture used across MARL. The contribution is the framework organized around it: a soft, permutation‑invariant, regularizer‑free, interpretable, and adaptable coordination graph topology. LITO is the only method competitive in both mixed and cooperative settings.
  • We would also like to clarify that the edge weights are not a post-hoc mechanism. The edge weights are computed from the attention (αij\alpha_{ij}) and are used in the value decomposition (Eqs. 8, 12). More importantly, the ablation (Fig. 5e–h) shows that our learned edge weights are semantically meaningful (task‑relevance‑driven, event‑driven reconfiguration; Fig. 4, 7), whereas post‑hoc weights computed from the only‑embeddings variant are often uninterpretable and sometimes random. Fixed‑topology baselines (complete, star), on the other hand, do not offer such interpretable coordination graphs. We therefore regard interpretability as a distinct contribution in its own right, valuable for analysis and trust in deployed multi-agent systems, and complementary to the performance gains.
  • We also thank the reviewer for spotting the typo in Q4: the statement should read that 66% of edges satisfy wij−1<10−3w_{ij} − 1 < 10^{-3} (though we mention that edges that will be pruned without +1 residual). We have corrected this in the Table 5 caption.

3-(a):

  • For Table 1, training and test policies are mutually exclusive. In Wolfpack, the learner-agent is trained alongside with GreedyPredator, GreedyProbabilistic, TeammateAware, DistilledCoopAStar, and GraphDQN, and tested on unseen Random, GreedyWaiting, GreedyProbabilisticWaiting, and TeammateAwareWaiting agents. In FortAttack, teammates and attackers are drawn from pretrained checkpoints ep0, ep220, ep650, and ep1240 during training, and from unseen checkpoints ep1600, ep2520, and ep5050 at test time. At the start of each episode, each agent is assigned a random checkpoint from the active set, and then a random guard or attacker model from it. We evaluate the final checkpoint of each seed with a maximum team size of 5, over 10 evaluation rounds across 16 parallel environments.

3-(b):

  • LITO has 0.80M trainable parameters in FortAttack and 0.39M in Wolfpack, compared with 0.41M and 0.25M for GPL and CIAO, respectively (the same number of parameters for both domains). The difference comes mainly from the MAT encoders in the value network and the agent model. All models remain small (under 1M parameters). In FortAttack, the additional 394K parameters come almost entirely from the two MAT encoders:
ComponentGPLLITODifference
Value-network MAT encoder-264,960+264,960
Agent model (MAT in place of GPL’s relational forward model)32,490153,936+121,446
Agent-model readout layer9,08816,512+7,424
LSTMs and utility MLPssamesame0

The increase is smaller in Wolfpack because its embedding dimension is 70 rather than 128. Notably, the agent-model share of this increase (about 129K parameters) does not affect performance: replacing LITO’s MAT agent model with GPL’s GNN agent model (GNN-version, Section 5.3) performs comparably to the full method in both environments. The remaining increase is the value-network MAT encoder, i.e., the attention-based interaction modeling itself.

We have updated the manuscript with the changes (in blue colored text).

We hope these responses address the reviewer’s concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.