acceptodds
Under review as a conference paper at ICLR 2027

Not All Coupling Matters: Value-Induced Belief Edges for Budgeted Coordination in Cooperative MARL

Abstract

Sparse coordination lets cooperative multi-agent reinforcement learning (MARL) scale by giving each agent a limited budget of teammate interactions. Under a fixed budget, the question is which teammates deserve that capacity. Existing edge-selection criteria mainly measure how strongly a teammate affects payoff and do not account for whether reducing uncertainty about that teammate would change the receiver's decision. The result is certainty-blind coordination: capacity goes to strongly coupled but predictable teammates, while uncertain, decision-pivotal ones are overlooked. We introduce Value-Induced Belief Edges (VIBE), which scores an edge by the expected value of resolving uncertainty about a teammate: given its belief over teammate actions, the receiver values evidence that would change its preferred action and improve its expected return. This decision value decomposes into the probability that the receiver switches actions and the gain from switching, and payoff variance can rank teammates in the opposite order. A centralised teacher approximates the decision value over a bounded set of joint-action probes, and a decentralised student learns the resulting rankings from receiver-local information. Selected edges direct relational processing of teammate features the receiver already observes, with no additional communication at execution. Extensive experiments on five benchmark families show that VIBE outperforms strong value-based, partner selection and coordination-graphs baselines. Our code is anonymously available at https://anonymous.4open.science/r/VIBE-D717

Primary area:
reinforcement learning
Keywords:
Cooperative Multi-Agent Reinforcement Learning, Budgeted Coordination, Belief-Conditioned Edge Selection, Value of Information

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iuv4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces Value-Induced Belief Edges (VIBE), which scores an edge by the expected value of resolving uncertainty about a teammate: given its belief over teammate actions, the receiver values evidence that would change its preferred action and improve its expected return.

Critical strengths
  1. The paper introduces a sharp and well-motivated decision-theoretic criterion for edge allocation.
  2. The empirical package is broad and includes useful mechanism-level diagnostics.
Critical weaknesses
  1. The mathematical presentation and notation are not yet publication-ready, which impedes verification.
  2. The empirical claims are stronger than the reported uncertainty and effect sizes support.
Questions for authors
  1. Please add a diagnostic experiment on CCT and small tabular tasks where the ideal value can be computed exactly, reporting top-1 agreement, Spearman correlation, and ranking inversions between practical teacher score and the ideal decision value.
  2. The “no additional communication” claim relies on teammate features already being present in the receiver’s observation. Please state this assumption explicitly and discuss what happens when those features are absent, delayed, or noisy.
3 comments
AuthorsOpenReview ↗

Response to Reviewer iuv4

We thank the reviewer for the positive assessment and for the concrete requests, all of which we have addressed in the revision.

W1. Mathematical presentation and notation are not publication-ready.

We agree and have revised the formal presentation throughout. In the Introduction we rewrote the ungrammatical sentence summarising the switching decomposition and now refer to Eq. 2 for the formal definition. In Sec. 2.1 we completed the optimisation problem with the explicit constraint |E_i^t| ≤ K. In Sec. 2.2 we made the prior-preserving tie-breaking rule used by Lemma 1 an explicit assumption.

The most substantive correction concerns Theorem 2. Its (1 − 1/e) guarantee for greedy selection relies on submodularity of F_i in the proof, but the original statement did not assume it. Our own appendix shows that F_i is not submodular in general: two independent bits can each have zero individual value but positive joint value. Theorem 2 now states submodularity as an explicit additional assumption for the greedy guarantee, separately from the modular case and the NP-hardness result. This is consistent with greedy guarantees for information-based selection, which likewise rely on submodularity conditions (Liao et al., 2009). The primary K = 1 setting does not depend on this guarantee.

Unresolved internal labels in Appendices A, B and F (e.g. 'teacher-score', 'student-input', 'igm-inclusion') were replaced by proper equation and section references.

W2. Empirical claims are stronger than the reported uncertainty supports.

We agree that several statements went beyond what the spreads support. The ± values in Tables 1–3 are standard deviations across the 10 training seeds, and we now say so in the Table 1 note. Using the per-seed results, we ran Welch's t-tests with Holm correction for a pre-specified family of comparisons (VIBE vs POW-QMIX and VIBE vs Variance-K across tasks) following recommendations for multi-run, multiplicity-aware evaluation of deep RL (Grądzki, 2026).

The tests change the picture in the direction the reviewer anticipated. Against the POW-QMIX backbone, VIBE's improvement remains significant after correction on CCT, corridor and SMACv2 Terran only. Against Variance-K, only the 10m vs 11m gain survives correction (+6.19 pp, p_Holm = 0.038); corridor (0.103), CCT (0.102) and Intersection (0.318) are significant only before correction, and none of the SMACv2 differences is significant. Some differences also go against VIBE: its return is below POW-QMIX on both Predator–Prey settings (39.29 vs 40.04; 38.58 vs 39.79), and its mean is below Variance-K on all three SMACv2 races.

We have revised the text accordingly. The Abstract no longer states that VIBE outperforms the baselines. Sec. 5.2 now reports the Predator–Prey and SMACv2 shortfalls, describes the gains as concentrated in coordination-intensive tasks rather than uniform, and summarises the Holm-corrected result; the full test procedure and results are in new App. G. The Intersection comparison in App. E.5 is qualified (p_Holm = 0.318). The Table 1 note now defines ±.

References.

[1] Liao W, Ji Q, Wallace WA. Approximate Nonmyopic Sensor Selection via Submodularity and Partitioning. IEEE TSMC-A. 2009;39(4):782-794.

[2] Grądzki P. Unstable Gains: Multiplicity-Aware Evaluation of Financial Deep Reinforcement Learning. J. Finance Data Sci. 2026;12:100205.

AuthorsOpenReview ↗

Response to Reviewer iuv4

Q1. Diagnostic on CCT and small tabular tasks where the ideal value can be computed exactly: top-1 agreement, Spearman correlation, and ranking inversions between the practical Teacher and the ideal decision value.

We have added this diagnostic on small tabular tasks; it was not run on CCT, and we say so explicitly. On CCT, Table 5 reports agreement with the ground-truth optimal edge (87.63% for VIBE vs 8.73% for Variance-K), not with exact Δ, and App. E.2 now states this distinction.

Tasks. Random tabular games (RTG) use n ∈ {3, 4, 6} agents with m ∈ {2, 3} actions and five families (Indep, CBC, Offset, Corr, HO), with 5 seeds × 400 games per configuration, 60,000 games in total. Beliefs are drawn from Dirichlet distributions covering different entropy, correlation and coupling levels. The two-chance task is a constructed mechanism test with m = 3, at n = 4 and n = 6, with 5 seeds × 2,000 instances per scale. In both, Δ is computed exactly by enumeration, and all criteria use the same instances and information inputs.

RTG results (Eq. 5 vs exact Δ). Top-1 agreement is 49.07%, Spearman ρ* is 0.451 and the inversion rate is 10.31%. Random selection reaches 34.44% and the best of four payoff-variance formulas 80.62% (the advantage-variance formula used by Variance-K reaches 79.63%). By family, top-1 / ρ* / inversion rate are: Indep 51.43 / 0.564 / 7.17%; CBC 50.72 / 0.528 / 9.10%; Offset 46.53 / 0.305 / 15.61%; Corr 51.49 / 0.571 / 6.16%; HO 45.16 / 0.289 / 13.50%. Ties are a major issue: the overall tie rate is 58.33%, mostly because in 49.66% of instances the switch gate fires for no teammate, so Spearman ρ is defined in only 34.66% of instances (29.63–40.71% across families). We report these rates alongside the correlations.

Two-chance task (n = 4 / n = 6). Top-1 agreement is 92.99% / 93.38%, Spearman ρ is 0.805 / 0.764 and the inversion rate is 0.62% / 0.12%, with tie rates of 36.32% / 45.32%. The best payoff-variance criterion reaches 38.37% / 46.12%. When the opportunity priors become unequal (η = 0.75), Eq. 5 falls to 61.59% / 62.96%, below payoff variance × entropy (91.80% / 92.47%).

The practical Teacher thus tracks the ideal ranking closely when decision-pivotal and strongly coupled teammates differ, but not in general random games, mainly because of joint-probe maximisation (see the factorial decomposition in our response to Reviewer iYUX, Q1). All three requested metrics, the task specifications and the factorial results are reported in new App. E.6.

AuthorsOpenReview ↗

Response to Reviewer iuv4

Q2. State the observation assumption explicitly, and discuss absent, delayed or noisy features.

We agree that this assumption should be explicit. Sec. 2.1 now states it formally: every teammate feature block o_{i→j} processed by the relational channel is part of the receiver's own observation. The "no additional communication" claim holds only under this assumption.

Absent features. We checked the deployed Student against the code used for the submitted runs. It masks the receiver itself and then takes the top-k of the remaining teammates' logits. It does not apply a separate visibility filter, so a teammate that is currently out of sight remains in the ranking and is represented by a zero-filled feature block rather than being removed from the candidate set. The availability mask is privileged and is used only by the Teacher, which renormalises its belief over the teammate's available actions (Sec. 3.2). An empty selection still gives zero relational payload. A consequence, which we now state, is that the Student can assign an edge to a teammate it cannot currently observe, in which case that edge processes a zero block and does not add information. We have corrected App. A.3, which previously could be read as excluding unobserved teammates before ranking. On Predator–Prey the practical effect is limited, because the feature blocks available to a receiver are those of predators inside its local observation grid, but the ranking itself is not restricted to that grid.

Delayed or noisy features. The selected edge processes whatever the receiver observes, so VIBE does not reduce the information available to the backbone. However, noise or delay can degrade both the pairwise belief encoder, which is conditioned on the receiver's recurrent state, and the usefulness of the routed payload. We have not tested this empirically and state it as a limitation.

Unobserved features. Where teammate features are not observed at all, applying VIBE would require an explicit communication channel, and its criterion would then act as a communication-scheduling rule. The appropriate comparison would then be with communication methods that schedule who communicates and when, often by estimating the value of information, and that must handle noisy messages, transmission delays and limited bandwidth (Liu et al., 2026), such as VBC (Zhang et al., 2019) and VIL2C (Zhang et al., 2026).

The assumption is stated in Sec. 2.1, and the limitations paragraph has been added to App. C.3.

We hope the revised manuscript addresses your concerns. If so, we would be grateful if you would consider revisiting your score. We are glad to provide further details.

References.

[1] Liu Z, Li Y, Wang J, Tu J, Hong Y, Li F, et al. Robust and efficient communication in multi-agent reinforcement learning. Chaos. 2026;36(2):022101.

[2] Zhang SQ, Zhang Q, Lin J. Efficient Communication in Multi-Agent Reinforcement Learning via Variance Based Control. NeurIPS 2019.

[3] Zhang et al. VIL2C: Value-of-Information Aware Low-Latency Communication for Multi-Agent Reinforcement Learning. AAAI 2026;40(35):29874-29882.

Reviewer_JQBGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes VIBE, a method that helps cooperative AI agents choose which teammates to focus on during multi-agent coordination problems. The method seeks to prioritize teammates whose uncertain behavior could change the agent’s decision and improve the outcome, using a centralized teacher with global information as a supervision target for a student with local information (past state, teammate observed features, etc)

Critical strengths
  1. There is a clear motivation, and i think the problem is relevant. Uncertainty of other agent's should not prevent an agent from paying attention to them, because they can potentially help in solving the task. Estimating that uncertainty is important.
  2. The paper gives a mathematical example showing why this selection rule can be better than choosing teammates based on how much their actions affect reward.
Critical weaknesses
  1. If the student accurately matches the teacher, this doesn't mean that the agent has learned which teammates are actually useful to the goal.

  2. A simpler method with the same architecture has higher average returns in all SMACv2 settings in App, Table 9.

  3. I feel the authors are missing references to existing MARL literature. For instance, InfoPG (https://arxiv.org/abs/2201.08484) dealt with this issue by enabling recursive reasoning towards pairs of agents. The measure of uncertainty in another agent, or its ability to help achieve the goal, happens implicitly in that framework. How is this better?

Questions for authors

Can you show, on small tasks where the ideal decision benefit can be calculated exactly, that your practical score reliably identifies the teammates whose information would most improve the agent’s decision?

2 comments
AuthorsOpenReview ↗

Response to Reviewer JQBG

We thank the reviewer for the thoughtful review and for highlighting the Variance-K comparison, which has led to a substantial revision of our claims.

W1. Matching the Teacher does not show that the agent has learned which teammates are useful.

We agree. A Student that reproduces the Teacher only inherits the Teacher's notion of usefulness, so distillation fidelity alone is not evidence that the selected teammates help the task. The revision sets out the evidence that does not rely on Student–Teacher agreement, and states plainly where it is limited.

(i) Ground truth on CCT. The hit rate in Table 5 (87.63% for VIBE vs 8.73% for Variance-K) is measured against the optimal edge derived from the known payoff tensor, not against the Teacher.

(ii) The Teacher against the exact decision value. The new exact-Δ study (new App. E.6; see our response to Reviewer iYUX, Q1) tests the Teacher itself. It recovers the Δ-maximising teammate in 92.99% / 93.38% of instances (n = 4 / 6) in a constructed two-chance task, where the best payoff-variance criterion reaches 38.37% / 46.12%. It does so in only 49.07% of random tabular games (best of four payoff-variance formulas: 80.62%, of which the advantage-variance formula used by Variance-K reaches 79.63%), and in 37.30% with a learned critic (Variance-K: 60.40%). A corrected Teacher that replaces joint-probe maximisation by belief-weighted expectation reaches 95.54% on fitted-critic RTG and 99.30% against enumeration of the same learned critic. On the states actually visited by the submitted models, original Eq. 5 agrees with that corrected Teacher on 91–94% of informative decisions (new App. H). The original Teacher is therefore not a general proxy for usefulness, and we no longer present Teacher agreement as evidence of it; the submitted tables are nevertheless not an artefact of that failure.

(iii) End-to-end outcomes. The Oracle in Table 3 is a privileged reference selector: under the same relational budget, it selects, among eligible teammates, the one with the highest receiver-specific decision value, computed on SMAC and Intersection from privileged state information and the learned critic. It is a privileged reference selector rather than a proven upper bound on return, and it is not exact Δ. VIBE stays close to it (corridor 82.59% vs 83.38%; Intersection 93.19% vs 94.20%). The Random-K, Proximity-K and Variance-K selectors share VIBE's backbone, relational pathway and edge budget (Fig. 4; Variance-K is now also in Table 1), and the compute-matched controls receive the same GPU-hours (see our response to Reviewer iYUX, W2). Against Variance-K the end-to-end evidence is mixed, as discussed under W2.

Theorem 5 explains why close Student–Teacher agreement matters: it bounds the return gap between the Teacher-graph and Student-graph policies by the probability that the two graphs disagree. It does not say whether the Teacher's graph is good; points (i)–(iii) address that. We added a sentence to Sec. 5.2 separating these levels of evidence, and we defined the Oracle precisely in App. D.

W2. Variance-K has higher returns on all SMACv2 settings.

We confirm the observation. Variance-K shares VIBE's backbone, probe interface, edge budget and relational pathway, and its mean return is higher on all three SMACv2 races (Protoss 19.32 vs 18.88, Terran 18.94 vs 18.79, Zerg 18.35 vs 17.54). In seed-level Welch tests with Holm correction (n = 10 per method), none of these three differences is significant, but the direction is consistent and persists when both methods receive the same 120 GPU-hours (compute-matched Variance-K: 19.40, 19.00, 18.40). The gap is therefore not an artefact of compute.

Across all ten tasks, VIBE is higher than Variance-K on six (CCT, both Predator–Prey settings, corridor, 10m vs 11m, Intersection), ties on 3s vs 5z, and is lower on the three SMACv2 races. After Holm correction, only the 10m vs 11m gain is significant (+6.19 pp, p_Holm = 0.038); corridor (0.103), CCT (0.102) and Intersection (0.318) are significant only before correction.

We therefore conclude that on SMACv2, payoff-variance selection is at least as effective as decision-value selection. We have not tested the following explanation directly, and offer it as a hypothesis: SMACv2's randomised start positions and unit types make payoff variation a good proxy for decision relevance, so the two criteria select similar teammates. We have moved Variance-K into Table 1, removed the claim that VIBE attains the highest mean on all SMACv2 races, and now discuss SMACv2 in Sec. 5.2 as a regime where the decision-value criterion offers no advantage.

AuthorsOpenReview ↗

Response to Reviewer JQBG

W3. Missing references, e.g. InfoPG. How is VIBE better?

We thank the reviewer for this pointer. We have added InfoPG (arXiv:2201.08484) and related work on recursive reasoning to Sec. 6. InfoPG lets agents exchange latent action guesses with their neighbours for k rounds before acting, so that each teammate's relevance is captured implicitly during policy learning. Level-K reasoning has also been built into MARL frameworks in applied settings (Li et al., 2026).

On "how is this better": we see VIBE as addressing a different and complementary question rather than as dominating recursive reasoning. InfoPG handles relevance implicitly, inside policy learning. VIBE asks which teammate should receive dedicated processing under a hard per-receiver budget of K relational edges, with no additional execution-time communication. Its criterion is explicit and inspectable: it decomposes into the probability that resolving uncertainty switches the receiver's action and the value of that switch (Lemma 1), which makes it possible to state and test when sensitivity-based relevance misranks teammates (Prop. 1; App. E.6). The two approaches can be combined: a decision-value selector could restrict which pairs receive recursive reasoning, which matters when the number of pairs grows quadratically with team size. We have not compared VIBE empirically with InfoPG, so we make no performance claim. The new Related Work paragraph states this contrast.

Question. On small tasks where the ideal decision benefit can be computed exactly, does the practical score reliably identify the teammates whose information would most improve the agent's decision?

We ran this analysis. The honest answer is: reliably in the regime that motivates VIBE, but not in general. We computed exact Δ (Eq. 2) by enumeration on 60,000 random tabular games (3–6 agents, 2–3 actions, five families) and on a constructed two-chance task (n = 4 and n = 6), and compared Δ̂ (Eq. 5) with Δ.

In the two-chance task, where decision-pivotal and strongly coupled teammates differ, Δ̂ identifies the Δ-maximising teammate in 92.99% (n = 4) and 93.38% (n = 6) of instances (ρ 0.805 / 0.764, inversion rate 0.62% / 0.12%, normalised regret about 0.5%). The best payoff-variance criterion reaches only 38.37% / 46.12%. When the opportunity priors become unequal (η = 0.75), however, Δ̂ falls to 61.59% / 62.96%, while payoff variance × entropy reaches 91.80% / 92.47%.

In random tabular games, Δ̂ identifies the Δ-maximising teammate in only 49.07% of instances (random selection: 34.44%; best of four payoff-variance formulas: 80.62%, of which the advantage-variance formula used by Variance-K reaches 79.63%). With a learned critic the figures are 37.30% for Δ̂ and 60.40% for Variance-K. A factorial decomposition attributes this mainly to joint-probe maximisation, which leaves the receiver's switch gate inactive for every teammate in about half of the instances, so that all scores tie at zero. The entropy weight and the mean-embedding approximation cost little.

This analysis was not run on CCT. The CCT result in Table 5 measures agreement with the ground-truth optimal edge from the known payoff tensor, not with exact Δ. Full results, including per-family results and ranking inversions, are in new App. E.6; the decomposition is described in our response to Reviewer iYUX, Q1. We have revised the claims in Secs. 1, 3.3 and 5.2 to reflect these findings.

We hope these responses and the revised claims address your concerns. If so, we would be grateful if you would consider revisiting your score. We are glad to provide further details.

References.

[1] Konan S, Seraj E, Gombolay M. Iterated Reasoning with Mutual Information in Cooperative and Byzantine Decentralized Teaming. ICLR 2022 (arXiv:2201.08484).

[2] Li J, Wang R, Zhu Y, Zhang M, Zhao X. MRISce: an interactive autonomous driving test scenario generation method based on multi-agent reinforcement learning. Accid Anal Prev. 2026;234:108611.

Reviewer_iYUXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes value-induced belief edges for selecting which teammates to attend to under a fixed relational-processing budget in cooperative MARL, based on the expected gain in return when evidence about a teammate changes the receiver’s preferred action. A centralized teacher approximates this ideal criterion using bounded joint-action probes, belief-mean action embeddings, and an entropy-weighted switching score, and a decentralized student then learns the resulting teammate rankings and routes only the selected teammates’ already-observed features through a relational channel. It has been validated through a constructed example of ranking reversal, theoretical analysis, experiments on five benchmark families, and ablation experiments on components and margin budgets.

Critical strengths
  1. The difference between payoff sensitivity and decision-relevant uncertainty is well motivated.
Critical weaknesses
  1. The practical teacher is not sufficiently connected to the decision-theoretic target.
  2. The comparative evidence does not yet establish an explicit advantage under matched computational resources.
Questions for authors
  1. how closely does Equation 5 reproduce the exact decision-value ranking? Please separate the effects of mean-embedding evaluation, joint-action maximization, and entropy weighting.
  2. Which ACORM implementation and configuration were used? Its official QMIX implementation performs K-means within representation training. Were these schedules and its representation-pretraining procedure preserved, and were their costs accounted for consistently?
4 comments
AuthorsOpenReview ↗

Response to Reviewer iYUX

We thank the reviewer for an expert and precise review. W1 and Q1 concern the same gap, so we answer them together.

W1 / Q1. How closely does Eq. 5 reproduce the exact decision-value ranking, and what does each approximation cost?

We agree that the submission connected Eq. 5 to Eq. 2 only partially. Sec. 3.3 and App. C.2 named the three departures, and the error bound in App. B.5 contains a surrogate term ε_sur that neither critic accuracy nor belief calibration controls. We have now measured this gap directly (new App. E.6).

What carries over exactly. Δ̂ preserves three properties of Δ (Theorem 1): it is nonnegative, it vanishes when the receiver's preferred action does not change, and it vanishes when the Teacher belief is deterministic.

Setup. Exact Δ is computed by enumeration on two task types. (i) 60,000 random tabular games (RTG) with n ∈ {3, 4, 6} agents, m ∈ {2, 3} actions and five families (Indep, CBC, Offset, Corr, HO); Dirichlet beliefs span a range of entropy, correlation and coupling; 5 seeds × 400 games per configuration. (ii) A constructed two-chance task in which decision-pivotal and strongly coupled teammates differ (m = 3, n ∈ {4, 6}, 5 seeds × 2,000 instances per scale).

Factorial design. Four switches each replace an exact operation with its practical counterpart. E: the critic is evaluated at the belief-mean embedding instead of integrating over the belief. J1: optimistic maximisation over the sender's action a_j instead of its expectation under the belief. J2: joint maximisation over the other teammates' actions instead of marginalisation. W: entropy weight instead of unit weight. F0000 is exact Δ. Every cell uses the same instances, critic, beliefs and probe set. The RTG critic is a multilinear table, for which E is exact, so cell F0111 reproduces the production Eq. 5 instance by instance; E is measured separately on nonlinear critics.

RTG cellApproximations onTop-1 (%)Norm. regret (%)
F0001W97.051.10
F0010J273.2023.41
F0011J2 + W72.3024.32
F0100J153.6141.51
F0101J1 + W55.2739.77
F0110J1 + J248.6547.03
F0111full Eq. 549.0746.57
  • Entropy weighting is nearly harmless on its own (97.05%).
  • Mean-embedding evaluation is minor: across 600 nonlinear critics, switching on E lowers top-1 agreement from 100% to 95.54% (−4.46 pp; NRMSE 0.0276%).
  • Joint-probe maximisation causes most of the loss. With J1 and J2 on, the switch gate is inactive for every teammate in 49.66% of instances, so all scores are zero and the ranking is a tie (removing the gate barely helps: 49.32%). Revealing a single teammate can leave the receiver's component of the joint optimum unchanged even when Δ > 0. Joint maximisation also makes Eq. 5 sensitive to receiver-independent payoff offsets, as noted in Sec. 3.3.
AuthorsOpenReview ↗

Response to Reviewer iYUX

(W1 / Q1, continued.)

Full Eq. 5 against exact Δ. On RTG, top-1 agreement is 49.07%, Spearman ρ* 0.451, pairwise inversion rate 10.31% and normalised regret 46.57%; random selection reaches 34.44% and the best of four payoff-variance formulas 80.62% (the advantage-variance formula used by Variance-K reaches 79.63%). With a learned critic (native POW recogniser, M = 8), top-1 agreement is 37.30%, against 60.40% for Variance-K. On the two-chance task, top-1 agreement is 92.99% / 93.38% (n = 4 / 6), ρ 0.805 / 0.764, inversion rate 0.62% / 0.12% and regret 0.49% / 0.51%; the best payoff-variance criterion reaches 38.37% / 46.12%. Here J2 alone gives 40.46% / 42.08%, and either J1 (91.15% / 91.50%) or W (J2 + W: 92.29% / 92.73%) corrects this bias. The two repair largely the same instances (Jaccard overlap 89.30% / 89.58%), so they are redundant within Eq. 5.

Where it breaks down. The parameter η controls how unequal the opportunity priors are, p_r = clip(p·exp(ηε_r), 0.02, 0.45). Optimistic maximisation (J1) scores an opportunity by its best case, whereas Δ weights it by its probability, so the two diverge as the priors become unequal. At η = 0.75, Eq. 5 falls to 61.59% / 62.96%, whereas payoff variance × entropy reaches 91.80% / 92.47%.

A corrected Teacher. The corrected Teacher (cell F1000) keeps the learned critic and the belief-mean embedding (E on), replaces optimistic and joint-probe maximisation by belief-weighted expectation (J1, J2 off) and removes entropy weighting (W off); it largely removes this failure. Against exact Δ it reaches 95.54% top-1 agreement on fitted-critic RTG (Spearman ρ 0.963, strict inversion 2.65%; 600 nonlinear critics, NRMSE 0.0276%) and 99.30% against enumeration of the same learned critic and beliefs on the resource game (ρ 0.993, inversion 0.35%; 5 training seeds). Against the environment's true payoff on that game the figures are lower, 72.07% top-1, ρ 0.657 and inversion 15.68%, which we attribute to critic and belief error rather than to the scoring rule: the 99.30% figure holds the critic and beliefs fixed and only changes the target. We will retrain corridor and 10m vs 11m with this corrected Teacher for the camera-ready version; we make no end-to-end claim for it yet.

AuthorsOpenReview ↗

Response to Reviewer iYUX

(Q1, continued.) Does the failure affect the submitted results? The submitted models were trained with the original Eq. 5, so we also asked how often Eq. 5 and the corrected Teacher disagree on the states those models actually visit (new App. H). For each of corridor, 10m vs 11m and Intersection we took the ten submitted final checkpoints, rolled out the evaluation policy, and sampled 2,000 transitions per seed. A decision is one (transition, receiver) pair with at least two eligible teammates; only informative decisions, where the corrected Teacher's maximum exceeds 10⁻⁶, are scored. All selectors share the same critic, beliefs and candidate set. Agreement means the chosen teammate lies in the corrected Teacher's maximal set (tolerance 10⁻⁹); ties receive partial credit, and an all-zero Eq. 5 score counts as a full tie.

TaskInformative (%)Eq. 5 (%)Eq. 5 all-zero (%)Student (%)Variance-K (%)Random (%)Eq. 5 regret (%)
corridor47.2291.194.1389.9670.5428.101.53
10m vs 11m42.9193.912.8591.7375.7917.881.05
Intersection60.4593.174.3490.0968.5138.631.25

On these states Eq. 5 agrees with the corrected Teacher on 91–94% of informative decisions, well above a variance selector that uses the same critic (68–76%) and above random, with normalised regret of 1.05–1.53%. The all-zero ties that affect 49.66% of RTG instances occur in only 2.85–4.34% of informative decisions here. The deployed Student tracks the corrected Teacher almost as closely (89.96–91.73%). The Random column equals the mean of one over the number of eligible teammates, which is a check on the candidate sets. This is agreement with the corrected reference on informative decisions, not a direct measurement of exact-Δ fidelity on these states. We read the 49.07% RTG result as a real property of Eq. 5 whose all-zero failure is rarely triggered here, and make no claim for tasks we did not replay.

Interpretation. The decision-value form and the entropy weight are not the bottleneck; the probe-maximisation step is. Receiver-side scores that use the same critic and beliefs but normalise over the receiver's own action (MI/KL) reach 98–99% in the main two-chance setting and are invariant to receiver-independent offsets. This identifies a concrete way to improve the Teacher. We report these controls in App. E.6 but make no end-to-end claim for them. Eq. 5 is therefore a faithful surrogate when decision-pivotal and strongly coupled teammates differ, and on the states our submitted models visit, but not a general estimator of Δ. We added a paragraph to Sec. 3.3 stating what carries over exactly and where fidelity is lost, revised App. C.2, and the Introduction no longer claims that the Teacher generally identifies decision-pivotal teammates. We did not run the enumeration diagnostic on CCT; Table 5 (87.63% for VIBE vs 8.73% for Variance-K) measures agreement with the ground-truth optimal edge from the known payoff tensor, which is a different quantity. Spearman ρ* in the RTG tables averages only instances with at least three candidates and a non-constant predicted ranking (defined in 34.66% of instances); the ρ reported for the corrected Teacher is the ordinary Spearman correlation. The two are not on the same footing, and App. E.6 states the distinction.

W2. Comparison under matched computational resources.

We agree that execution latency (Table 4) does not settle the training-side comparison, and we have added full resource accounting and a compute-matched control (new App. D.1, with the resource-accounting and compute-matched tables).

Environment interaction. All methods are compared at 10.00M RL-stage environment steps. For VIBE this comprises Stage I backbone training (0.50M) and Stage IV graph-conditioned learning (9.50M). Stages II–III (belief calibration, Teacher scoring, Student distillation) train from the replay buffer and collect no new interaction. The 10,050,000 entry in Table 8 is the runner's hard cap, not the amount consumed; we have relabelled it.

Training compute on 10m vs 11m (GPU-hours cover all stages, including replay-only stages, pretraining and clustering):

MethodGPU-hParams (M)Latency at n = 10 (ms)
POW-QMIX981.101.20
Variance-K1021.251.42
APS1081.181.524
SOP-CG1161.311.896
VIBE1201.251.498
CASEC1241.362.052
ACORM1321.421.62

VIBE's 120 GPU-h break down as Stage I 5, belief calibration 4, Teacher scoring 10, Student distillation 4 and Stage IV 97. VIBE therefore uses more compute than its backbone and the variance control, but less than CASEC and ACORM.

AuthorsOpenReview ↗

Response to Reviewer iYUX

(W2, continued.) Compute-matched control. We gave POW-QMIX 22 and Variance-K 18 additional GPU-hours of replay-only training (no additional environment steps), bringing both to 120 GPU-h. The control uses the same 10 training seeds as the main comparison:

Method (120 GPU-h)corridor10m vs 11mProtossTerranZergIntersection
POW-QMIX + 22 GPU-h79.6074.8018.2017.9016.6092.40
Variance-K + 18 GPU-h77.2072.4019.4019.0018.4090.60
VIBE82.5977.4318.8818.7917.5493.19

VIBE remains ahead of both on corridor, 10m vs 11m and Intersection. Variance-K remains ahead on the three SMACv2 races, as in the main comparison, so neither the gains nor the SMACv2 gap can be attributed to compute (see also our response to Reviewer JQBG, W2). We also corrected the 10-agent latency quoted in Sec. 5.2 to 1.498 ms, consistent with Table 4.

Q2. ACORM implementation, schedules and cost accounting.

We used the official repository (NJU-RL/ACORM, commit 67aa762e3ccc1d7333f77765d9780b2f30aca296), ACORM_QMIX variant. The paper's ACORM runs use the common learner settings of Table 8, the same settings as the other methods: Adam with learning rate 10⁻³, batches of 128 episodes, a 5,000-episode replay buffer, and ε annealed from 1.0 to 0.05 over 500,000 environment steps, with 8 parallel environments, γ = 0.99, TD(λ) = 0.6 and target networks synchronised every 200 learner updates. We kept ACORM's own representation machinery at the official defaults: cluster_num = 3, train_recl_freq = 200 learner calls, agent_embed_pretrain_epochs = 120, recl_pretrain_epochs = 100, multi_steps = 1, n_heads = 4, att_dim = 128, att_out_dim = 64, soft_temperature = 1.0. The contrastive loss has no separate temperature or weight and uses its own optimiser. The K-means schedule within representation training was left unchanged. App. D now states this split, so that Table 8's "common learner settings" are not read as replacing ACORM's pretraining schedule, and ACORM's official learner defaults (batch 32, learning rate 6 × 10⁻⁴, ε annealed over 80,000 steps) are not read as what we ran.

Cost accounting. As in the official code, pretraining episodes are not counted in total_steps: the official counter starts at the ordinary RL stage, and pretraining environment steps are accumulated separately from the actual lengths of the episodes collected during warmup and pretraining. Those interactions therefore come on top of ACORM's 10M RL-stage steps and are excluded from that counter, as App. D now states. Its 132 GPU-h include pretraining and K-means clustering. This is the same convention we apply to VIBE: replay-only Stages II–III consume no environment steps, but every stage counts towards GPU-hours. Because VIBE's own backbone stage counts towards its 10M steps, this convention, if anything, gives ACORM slightly more environment interaction than VIBE.

We hope these additions address your concerns, in particular the new fidelity analysis, which we believe makes the paper's claims more precise. If so, we would be grateful if you would consider revisiting your score. We are glad to provide further details.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to all reviewers

We thank all three reviewers for careful and constructive reviews. We have uploaded a revised PDF. The main changes are summarised below; each review is answered in its own thread.

1. Exact-decision-value study (new App. E.6; Secs. 3.3 and 5.2). Requested by iYUX (Q1) and iuv4 (Q1). We compute the exact Δ (Eq. 2) by enumeration on 60,000 random tabular games (RTG) and on a constructed two-chance task, and we decompose the practical Teacher (Eq. 5) into its approximations with a full factorial design. The results are mixed, and we report them as such. Eq. 5 recovers the Δ-maximising teammate in about 93% of two-chance instances, against 38–46% for the best payoff-variance formula, but in only 49.07% of random games, against 80.62% for the best of four payoff-variance formulas (79.63% for the advantage-variance formula used by Variance-K). The loss is localised: mean-embedding evaluation and entropy weighting cost little, and joint-probe maximisation causes most of it. We have narrowed our claims about the Teacher accordingly.

1b. Does this failure appear in the submitted models? (new App. H). We scored the states actually visited by the ten submitted final checkpoints of corridor, 10m vs 11m and Intersection with a corrected Teacher that replaces joint-probe maximisation by belief-weighted expectation (F1000 in the factorial notation: belief-mean embedding kept; optimistic and joint maximisation and entropy weight removed). Against that reference, Eq. 5 agrees on 91.19%, 93.91% and 93.17% of informative decisions, against 70.54%, 75.79% and 68.51% for a variance selector using the same critic, and its normalised regret is 1.53%, 1.05% and 1.25%. The zero-score ties that affect 49.66% of RTG instances occur in only 2.85–4.34% of informative decisions here. The corrected Teacher itself agrees with exact Δ in 95.54% of instances on fitted-critic RTG and in 99.30% against enumeration of the same learned critic on the resource game. Eq. 5 therefore agrees closely with the corrected reference on the replayed decisions; this is distribution-specific consistency, not a direct measurement of exact-Δ fidelity on those states.

2. Training-side resource accounting and a compute-matched control (iYUX W2). We report total GPU-hours, parameters and latency for all methods, with a stage-wise breakdown of VIBE's training cost. When POW-QMIX and Variance-K receive the same 120 GPU-hours as VIBE (replay-only, no extra environment steps), VIBE still leads on corridor, 10m vs 11m and Intersection, and Variance-K still leads on SMACv2.

3. Variance-K in the main table and calibrated claims (JQBG W2; iuv4 W2). The architecture-matched Variance-K selector is now in Table 1. All reported VIBE-versus-POW-QMIX and VIBE-versus-Variance-K comparisons use seed-level Welch's t-tests with Holm correction. We removed "outperforms the baselines" and "highest on all SMACv2 races", and we state that payoff-variance selection is at least as effective as decision-value selection on SMACv2.

4. Formal presentation (iuv4 W1). Theorem 2 now states submodularity as an explicit assumption for the (1 − 1/e) guarantee. The budget constraint |E_i^t| ≤ K and the tie-breaking rule are explicit, unresolved internal labels are replaced, and the 10-agent latency is corrected to 1.498 ms.

5. Clarifications. The ACORM configuration and cost accounting (iYUX Q2), a precise definition of the Oracle (JQBG W1), the observation assumption stated formally with its limitations (iuv4 Q2), and InfoPG and recursive reasoning in Related Work (JQBG W3).

Taken together, the revision presents decision-relevant edge allocation as a useful inductive bias for coordination-intensive tasks rather than a uniform improvement, and the practical Teacher as a surrogate that is faithful in the regime that motivates VIBE and on the states our submitted models actually visit, with a characterised and now quantified failure mode outside that regime.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.