- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes SIGMA, a multi-agent communication framework that uses a variational encoder to produce Gaussian message parameters from agents' local observations. An offline LLM-annotation stage supplies semantic targets for variational pretraining, and an inverse-variance fusion mechanism weights incoming messages based on their predicted uncertainty. The fused means are subsequently used by a QMIX-style MARL policy. Experiments on SMAC/SMACv2, LBF, and GRF evaluate task performance, communication budgets, uncertainty calibration, and transfer to unseen scenarios.
Critical strengths
-
The paper considers how agents should weight received information under partial observability. The empirical study spans multiple MARL benchmarks and includes communication-budget experiments, component ablations, transfer experiments, and uncertainty analyses.
-
Variance-weighted aggregation is computationally straightforward, and explicitly communicating a mean and uncertainty parameter offers a useful design for investigating confidence-sensitive message fusion.
Critical weaknesses
1. Limited novelty and incomplete positioning against closely related probabilistic and information-bottleneck MARL methods
The claimed contributions rely heavily on established techniques: Gaussian variational encoding, KL regularization, inverse-variance weighting, offline semantic pretraining, and value-based MARL. The main combination is potentially useful, but the manuscript does not convincingly explain which part constitutes a new methodological insight beyond assembling these components.
In particular, the Related Work should discuss several closely relevant methods more precisely:
- Bayesian Ego-graph Inference for Networked Multi-Agent Reinforcement Learning (NeurIPS 2025; BayesG) treats communication structure as a latent stochastic variable and optimizes variational edge masks under local observations. It infers which links should be active, whereas SIGMA assigns uncertainty-based weights to messages on available links. The mechanisms are distinct, but both use Bayesian/variational uncertainty to guide decentralized interaction; this distinction is important to the novelty argument.
- Bandwidth-constrained Variational Message Encoding for Cooperative Multi-agent Reinforcement Learning (AAMAS 2026; BVME) is particularly close to SIGMA's variational content encoding and KL-based compression claims. BVME already uses Gaussian message posteriors and information-bottleneck-inspired regularization under explicit dimensional constraints. SIGMA should explain what is new relative to this setup, beyond LLM semantic targets and inverse-variance fusion.
- Heterogeneous Information-Bottleneck Coordination Graphs for Multi-Agent Reinforcement Learning (HIBCG, arXiv:2605.17393, 2026) integrates information-bottleneck principles with heterogeneous coordination-graph structure and per-agent/message capacity control. This provides another relevant reference point for SIGMA's information-bottleneck and communication-efficiency motivation.
These are relevant comparisons for establishing the research gap, not automatically mandatory direct baselines: BayesG and HIBCG primarily address graph structure/capacity, whereas SIGMA emphasizes semantic priors and confidence-weighted aggregation. However, omitting their discussion leaves the contribution insufficiently differentiated from recent graph-based and variational MARL research.
2. The key uncertainty and Bayesian-fusion assumptions are not adequately justified
SIGMA interprets the variational encoder's learned variance as an estimate of partial-observability uncertainty and uses its inverse to weight messages. Yet a Gaussian encoder trained through reconstruction and KL regularization does not automatically provide calibrated epistemic uncertainty: its variance may reflect optimization, compression, or model misspecification rather than how reliable an agent's message is for a particular receiver.
Furthermore, the Gaussian product/precision-weighting argument assumes that different agents' messages are conditionally independent estimates of a shared latent quantity. In MARL, observations are correlated, and each encoder may represent different local information. It is unclear whether these assumptions hold sufficiently for the fusion rule to be described as optimal Bayesian aggregation rather than a useful heuristic.
There also appears to be a mathematical inconsistency in Eq. (5): if denotes standard deviation as defined in Eq. (1), the fused standard deviation should be (component-wise), or the fused variance should be . The displayed expression instead appears to set . The authors should verify the equation, figure, and implementation and clarify whether this is a notation error or affects the reported uncertainty calibration. The reported action network receives the fused mean only, so any consequences for policy performance should be distinguished from those for calibration.
Questions for authors
- What concrete methodological capability does SIGMA add beyond prior Bayesian/variational graph and message-learning approaches, especially BVME's Gaussian KL-regularized messages and HIBCG's bottleneck-based communication-capacity control?
- What justifies interpreting independently encoded local messages as conditionally independent estimates of the same latent variable? Can the authors verify Eq. (5) and clarify whether the fusion rule is a formal Bayesian estimator or a precision-weighted heuristic?
- How is the learned distinguished from representation compression/noise, and what evidence shows it is calibrated as sender reliability under shift?
- Since online updates omit the LLM semantic loss and TD gradients do not update the VAE, what ensures that the claimed semantic organization persists and remains relevant to subsequent policy learning?
5 comments
Responses by Authors
Part 2/3: Responses to Weakness 2 and Questions 2-3
Weakness 2 and Questions 2–3: Uncertainty Interpretation, Fusion Assumptions, and Calibration
Response: We thank you for these concerns. We agree that a VAE variance is not calibrated epistemic uncertainty, shared encoding does not guarantee conditional independence, and Eq.5 needs clearer notation. We describe σ as a relative confidence proxy and fusion as a precision-weighted product-of-experts aggregation with a conditional Bayesian interpretation, verify Eq.5, and add σ permutation and constant-σ controls.
Two claims about σ. The concern splits into whether σ reflects sender knowledge of global state (A) and whether it indicates receiver-specific trust (B). SIGMA supports A only as a relative proxy: σ ranks observations by how much they constrain the hidden state, not as a calibrated estimate of . Appendix C.3 reports with global-state reconstruction error, lower error using than only (0.23 vs. 0.41), stronger response to occlusion than additive noise, and ECE 0.11→0.04 after temperature scaling; these are empirical associations, not theoretical identities. σ is not an explicit compression-rate control or noise estimate; the occlusion-vs-noise contrast supports this. For B, a globally uncertain sender may still be most informative; SIGMA weights by per-dimension precision without modeling receiver-specific relevance, so σ is not receiver-specific reliability.
Fusion as Gaussian product of experts. A shared encoder guarantees a shared function, not that each latent dimension estimates the same quantity. Under a shared prior with conditionally independent observations, the joint posterior needs a prior correction, , rather than unconditional multiplication that double-counts the prior. When the prior is weak relative to message likelihoods, this correction is small, and Eq. (4) is a useful approximation; we state this condition rather than claiming an exact posterior. Inverse-variance weighting minimizes variance among linear unbiased combinations under unbiased messages, uncorrelated errors, and matching variances; we treat it as a statistically motivated approximation.
Eq.5 verification. We thank you for flagging Eq.5, and we realize the notation can be read in more than one way. The intended meaning is the fused precision, , equivalently , which is consistent with standard inverse-variance fusion and with the implementation. To remove the ambiguity, we will rewrite Eq.5 to show precision, variance, and standard deviation on separate lines. The action network receives only , so calibration affects reported uncertainty, not policy.
Temperature scaling calibrates reported variance but does not enter fusion weights; multiplying all variances by a positive scalar leaves normalized weights unchanged. Improved ECE therefore does not itself show that sender ranking or fusion decisions improve. Table 17 reports transfer-target calibration (ECE 0.13→0.06, NLL 1.18→0.92), supporting meaningful reported uncertainty under shift, not a perfectly calibrated sender-reliability estimator under arbitrary shift.
Inference-time controls. We permute σ across senders and fix all variances to a constant, without retraining.
| Inference condition | Win rate (%) | Δ vs. unmodified |
|---|---|---|
| SIGMA (original σ) | 84.5 | — |
| σ permuted across senders | 77.8 | −6.7 |
| σ fixed to constant | 76.3 | −8.2 |
Permuting σ across senders drops the win rate to 77.8%, showing that content-confidence matching matters. Fixing all variances to a constant drops it further to 76.3%, showing that a non-uniform confidence distribution also contributes. We also train a probe on local history only:
| Probe (no σ) | Pearson | Spearman | AUC |
|---|---|---|---|
| Hidden global-state prediction error | 0.74 | 0.72 | 0.71 |
Because the probe never sees or , this supports the proxy claim independently.
Task-relevant reliability check (2o_20b_vs_2r). We compare fusion weights with task-relevant error for enemy position. (i) Visible-Roach Overseer messages have lower error and higher weight than non-visible Baneling messages. (ii) For the same Overseer, occlusion raises error and lowers weight, so weight tracks current visibility rather than sender identity. (iii) Highly overlapping senders receive comparable weights, so inverse-variance fusion does not explicitly de-duplicate redundant messages; we treat this as a limitation. These checks support a task-relevant fusion value for σ without claiming calibrated posterior variance.
We will revise Eq.5 and Section 3.4, and state explicitly that σ is a relative confidence proxy whose benefit comes from content-confidence matching rather than calibrated absolute uncertainty.
Responses by Authors
Part 3/3: Response to Question 4
Question 4: What Ensures Semantic Organization Persists Online?
Response: We thank you for pointing this out. We agree that removing the online LLM semantic loss and blocking TD gradients from the VAE does not guarantee preservation of the pretrained semantic geometry. Dual-speed reduces but does not eliminate drift; we describe it as mitigation, not a guarantee. We report drift measurements and an inference-time intervention testing reliance on semantic organization.
Online, the VAE trains only with reconstruction and KL (), updates every episodes at a learning rate an order of magnitude smaller than the policy, and blocks TD gradients. This reduces parameter change but cannot guarantee unchanged semantic geometry; reconstruction may also pull the latent toward reconstruction-friendly directions that conflict with pretrained semantics. Appendix C.4/C.5 indirectly support this: freezing outperforms fine-tuning without slow updates (74.8 vs. 68.2 and 81.0 vs. 75.4), but these are not direct measurements.
Persistence: direct drift measurement. Using saved LLM annotations, we freeze policy snapshots at several stages and report semantic cosine between decoded and original embeddings, CKA between pretrained and online latents, and nearest-neighbor preservation for semantically similar observations. CKA and NN preservation are primary; cosine is secondary because decoder updates can confound it. We set a 0.90 "semantic-retention" threshold from the validation lower bound after pretraining. Four update schemes are compared under the same budget.
| Update scheme | Semantic cosine↑ | CKA↑ | NN preservation↑ | Win Rate (%) |
|---|---|---|---|---|
| Fully frozen VAE | 1.00 | 1.00 | 1.00 | 76.6 |
| Dual-speed (SIGMA) | 0.93 | 0.91 | 0.89 | 84.5 |
| Frequent VAE updates | 0.74 | 0.68 | 0.65 | 79.2 |
| Periodic semantic anchoring | 0.96 | 0.95 | 0.94 | 84.1 |
Dual-speed stays above 0.90 while frequent updates fall well below it. Preservation and performance are non-monotonic: frozen preserves best but performs lowest, while dual-speed preserves above the threshold and performs best. Preservation alone is insufficient; the representation must also adapt. The drop under frequent updates suggests semantic organization is relevant, not incidental. Periodic anchoring recovers most similarity but does not improve performance over dual-speed; the difference is within seed noise.
Relevance: inference-time semantic intervention. Drift measurement shows whether structure is retained, not whether the policy relies on it. We identify semantic dimensions with a linear probe predicting LLM embeddings; top- by probe weight are semantic. We permute only semantic dimensions of across samples, keeping , non-semantic dimensions, and policy unchanged; this breaks content-semantic correspondence while preserving marginals.
| Inference condition | Win Rate (%) | Δ vs. unmodified |
|---|---|---|
| Unmodified SIGMA | 84.5 | — |
| Semantic dimensions of permuted | 72.1 | −12.4 |
| Non-semantic dimensions of permuted | 83.6 | −0.9 |
The large semantic drop and small non-semantic drop indicate that the policy relies on semantic organization of the fused representation rather than treating it as a generic feature vector. The non-semantic drop is within seed noise. This addresses "remains relevant to subsequent policy learning" more directly than downstream win rate alone.
Mechanism decomposition. To separate update frequency from online loss, we compare three VAE update rules at the same frequency .
| Online loss | Semantic cosine ↑ | CKA ↑ | Downstream WR (%) |
|---|---|---|---|
| KL only | 0.88 | 0.84 | 82.4 |
| Reconstruction only | 0.83 | 0.78 | 82.9 |
| Reconstruction + KL (SIGMA) | 0.93 | 0.91 | 84.5 |
Reconstruction-only drifts most, consistent with the reconstruction objective pulling the latent away from semantic structure. KL-only also drifts, though less, because it pushes the posterior toward the prior; to keep this variant trainable, we freeze the decoder and use the same low KL weight as in SIGMA, so it does not collapse. Keeping both terms is more stable: the online gradient stays closer to the pretraining balance than either alone, which is why the combined loss drifts least. At dual-speed frequency, the loss combination is therefore the main factor preserving semantic structure; this does not show that frequency is irrelevant.
We therefore present semantic preservation as an empirically measured property of the dual-speed scheme, not as a theoretical guarantee.
We are grateful for these constructive suggestions. We hope these responses and the planned revisions address your concerns.
Responses by Authors
We sincerely thank you for the insightful and constructive comments. We address each concern below.
Part 1/3: Responses to Weakness 1 and Question 1
Weakness 1 and Question 1: Novelty and Positioning vs. Probabilistic and IB-based MARL
Response: We thank you for pointing this out. We agree the components are established and SIGMA is insufficiently differentiated from BayesG, BVME, and HIBCG. Our different angle on the variational variance is to use the same σ that shapes the latent code during training as the execution-time weight for message trust, so content and trust come from one encoder rather than two mechanisms. We will add a Related Work comparison and a mechanism control isolating σ-as-fusion-weight from variational encoding.
The three methods differ in what the uncertainty acts on:
The closest comparison is BVME. BVME samples zi=μi+σi⊙ϵ during training and feeds the sampled representation downstream, coupling stochastic encoding with task optimization; at evaluation it uses the deterministic mean. SIGMA transmits mi=(μi,σi) and uses σi−2 as the receiver-side fusion weight. The real difference is that BVME uses σ inside stochastic encoding and regularization, whereas SIGMA uses σ directly as a fusion confidence signal. The original manuscript described σ mainly as an uncertainty proxy; we will state explicitly that its receiver-side fusion role is the contribution relative to BVME.
Mechanism control. BVME has no official implementation, and its GNN backbone, end-to-end training, and bandwidth-constrained message dimension differ from SIGMA's QMIX backbone. A head-to-head comparison would mix encoding, training, and fusion factors and cannot isolate receiver-side confidence weighting. We instead compare five variants within the SIGMA backbone on 2o_20b_vs_2r. (i) SIGMA (full): Gaussian VAE, frozen/slow updates plus LLM semantics, inverse-variance fusion. (ii) SIGMA w/o LLM: same encoder and fusion without LLM supervision. (iii) SIGMA + E2E VAE: end-to-end training with TD gradients entering the VAE and inverse-variance fusion. (iv) SIGMA + E2E VAE + simple fusion: same end-to-end encoder but simple averaging, so σ is not used in fusion. (v) BVME-style: SIGMA's Gaussian VAE but samples z=μ+σ⊙ϵ during training, feeds the sampled message downstream under end-to-end training, and uses the deterministic mean at evaluation, matching BVME's evaluation protocol; σ is not used as a receiver-side weight.
The design isolates one factor at a time: (iii) vs (iv) isolates σ as a fusion weight, (i) vs (iii) isolates frozen encoder plus LLM semantics, and (v) tests sampled variational encoding without receiver-side weighting. The key comparison is SIGMA + E2E VAE (79.6) versus SIGMA + E2E VAE + simple fusion (73.8). With encoding and training fixed, using σ as a fusion weight yields a substantial gain, isolating receiver-side confidence weighting from variational encoding itself. SIGMA + E2E VAE (79.6) versus SIGMA (full) (84.5) shows that freezing the encoder and adding LLM semantics contributes further, so the gain is not due to end-to-end training alone. BVME-style (72.4) is comparable to the variant without receiver-side confidence weighting, consistent with sampled variational encoding without σ-based fusion not producing the same benefit. All variants use the same QMIX backbone, training budget, and actual bit budget. The BVME-style variant is not a full BVME reimplementation or a head-to-head baseline; it simulates BVME's key mechanism to isolate σ's fusion effect.
We will not claim a new Gaussian encoder or information-bottleneck objective, or better compression without direct comparison. We will make this positioning explicit in Related Work and the contribution statement.