- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes LJAM, a latent joint-action modeling approach for centralized multi-agent world models. The method employs FiLM to construct state-conditioned action representations and uses Slot Attention to aggregate individual agent actions into a fixed number of latent joint-action effects for subsequent dynamics prediction. Additionally, the authors introduce a prediction-error-aware confidence weighting mechanism to reweight actor–critic updates during policy learning. Experiments on MAMuJoCo and Bi-DexHands demonstrate that the proposed method outperforms existing baselines on multiple tasks.
Critical strengths
-
The paper addresses joint-action representation in multi-agent world models, which is a relevant and meaningful research problem.
-
The method is evaluated on two continuous-control benchmarks, MAMuJoCo and Bi-DexHands, and achieves competitive performance across multiple tasks.
Critical weaknesses
-
The proposed method mainly combines existing techniques, including FiLM, Slot Attention, and Cross-Attention. There is insufficient evidence that the learned slots capture distinct dynamical effects rather than simply serving as generic attention-aggregated features.
-
The authors hypothesize that joint actions contain dynamics-relevant redundancy, but this assumption is neither explicitly quantified nor directly validated. Correlations among actions do not necessarily imply redundancy in their dynamical effects, and a fixed number of latent slots does not guarantee effective information compression.
-
The paper devotes excessive space and equations to describing standard operations, resulting in a verbose and unfocused presentation that obscures the core technical contributions.
Questions for authors
- Confidence weighting only adjusts the actor–critic loss weights based on predicted errors. How does the method prevent errors from low-confidence transitions from propagating to other time steps through recursive return computation? Can the authors provide direct evidence that this mechanism effectively mitigates model bias?
3 comments
Regarding W3, the authors have not yet revised the manuscript to address this concern. The presentation remains overly verbose and unfocused, making it difficult to understand the proposed method and identify its core technical contributions. Therefore, my concern remains unresolved.
Further Manuscript Revisions to Address W3
Thank you for the follow-up. We understand that the previous revisions did not sufficiently clarify the method and its contributions. We have now uploaded a further revised manuscript with the following changes:
-
Rewritten contribution statement (Introduction, p. 2). We replaced the abstract “new view” and “latent effect factorization” descriptions with explicit statements of the design: state-conditioned action tokens, competitive slot aggregation for dynamics prediction, and horizon-normalized error-based weighting for imagined updates. We also distinguish these designs from the established components they employ.
-
Consolidated related work (Sections 2.1–2.2, pp. 2–3). We merged the previous multi-agent and sequential world-modeling subsections, shortened individual method descriptions while retaining all cited works, and consolidated the comparison with LJAM. The latent-action discussion has also been shortened and organized around its relationship to our modeling problem.
-
Removed repeated motivation (Sections 3–4.1, pp. 3–4). We deleted repeated discussions of joint-action modeling from the end of the preliminaries and the beginning of Section 4.1. The presentation now proceeds from the method overview directly to state-conditioned action representations and slot aggregation.
-
Reduced standard-operation exposition (Sections 4.1–4.2, pp. 4–5). Action encoding and FiLM parameter generation are now inline definitions accompanying one action-token equation. The separate fusion equations have been consolidated into an overall transition-model interface, and the observation decoder and reward/continuation interfaces are described inline. Relative to the earlier manuscript, numbered equations in the preliminaries and method sections have decreased from 17 to 12.
Importantly, Section 4.1 now explicitly explains the purpose of normalization across slots and subsequent renormalization across agents, including its distinction from standard cross-attention. This makes the design rationale visible alongside the equations.
These revisions aim to make the technical contributions identifiable without requiring the reader to work through repeated motivation or detailed descriptions of standard components. We would appreciate your reassessment of W3 based on the newly uploaded version.
Additional Evidence for Latent Joint-Action Modeling and Confidence Weighting
Thank you for the detailed feedback. We have uploaded a revised PDF with additional slot-intervention and return-reliability analyses, and streamlined the method presentation.
W1: Existing components and the interpretation of slots
We agree that FiLM, Slot Attention, and Cross-Attention are established techniques. Our contribution concerns their use in a state-conditioned joint-action representation for centralized dynamics prediction, rather than new attention operators.
Contextualized tokens represent individual actions conditioned on state; joint-action effects aggregate across agents for transition prediction. We added interventions that zero one slot or replace it with its reference-action representation, holding real histories and candidate actions fixed.
For the checkpoint in Figure 5(a), either intervention increases full-state prediction error for either slot, with all corresponding 95% episode-bootstrap intervals above zero. In a common PCA basis fitted to real transition effects, the slots also contribute differently across transition directions (Figure 5(b)). Appendix Tables 5–7 report additional checkpoints and state-variable-group results.
The cross-attention checkpoint shows nearly identical intervention effects across its two tokens, whereas LJAM exhibits differentiated contributions. However, another LJAM checkpoint primarily shows a dominant-slot pattern, and the cross-attention comparison is not fully matched in architecture and training history. We therefore interpret these results as functional evidence, without claiming independent physical factors or fixed semantics across checkpoints.
W2: Evidence for latent joint-action modeling
Our central question is whether latent joint-action representations provide more effective conditioning for transition prediction than raw joint actions.
The state-change prediction probe obtains an MSE of 0.0688 with latent representations versus 0.0875 with raw joint actions, a 21.4% reduction; state-only inputs yield 0.1299. Latent similarities also more closely resemble real next-state effect similarities (Figure 8), supporting a representation better suited to transition prediction.
Figure 6(a) further compares slot aggregation with standard cross-attention, an MLP mapping, and direct action-token conditioning. Slot aggregation achieves higher policy returns on the evaluated task. The new interventions additionally establish that the transition model uses information from both slots.
Together, these results support the predictive and downstream value of latent joint-action modeling. A fixed slot count provides a fixed-size token interface; our empirical justification rests on predictive quality and functional contributions, rather than token-count reduction alone.
W3: Verbose presentation of standard operations
We have streamlined the method section by combining action encoding and FiLM parameter generation into inline definitions, consolidating the fusion equations into an overall transition-model interface, and shortening the decoder and reward/continuation descriptions. Repeated motivation was removed, while retaining slot-normalization axes, the multi-step dynamics objective, and confidence-weight construction and application.
Q1: Recursive error propagation and direct evidence
Confidence weights reweight actor–critic losses without changing return recursion. Later model errors can therefore still affect earlier targets; the mechanism does not explicitly block this propagation.
We added a paired finite-return diagnostic on two frozen Ant-8×1 checkpoints. Model and environment branches share 100 initial snapshots from 40 test episodes and identical recorded action sequences. We accumulate rewards and their respective continuation signals over 15 steps with discount 0.99 and no critic bootstrap. This evaluates accumulated reward/continuation errors, rather than reconstructing training lambda-return targets.
Values are absolute finite-return discrepancies. Predicted weights are normalized by their sum; Shuffled reports the exact expectation under within-depth permutations. Paired differences favor Predicted over both controls, with 95% episode-bootstrap intervals above zero. Equal averaging across depths retains reductions of 1.49% and 1.39%, with ESS/N of 0.948 and 0.952 (Appendix Tables 8–10).
The effects are modest and vary across depths. They show that the weights preferentially emphasize samples with smaller accumulated finite-return errors, supporting their intended reliability-selection role. They do not establish that recursive propagation is prevented or that bootstrapped targets or policy gradients are unbiased.