- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the symmetry trap in multi-agent LLM coordination, where homogeneous agents acting concurrently without communication may collide on targets requiring division or diverge on targets requiring joint action. The authors propose Theory of Scene (ToS), a training-free reasoning schema that derives a division of labor from shared observations and public roles rather than predicting teammates' intentions. ToS uses role gating to distinguish overlapping from already divided ownership, and task coupling to identify joint, exclusive, or sequential dependencies. Agents then select actions through a shared ordering and role-based allocation. Experiments on the newly introduced DivvyBench and adapted versions of GovSim and Collab-Overcooked show substantial improvements over six baselines. The paper also provides probability bounds characterizing coordination under shared and agent-specific action distributions.
Critical strengths
-
Clear problem formulation and controlled evaluation. The symmetry trap captures two opposing coordination requirements: divergence on exclusive targets and convergence on joint targets. DivvyBench provides a controlled setting for evaluating both requirements within the same episode. ToS achieves nearly perfect completion and substantially outperforms the strongest baseline.
-
Substantial empirical improvements across different coordination settings. The same ToS reasoning schema improves performance on both shared-resource allocation and sequential cooking tasks. Experiments with additional model backbones, heterogeneous agents, observation perturbations, and component ablations provide useful evidence of its effectiveness and robustness.
Critical weaknesses
- The contribution of the structured reasoning schema is not sufficiently isolated from the explicit coordination rules. The ToS prompt directly instructs agents to use a canonical ordering and role-based allocation for different task couplings. Although Table 2 examines the role and task readings, it does not isolate the benefit of these intermediate reasoning fields from that of the supplied coordination rules. Table 3 compares direct action derivation with prediction-based action selection, but does not include a simpler, rule-matched direct-action baseline. Consequently, it remains unclear how much of ToS's advantage comes from its structured reasoning schema rather than the explicit coordination convention.
- The theoretical analysis does not establish a general limitation of Theory of Mind. The probability bounds characterize coordination under shared or agent-specific action distributions, but do not demonstrate that ToM inherently cannot escape the symmetry trap. In particular, a ToM policy could incorporate public roles and a shared allocation convention when predicting teammates' actions. Although the evaluated ToM baselines perform poorly, these results establish limitations of the tested reasoning schemas rather than a general impossibility for ToM. The abstract's categorical claim that ToM "cannot escape this trap" therefore appears stronger than the theoretical and empirical evidence warrants.
Questions for authors
- How does ToS compare with a compact baseline that receives the same shared observations, public roles, and coordination rules, but proceeds directly to planning and action selection without explicitly generating separate role and task readings? This comparison would help isolate whether the intermediate reasoning structure provides benefits beyond the supplied coordination convention.
- Is the claimed limitation of ToM intended to apply only to the evaluated prediction-based reasoning schemas? Given that the theoretical bounds do not exclude role-dependent action distributions, what assumption would prevent a role-aware ToM policy from predicting complementary or matching assignments under a shared convention?
5 comments
Dear Reviewer 9Eno,
We thank you again for your time and effort in reviewing our paper. In our response, we have conducted ablation studies on the reasoning schema and the coordination rules, including the rule-matched direct-action baseline (W1) and the compact baseline (Q1), benchmarked a role-aware ToM under a shared convention with its prediction accuracy and stuck rate (W2 & Q2), and revised the abstract's statement about ToM (W2).
With three days left in the discussion period (November 18, AoE), we would greatly appreciate hearing whether our response has addressed your concerns, and we would be glad to address any that remain.
Best regards,
The Authors
Thank you once again for your valuable comments on our submission. We have posted a summary of our rebuttal at the top of this page. With the discussion period ending on November 18 (AoE), we would greatly appreciate hearing whether our response has addressed your concerns, and if you are satisfied, we would be grateful for your consideration in updating the score. We sincerely look forward to your feedback.
Response to Reviewer 9Eno
We thank the reviewer for the careful and constructive review.
W1 & Q1. The Schema and the Stated Rules
We have run three variants, (i) ReAct + Coupling Rules, the rule-matched direct-action baseline in W1, (ii) ToS (Reasoning without Rules), the intermediate reasoning fields without the coordination rules in W1, and (iii) ToS (Compact Baseline), the compact baseline in Q1 that proceeds directly to plan and action without the role and task readings. None of the three variants reaches ToS across the three benchmarks, so the advantage of ToS needs all three parts below.
W2. The Propositions and the Abstract's Claim about ToM
We want to clarify that the propositions and the abstract's sentence about ToM are two separate claims. The propositions bound the per-step probabilities of any policy, ToM included, and Proposition 2 shows the importance of the public role, so a role-aware ToM is a potential solution under them. The abstract's sentence is an empirical claim about prediction-based schemas, in which a homogeneous agent's prediction of the others returns only its own reasoning (Section 3).
We have also run the policy the reviewer describes, Role-Aware ToM under a Shared Convention. The public role and the convention raise it above Classic ToM on DivvyBench, and it remains below ToS on all three benchmarks, because its action still depends on a prediction of the other agents, whereas ToS derives its action from the scene alone (Q2).
Q2. A Role-Aware ToM under a Shared Convention
We want to clarify that our claim is empirical and restricted to schemas that derive their action from a prediction of homogeneous agents, the more challenging and practical setting, as LLM agents behave alike even across distinct LLMs. In W2, we have shown that a role-aware ToM can predict matching or complementary assignments under a shared convention and remains below ToS on all three benchmarks. We therefore report the prediction accuracy and the stuck rate of the ToM schemas, where the role-aware ToM reaches the highest prediction accuracy and the lowest stuck rate of the ToM methods, yet its stuck rate remains above that of ToS, because its action still depends on the prediction.