- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces 510K, a four-player card game in which team membership depends on the initial deal and can be hidden from players. The authors separate four questions: whether relationship information is available, whether it can be decoded, whether a policy uses it, and whether it improves outcomes. They study these questions using belief probes, interventions on team-information inputs, and scripted players with different levels of team-awareness. Under the tested PPO configuration, removing explicit team information rarely changes the selected action. Making teammates team-aware improves the learner’s individual first-place rate. The paper also includes ablations of observations, reward settlement, discounting, and training populations, along with exploratory LLM experiments and a privileged-information search reference.
Critical strengths
Separating decodability from policy utilization is useful. The probes and behavioral interventions address different questions that outcome metrics alone cannot answer. This gives the benchmark a clear purpose: studying whether agents act on information about their relationships.
Holding the learner fixed while changing other players’ team-awareness helps identify where performance improvements come from. The same-deal comparisons reduce variation from card luck, and checking the effect on a second deal set strengthens the evidence for the teammate-awareness result.
Critical weaknesses
I am not yet convinced that the behavioral measures establish the claimed utilization bottleneck. Low action-flip rates are difficult to interpret without knowing how often the tested decisions offer meaningful team-dependent alternatives. Many states may call for the same action regardless of team identity. The raw deferral measure also depends on hand strength, game phase, and leader identity, as the authors acknowledge. Finally, the history probe has access to temporal evidence that is not directly comparable to the canonical feedforward policy’s input. The results show limited sensitivity on the measured channels, but the broader conclusion that recoverable relationship information goes unused needs stronger support.
The outcome results support a narrower interpretation than some of the paper’s claims suggest. In Table 5, teammate awareness increases individual first-place rate from 0.212 to 0.278, while finish-based team-win changes from 0.442 to 0.430. The table does not establish whether the reward increase from 78.2 to 80.8 is statistically reliable. The clearest benefit is therefore in individual placement. I also need clarification on Table 7, where first-place rates are 0.206 for DYNAMIC and 0.362 for OBVIOUS. These figures appear difficult to reconcile with statements that explicit revelation does not change outcomes.
Questions for authors
-
What concrete class of coordination problems is 510K intended to model, and which properties of the environment are essential to that connection? What does the reported utilization failure teach us about those problems beyond the limitations of the tested learner?
-
How do the DYNAMIC and OBVIOUS first-place rates in Table 7 relate to the claim that explicit revelation leaves outcomes unchanged? Which comparison isolates revelation while holding the other conditions fixed?
-
What fraction of decisions in the counterfactual test have multiple legal actions and a meaningful opportunity for team-dependent behavior? Does the low action-flip rate persist within that subset?
-
Does teammate awareness produce a statistically reliable improvement in terminal reward or finish-based team-win, beyond the reported improvement in individual first place? What exact changes to the scripted bots’ behavior implement team-awareness?
We thank the reviewer for the detailed and constructive review. The distinction between "sensitivity on the measured channels" and "the broader conclusion" is well taken; we agree with the narrower interpretation and have run the analyses requested.
Q3: meaningful opportunities. We repeated the team-indicator intervention and recorded the decision context. On the default OBVIOUS policy (5 seeds × 100 deals; 8,253 decisions with team indicators, pooled):
54% (4,490) have >1 legal action; 46% are forced passes. Removing the indicators changes 0.0% of forced decisions and 0.04% of choice decisions (2/4,490 pooled; per-seed mean 0.04%, range 0.0–0.2%); partner-leads 0.04%; opponent-leads 0.0%; pass↔play 0.0%. For RevealEnv p=1, choice-state change is 5.8% pooled (per-seed mean 5.2%, range 0.3–12.9%), partner-leads 3.9%. Even the full masked action distribution barely changes: mean KL(true||zero) is 9.4×10⁻⁵ in choice states (OBVIOUS) and 0.043 for RevealEnv p=1. A second deal set (seeds 2000–2099) reproduces the pattern: flip 0.0–0.1%, outcome reward difference 0.01 (p=0.37). Consistently, the raw deferral asymmetry is sample-sensitive (100 vs 300 episodes can change its sign), so we use it only as a behavioral channel, not as proof. Thus the low rate is not a denominator artifact; it persists where the team relation could change the action. Q2: Table 4 / revelation. We agree the DYNAMIC–OBVIOUS comparison is not a reveal manipulation: they are separately trained checkpoints, and OBVIOUS also makes the rule bots team-aware, so the other players change too. The revised manuscript relabels it as a cross-checkpoint capability comparison and removes it from the causal claim. The comparisons that hold other conditions fixed are: (i) reveal dial during DYNAMIC training: p=0 vs 1 first 0.212 vs 0.200, slope p=0.83; team-win 0.442 vs 0.460, slope p=0.47; (ii) within-policy counterfactual test on OBVIOUS: same policy, same 100 deals, indicators true vs removed vs shuffled: first 0.362/0.362/0.362, team-win 0.482/0.482/0.482, reward 69.88/69.88/69.88; the intervention changed the chosen action in two of 8,253 decisions, and terminal outcomes were identical. For the RevealEnv p=1 checkpoint, test-time zeroing gives first 0.200 vs 0.186 (p=0.21), team-win 0.460 vs 0.432 (p=0.23), reward 82.20 vs 80.36 (p=0.19): no reliable change.
Q4: teammate awareness and statistics. You are right that the original 100-deals/seed evaluation was underpowered for reward. We re-ran the same fixed-policy decomposition with 500 deals/seed (same 5 seeds). Teammate-aware vs ignorant on the primary deals: first +0.061 (p<0.0001; all five seeds positive), reward +2.39 (95% CI 1.35–3.44; p=0.0031), team-win −0.010 (p=0.071). On the fresh deal set: first +0.045 (p=0.0009), reward +6.78 (CI 3.38–10.17; p=0.0052), team-win +0.012 (p=0.234). So terminal reward is statistically reliable; finish-based team-win is not. The clean opponent-only contrast (all-aware vs teammate-aware) lowers reward by 13.47 and 13.08 (p<0.0001).
Bot implementation: teammate-aware bots use one extra rule: when their partner is the current trick leader and passing is legal, they pass; otherwise they play the rule bot's lowest legal combination. Opponents are unchanged. All-aware applies the same rule to every seat using its own team. This is a minimal one-rule intervention, which is why it isolates partner responsiveness; it is not a rich cooperative policy. When the learner is trained under teammate-aware bots, the same contrast gives first +0.064 (p=0.0112), reward +5.29 (p=0.0106), team-win +0.012 (p=0.3585).
Q1: class of problems / what it teaches. 510K instantiates latent-partner ad-hoc teamwork under coupled individual and team objectives: team membership is random at deal time (2v2 76.5%, 1v3 23.5%), hidden, decodable from the agent's observation and public history, and payoff-relevant through the team settlement. It isolates "who is with me" from "how to coordinate" (known teams in Hanabi/Overcooked) and from adversarial hidden-role games (Avalon/DeepRole). The essential properties are: stochastic hidden membership; public evidence sufficient for inference; reward indexed by the latent team; counterfactual interventions and ground-truth labels. The utilization failure teaches that availability/decodability are not sufficient; outcome metrics can mask non-use; standard PPO can achieve high reward via generic card play without conditioning on a payoff-relevant latent. This motivates auxiliary belief objectives, memory, and population training. We scope this to the tested learner; the revision adds a scope paragraph.
History probe. We agree the history probe is not the feedforward policy's input; it is an upper-bound diagnostic. The revised manuscript relabels it as "decodability by an idealized history probe" and not as "what the policy sees".