acceptodds
Under review as a conference paper at ICLR 2027

When Good Teammates Are Weaker Alone: Handoff Regret in Skill-Compatible AI

Abstract

A strong agent that shares control with a weaker partner can leave behind states the partner cannot handle, even when each of its own actions is sound. We study this through handoff regret, the value lost when a fixed partner rather than the agent takes the next action. The central object is the handed-over loss: the solo losses of the partner's actions, summed over its observation classes with the team's weights. Team loss relative to the solo optimum is exactly the agent's own error plus this loss averaged under the partner's rule; the least any rule on the same observations can pay is the value of the information the partner lacks, and the rest is its decision error. Refinement, as in Blackwell's comparison of experiments, is exactly the order under which the information term never rises in any task, but behaviour identifies the split only up to an interval whose ends respond to different interventions, and a construction at one end shows that the team gain per unit of solo loss can be unbounded. The same accounting yields partner-aware corrections to an existing value oracle with explicit error terms. In tag-team chess with Maia-2 partners and strict alternation, the correction beats an engine given the same search budget head to head (57.2% of points), while the extra search alone gains nothing. In navigation, interventions on the agent's planning, the partner's rule and its information each move the term the decomposition predicts; on human games, estimated handoff regret predicts losing moves better than standard difficulty measures, and adds to them.

Primary area:
reinforcement learning
Keywords:
human-AI collaboration, skill compatibility, handoff, value of information, information structures, Markov decision processes, partner modelling, chess

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_PtNYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper systematically studies the agent handover problem where a strong agent needs to figure out the best strategy when collaborating with a weak agent, because naively output the best action under the strong agent's policy is no longer the best move when the weak agent is not able to pick up the subsequent strategy on its own.

The paper provides a detailed theoretical analysis around handoff regret, and apply the proposed method in chess and navigation.

Critical strengths
  • The paper presents theoretical contribution, and also have a discussion over prior work (Expector) under the theoretical framework of this work.
  • The experiment is solid, with sufficient controls that well explain the source of gains. For example, the authors increase the compute for the baseline pair that does not use HAD's extra search and show that HAD still outperforms the pair.
Critical weaknesses
  • The proposed HAD method is on-par with Expector, but this issue is partly mitigated by the fact that the theory in this paper helps explaining the why it works. I also appreciate the fact that the authors are honest about this.
  • The method relies on opponent modeling, and in evaluations the authors use the same Maia model and Stockfish engine that the opponent pair actually uses. This is a strong assumption, and I would like to know how the system will perform when the opponent modeling is noisy.
Questions for authors
  • Maybe I have missed it, but do you have any results on the robustness to opponent modeling?
1 comment

Thank you for the review and for the direct question. Yes: the submission reports robustness to partner-model error on random MDPs (Appendix G, Table 14), and we have now run it in chess. With deliberately wrong partner and opponent models, the gain over the reference team holds in every condition.

W2 / Q1 (robustness to opponent and partner modelling).

do you have any results on the robustness to opponent modeling?

New runs with the settings of Section 5.1: Maia-2 juniors truly at 1500, strict alternation, the 800 held-out opening pairs, HAD-4 against the reference team R (raw engine plus the same junior). The senior is given a wrong model of its own junior or of the opposing junior: Maia-2 at the wrong rating, or mixed with a uniform random legal move.

Senior's model of the juniorswin share vs R (%) ± 95% intervalmean TV of the model
Maia-2 at the true rating (paper's configuration)58.6 ± 2.30
own junior modelled at 110057.1 ± 2.20.138
own junior modelled at 190057.4 ± 2.20.111
opposing junior modelled at 110057.5 ± 2.20.144
opposing junior modelled at 190057.7 ± 2.20.116
both models mixed with a uniform move at weight 0.357.2 ± 2.20.22

HAD-4 beats R under every wrong model, with every interval above 50%. Compared with the true-rating model in the same runs, the win share is 0.9 to 1.5 points lower, and no single difference is distinguishable from zero (Table 6 in Appendix D gives each with its interval). The last column is the mean total-variation distance between the assumed and the true move distribution at the positions where that junior moved. A rating error changes which moves the model ranks first; the uniform mixture keeps the ranking and flattens the probabilities.

The true-rating row is the paper's configuration, run alongside the wrong models on different machines and software builds from the submission; the submitted runs give 56.9 ± 2.2 on the same openings. In that environment we also reran all nine pre-registered comparisons: the pre-registered criterion holds again, with HAD-4 beating matched compute head to head at 57.3 ± 2.3 (submitted: 57.2 ± 2.3; Table 5).

Two further facts bear on the question. First, the senior's models were approximate in the submitted runs as well. It uses the top 4 moves of its own junior (0.83 of Maia-2's mass on average), the top 2 of the opposing junior, and a depth-10 reply for an opposing senior that searches at depth 12. In the head-to-head runs of Table 1 the opposing senior is in fact OA, the matched-compute engine or Expector, and HAD-4 scores 55.9 ± 2.3, 57.2 ± 2.3 and 50.7 ± 2.0 against them. Second, the theory has the same form: in Theorem 12 the value lost to a partner-model error ϵ\epsilon (worst-case total variation) is at most linear in ϵ\epsilon.

Table 6 and its description are in Appendix D of the revised PDF (in blue), with one sentence in Section 5.1.

W1 (on par with Expector). Correct, and this is what the paper claims: with full partner support and a two-step horizon the correction is Expector (Section 4), and head to head the two are level (50.7 ± 2.0%; 49.3 ± 1.9 in the rerun). What the paper adds to Expector is the account of why such a rule works. The decomposition places the partner's loss in chess in decision error at the states it is handed, which extra search cannot change: an engine given HAD-4's search budget gains nothing as a teammate (50.7 ± 1.9% against R), while HAD-4 beats it head to head (57.2 ± 2.3%, pre-registered). The analysis also gives explicit error terms for oracle error, partner-model error and truncated support (Theorem 12, Proposition 21), and states the correction for any value oracle and partner model, with scale and depth as parameters (Definition 11).

If the mismatch table answers your question, we would appreciate it if you would reconsider your rating in that light. We are glad to answer further questions.

Reviewer_pKBLOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies teams of two agents, a strong and a weaker agent. The critical failure mode is that actions that might be helpful if the strong agents plays alone result in sitations the weaker agent can't handle. The paper then decomposes the regret into the regret of the strong payer and the regret induced by creating situations the weaker agent can't handle. The authors propose handoff aware planning (HAD), which aims to account for expected mistakes in the weaker agent. Experiments are conducted in stricly alternating tag-team chess using stockfish as strong player and maia as weaker agent (human substitute).

Critical strengths
  • Elegant decompositions of the regret in a well motivated setting of increasing real world importance. Crucial for the community to engage.
  • Theory and practise align describe the phaenoenon in a coherent way. Chess is a well choosen domain.
Critical weaknesses
  • The paper is mainly using known tools to analyse an interesting setting. The experiments confirm the phaenomenon but show little gain of HAD over Expector.
  • Static partner is a rather strong assumption. Would encourage extensions to co learning to make for a more substantial contribution.
Questions for authors

Knowing the information/ decision error decompostions is a rather strong assumption. How to make this feasible in practise. What learning algorithms should be considered in the first place?

2 comments

Thank you for the review. Below: what is new, how the theory covers partners that adapt, and answers to the two questions.

W1a (known tools). The ingredients are classical, and the paper cites each where it uses it. The contribution is the object they are assembled around and three results about it.

  1. The handed-over loss and the exact three-term identity (Theorem 5). Performance-difference accounting and the value of information each give one piece. The theorem puts them on one object, the partner's solo losses summed over its observation cells with the team's own weights, and shows that the three ingredients of that pairing are the three remedies: planning, a better decision rule, more information.
  2. Identifiability (Proposition 10). The partition induced by the partner's behaviour is the coarsest information partition compatible with it. Behaviour therefore identifies the split between missing and misused information only up to an interval, and the two ends of the interval respond to different interventions (Figure 1c).
  3. An unbounded exchange rate from information alone (Theorem 8). A solo-for-team trade-off forced purely by the partner's missing information, with a team gain per unit of solo value given up that is unbounded. This shows how the finding that compatible agents are weaker alone can arise in an ordinary MDP.

Proposition 9 ties the information term to Blackwell's order: refinement is exactly the order under which it never rises, which makes "more information" a well-defined remedy. Theorem 12 and Proposition 21 add error accounting for corrections of an existing oracle, separating oracle error, partner-model error and truncated support.

W1b (little gain over Expector). Correct, and the paper claims none: with full partner support and a two-step horizon the correction is Expector, and head to head the two are level (50.7 ± 2.0%). What the chess experiment establishes is that the gain comes from partner-awareness and not from compute. HAD-4 beats an engine given the same search budget head to head (57.2 ± 2.3%, pre-registered; 57.3 ± 2.3 in a full rerun on different machines), while that engine gains nothing as a teammate (50.7 ± 1.9%).

W2a / W2b (static partner, co-learning). The decomposition is less restricted than "static" suggests, and the revision makes this explicit. A partner that updates a finite memory after every step by a fixed rule, and acts on its observation and that memory, is a fixed policy on the state augmented with its memory, and the solo values do not depend on that memory. Theorem 5, Corollary 6 and the refinement direction of Proposition 9 therefore hold for such adaptive partners (Proposition 23, Appendix K; a numerical test on 1,000 random instances gives residuals below 1e-12). We have added a sentence on this to Section 3.1 of the revised PDF, with the argument in Appendix K. The experiments use fixed partners, as Section 3.1 states. Co-learning of both policies is a different problem; at any fixed pair of policies the decomposition still applies as a diagnostic.

Q1a (is knowing the decomposition required?). No. The agent-side method, HAD (Definition 11), uses only a partner model and the existing value oracle. The split matters for choosing a remedy on the partner's side, and there Proposition 10 says what behaviour alone gives: an upper bound on the information term and a lower bound on the decision term, both computable from the partner's action distributions and the oracle (Proposition 20 covers approximate cells from an estimated policy). The split is exact when the partner's observation channel is known by design, as in chess, where the partner sees the whole board, and in navigation (Section 5.2). Otherwise an intervention tells the two ends of the interval apart, since they respond to different remedies (Figure 1c). We have added this as a paragraph after Proposition 10 (Section 3.4).

Q1b (which learning algorithms). By term of the decomposition:

  • Where the team goes (the agent's side): any planner or reinforcement-learning method on the team process, which is an ordinary MDP once the partner is folded into the dynamics (Section 3.1); or, when retraining is not possible, a decision-time correction of an existing value oracle, as HAD does.
  • Partner model: skill-conditioned imitation of the partner's behaviour (Maia-2 in chess).
  • Decision term: a better rule on the same observations (the Bayes rule of Section 5.2), or training the partner.
  • Information term: changing what the partner observes (disclosure, interface design), targeted at the cells where the information term is largest. In navigation, on top of the Bayes rule, this closes 26% of the regret in all at k=0k=0, against 20% for random cells (Section 5.2).

If this answers the concerns about the tools and the static partner, we would appreciate it if you would reconsider your rating.

I thank the authors for their comments and keep the score.

@AC, while this is a 3 it is the upper side of 3, so more a 3.5.

Reviewer_115QOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies why a strong AI agent can perform poorly when working with a weaker partner, even if its individual decisions are optimal. It introduces handoff regret and decomposes team performance loss into the agent’s own errors, information unavailable to the partner, and mistakes made by the partner despite having the relevant information. Based on this framework, the paper proposes different remedies, including partner-aware planning, improved partner decision-making, and providing additional information. Experiments on tag-team chess and navigation show that accounting for the partner can be more effective than simply increasing search, while analysis of human chess games demonstrates the predictive value of handoff regret. Overall, the paper offers a useful theoretical framework for understanding and improving coordination in AI systems that share control with other agents or humans.

Critical strengths
  1. The paper provides an exact decomposition of team loss into the agent’s own error, missing information, and misused information, with corresponding remedies. This is useful because it turns the qualitative observation that individually strong agents can be poor teammates into a quantitative framework. The accompanying results further clarify what can and cannot be inferred from observed partner behaviour.
  2. The paper supports its framework with theoretical results, empirical validation across tag-team chess, navigation, and human chess games. Results provide evidence that accounting for the partner can be more valuable than additional search alone.
Critical weaknesses
  1. It is unclear whether the decomposition and proposed interventions generalize to realistic collaboration tasks. In particular, the reported chess results would benefit from more detail on statistical robustness across partners, positions, and evaluation settings. This limits confidence in the broader practical claims.
  2. The operational distinction between missing and misused information may be difficult to identify in practice. The paper acknowledges that partner behaviour alone identifies this distinction only up to an interval, and that additional information or interventions are required. While this is a valuable theoretical observation, it raises a practical concern about whether this framework can reliably decide which remedy to apply when the partner’s decision rule is not directly observable.
Questions for authors

Refer to weakness.

2 comments

Thank you for the review. The chess result is robust on each axis the review names: across halves of the openings, positions, partners, evaluation settings, and a second computing environment. Details follow, then the answer on identifying the split.

W1b (statistical robustness of the chess results). The design: 1,000 opening positions generated for this study and disjoint from every opening used in development; each opening played once with each colour under common random numbers, with the opening pair as the unit of analysis; 95% intervals over pairs; the primary rule, the comparisons and the success criterion fixed in a dated note before any game on these openings. The 800 primary pairs were run as two separate halves, so each half replicates the other. Win share of the first team against the second, in %:

Comparisonall 800 pairsopenings 0-399openings 400-799
HAD-4 vs reference team R56.9 ± 2.255.4 ± 3.258.4 ± 2.9
HAD-4 vs matched compute (head to head)57.2 ± 2.357.1 ± 3.257.3 ± 3.2
matched compute vs R50.7 ± 1.951.0 ± 2.850.5 ± 2.7

Positions. By tercile of the engine's evaluation of the opening, HAD-4 against matched compute scores 59.3 ± 3.9, 54.7 ± 4.0 and 57.7 ± 3.9; a paired sign-flip permutation test over opening pairs gives p < 0.001 overall (Table 4, Appendix D).

Partners. Against R, HAD-4 scores 54.5 ± 3.1 with Maia-2 juniors at 1100, 56.9 ± 2.2 at 1500 and 59.8 ± 3.1 at 1900 (Table 2, Appendix D). In new runs where the senior's model of either junior is deliberately wrong (wrong rating, or mixed with noise), HAD-4 still beats R in all five conditions, with 57.1 to 57.7% and every interval above 50%, against 58.6 ± 2.3 for the true model in the same runs (Table 6).

Evaluation settings. Under a stricter adjudication rule HAD-4 scores 59.4 ± 3.2. Under the coin-flip schedule, where a handoff follows a senior move only half the time, it scores 51.2 ± 2.9 in the submitted runs, 54.1 ± 2.9 in a rerun, and 52.6 ± 2.1 over both (Table 7).

A second environment. We reran all nine pre-registered comparisons (800 pairs each) on different machines and software builds. The pre-registered criterion holds again: HAD-4 beats matched compute head to head with 57.3 ± 2.3 (submitted: 57.2 ± 2.3), and matched compute still gains nothing against R (51.2 ± 2.1). Eight of the nine comparisons lie within their submitted intervals, and all nine intervals overlap (Table 5).

W1a (realistic collaboration tasks). The decomposition (Theorem 5) is an identity for any Markov decision process with a fixed partner; nothing in it is specific to games, and Appendix K of the revised PDF extends it to partners that adapt by a fixed finite-memory rule. The experiments cover three settings chosen to exercise its three terms: chess with a model of human play as the partner, navigation with a learned partner whose information we control, and recorded human games. There the estimate predicts real players' losing moves better than nine standard difficulty features (AUROC 0.896 against 0.871), and the same holds on games played after the partner model's training period (0.906 against 0.883, Table 13).

W2 (telling missing from misused information in practice). The framework says what can be decided at each level of knowledge about the partner.

  1. The agent-side remedy needs no split. HAD (Definition 11) uses only a partner model and a value oracle.
  2. Behaviour alone gives two-sided bounds. By Proposition 10 the information term is at most its value IβI_\beta on the partition induced by the partner's behaviour, so the decision term is at least Edρ−Iβ\mathbb E_d \rho - I_\beta. Both are computable from the partner's action distributions and the oracle, and Proposition 20 bounds the effect of approximate cells on the information floor when the policy is estimated.
  3. The split is exact when the partner's observation channel is known by design: in chess the partner sees the whole board, so the information term is essentially zero, and in navigation we set the channel (Section 5.2). Otherwise an intervention tells the two ends of the interval apart, since they respond to different remedies (Figure 1c).

That behaviour alone cannot go further is itself a result of the paper (Proposition 10). We have added this as a short paragraph after Proposition 10 (Section 3.4).

Presentation. To make Section 3 quicker to read, we have added a one-glance table at its start (each term of the decomposition, what it measures, which remedy moves it, where it is tested and what the test found) and a notation table at the start of the appendix. All changes are in blue.

If the robustness evidence and the clarified use of the split address your concerns, we would appreciate it if you would reconsider your rating.

Reviewer_115QOpenReview ↗

Acknowledgement

I have gone through the responses. I would like to retain to my scores.

Reviewer_xc4zOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies shared control between a strong agent and a fixed, weaker partner. It defines handoff regret and decomposes team loss relative to the solo optimum into the agent's own error, loss due to the partner's missing information, and loss due to its decision rule. It characterizes how information refinement affects the decomposition, establishes limits on identifying the components from behavior, and constructs a setting where a small sacrifice in solo performance yields a much larger team benefit. The paper also analyzes partner-aware corrections to an existing value oracle, including oracle error, partner-model error, and support truncation. Experiments cover tag-team chess, controlled navigation interventions, and prediction of human chess errors from recorded games.

Critical strengths
  1. The paper provides a coherent and carefully qualified account of handoff failures. The decomposition separates the agent's own errors from the partner's information and decision limitations while accounting for the states the team visits. The information-order and identifiability results clarify both what additional information can accomplish and why behavior alone cannot identify the appropriate remedy. The counterexamples concerning correction depth and penalty scaling further prevent overly strong interpretations of the analysis.

  2. The empirical controls support the central claim that partner awareness can matter beyond additional search. In tag-team chess, HAD-4 scores 57.2 ± 2.3% head to head against the engine with a matched node budget. Held-out openings, paired colors, and controls for opponent modeling and generic position sharpness make this result persuasive. The navigation interventions and human-game prediction study provide complementary evidence, with the latter using game-level splits, bootstrap intervals, and a separate deeper oracle to check the evaluation.

Critical weaknesses
  1. The methodological advance is primarily explanatory rather than algorithmic. The theoretical framework combines performance-difference accounting, the expected value of perfect information, information refinement, and perturbed-contraction analysis. This synthesis is useful, but the decoder with full partner support recovers the existing Expector rule, and HAD-4 does not outperform Expector in their direct comparison. Under coin-flip control, HAD-4 also shows no clear gain over the reference team, whereas Expector does. The paper appropriately acknowledges these results; they limit the contribution as an improved decision-making method and make the case for acceptance depend chiefly on the explanatory framework.

  2. The main team-performance experiments provide limited evidence of robustness to realistic partner-model mismatch. In chess, the agent models the same Maia-2 policy and rating configuration used to generate its partner's moves. Navigation likewise gives the agent an exact model of the learned partner. The random-MDP mismatch experiments are informative, but they do not establish whether the practical gains survive errors in modeling a chess partner, individual behavioral differences, or adaptation during interaction. The results therefore support the controlled fixed-partner setting more strongly than deployment with a human teammate whose behavior is only approximately known.

  3. The human-game analysis does not establish predictive generalization beyond the partner model's training data. The evaluated games are from July 2014, within the stated 2013–2023 training period of Maia-2. This does not establish that these exact games were used for training, but it leaves their independence from the partner model's training corpus unresolved. The downstream game-level split and deeper-oracle check address different concerns and do not remove this limitation. The predictive improvements are useful observational evidence, but their relevance to previously unseen games or improved human–AI team outcomes remains limited.

  4. The decomposition's practical advantage for choosing interventions is not clearly established. The navigation study evaluates the information and decision terms using known observation partitions, exact action values, and exact occupancies. Even in this favorable setting, choosing disclosure cells by the information term performs essentially as well as the simpler policy-entropy heuristic: at the 10% disclosure budget, the regrets are 0.0753 versus 0.0755 for the shortest observation window and 0.0638 versus 0.0627 for the longest. Together with the demonstrated identifiability limitation, this leaves the framework stronger at explaining intervention outcomes than at showing how its additional quantities enable better intervention choices in practice.

Questions for authors

See weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer xc4z (1/2)

Thank you for the review. We ran the two experiments it calls for: partner-model mismatch in chess (W2) and the human-game analysis on games after Maia-2's training period (W3). Both support the paper. W1 and W4 concern what the paper claims, and we answer them first.

W1 (explanatory rather than algorithmic). The paper claims no better decision rule than Expector's, and Section 5.1 says so; with full partner support and a two-step horizon the correction is Expector (Section 4), and head to head the two are level (50.7 ± 2.0%; 49.3 ± 1.9 in a rerun). Its claims are the decomposition, the identifiability result, and corrections of an existing oracle with explicit error terms. Three results belong to the account and not to the rule.

  • Why extra search does not help a team. The partner sees the whole board, so its loss is essentially all decision error at the states it is handed, and only the agent's choice of where the team goes changes those states. Matched compute gains nothing as a teammate (50.7 ± 1.9% against R); HAD-4 beats it head to head (57.2 ± 2.3%, pre-registered; 57.3 ± 2.3 in a full rerun on different machines, Table 5). The mechanism shows in the losses: under HAD-4 the junior's loss per move falls from 0.131 to 0.107, while the senior's own rises from 0.008 to 0.023.
  • Why the compatible agent is weaker alone. Hamade et al. (2024) found compatible agents weaker in solo play. Playing every move itself while keeping its team assumptions, HAD-4 scores 5.9% against the raw engine; told that no partner will move, it recovers to 35.7% (Section 5.1, "Alone"). In chess the weakness alone is mostly planning for a partner who is absent, not the solo-for-team trade of Theorem 8.
  • A shift that accuracy does not see. In navigation a longer memory window cuts the information term by 38% while the learned partner's decision term rises by 20%; shortest-path accuracy barely moves (75.4% to 77.2%).

The error accounting (Theorem 12, Proposition 21) also identifies support truncation as a design choice. In the primary runs the less truncated estimates score higher against R: paired with top-4 HAD-4, +2.0 ± 3.0 for the top 8, +3.2 ± 3.0 for Expector's renormalisation and +5.6 ± 3.0 for a support grown to 0.9 of the mass. Head to head that adaptive support is level with Expector (48.8 ± 2.0, Table 7).

Coin flip. A rerun of this schedule (400 pairs) gives HAD-4 54.1 ± 2.9 and Expector 53.9 ± 2.9 against R: both gain, and they are level. Over the submitted runs and the rerun, HAD-4 scores 52.6 ± 2.1 and Expector 55.0 ± 2.3 (Table 7; one sentence added to Section 5.1). Over both runs HAD-4 is above 50% under coin flip, by less than under strict alternation, where a handoff follows every senior move.

W4 (choosing interventions). The claim tested in navigation is that each remedy moves the term the decomposition assigns to it, and it holds: the Bayes rule lowers the decision term and leaves the information term unchanged, disclosure lowers the information term, and planning by the agent moves the occupancy (Figure 3b, Table 9). That is a statement about which kind of intervention acts on which part of the loss. A behavioural statistic cannot make it: by Proposition 10, two partners with identical behaviour, and hence identical policy entropy, can need different remedies (Figure 1c).

For ranking cells within one intervention, entropy does as well as the information term in this environment, as Section 5.2 reports; on top of the Bayes rule, both close 26% of the regret in all at k=0k=0, against 20% for random cells. For use without a known partition, the paragraph added after Proposition 10 (Section 3.4) states what behaviour alone gives: the upper bound IβI_\beta on the information term and the lower bound Edρ−Iβ\mathbb E_d \rho - I_\beta on the decision term, with the split exact when the observation channel is known by design; otherwise an intervention tells the two ends of the interval apart.

AuthorsOpenReview ↗

Response to Reviewer xc4z (2/2)

W2 (partner-model mismatch). New runs with the settings of Section 5.1 (juniors truly at 1500, strict alternation, the 800 held-out opening pairs) give the senior a wrong model of its own or the opposing junior:

Senior's model of the juniorswin share vs R (%) ± 95% intervalmean TV of the model
Maia-2 at the true rating (paper's configuration)58.6 ± 2.30
own junior modelled at 110057.1 ± 2.20.138
own junior modelled at 190057.4 ± 2.20.111
opposing junior modelled at 110057.5 ± 2.20.144
opposing junior modelled at 190057.7 ± 2.20.116
both models mixed with a uniform move at weight 0.357.2 ± 2.20.22

HAD-4 beats R under every wrong model, with every interval above 50%. Compared with the true-rating model in the same runs the win share is 0.9 to 1.5 points lower, and no single difference is distinguishable from zero (Table 6, Appendix D, gives each with its interval). TV is the mean total-variation distance between the assumed and the true move distribution. A rating error changes which moves the model ranks first; the uniform mixture keeps the ranking and flattens the probabilities. The true-rating row is the paper's configuration, run alongside the wrong models on different machines from the submission; the submitted runs give 56.9 ± 2.2 on the same openings.

On individual players, the closest evidence is Section 5.3: the Maia-2 estimate predicts real players' losses, now also after its training period (W3). On adaptation during play, Appendix K of the revised PDF shows that the decomposition covers a partner whose finite memory is updated by a fixed rule: on the state augmented with that memory it is a fixed policy, so Theorem 5 holds there (Proposition 23; Appendix K also gives the conditions under which Theorem 12 transfers). The experiments use fixed partners, and navigation gives the agent the learned partner as an exact model, as Section 3.1 and Appendix E state.

W3 (human games inside Maia-2's training period). We reran the pipeline unchanged (same bands, filters, features, split and bootstrap) on the January 2025 file of the same database, after Maia-2's training period: 1,050 games and 16,275 positions, with 7,681 test positions from 497 games. The result replicates. The estimate alone has AUROC 0.906 against 0.883 for the nine-feature model, and adding it raises the AUROC by 0.018 to 0.026 and the AUPRC by 0.033 to 0.064 (95% bootstrap intervals over games), as in the submitted analysis (0.018 to 0.025 and 0.025 to 0.058). The gain is positive in each rating band and under the deeper re-scoring (Table 13, Appendix F; one sentence added to Section 5.3). As Section 5.3 states, this is evidence about real players' losses, not about team outcomes.

If these results address W2 and W3, we would appreciate it if you would take them into account in your final assessment.

Comments on the paper

2 comments

General Response (1/2)

We thank the four reviewers. This comment answers the points raised by more than one of them; each reviewer thread answers the rest. The revised PDF is uploaded: every change is an addition, in blue, and no submitted text was removed. Table, proposition and appendix numbers in all responses refer to the revised PDF.

New evidence since submission. (a) All nine pre-registered chess comparisons rerun on different machines: the pre-registered criterion holds again (Table 5). (b) The chess gain under deliberately wrong partner and opponent models (Table 6). (c) The human-game result on games after Maia-2's training period (Table 13). (d) The decomposition for partners that adapt by a fixed finite-memory rule (Proposition 23, Appendix K).

1. The paper claims an account of partner-aware play and corrections with explicit error terms, not a better rule than Expector's, and the experiments test that account (Reviewer PtNY W1, Reviewer pKBL W1b, Reviewer xc4z W1). With full partner support and a two-step horizon the correction is Expector, and head to head the two are level (50.7 ± 2.0%); the paper says so. The account explains why extra search does not help a team while partner-aware choice does. In chess the partner's loss is decision error at the states it is handed, which only the agent's choice of where the team goes can change: matched compute gains nothing as a teammate (50.7 ± 1.9% against the reference team R), and HAD-4 beats it head to head (57.2 ± 2.3%, pre-registered). It also explains why the compatible agent is weaker alone: it keeps planning for an absent partner (5.9% alone, 35.7% once told that no partner will move).

2. The chess gain is not a compute effect, and it is robust: across halves and positions, in a second computing environment, and under wrong models of both juniors (Reviewer 115Q W1b, Reviewer PtNY W2, Reviewer PtNY Q1, Reviewer xc4z W2). The 800 pre-registered pairs were run as two halves: HAD-4 against matched compute scores 57.1 ± 3.2 and 57.3 ± 3.2, and every tercile of opening evaluation is above 50% (Table 4). A full rerun of the nine pre-registered comparisons on different machines and software builds gives 57.3 ± 2.3% for the criterion (submitted: 57.2 ± 2.3%), with eight of the nine comparisons inside their submitted intervals and all nine intervals overlapping (Table 5). With the senior's model of its own or the opposing junior set to the wrong Maia-2 rating, or mixed with uniform noise (mean total-variation error 0.111 to 0.22), HAD-4 still beats R in every condition on the 800 pairs: 57.1 to 57.7%, every interval above 50%, against 58.6 ± 2.3% for the true model in the same runs (Table 6).

3. The split between missing and misused information is usable without observing the partner's rule (Reviewer 115Q W2, Reviewer pKBL Q1a, Reviewer pKBL Q1b, Reviewer xc4z W4). The agent-side correction needs no split: HAD uses only a partner model and an oracle. Behaviour alone bounds both terms (Proposition 10), and the split is exact when the partner's observation channel is known by design; otherwise an intervention tells the two ends of the interval apart. The paragraph added after Proposition 10 (Section 3.4) states this recipe. That behaviour cannot identify more is a result of the paper: two partners with identical behaviour can need different remedies (Figure 1c). In navigation each intervention moves the term the decomposition assigns to it (Figure 3b). For ranking disclosure cells, the entropy of the partner's policy does as well as the information term in that environment, as Section 5.2 reports.

4. The decomposition covers partners that adapt by a fixed rule with finite memory (Reviewer pKBL W2a, Reviewer pKBL W2b, Reviewer xc4z W2). Such a partner is a fixed policy on the state augmented with its memory, and solo values do not depend on that memory, so Theorem 5, Corollary 6 and the refinement direction of Proposition 9 hold there (Proposition 23, Appendix K; numerical residuals below 1e-12 on 1,000 random instances). The experiments use fixed partners, as Section 3.1 states.

General Response (2/2)

Revisions:

ChangeWhere in the revised PDFFor
At-a-glance table of the decomposition; notation tablestart of Section 3; start of the appendixReviewer 115Q
Using the split in practiceSection 3.4, after Proposition 10Reviewer 115Q, Reviewer pKBL, Reviewer xc4z
Partners that adapt by a fixed rule with finite memorySection 3.1; Appendix K (Proposition 23)Reviewer pKBL, Reviewer xc4z
Rerun of the nine pre-registered comparisonsSection 5.1; Table 5 (Appendix D)Reviewer 115Q, Reviewer PtNY, Reviewer xc4z
Wrong partner and opponent modelsSection 5.1; Table 6 (Appendix D)Reviewer 115Q, Reviewer PtNY, Reviewer xc4z
Split halves and opening tercilesTable 4 (Appendix D)Reviewer 115Q
Adaptive support against Expector; coin-flip rerunSection 5.1; Table 7 (Appendix D)Reviewer PtNY, Reviewer pKBL, Reviewer xc4z
Human games after Maia-2's training periodSection 5.3; Table 13 (Appendix F)Reviewer xc4z

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.