- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies deception in four-agent chess collectives. It finds that asking agents to identify imposters can introduce suspicion, scapegoating, and extra reasoning costs, and proposes an external detector with silent removal instead. It also introduces GAMBIT, a benchmark of 37,352 labeled instances, and evaluates ANIL-based few-shot adaptation to unseen attack strategies.
Critical strengths
- The paper examines the costs of defense, including when no attacker is present. The benign and weak-but-honest agent conditions help expose a useful failure mode: agents can confuse poor reasoning with malicious intent. The accompanying transcripts make this problem concrete.
- The detector evaluation goes beyond zero-shot performance. ANIL and SFT have similar offline F1, but differ substantially after adaptation under the 1:5 query setting. This is a useful demonstration that offline accuracy alone can miss an important capability.
Critical weaknesses
- The experiments do not cleanly establish the central recommendation. The reporting prompt explicitly encourages agents to prefer moves supported by trusted analysts (App. C.2.9), which can itself induce majority following; App. D.1 acknowledges this. Meanwhile, silent removal uses a persistent imposter, whereas the reporting condition reselects the imposter each turn (§3). These differences make it hard to attribute the advantage to keeping detection outside deliberation. Awareness also helps under attack in Table 1, so the recommendation needs to account for that benefit alongside its benign-case cost.
- The adaptation result is much less robust than the headline suggests. Under the 1:75 full-log query setting, ANIL adaptation decreases macro ΔF1 by 0.134, versus a decrease of 0.033 for SFT (App. G.7). Prior correction may help, but it is not evaluated, and the live overseer experiments use the offline checkpoint. The paper therefore shows promising adaptation under a particular evaluation setting, but does not yet show that this improves deployed defense.
Questions for authors
-
Does silent removal still outperform reporting when both use the same imposter persistence, and the report provides detector scores without instructing agents which move to prefer? What is the benign-case cost of silent removal?
-
Does the proposed prior correction restore positive adaptation gains at 1:75? Please report the resulting precision, recall, and F1.
-
How much of the reported 14× training-time advantage remains on matched hardware? App. I.5 reports different GPU memory capacities for SFT and 3B ANIL.
4 comments
Thanks for addressing my issues, I have also checked the revised version of the paper. I still have two quick follow-up questions:
-
On the 1:75 query setting(Weakness 2): Low precision means many false alarms, which will kick out honest agents in real play. Since you proposed the prior correction formula in App G.7, why not just test it and give the actual precision, recall, and F1 numbers?
-
Regarding to weakness 1 on Ovs2, it seems like your new test on page27, Table 14 shows that Ovs2 drops to -171.5 CP if the imposter is not permanently kicked out. This means the gain comes from permanent expulsion, not from keeping it silent. What is the final CP score if you run Ovs2 in a fully benign game (suppose we do not count the imposter here)?
Thank you for following up.
On the 1:75 query setting(Weakness 2): Low precision means many false alarms, which will kick out honest agents in real play. Since you proposed the prior correction formula in App G.7, why not just test it and give the actual precision, recall, and F1 numbers?
The ANIL detector as deployed in the main body (Section 4, "External detection") already operates at a very low false-flag rate: 0.03% of honest agent-turns in benign games and 0.08% in attacked games (App. D.5, Table 11), and no honest agent was flagged while the imposter was present (App. D.8). At this operating point, silent removal raises the collective's median final CP from −284.5 to +262 (Table 1), so its threshold is acceptable for practical applications.
We ran the prior correction proposed in App. G.7, in the full-log setting (62 strategies × 5 seeds, using each query's true base rate). The results are negative:
| ANIL, full log | Precision | Recall | F1 |
|---|---|---|---|
| Before recalibration | 0.819 | 0.480 | 0.548 |
| Recalibrated | 0.362 | 0.693 | 0.413 |
| + prior correction (App. G.7) | 0.746 | 0.300 | 0.395 |
The correction removes most false alarms, but over-corrects: recall falls below that of the detector before recalibration. Looking closer, we realize that the advice in App. G.7 was poorly formulated: the formula takes the balanced support set as the reference prior, whereas ANIL meta-trains the head to perform on 1:5 query sets after the update, so the shift it applies is too large.
This does not weaken our core claims. The detector of the live games is not recalibrated, so their results stand as reported, and the adaptation result of Section 5 is measured in the regime the detector is meta-trained for. ANIL remains a valid and robust method as described in the main body. We corrected the advice of App. G.7 accordingly in the revised paper.
Regarding to weakness 1 on Ovs2, it seems like your new test on page27, Table 14 shows that Ovs2 drops to -171.5 CP if the imposter is not permanently kicked out. This means the gain comes from permanent expulsion, not from keeping it silent. What is the final CP score if you run Ovs2 in a fully benign game (suppose we do not count the imposter here)?
Thank you for this question. You are right that the gain under attack comes from removing the imposter for the rest of the game, which App. D.9 states. Silence matters for the benign case: until the detector flags someone, Ovs$_2$ shows the agents exactly the same prompts as Col, so a benign Ovs$_2$ game is a Col game unless an honest agent is wrongly flagged. On the 20 benign Col + Ovs$_1$ games, the same detector flagged an honest agent once (at the collective's 2nd decision of 20, probability 0.52). A benign Ovs$_2$ collective would therefore play exactly as Col (median +198 CP, Table 1) in 19 of these 20 games (since Ovs$_2$ uses the same detector as Ovs$_1$. The resulting CP score is +153, close to the Col +198 CP baseline. Ovs$_2$ therefore adds a marginal performance loss on benign games.
Thank you again for acknowledging that our previous answers addressed your issues. As these follow-up questions are now answered as well, and the paper was revised accordingly, would you consider raising your score from 2 to 3?
Thanks for the replies, those new comments have addressed my concerns now, I will raise my socre.
We thank the reviewer for their constructive and detailed feedback. We performed additional experiments and revised the paper accordingly (changes in blue). Please find the answers to the weaknesses and questions below.
We ran the same experiment without that instruction (to prefer moves supported by trusted analysts), and it didn't change anything to the conclusions.
We follow the reviewer's suggestion: we ran Ovs$_2$ again (silent removal), but put the imposter back at every turn. This makes it less realistic, but better comparable to the Ovs$_1$ experiment (reporting instead of silent removal). We find that the silent removal per turn is worse (in median CP), but without the additional token budget of Ovs$_1$. We conclude that expelling the agent from the game is what brings a true advantage in Ovs$_2$, which is still a valid and useful result. These results were added to the updated paper, in App. D.9 (Table 14, page 27; the new setting is indicated as 'without trust instruction').
Regarding the benefits of the awareness experiments: we wanted to stress the fact that awareness damages the overall performance, as we have to expect most collective experiments do not contain an imposter, and a benevolent collective considerably suffers from the awareness instructions. Awareness indeed helps under attack, but only when an imposter (or dummy agent) is present. In general (i.e., also in the benevolent case) awareness considerably rises the token budget. Silent removal from the first detection onwards (Ovs$_2$) does better than awareness under attack and changes nothing in benign settings, so it dominates awareness whatever the attack probability. Therefore, our central recommendation (added in a new paragraph in the Conclusion) is: We do not recommend to use awareness instructions, given the extra cost in benign settings and since silent removal after external detection is at least as effective under attack, without this cost.
Thank you for this remark. A closer look shows that the two ΔF1 values do not measure the same thing. F1 is not invariant to the class mix: in the full-log setting, each query holds only a few imposter instances but every honest instance of the test set. Even a very small share of false alarms on honest instances then outnumbers the imposters that are caught, so precision, and with it F1, collapses. F1 in this setting therefore reflects how often the detector raises a false alarm, which depends on its decision threshold, rather than how well it recognizes the imposters. The two ΔF1 values thus measure different things. Recall, which is independent of the mix, rises after recalibration (0.478 → 0.693): the detector does learn the new attack in both settings. SFT loses less only because it barely adapts (recall 0.476 → 0.497). Therefore, our results about ANIL remain robust. We made it clear in our revised App. G.7.
Yes: the cumulative report is that condition: +46.5 CP against +262 for silent removal. Benign-case cost: prompts are unchanged without a flag; the detector flags only 0.03% of honest agent-turns in benign games (App. D.5), and none while the imposter was present (App. D.8).
Because we established that the problem at 1:75 was due to a misinterpretation of the App. G.7 results rather than a robustness problem of our ANIL method, this correction was no longer relevant to apply.
That is correct, but measured in training passes for hardware-independent comparisons, ANIL training is still considerably more efficient than SFT. Specifically, SFT needs 3.8× more passes than ANIL to reach its best checkpoint (45,792 vs. 12,000) and 2.7× more in total (114,480 vs. 42,000). We report these counts in App. G.9.3.
We thank the reviewer for acknowledging the strengths of our paper, and hope these additional insights take away any hesitation to support the publication of our work.