- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper studies multi-agent collusion detection through internal model activations. It combines per-agent activation probes with group-level detection statistics and introduces NARCBench to evaluate generalization across committee deliberations, adversarial workflows, and steganographic communication. Results show strong transfer across instructed-collusion environments, while transfer to emergent collusion remains partial.
Critical strengths
The evaluation provides compelling evidence that internal activations offer a complementary monitoring signal for covert coordination. In particular, transferring detectors from committee deliberations to steganographic blackjack achieves near-perfect AUROC with the best method on each model. Testing structurally different tasks and emergent collusion also helps establish the practical boundaries of the approach.
Critical weaknesses
Cross-model portability remains insufficiently established. The main experiments train a separate probe on each model’s activations, so their zero-shot results primarily demonstrate cross-environment generalization. They do not establish that a detector trained on Qwen can be directly applied to Llama. Although Appendix N reports an exploratory cross-model experiment, the very limited positive samples and insufficient explanation of representation alignment make its conclusions difficult to assess. This leaves applicability to changing or heterogeneous agent systems uncertain. Clarifying the cross-model implementation and evaluating frozen-detector transfer against model-specific retraining would strengthen the paper.
Questions for authors
How is the cross-model performance?
3 comments
Thanks for the authors' response. The new results help a lot in improving the soundness of this paper. I will raise the soundness score and keep the overall one.
New Rebuttal Version
We have posted a revised version of our paper for the rebuttal phase. We hope it addresses your concerns.
In addition to addressing the concerns raised/commitments we made for specific revisions, it substantially strengthens and provides further detail on the theoretical and statistical foundations of our approach and setting. It expands our hypothesis-testing framework into a full environmental setting we call HELIOS, in the main body and Appendix A.
We hope you will take a look, and if it addresses your concerns, we also hope that you will consider revising your score appropriately.
Additionally, we previously posted a reply to reviewer N7D6 on the relevance of our approach and detection procedure that you may find useful. We hope you will take a look at it as well. https://openreview.net/forum?id=zNolMfwGC7¬eId=1JESCAd0bE
Thank you for your help in improving the paper!
Response to reviewer nAE8
We thank the reviewer for their time and effort in reviewing our work and the acknowledgement of our paper’s strengths, including the strong performance of our probing methods on the out-of-distribution steganography dataset. We also appreciate the constructive feedback provided and welcome the opportunity to provide clarification and new experimental results.
As the reviewer notes, the cross-model experiment was an exploratory experiment in our Appendices and we do not consider its result within our core contributions. The experiment in Appendix N shows poor probe performance, however, it is uncertain whether this is due to the limited positive samples, the generalisation to emergent collusion or if it is due to the cross-model set-up.
Even though more thorough experiments on cross-model probing is left to future work, we provide below some further results where we use probes trained on the CORE data generated by Qwen-32B and test them on TRANSFER and STEGO data generated by the other three models. To account for the dimension size variations in the different models’ activation spaces, we perform dimension truncation. Most probe methods perform poorly, however cross model performance on the steganography dataset is still high when transferred to the Llama-70b model. These results provide support for further exploration of cross-model performance.
Qwen-32B to OSS-20B
Qwen-32B to DSR1-distilled-32B
Qwen-32B to Llama-70B