acceptodds
Under review as a conference paper at ICLR 2027

Coach-Guided Autonomous Reflection Improves Deception and Detection in Multi‑Agent Werewolf Games

Abstract

Previous studies have demonstrated that large language models (LLMs) are capable of generating deceptive statements in interactive settings, whereas their ability to detect deception and improve these capabilities through experience remains comparatively under-explored. We present an autonomous reflection framework for multi-agent Werewolf games that enables LLM agents to extract, organize, and reuse strategic skills for both deception generation and detection. The framework introduces a Coach Agent that automatically extracts reusable skills from gameplay trajectories, removes redundant or low-quality strategies, and dynamically injects curated skills into subsequent games. To assess strategic competence beyond game outcomes, we further propose a dual-path game intelligence evaluation framework that combines Monte Carlo simulation with LLM-based judgment. Empirical evaluations across baseline games demonstrate a balanced win rate with 78.3% deception detection accuracy. Coach-guided skill learning improves the win rate of the enhanced faction to 65–66%, while maintaining balanced competition when both factions are enhanced and yielding consistently higher game intelligence scores. Cross-model experiments reveal that while detection enhancement transfers robustly across various model architectures, deception production skills show model-dependent performance. Overall, the proposed framework complements win-rate-based evaluation and establishes a practical paradigm for continual strategic learning and game intelligence evaluation in complex multi-agent environments.

Primary area:
reinforcement learning
Keywords:
Large Language Models, LLM Agents, Multi-Agent Systems, Reinforcement Learning, Experience-Driven Learning, Strategic Reasoning, Deception, Multi-Agent Games

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_Xv5uOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors study how to improve the agent's ability of deception generation and detection in adversarial natural language-based interaction. They use the multi-agent Werewolf game as the benchmark and propose a coach aegnt to extracts reusable skill from historical trajectories. Then they create a rule-based Monte Carlo simulator to evalute the performance of individual actions and use LLM to analyze the game trace qualitatively. Based on these evaluation, the agent dynamically inject better skills into future games. Experiments indicate that the refined skills repository can improve the performance of both factions.

Critical strengths
  1. This paper proposes a framework that extracts deception generation and detection skills from multi-agent interaction trajectories and refines these skills without updating model parameters. The Monte Carlo simulator omits LLM calls and thus the evaluation is lightweight and practical.

  2. This paper conducts substantial experiments and provides details about the simulator design.

Critical weaknesses
  1. Given the skill repository and evaluation, how to refine the skill usage is unclear. The authors mainly describe the details regarding extracting, deduplicating, and filtering skills. This makes the claimed continual strategic learning vague.

  2. The Monte Carlo simulator uses predefined strategies rather than LLM calls for rollouts. However, the natural-language interactions and LLM decisions are critical to deception and reasoning in Werewolf. Although some experiments show that the simulator achieves reasonable win-rate prediction accuracy, this does not necessarily mean that it can accurately evaluate the contribution of individual actions. Moreover, its prediction performance is sensitive to the strategies being used in simulation.

Questions for authors
  1. How is the skill selection refined based on the designed dual-path evalution? The authors mention skill usage frequency and faction-aligned outcomes. How do these metrics affect the update of skill repository and skill choice?

  2. How reliable is the Monte Carlo simulator for evaluation? The experiments validate its ability to predict game outcomes from intermediate states. However, this is different from evaluating whether an action is strategically good. Could the authors provide stronger evidence that the simulator can reflect actual strategic advantages? For example, you may compare with LLM-based counterfactual rollouts.

  3. Does the BalancedStrategy calibration affect the learning of skills? Specifically, the BalancedStrategy used in simulator is tuned to produce 50/50 faction win rates. If the simulator scores are used for skill selection or refinement, could this introduce a bias toward skill refinement and potentially contribute to the near-50/50 win rates when both factions use refined skills in Table 3?

Reviewer_pvR4OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

1: An agent framework for werewolf.

Critical strengths

No strengths

Critical weaknesses
  1. I recommend a desk rejection for a submission of such low quality.

  2. The authors claim to introduce a new Werewolf benchmark, yet the paper never explains how the benchmark was constructed, how calibration was performed, or how the test data were selected. It also includes no ablation study showing that the test data did not appear in the pretraining data.

  3. The paper’s literature review is insufficient. The authors omit a substantial body of recent and closely related work, and the paper cites only 24 references in total.

  4. I do not intend to spend more time on a further review of this manuscript. If the AC needs me to provide detailed weaknesses, please leave a comment.

Questions for authors
  1. Did you really read this "paper" after claude code finished???

Comments on the paper

1 comment
AuthorsOpenReview ↗

ablation experiment

We thank the reviewer for the detailed and constructive comments. We address some points below. Because DeepSeek removed the v4.0-flash endpoint, we re-ran the main experiments with v4.1-flash and use it for ablations below; the skill repository was likewise re-trained from 100 games under the new model. The results are almost consistent with those reported in the original submission: no-skill good-faction win rate 48.0% (48/100), detection accuracy 76.2%; good-only skills 66.0% (33/50), detection accuracy 84.4%.

1. Seed, role assignment, and opponent control.

We confirm that DeepSeek’s API does not expose a seed parameter and that we do not claim token-level reproducibility. To address the reviewer’s concern about experimental separability, we clarify the following. (i) Evaluation games are disjoint from the 100 games used to train the skill repository; the repository is frozen before any evaluation game is generated, and no evaluation trajectory enters skill extraction. (ii) Role assignments are randomized independently per game under a per-game random seed and roles are systematically rotated across conditions. (iii) Opponent agents use the same model version, prompt, and decoding settings across all conditions; the only difference between conditions is whether the target faction loads the skill prompt.

The skill repository is trained exclusively on a 100-game learning pool and frozen before evaluation. No evaluation game is used for skill selection, early stopping, or threshold tuning. Win rates with Wilson 95% confidence intervals are: no-skill 48.0% [38.4, 57.8]; good-only skills 66.0% [52.2, 77.6]. The between-condition difference is +18.0 pp [bootstrap 95% CI: +3.1, +32.9].

2. Ablations of Coach components.

We conducted ablations for de-duplication and review under the same full 100 logs, the same seed set (1–50), and a matched prompt width of 340 characters:

Condition Good-faction win rate Detection accuracy

Full skill repository 66.0% (33/50) [52.2, 77.6] 79.6%

Length-matched neutral placebo 42.9% (21/49) [29.8, 57.1] 74.4%

No review 64.0% (32/50) [50.1, 75.9] 83.9%

No de-duplication 50.0% (25/50) [36.6, 63.4] 74.6%

The placebo condition is length-matched to the skill prompt and serves as the equal-length strategy prompt control. The full repository outperforms the placebo by +23.1 pp in win rate and +5.2 pp in detection accuracy, indicating that the improvement is not merely due to prompt length. Removing review or de-duplication reduces both win rate and detection accuracy relative to the full repository, indicating that both components contribute. We did not ablate skill extraction and retrieval separately because they are the core pipeline steps.

3. Appendix

Appendix A.2

The earlier appendix version incorrectly duplicated the event in the subsequent discussion. We have replaced the example with the corrected log excerpt and added a note clarifying the resolution order.

Guard role and six-role configuration.

The Guard role was a configurable option in an earlier code version but was not active in any reported experiment. The main experimental configuration contains five roles throughout. We have removed the Guard-related content from the appendix so that it describes only the configurations actually used.

We acknowledge that there are issues with the description of AUC experiments in the appendix, and continuous learning stage performance curve of coach skills would be a valuable addition. Due to the limited rebuttal period, we are unable to complete it at this time, but we have noted it as a priority for the revision.

We thank the reviewer again for pointing out these issues.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.