- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces MineAmongUs, a 3D version of Among Us built in Minecraft, where VLM agents engage in social deduction through visual perception, navigation, and dialogue. It investigates how agents deceive other players through behaviors such as faking tasks, avoiding witnesses, and fabricating alibis.
The authors also develop ARIA, a modular agent harness with configurable components such as memory, planning, and reflection, along with a taxonomy of 23 verbal and nonverbal deceptive behaviors. Through 192 harness ablation games and 1,568 cross-model games, the paper examines how different models and agent configurations affect win rates and deception strategies.
Critical strengths
- Compared with text-based AmongAgents and the Minecraft multi-agent benchmark SoMi-ToM, this work combines hidden identities, spatial actions, and social interaction in a single environment. This introduces meaningful evaluation dimensions beyond simply changing the game setting.
- The taxonomy of 23 deceptive behaviors provides more insight than game win rates alone, distinguishing verbal deception, behavioral disguise, and strategies surrounding kills. Human and LLM annotations also show reasonably good agreement.
- The experiments cover 14 VLMs and 1,568 cross-model games, along with component ablations and behavioral analyses. The benchmark has value as an experimental platform for studying deception in multi-agent systems.
Critical weaknesses
- It is difficult to separate model-driven deception from behavior built into the harness. Witness checks and scripted actions already implement some reported strategies. Although the minimal module templates remove tactical advice, the shared planner still suggests fake tasks and self-reporting. Using private action narration for annotation adds to this concern.
- Winning does not necessarily mean deceiving others successfully. Only 7 of the 105 crewmate wins in Table 4 come from voting. The reported associations may therefore reflect general gameplay ability or killing proficiency. Direct evidence that deceptive behavior changes another player’s beliefs or decisions is missing.
- Cross-model comparisons also depend on different inference settings and potentially different fallback rates. These factors make it hard to attribute the results to the models themselves.
- The No RGB ablation supports a role for vision, but the main experiments still rely on privileged spatial tracking. The conclusions should be limited to this setting.
Questions for authors
- Which behaviors come from model decisions, and which come from fixed scripts or tactical prompts?
- Can you report fallback rates and control inference settings in key comparisons?
- Can you show whether observing a deceptive action changes another player’s suspicion or vote?
4 comments
Response to reviewer oKT8 (2/3)
(Q3-oKT8) Different inference settings and fallback rates
TL;DR. We comprehensively audited fallback rates and conducted additional experiments with controlled inference (temperature) settings. The results show that fallbacks are generally infrequent and our main findings remain largely consistent across the tested inference settings.
To address this concern, we conducted (1) a comprehensive audit of recorded fallbacks across all 1,568 RQ2 games and (2) an additional 200-game experiment controlling inference settings.
(1) Fallback Rate Analysis
We examined fallback mechanisms throughout ARIA, covering 14 decision-stage fallback sites (including planning, kill-related checks, reporting, movement, mission selection, surveillance, meetings, voting, memory, and reflection) and one action-generation fallback site (e.g., MOVE/MISSION code generation). Across all 1,568 games, fallback rates were 0.66% at the decision stage and 0.50% at the code-generation stage, as follows:
| Category | Fallback behavior | Rate |
|---|---|---|
| KILL | Defer the kill | 0.40% (52/12,927) |
| REPORT | Report the nearest body | 0.48% (32/6,618) |
| MOVE | Random movement | 0.55% (286/51,927) |
| MISSION | Select the first remaining mission | 0.88% (824/93,316) |
| Code generation | Use a predefined movement program | 0.50% (726/144,516) |
Fallbacks were generally infrequent but varied across models. 10 of the 14 VLMs exhibited decision-stage fallback rates below 0.3% in both roles. Higher rates were concentrated in Gemma4-31B (4.1-4.2%), Qwen3.5-27B (1.6-3.1%), and Kimi-K2.5 (1.4-2.1%). These results indicate that recorded fallbacks are uncommon overall, although their model-dependent frequency remains a consideration when interpreting cross-model differences.
(2) Controlling Inference Settings
For reasoning effort, we used ’None’ where supported, or the minimum available reasoning level otherwise, since supported options differ across model APIs.
For temperature settings, some models require a fixed temperature of 1.0 (i.e., GPT-5-mini, GPT-5, and GPT-6Astra) and others evaluated at 0.1 or 0.7. To examine the sensitivity to temperature, we reran the RQ2 evaluation for the 5 models originally evaluated at temperature 0.7 (Qwen3.5-9B, Qwen3.5-27B, Qwen3.6-27B, Gemma4-26B-A4B, and Gemma4-31B), changing their temperature to 0.1. We evaluated all 5 models against one another across 4 cases and 2 repetitions, yielding 200 additional games, and compared them with the corresponding 200 games from the original RQ2 evaluation.
Game outcomes. The overall imposter WR remained similar, at 54.5% under temperature 0.1 versus 53.5% in the original experiments. Importantly, the model ranking remained highly consistent, with the top-3 models retaining the same ordering.
| Case | Temperature 0.1 | Original | Difference |
|---|---|---|---|
| Overall | 109/200 (54.5%) | 107/200 (53.5%) | +1.0 pp |
| 1A | 30/50 (60%) | 25/50 (50%) | +10 pp |
| 1B | 20/50 (40%) | 20/50 (40%) | 0 pp |
| 2A | 27/50 (54%) | 31/50 (62%) | -8 pp |
| 2B | 32/50 (64%) | 31/50 (62%) | +2 pp |
Deceptive behavior patterns. We additionally applied our deception-atom judge to all 200 new games and compared the results with the original games. The average number of observed atoms per game remained similar (54.77 vs. 55.95), with a difference of −1.18 (95% CI: -5.38 to +3.02). Both verbal and non-verbal atom frequencies were also comparable. Furthermore, the distribution of the 23 deceptive atoms remained highly consistent (Pearson r = 0.998) when comparing their aggregate per-game frequencies across the two settings.
| Atoms per game | Temperature 0.1 | Original | Difference (95% CI) |
|---|---|---|---|
| All atoms | 54.77 | 55.95 | -1.18 (-5.38, +3.02) |
| Non-verbal | 33.31 | 33.97 | -0.66 (-3.82, +2.51) |
| Verbal | 21.45 | 21.98 | -0.53 (-3.12, +2.07) |
To summarize, these analyses provide additional evidence for the robustness of our cross-model findings. The fallback audit shows that recorded fallback events are relatively rare overall, while identifying a small number of models with higher rates. Furthermore, the controlled temperature experiment reproduces similar aggregate WR, model rankings, and deception-atom distributions.
Response to reviewer oKT8 (3/3)
(Q4-oKT8) Scope of privileged spatial tracking
We agree that our findings should be interpreted within the privileged-state setting, and we have already explicitly acknowledged this scope in both the main paper and the Limitations section.
Existing ablation evidence.
In §4.1 and Appendix G.1-G.7, we systematically investigate why privileged spatial tracking is necessary for sustained gameplay. Beyond the No-RGB ablation, we compare 4 state representations: privileged, priv36 (local persistent tracking), No-RGB, and ego. The results reveal two distinct bottlenecks: (1) removing persistent spatial tracking prevents agents from maintaining targets, while (2) removing RGB perception collapses kill execution even when full-map coordinates remain available.
Why privileged state is necessary.
As explained in Appendix G.5-G.7, privileged state provides persistent spatial localization but does not replace visual perception. RGB observations remain necessary for witness detection, target verification, dead body discovery, and crewmate surveillance. Our choice reflects current VLM limitations in egocentric spatial perception and temporal tracking (Du et al., 2024; Yang et al., 2025a,b; Ravi et al., 2025; Ahn et al., 2025; Zhang et al., 2026). Providing structured spatial information alongside visual observations is also an established practice in embodied AI research (Savva et al., 2019; Szot et al., 2021; Puig et al., 2024; Chang et al., 2025).
Scope of our conclusion.
We already state in §4.1 why the remaining experiments adopt a privileged state, discuss this choice in Appendix G.7, and explicitly acknowledge it in the Limitations section. Our conclusions therefore concern embodied deception under a spatial-tracking scaffold, not fully egocentric autonomous gameplay. We regard overcoming this limitation as an important direction for future VLM-agent research.
References
[1] Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, ICLR 2023
Kind Reminder to reviewer oKT8
Dear Reviewer,
Thank you very much for the time and effort you have dedicated to reviewing our paper. As the discussion period is drawing to a close, we kindly remind you that there is less than one day remaining.
We have carefully addressed the concerns raised in the review, and we would greatly appreciate it if you could kindly reconsider your scores or let us know if there are any remaining points that need further clarification. We are happy to provide additional materials or experimental results if needed.
Thank you again for your time and consideration.
Best regards,
The Authors
Response to reviewer oKT8 (1/3)
We greatly appreciate the reviewer's thoughtful feedback on our paper.
Response to Weaknesses and Questions
(Q1-oKT8) Separate model-driven deception from harness-driven deception
Why we provide tactical scaffolding.
Our motivation was to enable VLM agents to participate in sustained, human-like gameplay. In our initial development, agents struggled to perform coherent gameplay and spontaneously develop deception strategies, particularly in the non-verbal channel. We therefore introduced tactical scaffolding to support these behaviors and systematically study how model capabilities (RQ2) and harness configurations (RQ1) shape deception. Accordingly, our main experiments characterize deception exhibited by harnessed VLM agents, rather than purely model-emergent deception. Besides, private action narration is primarily used to help the LLM judge identify non-verbal behaviors that are difficult to infer from gameplay logs alone, rather than as a fixed script or tactical prompt prescribing those behaviors.
Additional experiment: minimal-harness ReAct agents.
To investigate which behaviors models can exhibit without ARIA's scaffolding, we conducted an additional experiment using GPT-6-Astra and Muse-Spark-1.3, with 3 games per model (6 games total). Specifically, we removed ARIA's tactical modules and shared planner, retaining only a basic ReAct [1] loop and the original role instructions. Note that agents received egocentric visual observations without any privileged information.
Results.
Even without scaffolding, the models exhibited 6 of the 11 non-verbal deception atoms across 6 games. GPT-6-Astra exhibited 4 types (A, C, E, F), while Muse-Spark-1.3 exhibited 6 (A, B, C, E, F, G).
(Counts are non-verbal atom instances across 3 games per model.)
The gameplay logs provide further evidence of model-directed non-verbal deceptive behaviors without explicit tactical instructions. For example, agents moved away immediately after all 18/18 kills, and explicitly considered the presence of other players in 16/18 kill-related thoughts.
However, the resulting gameplay was considerably limited. None of the 6 games reached a meeting phase, preventing meaningful evaluation of verbal deception. The models also exhibited only a subset of the non-verbal taxonomy, with no observed joint motor coordination or reporting/meeting-related strategies. Thus, while some model-driven deceptive behaviors arise, the agents still struggle to sustain the full social-deduction gameplay needed for more elaborate deception.
In summary, these findings clarify the distinction between model-driven and harness-driven deception. The full ARIA harness explicitly supports certain strategies, while the minimal-harness experiments show that some non-verbal behaviors can also emerge from model-directed gameplay.
(Q2-oKT8) Winning (general gameplay ability) vs. deceiving others successfully
Please refer to our response to ‘(Q1-general) Gameplay competence vs. Deception effectiveness’.