acceptodds
Under review as a conference paper at ICLR 2027

Fake Player: Simulating Player Behavior to Distill Dialogues for LLM-based NPC Training

14 views

Abstract

In the era of large language models (LLMs), games increasingly deploy LLM-based role-playing NPCs to replace traditional scripted NPCs, enabling more intelligent and dynamic interactions. To ensure persona consistency and output stability, these NPCs require fine-tuning for alignment, utilizing training data with dual dimensions: persona-aligned assistant responses and diverse, authentic user inputs reflecting real player behaviors. However, existing research prioritizes persona consistency in NPC responses while neglecting the diversity and authenticity of user-side inputs. This critical gap leads to NPC responses that are misaligned with genuine player interactions, significantly impairing player immersion and experience. Human annotation struggles to address this gap due to its inability to comprehensively cover the vast spectrum of player behaviors. Moreover, practical deployment constraints strongly favor small-parameter LLMs for NPCs, making data quality paramount. To bridge this gap, we propose: 1) Fake Player: A multi-agent LLM distillation framework where collaborative agents simulate expression-constrained human players to distill diverse, human-aligned dialogue data from large LLMs; 2) Distill Bench: A standardized benchmark for quantitatively assessing distilled data quality, bypassing costly NPC retraining. Extensive experiments on upstream data quality and downstream NPC performance demonstrate that Fake Player produces higher-quality dialogue data and yields greater improvements in small-parameter NPC models than existing data synthesis methods. The consistent ranking across the two evaluation stages supports using Distill Bench to guide data selection and refinement before fine-tuning, offering a practical way to reduce costly trial-and-error training.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
LLM, Role-Playing NPCs, Data Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vPXaOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes Fake Player, a three-agent framework consisting of an Inner Monologue Agent, an Intent Analysis Agent, and a Typing Agent that jointly simulate human players to generate NPC training dialogues from strong LLM then SFT a small model.

Critical strengths

By simulating the player with an Inner Monologue Agent, an Intent Analysis Agent, and a Typing Agent, the framework produces more human-like and diverse player-side utterances than the ICL and RPA baselines.

Critical weaknesses
  1. NPC dialogue optimization is a highly vertical application domain, and the proposed solution is overly simplistic. Thus, this work is not well-suited for submission to ICLR.

  2. As shown in Figure 1, NPC dialogue needs to handle a wide variety of unexpected player inputs, and every response must account for multi-level information spanning the character, scenario, and game context. The data-generation harness is therefore crucial. Yet the paper addresses this with only the Inner Monologue / Intent Analysis / Typing agent pipeline, which is too simple to resolve the core challenges of NPC dialogue.

Questions for authors

N/A

1 comment

Thank you for the review. We respectfully disagree that the application domain and simplicity of the method are sufficient reasons to dismiss its contribution.

Our work addresses a player-side training-distribution mismatch that has received limited attention in NPC training. LLM-generated user inputs tend to be formal, verbose, and contain several intentions, while real users often send short, colloquial messages and switch intentions during conversation. This mismatch affects the trained model’s ability to understand and respond to users. Games provide our application setting, but the same problem is relevant to conversational and personal agents.

Character, scenario, and game context are included in the interaction setup. Fake Player improves how players express their intentions within that context, generating more realistic and diverse inputs for NPC training. Its simplicity does not diminish the value of addressing this problem. With the same Qwen2.5-7B backbone, Fake Player achieves a downstream score of 73.10, compared with 67.00 for RPA and 53.29 for ICL.

The method has also been deployed in a commercial game serving over one million players. Post-deployment questionnaires showed a 15% increase in satisfaction with NPC interactions. We believe these experimental and deployment results provide concrete grounds for assessing the contribution of a simple, effective approach.

Reviewer_e9gBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explores how single-player single-NPC actions could be made more dynamic and accurate. Authors cite previous gap where NPC responses are prioritized to be consistent, but do not account for the variability in user-side input. To address, they create FakePlayer, a distillation framework where LLM systems are used to interact with NPC agents and create conversation threads. They also introduce distilbench, a benchmark with 3 metrics (topic relevence, data diversity, human-ness) to evaluate NPC agents' synthetic training data. Authors find that generating data for distillation with FakePlayer creates better fine-tuned NPC models under their framework compared to exisiting approaches like RPA, ICL.

Critical strengths
  1. This paper is extremely timely, especially with distillation used to train LLM NPCs. Authors provide a good upstream method to create proper distillation data with FakePlayer and also establish its downstream usefulness on fine-tuning a real LLM for NPC use.
  2. Good use of baselines from existing literature to compare the method against. Authors do good job in showing (Fig 2) what the difference is between Fake Player and ICL/RPA common methods.
Critical weaknesses
  1. I have one main weakness. This paper's benchmark distillbench relies quite heavily on LLM-as-judge for 2 metrics. One of these metrics is human-ness. I hence would expect some direct agreement metric between manual human annotators and judges in either main text or appendix, instead of rankings. Since the full evaluation of distillbench and also the evaluation of NPCs fine-tuned are built on this, I would expect an even comparison to know that the prompts chosen for the NPCs demonstrate the same instance-level agreement as humans. This is my understanding after looking through main paper and appendix, and I encourage the authors to debrief me if my understanding is incorrect.
Questions for authors
  1. What component contributes to FakePlayer's better performance? is it the intent agent, the messaging agent, or the inner monologue? Curious on what authors' justifications are for this architecture.
2 comments

Thank you for the positive assessment and for suggesting an instance-level comparison. Three annotators evaluated 150 chit-chat dialogues, with 50 from each method. We compared their mean Human-Likeness ratings with the existing LLM scores for the same dialogues. The scores showed a Spearman correlation of 0.703, with a 95% bootstrap confidence interval of [0.655, 0.749], providing an instance-level comparison alongside the method rankings reported in the paper.

Regarding the architecture, the three agents model different parts of player behavior. The Inner Monologue Agent generates what the player wants to express. The Intent Analysis Agent organizes these thoughts and manages intent switching across turns. The Typing Agent converts the selected intent into a concise message. This design reflects how players may have several thoughts but express only part of them in each message. In our ablation, removing the Intent Analysis Agent and intent stack reduced Human-Likeness from 69.80 to 67.46, supporting the contribution of intent management.

We also report deployment in a commercial game serving over one million players. Following deployment, player questionnaires showed a 15% increase in overall satisfaction with NPC interactions. Dialogue guidance increased from 70 to 90, and character consistency from 80 to 85 on a 100-point scale. These results provide additional evidence of the method's practical value in interactions with real players.

Reviewer_e9gBOpenReview ↗

thank you for response

I thank the authors for their response and for providing me with the spearman correlations. However, I am not fully convinced that there is actually agreement on the same aspects of "humanness". Ie you should consider the validity of the rubric itself (what you are producing humanness scores based on) rather than consistency with that rubric (which at higher level, spearman would answer).

My primary reason for requesting a human agreement was to understand what dimensions (typo, punctuation) humans and evaluators agreed on for assessing humanness. I see another reviewer stated that from their prompt inspection, there are issues with modifying the text until it fits a better evaluator score (from Typing Agent). For a revision, I would highly suggest finding an external mechanism to optimize on. Increasing score from a signal that is constrained to superficial detectors (again punctuation, etc) is likely to overfit to an evaluator and makes the problem trivial.

Due to these concerns, I maintain my score.

Reviewer_xPzLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Fake Player, a three-agent framework (inner monologue, intent stack, typing) that simulates concise, single-intent players to generate diverse, human-like dialogue data for NPC fine-tuning, along with Distill Bench for assessing data quality without retraining. A 7B model fine-tuned on this data outperforms ICL and RPA baselines and is further validated on a public benchmark and a commercial deployment. However, the gains are not uniform across generation backends, human-likeness remains below that of real players, and the evaluation relies heavily on similar LLM judges.

Critical strengths
  1. The paper identifies an overlooked issue in existing research on NPC training data, namely the diversity and authenticity of player-side inputs, and accordingly designs a three-agent framework that simulates real players’ expression constraints and intent switching.
Critical weaknesses

Critical Weaknesses:

  1. The evaluation criteria largely overlap with the method’s own design. The human-likeness score rewards brevity, the absence of sentence-final punctuation, and the presence of typos, which is almost exactly what the Typing Agent prompt requires. The DeepSeek-V3 model that plays the player in the downstream evaluation is likewise instructed to be brief and to include typos. An NPC trained on Fake Player data therefore faces test inputs drawn from the same distribution as its training data, and its large margin over ICL may owe more to distribution matching than to data quality itself.

  2. The paper presents the intent stack as a core component, but the ablation shows that removing it lowers the overall score by only about 1.1 points, and the ablated variant still outperforms both RPA and ICL. This suggests that most of the gain may come from the Typing Agent’s step of shortening utterances rather than from the intent stack. Moreover, the ablation is conducted only on upstream metrics, without downstream NPC training. Finally, downstream validation is limited to a single NPC, so the evidence for generalizability is weak.

Questions for authors
  1. Both the simulated player in the downstream evaluation and the human-likeness criteria favor short inputs with typos, which closely aligns with the design of the Typing Agent. Would Fake Player’s advantage still hold if real players’ dialogues were used as test inputs instead?

2.The paper evaluates several generation backends but does not systematically establish whether advances in teacher-model capabilities reduce the need for specialized player simulation.

Would a more capable contemporary LLM with direct role-playing instructions generate equally realistic and diverse conversations without the proposed three-agent architecture?

3.The downstream experiments focus primarily on one NPC setting, while PersonaArena evaluates only three personas.

How can the authors establish that their findings generalize across game genres, character personalities and conversational environments? Would the approach remain effective for adversarial players, multi-party conversations or highly emotional interactions?

  1. On PersonaArena, Fake Player improves the overall score from 2.18 to 2.35 over RPA, an absolute gain of only 0.17 on a five-point scale.

Is this difference statistically significant across independent dialogue samples, simulation seeds and evaluators? More importantly, is the improvement perceptible to human players?

4 comments

Thank you for recognizing the importance of player-side inputs. We would like to clarify the following points.

1. Human-likeness and real-player interactions. Our goal is to bring synthetic player inputs closer to the inputs NPCs encounter in practice. The RPA baseline also receives player-expression constraints, and the appendix further evaluates RPA-refine with explicit concise, human-like expression instructions. RPA-refine achieves a Human-Likeness score of 63.28, compared with 69.80 for Fake Player, while its diversity score is 57.30 compared with 67.28. These results show the benefit of organizing intentions alongside controlling expression. We also report real-player feedback from commercial deployment: post-deployment questionnaires showed a 15% increase in satisfaction with NPC interactions.

2. Stronger teachers and direct role-playing. This is precisely the comparison provided by our RPA baseline. We evaluate direct role-playing and Fake Player using the same three generation backends: DeepSeek-V3-250324, DeepSeek-R1-250528, and Doubao-Seed-1.6-250615. Fake Player achieves higher overall data-quality scores with all three: 70.39 versus 66.65, 75.89 versus 71.02, and 67.77 versus 64.81, respectively. Thus, the benefit of player simulation persists across the strong teachers evaluated in our paper.

3. Intent management and transfer across personas. Removing the Intent Analysis Agent and intent stack reduces Human-Likeness from 69.80 to 67.46 and the overall score from 70.39 to 69.26. The remaining agents still provide useful player simulation; intent management adds support for switching between and returning to conversational goals. These ablation scores measure upstream data quality. For downstream transfer, PersonaArena evaluates the trained model under different character profiles and social settings. Fake Player scores above both ICL and RPA for each of the three personas after averaging the judges. This provides evidence of transfer beyond the original game NPC; adversarial and multi-party interactions would require their own evaluation.

4. The 0.17-point improvement and player experience. The PersonaArena overall score increases from 2.18 with RPA to 2.35 with Fake Player, with improvements in five of eight dimensions. These scores average three judges over the evaluated personas, and Fake Player scores higher than RPA for each persona. The commercial deployment provides separate evidence about player experience: dialogue guidance increased from 70 to 90 and character consistency from 80 to 85 on a 100-point scale, alongside the satisfaction increase reported above.

Thank you for the response. The clarifications are helpful, but I still have a few concerns:

  1. Both the training dialogues and the main test inputs are model-generated and favor brief expressions. It remains unclear whether Fake Player would still outperform the baselines with real-player inputs (Q1). 2. The ablation removes the Intent Analysis Agent and intent stack together and reports only upstream results. The contribution of the stack and its effect on downstream NPC performance remain unclear (W2).

Thank you for the follow-up. We ran two additional experiments to address your questions.

  1. Real-player inputs.​ In a pilot study, we invited three human annotators to interact with the three anonymized NPCs across five scenarios. For each scenario, the participant sent the same opening message to all three NPCs, then continued each four-turn conversation based on that NPC's replies. Using the same six evaluation dimensions as in the paper, the three annotators rated each system and we report the averaged scores: Fake Player 73.33, RPA 55.00, and ICL 41.67 out of 100. Fake Player scored higher than ICL in all five scenarios and higher than RPA in three.

  2. Intent stack and downstream performance.​ The Intent Analysis Agent (IAA) manages player intentions through the intent stack. Table 13 ablates this entire step. We apologize for the ambiguity and will clarify this in the paper. For the downstream comparison, we trained a Qwen2.5-7B NPC on data generated without IAA and evaluated both models on 50 scenarios. FP w/o IAA scored 69.38, while FP scored 73.10. This shows that the impact of IAA on downstream performance is critical, and is consistent with the upstream evaluation.

Thank you to the authors for their response. Taking into account the authors' responses above and the comments from other reviewers, I think that my current score is justified. Therefore, I will maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.