acceptodds
Under review as a conference paper at ICLR 2027

: Scaling Turing-Style Evaluation to Optimize Human Simulation in Dialogue

12 views

Abstract

Realistic user simulation is essential for developing and evaluating conversational AI agents, yet measuring and optimizing simulation fidelity remains challenging. Human Turing tests assess whether simulated behavior appears human, but are costly to use for iterative optimization, motivating scalable evaluation with Machine-Turing judges. We introduce , a human-grounded co-evolutionary framework for improving user simulators and Machine-Turing judges through prompt optimization, without updating model parameters. Machine-Turing judges provide rewards and diagnostic feedback to improve simulators, while human Turing judgments and explanations guide judge refinement. The refined judge then provides feedback for further simulator improvement. In customer-service dialogue, simulator optimization improves Turing win rates under both held-out LLM judges and human annotators. Incorporating human explanations improves held-out judge accuracy, and optimizing against the refined judge further improves simulator human-perceived human-likeliness. These results demonstrate the effectiveness of \turing in jointly improving user simulators and their evaluators through human-grounded co-evolution. The dataset would be released upon acceptance.

Primary area:
generative models
Keywords:
User Simulation, Sim-to-Real, Turing Test, Conversational AI, LLM Evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_bAa7OpenReview ↗
Rating
2(−1)
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a co-evolution approach for improving both user simulator realism and judges of realism, done solely via prompting. The authors's approach is based in a pairwise turing setup, where a judge must determine which of two dialogues was human vs. machine generated. The co-evolution approach iteratively improves the simulator using judge rationales, and then improves the judge using information about human vs. machine origins. The results show that the approach works quite well!

Critical strengths

The idea is great, the infusing of human data at each iteration is nice, and the results seem quite exciting!

Critical weaknesses
  • Evaluation setup validity: I have some concerns of the validity of some of the evaluations.
    • External validity: the setup is not really a Turing test; the real turing test is judging whether one conversation is real or AI, but here, it's much more simplified because the judgment is contrastive. While that doesn't negate the results, more discussion if needed for how this can be useful for evaluating simulators without human references.
    • Circularity: This may be due to the writing issues (below) but I was confused whether the human judges in §2 were the same as the ones in §3, which could cause an issue of circularity (same humans who originally did the Turing task are the ones being improved upon)? Similarly, it seems that the "human-rationale-assisted judge optimization" uses some of the human-generated evaluation signal to improve, which makes it a bit less valid. I would appreciate some clarification and discussion to indicate how this is avoided.
    • Win rate: I am not quite sure win for simulator is the goal; the goal should be for the human and the AI to be completely indistinguishable, not for the AI to appear "more human" (I'm not sure that even makes sense?). More discussion and perhaps results with a metric that aims for indistinguishability might be good.
  • Low human agreement on turing task: Table 4 & §2.2 shows that LLM-human agreement is quite low, and so is human-human. This raises the question of whether the foundation of the paper is meaningfully valid enough... I wish authors had discussed these low agreement rates a bit more.
  • Baselines: While I appreciate that the authors focus purely on prompt-based improvements, the impact and comparability of the paper is hindered by the lack of comparisons with models trained for user simulation (e.g., OSim/Odyssim, UserLM, CoSER). Having these results would help readers understand how prompting compares to training and which option is most likely to help with building truly good user simulators.
  • Writing: The paper's writing and organization could be improved.
    • Right now, §2 contains a mix of results and methods, it's a bit confusing. I'd suggest that the authors present their approach and methodology in §2, and relegate results to section §3, where they can frame each key result as a specific research question.
    • Authors should make sure to use \citet for in-line citations but \citep for parenthetical citations (e.g., L68, 73).
    • Table 2 is too small font-wise
    • Table 3 should probably be a confusion matrix, to also indicate how many dialogues pass the Turing test but not RNR. Right now the stats are a bit hard to interpret.

Post rebuttal: I think this paper has an amazing idea, but the execution is just not valid enough imo. I therefore lower my score 3->2, based on the rebuttal not addressing my concerns and even making them more crucial. I do believe with a careful pass and slight redesign of the experiments this paper will be amazing!

Questions for authors

Out of curiosity, how would the results look of pitting two AI-AI dialogues against each other? Can this method be used for comparing simulators against others?

2 comments

We thank the reviewer for recognizing our idea and experimental results.

1. External validity

Our setup follows common Turing-test practice, including the setup in Turing’s original article.

As described in [1], “Player C, the interrogator, is tasked with determining which player – A or B – is a computer and which is a human using only the responses to written questions.”

2. Circularity

We carefully separate data sources and the uses of human signals to avoid leakage.

To clarify the presentation, the human judges whose rationales are used for human-rationale-assisted judge optimization in Section 2 are the same annotators who evaluate the simulator in Section 3. This does not create circularity because the evaluation precedes judge refinement: we first evaluate the simulator optimized only against the seed judge, before any human-rationale distillation. We then use the rationales collected during that evaluation to refine the judge.

Human data# annotatorsUsed forNever used for
Paper: Human-AI conversations data collection9Reference dataJudging their own tasks
Paper: human Turing study (Tables 4, 6)8Evaluating simulators; validating LLM judges; rationales for judge refinement-

3. Why is a higher Turing win rate better?

A higher Turing win rate indicates that the simulator is more frequently perceived as human.

We cannot directly measure the humanlikeness of user simulators. What the Turing test and our method measure is how consistent the simulator’s behaviors are with human reviewers’ mental models of humanlikeness [4]. Therefore, a win rate above 50% means that the simulator is more frequently perceived as human.

Meanwhile, a win rate of 50% alone does not establish indistinguishability from humans, because aggregate results can conceal systematic differences across scenarios. For example, a simulator consistently selected as human in half the scenarios and consistently identified as AI in the other half would also achieve an overall win rate of 50%.

Moreover, papers such as [2] and [3] interpret higher win rates as a sign of more human-like simulators. We follow their practice.

4. Low human agreement

Different judges use different clues and have different mental models of humanlikeness [4]. Accordingly, agreement is around 60% in our experiments.

5. Fine-tuned baselines

Our optimized prompt achieves a higher TWR than the fine-tuned baselines.

Their model sizes are small, so we do not directly compare them with our results based on closed-source LLMs.

SimulatorSonnet 4.6 (held-out judge)Opus 4.8 (held-out judge)TWR (mean)
UserLM-8B0.3780.2790.328
OSim-8B0.3210.2860.303
CoSER-8B0.2070.1540.181
GPT-5.5 + our optimized prompt (ours)0.7270.6620.694

6. Confusion matrix for Table 3

The conclusion remains the same: RNR is a weaker filter.

An RNR pass indicates that the dialogue is judged as human. The table shows that many dialogues judged as human by RNR are identified as AI by the Turing judge in both presentation orders.

SimulatorRubricTuring: caught in both ordersSplitFooled in both orders
GPT-5-minipass (87)7485
GPT-5-minifail (6)600
GPT-5.5pass (91)432622
GPT-5.5fail (2)200

7. Comparing two AI–AI dialogues

Comparing two AI-generated dialogues is feasible, but falls outside our Turing-test-style evaluation because the judgments cannot be validated against a real human reference.

References

[1] Pinar Saygin, A., Cicekli, I. & Akman, V. Turing Test: 50 Years Later. Minds and Machines 10, 463–518 (2000).

[2] Wang, Yingshan Susan et al. “Learning User Simulators with Turing Rewards.” ArXiv abs/2606.19336 (2026).

[3] Hathidara, Ashutosh et al. “MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likeness.” Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (2026).

[4] C. R. Jones & B. K. Bergen. Large language models pass a standard three-party Turing test. Proc. Natl. Acad. Sci. U.S.A. 123 (21), e2524472123 (2026). https://doi.org/10.1073/pnas.2524472123

Thank you for responding to my review. However, the rebuttal largely doesn't address my concerns, making me doubt the validity of the paper more than during my first pass.

W1: External validity / usefulness of contrastive Turing test

You're right that the original Turing test was contrastive. But that does not negate my point that this is an easier setup compared to a single-input human vs. AI classification. More discussion is still needed for how this can be useful for evaluating simulators without human references.

W2: Circularity of human evaluations

I still think that there is bleed over from what the authors described in their rebuttal. Authors should conduct a version of the experiments where the human judges are completely separate, because humans have low agreement.

W3: Why is a higher Turing win rate better?

Even though previous work does this, we should always think critically about adopting evaluation setups from previous works. In this case, in terms of construct validity of evaluation, since the goal of the evaluator is "determine which one is human vs. AI", if the evaluator consistently judges the AI as human, then you're not measuring the right construct, and could possibly even suggest the evaluator is overfitting to spurious correlations in the AI trajectories. This is a big flaw in the experimental design.

W4: Low human agreement

This is not addressed in the rebuttal. I want more discussion on this and how this affects the validity of the findings.

W5: User sim baselines.

Thanks for computing this. In retrospect, you're right, since these are smaller models, they likely won't compete against your frontier model setups. But still a useful thing to compute imo, so thank you!

W6: Confusion matrix

Thanks!

W7: AI-vs-AI

Sure, that makes sense.

Reviewer_j9KvOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper aims to introduce a framework to evaluate and improve the human-perceived human-likeness of LLM based simulators by using a co-evolving judge-simulator structure.

Critical strengths

The paper's motivation is strong because there is a lot of demand for algorithmic fidelity when using LLMs for human-behavior simulations. The idea behind using a feedback loop, or iterative process, to improve algorithmic fidelity is also interesting. However, I think the work has several weaknesses in the way the idea is applied. The one result which seems to be interesting is the fact that (Table 6), with seed, humans would say AI-written conversation is the human one only 43% of the time, while after calibration they are at 50% chance (which, by the way, seemed to be the initial result, so I am a bit confused by the initial TWR=43% compared to the 0.5 accuracy of human judges in Table 4: are they coherent? Shouldn't TWR also be around 50%?)

Critical weaknesses

I have several important concerns with this paper:

  1. I think the authors should engage more with the algorithmic fidelity literature, e.g., Argyle LP, Busby EC, Fulda N, Gubler JR, Rytting C, Wingate D. Out of One, Many: Using Language Models to Simulate Human Samples. Political Analysis. 2023;31(3):337-351. doi:10.1017/pan.2023.2 N. Pagan, P. Törnberg, C.A. Bail, & C. Barrie, AI-generated language remains systematically distinguishable from human communication, Proc. Natl. Acad. Sci. U.S.A. 123 (42) e2531748123, https://doi.org/10.1073/pnas.2531748123 (2026). Bouleimen, A., De Marzo, G., Kim, T. et al. The collective turing test: large language models can generate realistic multi-user discussions. Sci Rep 16, 32760 (2026). https://doi.org/10.1038/s41598-026-62286-9
  2. The idea of using an LLM (even if a different one) to detect other LLMs and to self-improve is, in my opinion, weak. This is already clear from the fact Table 4, showing that the models (as well as the human) have basically 50% accuracy (presumably on a balanced dataset of human / AI-generated content), meaning they cannot distinguish more than flipping a coin. However, ML-models trained on stylistic features, as well as BERT-based classifiers, can do much much better than that (see the Pagan et al. (2026) paper mentioned above, showing that detection accuracy is at around 70-80%, even after calibration). Basically: why shall we improve algorithmic fidelity if the judges are already failing in the detection? I think the answer is: we should use better judges, in the first place. Connected to the above: the paper shows that, through this coupled evolution, the generation reaches a point in which AI-generated text is perceived as the human one in more than 50% of the times! (And, by the way, here it is also a bit confusing, because the authors change metric...!). But, anyways, was the goal that of producing text which is perceived as more human than genuinely human-written text? I think the goal should just be that it can be confused with human-written text at chance rate.
  3. Ultimately, it is not clear what the paper achieves. A mechanism in which the generator is instructed to basically game the judge so that it exploits some features that make the generated text be judged as human more than genuinely human-written text?
  4. Even so, I wonder what the generator learns to produce. Possibly more homogenized text, such that it is guaranteed to be scored high in human-likeness? They should check that. Basically, they are optimizing over the same metrics they are measuring (even though they might be using different LLMs as judges), but they should be measuring performance with different tests (for example NLP-based detectors, or ML models).
  5. Also, the human evaluation is very poor and just preliminary (with only 3 human annotators, a total of 300 conversations, also written by (the same?) 3 human annotators?
  6. I cannot fully translate between accuracy and TWR.

Minor:

  • Table 2: they are reporting outside-taxonomy cues. What about the inside ones? The table itself could also be more readable.
  • In the results of Table 3 and 4, it would be better to see a proper confusion matrix, not just the accuracy, especially if the dataset is not balanced.
  • Also, in Table 7, I don't get the Simulator Win Rate (Gemini), and how to interpret it.
  • The related work is spread between Section 2 and Section 4, and it misses important references on algorithmic fidelity.
Questions for authors

I think clarifying the use of TWR / Accuracy (and the balancing of the dataset) would be important

3 comments

We thank the reviewer for recognizing the strong motivation and the potential of an iterative feedback loop. We address the concerns below.

1. Engagement with algorithmic fidelity literature (W1, Minor 4)

We will incorporate the suggested works into the related work and discussion.

2. Pairwise Turing evaluation versus classifier-based AI detection (W2, W4)

Our pairwise Turing evaluation measures how often a simulator is selected as human, whereas standalone AI detection measures classification correctness.

The two evaluations ask different questions. A standalone classifier asks, “Is this conversation AI-written?” Turing² instead asks, “Given a human conversation and a simulated conversation for the same task, which is perceived as human?” This is a relative judgment between task-matched conversations.

We will add citations and clarify this distinction, including a discussion of the BERT-based results reported by Pagan et al. (2026). Because the evaluation protocols differ, their reported detection accuracy is not directly comparable to the performance of our LLM Turing judges.

Rebuttal Table 1. Comparison of evaluation settings.

EvaluationWhat the judge seesInterpretation of the score
Our pairwise Turing testOne human conversation and one generated conversationHow often the generated conversation is selected as human within the pair
Detector in Pagan et al. (2026)A single conversationWhether the conversation is correctly classified as human or AI

What we want to achieve

A reward above 0.5 means that simulated traces are selected as human more often than human traces. Thus, our reward is a simulator fooling rate, or win rate. Our objective, discussed in Sections 1–3, is to improve human-perceived humanlikeness.

3. Why is a higher Turing win rate better? (W2, W3, W4)

A higher Turing win rate indicates that the simulator is more frequently perceived as human.

We cannot directly measure the humanlikeness of user simulators. What the Turing test and our method measure is how consistent the simulator’s behaviors are with human reviewers’ mental models of humanlikeness [4]. Therefore, a win rate above 50% means that the simulator is more frequently perceived as human.

Meanwhile, a win rate of 50% alone does not establish indistinguishability from humans, because aggregate results can conceal systematic differences across scenarios. For example, a simulator consistently selected as human in half the scenarios and consistently identified as AI in the other half would also achieve an overall win rate of 50%.

Moreover, papers such as [2] and [3] interpret higher win rates as a sign of more human-like simulators. We follow their practice.

4. Overfitting, reward hacking, and data leakage (w2, W3, W4)

Across multiple evolution rounds, we find no evidence of overfitting or reward hacking under held-out judges and human evaluation.

As discussed in Sections 1–3, our goal is a simulator with higher human-perceived humanlikeness. We evaluate whether improvements carry over to held-out judges and humans across multiple evolution rounds. Held-out judges and Human TWR both increase across rounds, providing no evidence of either form of reward hacking in these experiments.

Rebuttal Table 2. Evaluation across evolution rounds.

Evolution round ttHeld-out judge TWR ↑Human TWR ↑
10.470.48
20.580.49
50.590.52

We also carefully separate data sources and their uses to avoid data leakage.

Rebuttal Table 3. Human data sources and their uses.

Human data# annotatorsUsed forNever used for
Paper: Human-AI conversations data collection9Reference dataJudging their own tasks
Paper: human Turing study (Tables 4, 6)8Evaluating simulators; validating LLM judges; rationales for judge refinement-

5. Human evaluation participants (W5)

The studies involve 9 writers, 8 annotators for the paper’s evaluation.

As detailed in Rebuttal Table 3, 9 people contributed to dataset generation and 8 annotators participated in the paper’s human evaluation.

6. Relationship between TWR and accuracy (W6)

TWR measures the simulator’s success, while accuracy measures the judge’s success.

For the same forced-choice pairwise judgments, the two metrics satisfy:

TWR=1−Accuracy.\mathrm{TWR}=1-\mathrm{Accuracy}.

They express the contrasting objectives of the simulator and judge, which co-evolve in our framework.

Minor 1. Inside taxonomy

Please refer to [1] for inside-taxonomy cues. We analyzed outside-taxonomy cues to demonstrate that our Turing evaluation can uncover novel cues not covered by existing behavioral taxonomies.

Minor 2. Confusion matrix for Table 3 and 4

For Table 3, the conclusion remains the same: RNR is a weaker filter.

SimulatorRubricTuring: caught in both ordersSplitFooled in both orders
GPT-5-minipass (87)7485
GPT-5-minifail (6)600
GPT-5.5pass (91)432622
GPT-5.5fail (2)200

Table 4 reports agreement between LLM judges and human annotators, rather than classification accuracy against ground-truth labels; a standard classification confusion matrix therefore does not directly apply.

Minor 3. Interpretation of “Simulator Win Rate (Gemini)”

This is the simulator’s Turing win rate (TWR) evaluated by the Gemini judge. It measures how often Gemini selects the simulated conversation as human when paired with a task-matched human conversation.

References

[1] Zhou, Xuhui et al. “Mind the Sim2Real Gap in User Simulation for Agentic Tasks.” ArXiv abs/2603.11245 (2026).

[2] Wang, Yingshan Susan et al. “Learning User Simulators with Turing Rewards.” ArXiv abs/2606.19336 (2026).

[3] Hathidara, Ashutosh et al. “MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likeness.” Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (2026).

[4] C. R. Jones & B. K. Bergen. Large language models pass a standard three-party Turing test. Proc. Natl. Acad. Sci. U.S.A. 123 (21), e2524472123 (2026). https://doi.org/10.1073/pnas.2524472123

I thank the authors for their response. Unfortunately, it did not fully resolve my concerns about the objective of the paper, which, as I now understand it, is to make AI-generated text appear more human-like than actual human text. Judging from the other reviews, this concern seems to be shared.

My difficulty is mainly conceptual: it is hard to see how AI-generated text could be more human-like than text that is fully written by humans. A more plausible explanation, in my view, is that the generator learns to exploit the weaknesses of the LLM judges and the human annotators. Both are known to have limited detection ability, and Table 4 also seems to confirm this. That would be an interesting finding in itself, and it may be worth looking into with stronger detectors (e.g., trained ML classifiers) to understand what drives this counterintuitive result.

Relatedly, I do not think my earlier comment on the accuracy of the LLM judges and human annotators has been fully addressed. The response states that “Table 4 reports agreement between LLM judges and human annotators, rather than classification accuracy against ground-truth labels; a standard classification confusion matrix therefore does not directly apply.” However, the table does report judge accuracy, and it is around 0.5 across all rows. If I read this correctly, on a balanced dataset of AI and human texts, the judges perform at roughly chance level. This would have two implications: (A) according to these judges, AI-generated text is already on par with human-written text; and (B) using these judges to evaluate the loop method may set a relatively low bar. I would appreciate it if the authors could clarify or expand on this point. For now, I keep my score.

Finally, I agree with the authors that “a win rate of 50% alone does not establish indistinguishability from humans, because aggregate results can conceal systematic differences across scenarios. For example, a simulator consistently selected as human in half the scenarios and consistently identified as AI in the other half would also achieve an overall win rate of 50%.” I would only add that, in my view, a 50% win rate is still a necessary condition, and a baseline worth checking within each scenario-specific subset too.

Reviewer_jYHcOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces a new method to improve “human perceived humanness” of AI simulators. The primary approach is to have AI users and AI generators interact and “co-evolve” with each other. The judges help the AI user become more accurate and in turn the AI user helps the judge strengthen its criticism decisions. Autgors test this approach on customer service datasets as they benefit most from AI simulators and show significant improvements without retraining or fine-tuning. The paper poses this Turing^2 as a promising way to improve simulations under human eye, but some extra validation should be done in related domains (ie advice giving, virtual therapy, etc) to see full effect.

Critical strengths
  1. Large gains are obtained in comparison to several different baselines. Authors show roughly 20–27% gains with evaluation across humans and AI judges.
  2. In comparison to approaches that use fine-tuning, the paper uses a novel “co-evolution” paper that has not been explored much in other paper’s (to my exposure). It is more efficient and feasible compared to re-training. I see this being especially useful to proprietary models which don’t allow fine-tuning.
Critical weaknesses
  1. To my knowledge, authors show only 1 full round of both judge’s improvement. I suggest there be some demonstration of multiple rounds. Perhaps a table with t=1, 2, 5, etc co-evolution rounds would be useful to understand the long-term effects of this co-evolution and if the judges reward hack to each other.
Questions for authors

How have you validated that reward hacking does not occur? To my understanding in fig2, multiple rounds receive multiple feedbacks, so this could cause some material overfitting to the judges? I understand that a separate judge was used and only 1 round/cycle was shown, but perhaps we could use the table I propsoed in weakness 1 to see the extremeness of this across optimization cycles (t=1,2,5)?

2 comments
AuthorsOpenReview ↗

Demonstration of multiple rounds

We thank the reviewer for recognizing the large gains across human and AI evaluations, the novelty of our co-evolution approach, and its efficiency and feasibility compared to retraining, especially for proprietary models that do not allow fine-tuning.

Evaluation across multiple evolution rounds provides no evidence of reward hacking under either held-out LLM judges or human annotators.

Our goal is human-perceived human-likeliness, so we consider it reward hacking when a simulator’s gain under its reward judge does not carry over. This takes two forms: (a) overfitting to the judge used as the reward, tested with held-out judges; and (b) exploiting a preference shared by all LLM judges, tested with human annotators.

To test these two forms of reward hacking across multiple rounds, we conduct a multi-round evolution experiment and evaluate the simulators using held-out LLM judges and new human annotators. The held-out LLM judges are not involved in any evolution cycle, and the human annotators were never involved in labeling or previous judgments. During evolution, we continue to incorporate the judge’s own prediction failures into judge refinement, but introduce no additional human rationales beyond those used in Table 7. This restriction reflects limited annotator availability: annotators whose rationales are used to refine the judge must be excluded from subsequent human evaluation.

Evolution round ttHeld-out LLM judge TWR ↑Human TWR ↑
10.470.48
20.580.49
50.590.52

TWR under held-out LLM judges increases across rounds, providing no evidence of form (a) reward hacking in these experiments. Human TWR also increases across rounds, providing no evidence of form (b) reward hacking.

Reviewer_jYHcOpenReview ↗

Thank you for the update - a question

I thank the authors for running my suggested experiment. I want to clarify a few things. Despite being "held out" to some extent there may be similarities between the jduges and what is in the evaluator, for example prompts, criterion, etc. It's not fully clear to me based on response above how these were decoupled. Could you provide the prompt used in this held out judge and the evaluator and delineate how they are meaningfully different, so this could be examined?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.