- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AnthroDial, a framework for developing and evaluating anthropomorphic social agents in continuous, open-ended interaction. It contains four main components: (1) MindFlow, an event-driven interaction harness with a private Mind Buffer that allows agents to autonomously schedule, revise, cancel, and send messages rather than following strict turn-taking; (2) CAPS-Eval, a theory-motivated hierarchical evaluation framework covering validity constraints, per-turn behavior, and holistic interaction quality; (3) SEEDS, a persona–scenario expansion pipeline for synthesizing training environments; and (4) DiAPO, an adaptive policy optimization method that reweights reward dimensions according to estimated capability deficits and task–capability matching. The authors evaluate several frontier and open-weight models on Everyday Chat, Game Interaction, and Long-Horizon Character scenarios, and train Qwen3.5-9B using SEEDS-based SFT followed by DiAPO. The paper reports substantial improvements under CAPS-Eval, human preference for MindFlow over conventional turn-based interaction, and reasonable agreement between automatic judges and human rankings.
Critical strengths
- MindFlow provides a useful interaction abstraction beyond strict turn-taking. The paper correctly identifies that realistic social interaction involves not only deciding what to say, but also whether and when to communicate. The private draft buffer and event-driven virtual clock provide a simple and interpretable mechanism for delayed responses, proactive follow-ups, message revision/cancellation, and autonomous conversational closure. I found this formulation useful, and the human comparison against a turn-based harness provides some evidence that the interaction scaffold matters.
- The paper is unusually complete as an end-to-end system and provides extensive diagnostics. The work connects interaction generation, multidimensional evaluation, data synthesis, SFT, and RL, and evaluates several model families across three settings. The authors also include detailed rubrics, ablations of DiAPO components, and judge-robustness checks. These make the empirical artifact more useful than a benchmark consisting only of aggregate leaderboard scores.
Critical weaknesses
- I do not yet see a sufficiently well-isolated scientific contribution beyond the integration of several reasonable components. The paper presents MindFlow, CAPS-Eval, SEEDS, and DiAPO as a unified framework, but it is difficult to identify the central scientific result that should be taken away from the work. MindFlow is a plausible event-driven social interaction harness; CAPS-Eval is an expert-designed evaluation rubric; SEEDS is a synthetic persona/scenario expansion pipeline; and DiAPO is an adaptive reward-weighting/curriculum mechanism based on current capability estimates. Each component is reasonable, but the current experiments mainly establish that the complete system can be built and optimized, rather than revealing a new phenomenon about LLM social behavior or establishing a substantially new learning principle. In particular, I would have found the paper considerably more compelling if MindFlow were used as an experimental instrument to establish a clear result such as: once models must autonomously decide whether and when to communicate, frontier LLMs exhibit systematic failure modes that are hidden by turn-based evaluation. The current paper instead moves quickly from introducing the interaction scaffold to building an evaluation and training pipeline around it. Similarly, for DiAPO, the paper does not sufficiently establish why the proposed ZPD-inspired weighting is fundamentally preferable to simpler adaptive weighting, hard-example mining, curriculum learning, or inverse-performance weighting. As a result, the work currently reads more as a well-engineered unified system than as a focused ICLR contribution.
- The training and evaluation objectives are too tightly coupled to support the strongest claims about improved anthropomorphic capability. CAPS-Eval is not only the final evaluation framework: the same evaluation dimensions are used to filter synthesized trajectories for SFT, and they directly provide the dimension-level reward signal for DiAPO. The appendix further states that the expert-designed evaluation dimensions are shared across construction, trajectory filtering, and downstream evaluation. This creates a substantial risk that the very large post-training gains primarily demonstrate optimization toward CAPS-Eval rather than a general improvement in human-like social interaction. This concern is especially important because the reported gains are extremely large (e.g., the Qwen3.5-9B Long-Horizon Character Score rises from 0.212 to 0.728 after SFT and to 0.993 after DiAPO). Judge agreement and the MindFlow-vs-turn-based human study are useful sanity checks, but they do not provide an independent evaluation of whether the trained model generalizes to human-likeness criteria that were not involved in data filtering or reward construction. For a submission whose primary area is datasets and benchmarks, I would expect substantially stronger evidence of construct validity and out-of-distribution validity, such as blinded human evaluation of the trained checkpoints, evaluation on independently designed third-party benchmarks/rubrics, or transfer to real human interaction data not generated or selected through CAPS-Eval.
Questions for authors
- Independent evaluation of the trained models. Can the authors evaluate the Base, SFT, GRPO, and DiAPO checkpoints using a genuinely independent evaluation process whose criteria were not used for SFT filtering or RL reward construction? For example, this could be a sufficiently powered blinded human evaluation, an independently designed rubric, or an external social/anthropomorphism benchmark. A result showing that the large DiAPO gains persist under such an evaluation would materially change my assessment.
- What scientific phenomenon does MindFlow reveal that cannot be observed under turn-based interaction? Can the authors provide controlled experiments that separately manipulate autonomous initiation, response delay, follow-up without an incoming message, draft revision/cancellation, and autonomous termination? I would particularly like to know whether allowing models to control these dimensions exposes systematic failure modes of frontier models, rather than only improving perceived naturalness.
- What part of DiAPO requires the proposed ZPD formulation? Can the authors compare DiAPO against strong simpler adaptive baselines, such as inverse-current-performance weighting, uncertainty-based weighting, hard-example mining, or standard curriculum sampling, under the same rollout and compute budget? If the ZPD matching term provides a robust advantage over these alternatives, that would strengthen the algorithmic contribution considerably.
8 comments
Response to Reviewer NW6E (2/5)
2. DiAPO: DiAPO consists of two core components, CDT and ZPD. CDT measures the deficit of a dimension from its historical capability estimate , so that the reward weight is proportional to its remaining headroom (, where preserves signals for mastered dimensions). ZPD follows the Zone of Proximal Development principle to estimate the range in which the model learns most effectively, and gates the weight by how close the rollout-group mean is to the center of that range, , while asymmetrically penalizing overly easy and difficult samples.
CDT itself is a simple form of adaptive weighting under hard-example mining; the core of this paper is the introduction of ZPD on top of it, so that the model attends not only to hard cases but also, progressively, to the region just beyond its current capability. Multiplying the two terms () confines the weight to dimensions that are both underdeveloped and within reach, which yields the balanced multidimensional improvement targeted in this paper.
To validate the proposed method, we additionally compare two baselines: Fast-RL [1] and MAESTRO [2]. The ablation and baseline results verify that DiAPO outperforms previous adaptive weighting algorithms. The core reason is that previous adaptive weighting algorithms emphasize difficulty-adaptive learning but neglect the optimal learning range of the model's own capability, a gap that ZPD fills.
Table 3: Comparison with adaptive-weighting baselines on Qwen3.5-9B (Score and ACC@95 in %; Qwen3.5-397B-A17B judge).
| Method (Qwen3.5-9B) | Everyday Score | Everyday ACC@95 | Game Score | Game ACC@95 | CLAM Score | CLAM ACC@95 |
|---|---|---|---|---|---|---|
| + GRPO | 97.31 | 72.00 | 98.79 | 88.14 | 98.20 | 74.00 |
| + Fast-RL | 97.35 | 68.00 | 98.59 | 86.44 | 97.06 | 66.00 |
| + MAESTRO | 97.34 | 66.00 | 98.07 | 84.75 | 97.69 | 74.00 |
| + CDT only | 97.58 | 68.00 | 98.99 | 89.83 | 97.75 | 70.00 |
| + ZPD only | 97.51 | 70.00 | 99.06 | 94.92 | 98.14 | 82.00 |
| + DiAPO | 98.01 | 76.00 | 99.26 | 96.61 | 99.30 | 92.00 |
[1] Li J, Zhang H, Zhang F, Chang TW, Kuang K, Chen L, Zhou J. Optimizing language models with fair and stable reward composition in reinforcement learning. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024 Nov (pp. 10122-10140).
[2] Zhao Y, Wang H, Ding X, Ouyang Y, Cai B, Xiong K, Gao J, Sun Z, Du L, Qin B, Liu T. Maestro: Meta-learning adaptive estimation of scalarization trade-offs for reward optimization. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2026 Jul (pp. 22267-22283).
Response to Reviewer NW6E (3/5)
W2: The training and evaluation objectives are too tightly ... generated or selected through CAPS-Eval.
Response: We thank the reviewer for raising this important concern about construct validity. We address it from three complementary angles: (i) the rubric itself is theoretically grounded and empirically aligned with human annotation; (ii) real human conversations provide an external anchor for the scale; and (iii) the gains transfer to third-party rubrics whose dimensions were fixed before our training and played no role in data filtering or reward construction.
(i) The rubric is grounded in psychology and aligns with human annotation. Each dimension instantiates one CAPS component—persona (), situational understanding (), affective adaptation (), and goal-directed participation ()—so the checklist covers the anthropomorphism construct rather than ad-hoc criteria. Figure 7 confirms the alignment: human annotators and the two automatic judges give the nearly same ordering at both ends.
(ii) Real human game conversations anchor the scale from outside. We score the in-game conversations collected from the game company—the source material of Game Interaction as below. Real conversations rank first on Score, on per-turn evidence and on every pass rate, and their holistic score stays at the level of the strongest model.
Table 4: Models (Table 1, Game Interaction) and company-collected real conversations under the same rubric (best per column in bold).
| Model (Game Interaction) | Score | Per-Turn | Holistic | ACC@85 | ACC@90 | ACC@95 |
|---|---|---|---|---|---|---|
| Real conversations | 0.9150 | 0.9170 | 0.9129 | 67.80% | 40.68% | 25.42% |
| Claude-4.6 Thinking | 0.8290 | 0.7454 | 0.9135 | 23.73% | 5.08% | 0.00% |
| Gemini-3.5 Flash | 0.8490 | 0.7782 | 0.9200 | 28.81% | 13.56% | 3.39% |
| GPT-5.5 | 0.7010 | 0.5811 | 0.8201 | 0.00% | 0.00% | 0.00% |
| DeepSeek-V4-Flash | 0.7520 | 0.6253 | 0.8782 | 0.00% | 0.00% | 0.00% |
| Qwen3.5-397B-A17B | 0.8360 | 0.7649 | 0.9074 | 25.42% | 6.78% | 0.00% |
| Qwen3.6-35B-A3B | 0.7970 | 0.7345 | 0.8587 | 10.17% | 1.69% | 0.00% |
| Qwen3.5-9B | 0.6620 | 0.5914 | 0.7320 | 0.00% | 0.00% | 0.00% |
(iii) The gains transfer to third-party rubrics. Following HumanLLM [3] and SimBench [4], we extract their rubric into our evaluation and score the same checkpoints with it as below. This provides strong evidence that the improvements from training under our evaluation framework generalize effectively and remain compatible with other evaluation dimensions.
Table 5: The same four checkpoints under CAPS-Eval (Everyday Chat) and under the HumanLLM and SimBench rubrics.
| Checkpoint (Qwen3.5-9B) | CAPS Score | CAPS Per-Turn | CAPS Holistic | CAPS ACC@85 | CAPS ACC@90 | CAPS ACC@95 | HumanLLM Score | HumanLLM Per-Turn | HumanLLM Holistic | HumanLLM ACC@85 | HumanLLM ACC@90 | HumanLLM ACC@95 | SimBench Score | SimBench ACC@60 | SimBench ACC@65 | SimBench ACC@70 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Base | 0.6410 | 0.7261 | 0.5558 | 12.00% | 4.00% | 0.00% | 67.61 | 77.28 | 57.94 | 10.00% | 6.00% | 4.00% | 43.11 | 28.00% | 24.00% | 20.00% |
| + SFT | 0.9210 | 0.9677 | 0.8737 | 80.00% | 52.00% | 26.00% | 93.68 | 92.05 | 95.31 | 88.00% | 70.00% | 36.00% | 68.66 | 70.00% | 58.00% | 44.00% |
| + SFT + GRPO | 0.9731 | 0.9962 | 0.9501 | 98.00% | 88.00% | 72.00% | 95.13 | 94.91 | 95.35 | 96.00% | 72.00% | 28.00% | 74.64 | 92.00% | 88.00% | 72.00% |
| + SFT + DiAPO | 0.9801 | 0.9966 | 0.9636 | 100.00% | 98.00% | 76.00% | 95.55 | 94.76 | 96.35 | 100.00% | 86.00% | 30.00% | 75.55 | 92.00% | 90.00% | 84.00% |
[3] Wang X, Yang J, Li W, Xie R, Huang JT, Gao J, Huang S, Kang Y, Guo Y, Feng H, Xiao Y. HumanLLM: Benchmarking and Improving LLM Anthropomorphism via Human Cognitive Patterns. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2026 Jul (pp. 38486-38517).
[4] Hu T, Baumann J, Lupo L, Collier N, Hovy D, Röttger P. SimBench: Benchmarking the ability of large language models to simulate human behaviors. In International Conference on Learning Representations, 2026 Apr (Vol. 2026, pp. 28290-28341).
Response to Reviewer NW6E (1/5)
We thank the reviewer for the detailed and constructive feedback. We address each concern below.
Response:
Regarding the concern that the scientific contribution of each component is not sufficiently isolated, we describe the design rationale and the validation evidence for MindFlow and DiAPO separately.
1. MindFlow: The main text already provides targeted experimental analyses of MindFlow: (i). Basic capabilities that small models lack. §5.2 shows that for weaker models such as Qwen3.5-9B, a large per-turn–holistic gap indicates that structural failures such as repetition, malformed output, and AI-identity leakage remain a major bottleneck. (ii). Difficulties shared by large models. As shown in §5.2 and Figure 3, under MindFlow the problems concentrate on four dimensions: context consistency (E1, down to 0.77 for Qwen3.5-397B on Everyday Chat and 0.59 on Long-Horizon Character), scenario–relationship fit (E2, also degrading on Long-Horizon Character), unpunctuated chat style (D11, at most 0.54 even for the strongest models), and fragmentation and immediacy (E8, markedly lower on Game Interaction).
To show more fully that MindFlow exposes model problems, we sample four models and measure the failure rate of every dimension (4 models × 50 scenarios × both roles A/B = 400 cases in theory, 397 valid cases after removing L0-fatal cases; a case counts as one failure on a dimension if any checklist item of that dimension is not satisfied; the models are GPT-5.5, DeepSeek-V4-Flash, Gemini-3.5 Flash, and Qwen3.5-397B-A17B).
Comparing MindFlow with the turn-based baseline, MindFlow substantially lowers the failure rate of most dimensions, in particular persona expression (D3, 46.10% → 3.53%), segmentation and rhythm (D5, 30.98% → 2.27%), chat naturalness (D1, 26.45% → 1.26%), and knowledge and action boundaries (D4, 22.67% → 0.76%). The mechanism requires the model itself to take responsibility for turn-taking, pacing, and persona consistency instead of letting a fixed turn-taking protocol absorb them, so failures caused by forced immediate replies are largely eliminated.
We nevertheless find that models remain weak on D2 and E1 under MindFlow.
D2 requires replies not to be overly redundant, yet models still over-interpret or over-explain, and their replies are not concise enough or have too low an information density; this is especially visible under successive follow-up questions, where models tend to re-explain earlier speech acts.
E1 concerns context consistency: models confuse earlier events, produce self-contradictory facts, or repeat meaningless dialogue in order to avoid errors. Notably, the failure rate of E1 rises from 14.36% under the turn-based baseline to 20.40% under MindFlow: once a model may revise a draft, postpone, or cancel a message, consistency between what has been sent and what is planned also becomes a critical bottleneck.
These are the aspects in which we hope anthropomorphic behavior can be further improved. We also note that CAPS-Eval and SEEDS are conventional in form — an expert rubric and an environment-expansion pipeline — and we do not claim either as a novel mechanism: they are the instruments that make the failure modes measurable and the training environments diverse.
Table 1: Case-level failure rate of each dimension under MindFlow (4 models × 50 scenarios × both roles A/B; Qwen3.5-397B-A17B judge).
Table 2: Case-level failure rate of each dimension under the turn-based baseline (same cases as Table 1).