- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Cog-Sim, a user simulator with explicit cognitive and affective states and constrained transitions, and CSTPO, a policy optimization method that assigns separate advantages to strategy selection and utterance generation using shared-prefix branching.
Experiments on emotional support, donation persuasion, and negotiation report improvements over SFT across two model sizes, competitive performance against GPT-5.5-based baselines, and positive human preferences on several task-relevant dimensions.
Critical strengths
-
A coherent approach to strategy and utterance optimization. Separating strategy selection from its linguistic realization addresses a meaningful limitation of strategy-only planners. Recoverable simulator states allow alternative actions to be compared from the same starting context, providing a reasonable basis for the proposed credit assignment.
-
Encouraging results across multiple settings. Improvements over SFT are consistent across three dialogue tasks and two model sizes. The blind human evaluation provides complementary evidence of better emotional identification, comforting, and persuasiveness, while also reporting dimensions where the method performs worse.
Critical weaknesses
-
Simulator realism is not sufficiently validated. Monotonic dose–response behavior shows that Cog-Sim reacts to stronger interventions, but does not establish that these reactions resemble those of real users. The manually specified transition rules also lack calibration or sensitivity analysis.
-
The simulator’s contribution to policy learning is unclear. Both the full model and “w/o Cog-State” use Cog-Sim to generate user replies; the ablation only removes hidden cognitive information from the critic. This shows that additional critic information helps, but does not establish whether training with Cog-Sim produces better policies than training with a simpler simulator.
-
Evaluation remains tied to the training simulator. Policies are trained and evaluated with Cog-Sim, leaving their performance under other user behaviors uncertain. The human evaluation provides useful evidence about dialogue quality, but annotators assess simulated conversations rather than interact with the policies.
-
The main performance comparisons lack uncertainty estimates. Some headline differences are small, including .460 versus .450 on P4G and .491 versus .489 on CB. Without paired confidence intervals or results across independent training runs, it is difficult to assess whether these gains are robust.
-
The hierarchical-critic ablation needs clearer specification. “w/o Cog-Critic” does not sufficiently explain its advantage construction or use of branch data. These details, together with confirmation of matched rollout budgets, are needed to assess whether the gains come specifically from hierarchical credit assignment.
Questions for authors
-
Does training with Cog-Sim produce better policies than training the same optimizer with a simpler simulator, when both are evaluated under the same conditions?
-
How are advantages constructed and branch data used in “w/o Cog-Critic”? Are its training and rollout budgets matched to the full method?
-
How are the Gaussian parameters in Appendix G estimated? What source of variability do the plotted bands represent?
-
Can you provide paired confidence intervals for the main task-return comparisons? How consistent are the gains across independent training runs?
-
What prompts, reasoning settings, and generation budgets are used for the GPT and Qwen baselines? Please clarify which baselines have thinking enabled.
Follow-up on Rebuttal and Additional Evidence
Dear Reviewer,
Thank you again for your thoughtful review. We hope our rebuttal and additional experimental evidence have addressed your concerns. We would greatly appreciate your feedback on whether any questions remain and would be happy to provide further clarification.
Thank you for your time and consideration.