- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a two-stage framework for personality-conditioned dialogue generation, which first maps natural-language character descriptions into continuous OCEAN scores, and then uses five trait-specific LoRA modules to control dialogue generation.
Critical strengths
S1:Converting natural-language character descriptions into continuous personality control is a reasonable and practically relevant direction.
S2: The work refers to personality inference, controllable dialogue generation, which is end-to-end evaluation within a unified framework.
Critical weaknesses
W1: The paper directly adopts the pipeline “natural-language description → OCEAN → dialogue,” but does not clearly explain the motivation of OCEAN as an intermediate representation, why not directly consider conditioning dialogue generation on the original character description.
W2: The related works are about personality inference and personality-conditioned generation, which neglects the closely related methods like PsychAdapter, BIG5-CHAT, and personality steering methods, please explain the reason.
W3: OCEANDialogue is mainly compared with numerical zero-shot prompting, but omitting several SOTA baselines. In addition, the comparison of “character description → dialogue” and“character description → OCEAN → dialogue.” is neglected. The contribution of the counterfactual objective and the use of five independent trait-specific LoRA modules are also not unclear.
W4: The experimental settings are different, such as retrieval corpus and dialogue training data, and several components rely on LLM-generated descriptions or responses, please give detailed illustration about them.
W5: The evaluation metrics like six-level control slopes, cross-scenario recovery and automatic personality scoring are insufficiently discussed for why use them.
W6: From the results, we can observe the clear improvements of OCEANDialogue for Openness, Conscientiousness and Extraversion, except Agreeableness and Neuroticism, and thus I want to know the reason about them .
Questions for authors
Q1: Why is OCEAN necessary as an intermediate representation? Can you provide a direct comparison of "character description → dialogue" against "character description → OCEAN → dialogue" to show the advantage of the intermediate step over directly conditioning on the original description?
Q2: What specific information does OCEAN capture that the raw description does not, and does the two-stage pipeline lose or distort any description-specific detail?
Q3: How does your approach technically differ from PsychAdapter, BIG5-CHAT, and personality steering methods? Please give a mechanism-level comparison rather than a brief overview, and clarify where this work sits in the development of the research direction.
Q4: The main comparison is against numerical zero-shot prompting. Can you add stronger personality-control baselines (e.g., PsychAdapter, BIG5-CHAT, steering methods) for comparison?
Q5: Can you directly compare "character description → dialogue" vs. "character description → OCEAN → dialogue" as the ablation?
Q6: Can you isolate the contribution of the counterfactual objective and of the five independent trait-specific LoRA modules, rather than focusing the appendix on LoRA rank and scaling?
Q7: The development set, retrieval corpus, dialogue training data, and confirmatory set are built in rather different ways, and several components rely on LLM-generated descriptions/responses. Can you analyze how these distributional differences affect generalization (e.g., train-on-one-source, test-on-another)?
Q8: Why are six-level control slopes, cross-scenario recovery, and automatic personality scoring the most appropriate measures of continuous personality control? Can you justify them and report their validity/reliability relative to alternatives?
Q9: Improvements are clear mainly for Openness, Conscientiousness, and Extraversion, while Agreeableness does not improve and Neuroticism performs worse. Can you explain why, and provide details about the method achieves effective control across all five dimensions? If not, can the claims be revised to reflect the current scope?
3 comments
Response to Reviewer 4Nww (Part 2/3)
Response to Reviewer 4Nww (Part 2/3)
Reviewer Concern:
W3: The contribution of the counterfactual objective and the use of five independent trait-specific LoRA modules are also not unclear.
Q6: Can you isolate the contribution of the counterfactual objective and of the five independent trait-specific LoRA modules, rather than focusing the appendix on LoRA rank and scaling?
Response:
We have added an ablation study to evaluate the contributions of counterfactual training and independent trait-specific LoRA modules. The results show that the full model achieves higher overall personality controllability than CE-only and both shared LoRA configurations, with statistically significant improvements over CE-only and shared rank-40 LoRA in the high–low effect. We have included the overall ablation results in the main paper and trait-wise analysis in the appendix.
Reviewer Concern:
W5: The evaluation metrics like six-level control slopes, cross-scenario recovery and automatic personality scoring are insufficiently discussed for why use them.
Q8: Why are six-level control slopes, cross-scenario recovery, and automatic personality scoring the most appropriate measures of continuous personality control? Can you justify them and report their validity/reliability relative to alternatives?
Response:
Our evaluation metrics capture complementary aspects of continuous personality control. Six-level regression slopes quantify the magnitude and direction of perceived trait changes, and Spearman correlation measures monotonic alignment. A method can exhibit a large positive slope despite inconsistent ordering at intermediate levels, which Spearman correlation helps identify. As an additional validation, we computed Kendall's on the same six-level evaluation scores used for Spearman correlation. Both measures show consistent trends, with OCEANDialogue improving over zero-shot prompting for Openness (0.123 to 0.242), Conscientiousness (0.170 to 0.303), and Extraversion (0.183 to 0.385), but not for Agreeableness or Neuroticism. These findings show that our conclusions about personality control are consistent across two rank-based evaluation measures.
Cross-scenario recovery measures how consistently positive control effects emerge when aggregating responses across different conversational contexts. Since this is a supplementary aggregation analysis, we have moved it to the Appendix.
Automatic personality scoring enables consistent evaluation of perceived trait expression across control levels and scenarios. To assess its reliability, we conducted cross-model comparisons between Claude Sonnet 4.5 and GPT-4.1, human perceptual evaluations, and judge-prompt sensitivity analyses. The two AI evaluators show strong agreement on high–low personality judgments (83.67–97.40% across traits), while human evaluations confirm that intended personality differences are perceptible (65.9% of non-tie judgments). However, prompt sensitivity indicates that absolute scores depend partly on evaluation design.
Reviewer Concern:
W6: From the results, we can observe the clear improvements of OCEANDialogue for Openness, Conscientiousness and Extraversion, except Agreeableness and Neuroticism, and thus I want to know the reason about them.
Q9: Improvements are clear mainly for Openness, Conscientiousness, and Extraversion, while Agreeableness does not improve and Neuroticism performs worse. Can you explain why, and provide details about the method achieves effective control across all five dimensions? If not, can the claims be revised to reflect the current scope?
Response:
We have expanded the Discussion to explain the differences across traits and clarify the scope of our claims. For Agreeableness, zero-shot prompting already provides relatively strong graded control, which potentially limits further improvement. For Neuroticism, prior research suggests that personality expression is particularly sensitive to situational context, including stressful interactions. We also acknowledge that our counterfactual objective distinguishes opposing personality conditions but does not explicitly enforce monotonicity across intermediate values. We have revised our claims to emphasize that OCEANDialogue significantly improves continuous control for Openness, Conscientiousness, and Extraversion, but does not consistently outperform zero-shot prompting for Agreeableness and Neuroticism.
Response to Reviewer 4Nww (Part 3/3)
Response to Reviewer 4Nww (Part 3/3)
Reviewer Concern:
Q2: What specific information does OCEAN capture that the raw description does not, and does the two-stage pipeline lose or distort any description-specific detail?
Response:
OCEAN represents personality along five continuous dimensions, providing an interpretable representation of personality traits inferred from natural-language descriptions. However, character descriptions may contain specific behavioral details that are not fully captured by these five dimensions.
Our direct-conditioning comparison supports this trade-off. OCEAN-based conditioning achieves significantly better OCEAN fidelity, appropriateness, and naturalness, whereas direct-description conditioning achieves significantly higher description fidelity. No statistically significant difference is observed in behavioral signature. These results indicate that OCEAN provides a structured representation for personality control but does not preserve all character-specific information from the original descriptions.
Reviewer Concern:
Q7: The development set, retrieval corpus, dialogue training data, and confirmatory set are built in rather different ways, and several components rely on LLM-generated descriptions/responses. Can you analyze how these distributional differences affect generalization (e.g., train-on-one-source, test-on-another)?
W4: The experimental settings are different, such as retrieval corpus and dialogue training data, and several components rely on LLM-generated descriptions or responses, please give detailed illustration about them.
Response:
We added a table to Appendix summarizing all data sources. We clarified the differences among the data sources and examined cross-source generalization through the confirmatory evaluation. NaturalOCEAN's configuration and calibration slopes are determined on the development set, which contains LLM-generated descriptions and reference scores. The confirmatory holdout uses descriptions generated by a different LLM and crowdsourced personality ratings. We retain the development-set calibration slopes and apply label-free batch centering without using holdout reference scores. Calibrated R² improves for four of the five traits on the holdout, providing evidence of transfer across these data sources.
OCEANDialogue is trained on Llama-generated responses and evaluated on held-out conversational contexts using independent LLM judges and human raters. While these evaluations demonstrate generalization across different data sources, they do not isolate the effects of individual distribution shifts. Evaluation using human-written character descriptions remains a limitation.
Response to Reviewer 4Nww (Part 1/3)
Response to Reviewer 4Nww (Part 1/3)
We thank the reviewer for their thoughtful feedback and suggestions. We have substantially revised the paper to address their concerns. The revised material is highlighted in blue in the paper. Our responses to individual comments are as follows.
Reviewer Concern:
Response:
We use OCEAN as an intermediate representation not because it is necessary for generating dialogue, but because it provides a structured and interpretable way to control personality. Different character descriptions can be mapped into the same five-dimensional personality space, regardless of their wording. Once a description is converted into an OCEAN profile, we can adjust individual traits without rewriting the original description. This also allows us to separate personality inference from dialogue generation.
To evaluate this approach, we compared direct-description prompting, direct OCEAN prompting, and our learned OCEAN controller. The learned controller achieves the highest high–low effects for four of the five traits and the highest Spearman correlations for two. We also compared the complete direct-description and OCEAN-based pipelines using 100 characters across five conversational scenarios, resulting in 500 matched response pairs. OCEAN-based conditioning achieves significantly better OCEAN fidelity (−0.414), appropriateness (−0.587), and naturalness (−0.613), whereas direct-description conditioning achieves significantly higher description fidelity (+0.478).
These results also highlight a trade-off. While OCEAN provides a compact, interpretable representation for personality control, it does not preserve all the character-specific details in the original descriptions. Our goal is not to replace direct-description conditioning in all applications, but to enable systematic and reusable control over individual personality traits.
We have included the reason behind using OCEAN in the Introduction and specified the trade-off as a limitation in the Discussion.
Reviewer Concern:
Response:
We have added comparisons with methods that support continuous personality control, Persona Vector and PsychAdapter. We evaluate all three systems under the same conditions using 100 characters and six control levels for each OCEAN trait. OCEANDialogue achieves a higher overall high–low difference (17.25) than Persona Vectors (5.71) and PsychAdapter (9.93), with statistically significant improvements over both baselines. It also achieves the highest Spearman correlation (0.375), with a statistically significant improvement over PsychAdapter. The trait-wise results show stronger control for Openness, Conscientiousness, and Extraversion, while Agreeableness and Neuroticism remain more challenging. We report the overall results in Section 4.2 and the trait-wise results in Appendix.
We focused our competitive evaluation on methods supporting continuous personality control. Since BIG5-CHAT uses discrete high- and low-trait conditions, we haven't included it in the evaluations.
We also included all these methods in the Related Work section and provided the technical differences.