- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This work proposes a two-stage method for continuous big five personality control in LLMs. In the first stage, it trains an encoder that maps a target profile to soft prompts for a frozen LLM, using a dialog loss and a survey loss (i.e., questionnaire answers derived from the target). The second stage applies DPO with an LLM-judge reward for naturalness and personality agreement. The method is evaluated on three backbone models using 100 dialogs (rated by LLM judges and 10 samples dialogs rated by humans) and two held-out questionnaires (TIPI and BFI-2).
Critical strengths
This work evaluates personality control on two axes in multi-turn dialogs: (i) trait expression and (ii) dialog naturalness. Prior work (e.g., persona vector) mostly reports trait scores and does not consider dialog naturalness, although many recent works on model steering show increase in perplexity. The evaluation of this work shows a tradeoff that strong steering may hide (e.g., steering reaches high trait accuracy but low on naturalness). In my opinion, this is the main strength of the paper.
Critical weaknesses
W1: The evaluation is not rigorous to support the main claims of the paper. The held-out questionnaires share items with the training ones (e.g., “Worries a lot” and “Tends to be quiet” appear in both BFI and BFI-2). The author should investigate how many test items overlap with the training items, and do the questionnaire results hold without them? The sample size is also too small in my opinion (i.e., 100 dialogs per method and backbone model, which is about 20 per trait). The human study has only 10 dialogs per method. With such super small test size, additional statistical tests should be conducted, confidence intervals should be reported, and if possible, test size should be increased. The proposed method is not a clear winner (mostly mixed results). For example, on Gemma-2-9B, PsychAdapter has higher trait accuracy (78.7 vs. 76.3) and prompting has higher naturalness (54.1 vs. 39.1). In the human study, the proposed method has the best trait accuracy by only 0.2 points over steering.
W2: The baselines may be weaker than the ones in the literature. The reason could be that many important details about baselines are missing. For example, the paper does not provide the prompt, variant, or score scale used for the big5-scaler baseline. Stronger prompting methods also exist, such as the adjective and qualifier prompts (https://www.nature.com/articles/s42256-025-01115-6) and (https://arxiv.org/abs/2206.07550). The steering strength also seems too high. Its scale is not reported. Moreover, this work adds vectors to the middle half of the layers, while many recent works steer one layer.
W3: In my opinion, the survey loss is the only novel contribution of this work, and its efficacy is only partly isolated. The encoder is conditional prompt tuning, which has been studied in the literature. The DPO has also been used to shape personality (e.g., big5-chat). The survey loss is ablated for the dialog metrics on one backbone model only. The questionnaire results need this ablation, since the survey loss trains on the same kind of answers. What are the questionnaire results without the survey loss? The authors can also explore whether adding the survey loss to PsychAdapter or DPG close the gap?
Questions for authors
Please see weakness section.
3 comments
Q7. Baseline Implementation and Experimental Settings
Response. To clarify the reproducibility and fairness of baseline comparisons, we provide the shared experimental settings and method-specific configurations.
Common settings. All backbone models were frozen. The trained methods used the same 46,229-sample dialogue corpus and 3,000 training steps, with an effective batch size of 8 (batch size 2 × gradient accumulation 4), AdamW (weight decay 0.01), bf16, and seed 42. Dialogue generation used 10 turns, up to 120 new tokens per turn, temperature 0.7, top-p 0.9, repetition penalty 1.15, and no-repeat 4-gram. Each method was evaluated on 100 dialogues per backbone (10 per topic).
Method-specific configurations.
| Method | Configuration |
|---|---|
| Prompting (Big5-scaler) | Big Five scores provided as persona text (simple format); no training |
| Steering | Covariance-based directions from 300 random profiles, using trait scores and final-token hidden states; unit-normalized and injected into the residual stream at 25–75% layer depth; coefficients 1.0 / 16 / 1.0 (Llama/Gemma/OLMo) |
| PsychAdapter | Two-layer MLP (5 → 512 → 512 → layers × hidden), tanh-bounded residual injections at 25–75% layer depth; scale 8; learning rate 5e-4 |
| DPG | Hypernetwork generating rank-8 LoRA for attention output projections (o_proj) across all layers; scale 0.5; learning rate 2e-4 |
| FATE-5 | Stage 1: 64-token soft-prompt encoder trained for 3,000 steps, with dialogue/survey loss weights 0.6/0.4 after 300 warm-up steps; training questionnaires BFI and IPIP-NEO-120. Stage 2: DPO with LoRA r = 16, α = 32, β = 0.1, and 200 training steps |
TIPI and BFI-2 were held out from training.
Most of these implementation details, including baseline architectures, training hyperparameters, and generation settings, are already provided in Appendix B. We refer the reviewer to Appendix B for further details.
Q3. Steering Coefficient Sweep (Naturalness–Accuracy Trade-off)
We provide additional experimental results and a detailed discussion of this point in our response to Reviewer wGWb (Q3).
Q8. Ablation on Questionnaire-based Evaluation
Response. We extended the ablation study in Section 5.4 (Table 3) to questionnaire-based evaluation on Llama-3-8B. Results include recovery RMSE (95% CI), monotonicity (rigid/natural), and dialogue trait accuracy/naturalness.
| Variant | RMSE [95% CI] | Monotonicity r | Trait / Nat. |
|---|---|---|---|
| FATE-5 | 10.5 [8.8, 12.5] | 0.99 / 0.99 | 75.3 / 43.0 |
| − Survey loss | 23.1 [20.1, 25.8] | 0.87 / 0.90 | 51.3 / 34.2 |
| − Encoder (+DPO) | 24.0 [21.4, 26.6] | 0.78 / 0.87 | 60.0 / 48.8 |
| − Encoder − DPO | 24.6 [21.2, 28.1] | 0.33 / 0.79 | 69.0 / 46.2 |
| − DPO | 10.7 [9.2, 12.5] | 0.99 / 0.99 | 71.3 / 37.5 |
| λ = 0 | 13.3 [11.5, 15.3] | 0.98 / 0.98 | 62.7 / 19.4 |
| λ = 2 | 10.4 [8.7, 12.6] | 0.99 / 0.99 | 78.7 / 32.5 |
| DPO → SFT | 11.1 [9.6, 13.0] | 0.99 / 0.99 | 69.7 / 41.3 |
Interpretation. Removing survey loss or the encoder increases RMSE from 10.5 to 23.1 or 24.0, respectively, demonstrating their importance within FATE-5. Removing DPO barely affects recovery (10.7) but reduces naturalness (43.0 → 37.5), supporting its role in improving dialogue quality.
Reward sensitivity. λ = 2 improves trait accuracy (78.7) but reduces naturalness (32.5). Across four seeds, λ = 0.2 achieved 75.0±2.9 / 39.7±1.0 versus 72.9±3.0 / 40.1±2.2 for λ = 1, with similar RMSE (10.5 vs. 10.4).
Revisions. We will extend Section 5.4 with these questionnaire-based ablations.
Q9. Survey Loss Applied to PsychAdapter and DPG
Response. We applied FATE-5's survey supervision to PsychAdapter and DPG across three backbones, using identical training data, steps, and loss scheduling (0.6 dialogue + 0.4 survey after 300 warm-up steps). Two training questionnaires were used; TIPI and BFI-2 remained held out.
Results. Arrows indicate performance before → after survey supervision. Columns report recovery RMSE, rigid-phrasing monotonicity, and dialogue trait accuracy/naturalness.
| Backbone | Method | RMSE | Monotonicity r | Trait / Nat. |
|---|---|---|---|---|
| Llama | PsychAdapter + Survey | 28.7 → 9.0 | −0.10 → 0.99 | 77.0 / 14.7 → 78.3 / 7.5 |
| Llama | DPG + Survey | 28.5 → 11.6 | 0.20 → 0.98 | 66.7 / 33.8 → 78.0 / 33.7 |
| Llama | FATE-5 | 10.5 | 0.99 | 75.3 / 43.0 |
| Gemma | PsychAdapter + Survey | 39.7 → 8.5 | 0.45 → 0.99 | 78.7 / 28.2 → 76.7 / 23.2 |
| Gemma | DPG + Survey | 31.0 → 7.8 | 0.45 → 0.98 | 63.0 / 34.8 → 81.7 / 33.7 |
| Gemma | FATE-5 | 8.9 | 0.99 | 76.3 / 39.1 |
| OLMo | PsychAdapter + Survey | 27.3 → 9.9 | 0.76 → 0.98 | 69.7 / 21.0 → 71.0 / 14.1 |
| OLMo | DPG + Survey | 28.9 → 15.8 | 0.43 → 0.98 | 69.7 / 26.9 → 73.0 / 27.9 |
| OLMo | FATE-5 | 12.4 | 0.97 | 72.7 / 45.4 |
Interpretation. Survey supervision reduces baseline RMSE from approximately 27–40 to 8–16, demonstrating its transferability beyond FATE-5. Nevertheless, FATE-5 achieves higher naturalness than both survey-enhanced baselines across all backbones, supporting the complementary contributions of survey supervision and the encoder–DPO framework.
For results combining survey loss and DPO on baselines, see Q12.
Revisions. We will include these results and clarify that questionnaire-based fidelity improvements primarily arise from survey supervision.
We will incorporate these findings and clarify the distinction between questionnaire-based calibration and conversational trait expression, ensuring our claims remain aligned with the experimental evidence.
We appreciate the reviewer's careful evaluation and constructive feedback. We have conducted additional experiments and analyses to address the key concerns, including evaluation reliability, continuous personality control, and dialogue-level trait expression. We provide detailed responses below and will incorporate the additional results and clarifications into the revised manuscript.
Q6. Questionnaire Evaluation After Removing Overlapping Items
Response. To examine potential overlap between training and evaluation questionnaires, we conducted an item-level similarity analysis and re-evaluated personality recovery after removing overlapping items.
Overlap detection. We computed embedding-based cosine similarities between held-out items (TIPI and BFI-2) and training questionnaire items. We applied three exclusion thresholds:
Recovery RMSE after overlap removal. The following table reports recovery RMSE under natural phrasing. FATE-5 was evaluated at all three thresholds, while baseline results are reported for L1.
Interpretation. Removing overlapping items moderately increased FATE-5's RMSE by 2.7/1.8/3.9 points at L3 (Llama/Gemma/OLMo), indicating some sensitivity to item overlap. Nevertheless, FATE-5 maintained low recovery errors across all three backbones and outperformed every baseline under the matched L1 evaluation. At L3, FATE-5 also preserved strong monotonicity (r = 0.97–0.99), demonstrating consistent continuous personality control even after removing semantically similar items.
Revisions. We will report the overlap detection criteria and include both full-set and deduplicated questionnaire results in the revised manuscript.
Q10. Interpreting the Mixed Results
Response. FATE-5 aims not to maximize trait accuracy or naturalness individually, but to achieve a favorable balance between both. High trait accuracy is insufficient if dialogue becomes unnatural, while high naturalness is insufficient if the intended personality is not expressed.
For example, on Gemma, PsychAdapter achieves higher trait accuracy (78.7 vs. 76.3) but lower naturalness (28.2 vs. 39.1), whereas Prompting achieves higher naturalness (54.1 vs. 39.1) but lower trait accuracy (65.3 vs. 76.3). Similarly, in human evaluation, FATE-5 achieves comparable trait accuracy to Steering (78.7 vs. 78.5) with substantially higher naturalness (67.8 vs. 20.2).
Thus, our main contribution is maintaining competitive personality expression while substantially improving dialogue naturalness, rather than outperforming every baseline on each metric independently.