acceptodds
Under review as a conference paper at ICLR 2027

FATE-5: Faithful Alignment of Trait Expression for Continuous Big Five Personality Control

Abstract

Controlling personality in large language models is important for building conversational agents that can be customized to different users and interaction settings. However, existing personality control methods often face a tradeoff between strong trait expression and natural dialogue generation. We introduce FATE-5, Faithful Alignment of Trait Expression for the Big Five, a two-stage framework that separates personality encoding from naturalness alignment. FATE-5 first learns continuous soft prompt representations of Big Five profiles using dialogue supervision and an auxiliary survey objective. It then improves dialogue naturalness through preference alignment based on both dialogue quality and personality agreement. Across three language model backbones, FATE-5 maintains strong trait expression while achieving substantially higher dialogue naturalness than other trait-strong methods. On held-out questionnaires, it achieves monotonicity correlations of 0.97-0.99 and recovery RMSE of 8.9-12.4, substantially improving over the evaluated baselines. These results demonstrate effective continuous personality control while maintaining natural conversational behavior.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Personality Control, Big Five Personality, Continuous Personality Control, Preference Alignment, Psychometric Evaluation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2tGvOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This work proposes a two-stage method for continuous big five personality control in LLMs. In the first stage, it trains an encoder that maps a target profile to soft prompts for a frozen LLM, using a dialog loss and a survey loss (i.e., questionnaire answers derived from the target). The second stage applies DPO with an LLM-judge reward for naturalness and personality agreement. The method is evaluated on three backbone models using 100 dialogs (rated by LLM judges and 10 samples dialogs rated by humans) and two held-out questionnaires (TIPI and BFI-2).

Critical strengths

This work evaluates personality control on two axes in multi-turn dialogs: (i) trait expression and (ii) dialog naturalness. Prior work (e.g., persona vector) mostly reports trait scores and does not consider dialog naturalness, although many recent works on model steering show increase in perplexity. The evaluation of this work shows a tradeoff that strong steering may hide (e.g., steering reaches high trait accuracy but low on naturalness). In my opinion, this is the main strength of the paper.

Critical weaknesses

W1: The evaluation is not rigorous to support the main claims of the paper. The held-out questionnaires share items with the training ones (e.g., “Worries a lot” and “Tends to be quiet” appear in both BFI and BFI-2). The author should investigate how many test items overlap with the training items, and do the questionnaire results hold without them? The sample size is also too small in my opinion (i.e., 100 dialogs per method and backbone model, which is about 20 per trait). The human study has only 10 dialogs per method. With such super small test size, additional statistical tests should be conducted, confidence intervals should be reported, and if possible, test size should be increased. The proposed method is not a clear winner (mostly mixed results). For example, on Gemma-2-9B, PsychAdapter has higher trait accuracy (78.7 vs. 76.3) and prompting has higher naturalness (54.1 vs. 39.1). In the human study, the proposed method has the best trait accuracy by only 0.2 points over steering.

W2: The baselines may be weaker than the ones in the literature. The reason could be that many important details about baselines are missing. For example, the paper does not provide the prompt, variant, or score scale used for the big5-scaler baseline. Stronger prompting methods also exist, such as the adjective and qualifier prompts (https://www.nature.com/articles/s42256-025-01115-6) and P2P^2 (https://arxiv.org/abs/2206.07550). The steering strength also seems too high. Its scale is not reported. Moreover, this work adds vectors to the middle half of the layers, while many recent works steer one layer.

W3: In my opinion, the survey loss is the only novel contribution of this work, and its efficacy is only partly isolated. The encoder is conditional prompt tuning, which has been studied in the literature. The DPO has also been used to shape personality (e.g., big5-chat). The survey loss is ablated for the dialog metrics on one backbone model only. The questionnaire results need this ablation, since the survey loss trains on the same kind of answers. What are the questionnaire results without the survey loss? The authors can also explore whether adding the survey loss to PsychAdapter or DPG close the gap?

Questions for authors

Please see weakness section.

3 comments

We appreciate the reviewer's careful evaluation and constructive feedback. We have conducted additional experiments and analyses to address the key concerns, including evaluation reliability, continuous personality control, and dialogue-level trait expression. We provide detailed responses below and will incorporate the additional results and clarifications into the revised manuscript.


Q6. Questionnaire Evaluation After Removing Overlapping Items

Response. To examine potential overlap between training and evaluation questionnaires, we conducted an item-level similarity analysis and re-evaluated personality recovery after removing overlapping items.

Overlap detection. We computed embedding-based cosine similarities between held-out items (TIPI and BFI-2) and training questionnaire items. We applied three exclusion thresholds:

  • L1 (≥ 0.99): 19 BFI-2 items removed (semantic overlap).
  • L2 (≥ 0.80): 23 BFI-2 items removed.
  • L3 (≥ 0.70): 35 BFI-2 and 2 TIPI items removed.

Recovery RMSE after overlap removal. The following table reports recovery RMSE under natural phrasing. FATE-5 was evaluated at all three thresholds, while baseline results are reported for L1.

MethodLlamaGemmaOLMo
FATE-5 (Full → L1 → L2 → L3)10.5 → 11.1 → 11.7 → 13.18.9 → 9.5 → 9.7 → 10.712.4 → 14.1 → 14.2 → 17.4
Prompting (Full → L1)20.7 → 25.118.0 → 20.425.4 → 25.9
Steering (Full → L1)28.8 → 34.428.6 → 36.426.8 → 32.7
PsychAdapter (Full → L1)28.8 → 32.839.8 → 42.327.3 → 29.3
DPG (Full → L1)28.5 → 29.830.8 → 33.128.7 → 31.3

Interpretation. Removing overlapping items moderately increased FATE-5's RMSE by 2.7/1.8/3.9 points at L3 (Llama/Gemma/OLMo), indicating some sensitivity to item overlap. Nevertheless, FATE-5 maintained low recovery errors across all three backbones and outperformed every baseline under the matched L1 evaluation. At L3, FATE-5 also preserved strong monotonicity (r = 0.97–0.99), demonstrating consistent continuous personality control even after removing semantically similar items.

Revisions. We will report the overlap detection criteria and include both full-set and deduplicated questionnaire results in the revised manuscript.


Q10. Interpreting the Mixed Results

Response. FATE-5 aims not to maximize trait accuracy or naturalness individually, but to achieve a favorable balance between both. High trait accuracy is insufficient if dialogue becomes unnatural, while high naturalness is insufficient if the intended personality is not expressed.

For example, on Gemma, PsychAdapter achieves higher trait accuracy (78.7 vs. 76.3) but lower naturalness (28.2 vs. 39.1), whereas Prompting achieves higher naturalness (54.1 vs. 39.1) but lower trait accuracy (65.3 vs. 76.3). Similarly, in human evaluation, FATE-5 achieves comparable trait accuracy to Steering (78.7 vs. 78.5) with substantially higher naturalness (67.8 vs. 20.2).

Thus, our main contribution is maintaining competitive personality expression while substantially improving dialogue naturalness, rather than outperforming every baseline on each metric independently.

Q7. Baseline Implementation and Experimental Settings

Response. To clarify the reproducibility and fairness of baseline comparisons, we provide the shared experimental settings and method-specific configurations.

Common settings. All backbone models were frozen. The trained methods used the same 46,229-sample dialogue corpus and 3,000 training steps, with an effective batch size of 8 (batch size 2 × gradient accumulation 4), AdamW (weight decay 0.01), bf16, and seed 42. Dialogue generation used 10 turns, up to 120 new tokens per turn, temperature 0.7, top-p 0.9, repetition penalty 1.15, and no-repeat 4-gram. Each method was evaluated on 100 dialogues per backbone (10 per topic).

Method-specific configurations.

MethodConfiguration
Prompting (Big5-scaler)Big Five scores provided as persona text (simple format); no training
SteeringCovariance-based directions from 300 random profiles, using trait scores and final-token hidden states; unit-normalized and injected into the residual stream at 25–75% layer depth; coefficients 1.0 / 16 / 1.0 (Llama/Gemma/OLMo)
PsychAdapterTwo-layer MLP (5 → 512 → 512 → layers × hidden), tanh-bounded residual injections at 25–75% layer depth; scale 8; learning rate 5e-4
DPGHypernetwork generating rank-8 LoRA for attention output projections (o_proj) across all layers; scale 0.5; learning rate 2e-4
FATE-5Stage 1: 64-token soft-prompt encoder trained for 3,000 steps, with dialogue/survey loss weights 0.6/0.4 after 300 warm-up steps; training questionnaires BFI and IPIP-NEO-120. Stage 2: DPO with LoRA r = 16, α = 32, β = 0.1, and 200 training steps

TIPI and BFI-2 were held out from training.

Most of these implementation details, including baseline architectures, training hyperparameters, and generation settings, are already provided in Appendix B. We refer the reviewer to Appendix B for further details.


Q3. Steering Coefficient Sweep (Naturalness–Accuracy Trade-off)

We provide additional experimental results and a detailed discussion of this point in our response to Reviewer wGWb (Q3).

Q8. Ablation on Questionnaire-based Evaluation

Response. We extended the ablation study in Section 5.4 (Table 3) to questionnaire-based evaluation on Llama-3-8B. Results include recovery RMSE (95% CI), monotonicity (rigid/natural), and dialogue trait accuracy/naturalness.

VariantRMSE [95% CI]Monotonicity rTrait / Nat.
FATE-510.5 [8.8, 12.5]0.99 / 0.9975.3 / 43.0
− Survey loss23.1 [20.1, 25.8]0.87 / 0.9051.3 / 34.2
− Encoder (+DPO)24.0 [21.4, 26.6]0.78 / 0.8760.0 / 48.8
− Encoder − DPO24.6 [21.2, 28.1]0.33 / 0.7969.0 / 46.2
− DPO10.7 [9.2, 12.5]0.99 / 0.9971.3 / 37.5
λ = 013.3 [11.5, 15.3]0.98 / 0.9862.7 / 19.4
λ = 210.4 [8.7, 12.6]0.99 / 0.9978.7 / 32.5
DPO → SFT11.1 [9.6, 13.0]0.99 / 0.9969.7 / 41.3

Interpretation. Removing survey loss or the encoder increases RMSE from 10.5 to 23.1 or 24.0, respectively, demonstrating their importance within FATE-5. Removing DPO barely affects recovery (10.7) but reduces naturalness (43.0 → 37.5), supporting its role in improving dialogue quality.

Reward sensitivity. λ = 2 improves trait accuracy (78.7) but reduces naturalness (32.5). Across four seeds, λ = 0.2 achieved 75.0±2.9 / 39.7±1.0 versus 72.9±3.0 / 40.1±2.2 for λ = 1, with similar RMSE (10.5 vs. 10.4).

Revisions. We will extend Section 5.4 with these questionnaire-based ablations.


Q9. Survey Loss Applied to PsychAdapter and DPG

Response. We applied FATE-5's survey supervision to PsychAdapter and DPG across three backbones, using identical training data, steps, and loss scheduling (0.6 dialogue + 0.4 survey after 300 warm-up steps). Two training questionnaires were used; TIPI and BFI-2 remained held out.

Results. Arrows indicate performance before → after survey supervision. Columns report recovery RMSE, rigid-phrasing monotonicity, and dialogue trait accuracy/naturalness.

BackboneMethodRMSEMonotonicity rTrait / Nat.
LlamaPsychAdapter + Survey28.7 → 9.0−0.10 → 0.9977.0 / 14.7 → 78.3 / 7.5
LlamaDPG + Survey28.5 → 11.60.20 → 0.9866.7 / 33.8 → 78.0 / 33.7
LlamaFATE-510.50.9975.3 / 43.0
GemmaPsychAdapter + Survey39.7 → 8.50.45 → 0.9978.7 / 28.2 → 76.7 / 23.2
GemmaDPG + Survey31.0 → 7.80.45 → 0.9863.0 / 34.8 → 81.7 / 33.7
GemmaFATE-58.90.9976.3 / 39.1
OLMoPsychAdapter + Survey27.3 → 9.90.76 → 0.9869.7 / 21.0 → 71.0 / 14.1
OLMoDPG + Survey28.9 → 15.80.43 → 0.9869.7 / 26.9 → 73.0 / 27.9
OLMoFATE-512.40.9772.7 / 45.4

Interpretation. Survey supervision reduces baseline RMSE from approximately 27–40 to 8–16, demonstrating its transferability beyond FATE-5. Nevertheless, FATE-5 achieves higher naturalness than both survey-enhanced baselines across all backbones, supporting the complementary contributions of survey supervision and the encoder–DPO framework.

For results combining survey loss and DPO on baselines, see Q12.

Revisions. We will include these results and clarify that questionnaire-based fidelity improvements primarily arise from survey supervision.


We will incorporate these findings and clarify the distinction between questionnaire-based calibration and conversational trait expression, ensuring our claims remain aligned with the experimental evidence.

Reviewer_9Az6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FATE-5, a two-stage approach for controlling Big Five personality traits in LLMs while maintaining dialogue naturalness. The first stage trains a personality-conditioned soft prompt encoder using dialogue supervision and an auxiliary questionnaire objective. The second stage applies DPO with a reward that combines naturalness and agreement with the target personality profile.

The authors evaluate FATE-5 on three model backbones using multi-turn dialogues, LLM and human judgments, and held-out personality inventories. The method achieves strong monotonicity and substantially lower questionnaire recovery error than the evaluated baselines. The dialogue results also suggest that FATE-5 offers a better balance between trait expression and naturalness than several of the tested intervention methods.

Critical strengths
  • The results on continuous personality control are strong and consistent across the three evaluated backbones. FATE-5 achieves monotonicity correlations of 0.97–0.99 and substantially lower recovery RMSE than the baselines on held-out questionnaires. Using different inventories for training and evaluation is a reasonable choice and provides evidence that the model can transfer its learned questionnaire response behavior to unseen items.

  • I appreciate that the paper evaluates both personality expression and dialogue naturalness, rather than relying only on questionnaire scores. The human evaluation is a useful addition, and the ablations provide evidence that survey supervision and preference alignment contribute differently to the final results. In particular, the improvement from the encoder-only model to the full model suggests that preference alignment can recover some naturalness without losing the learned personality control.

Critical weaknesses
  • Confounded baseline comparisons. FATE-5 uses additional survey supervision and DPO, while the trained baselines receive only dialogue supervision. The reported improvements therefore cannot be clearly attributed to the proposed architecture. Some baselines also differ from their original implementations, particularly the covariance-based steering method used in place of PERSONA's original approach.

  • Limited validation of continuous personality control in natural dialogue. The strongest results come from questionnaire evaluations, which are closely related to the survey supervision used during training. The dialogue evaluation only measures relative trait ordering between two agents, rather than whether their conversations reflect the intended continuous trait intensities or full five-dimensional profiles. Thus, the claimed fidelity of continuous personality control in open-ended dialogue remains insufficiently validated.

Questions for authors
  • Can the authors provide a baseline comparison in which the same survey objective and DPO procedure are applied to other conditioning architectures? This would help determine whether the main advantage comes from the soft prompt encoder or from the additional training objectives.

  • How closely do the reimplemented steering and PsychAdapter baselines follow their original methods? In particular, why is covariance-based steering used instead of the direction extraction proposed in PERSONA? Have the authors evaluated different steering strengths to compare the full trait accuracy–naturalness trade-off?

  • Can the authors evaluate recovery of continuous Big Five profiles directly from generated conversations, rather than from questionnaire responses? Ideally, this would include profiles where several traits vary simultaneously, with independent raters estimating their intensities without access to the target scores.

2 comments

We appreciate the reviewer's careful evaluation and constructive feedback. We have conducted additional experiments and analyses to address the key concerns, including evaluation reliability, continuous personality control, and dialogue-level trait expression. We provide detailed responses below and will incorporate the additional results and clarifications into the revised manuscript.


Q5. DPO on Baselines and Covariance-based Steering

Response. To examine whether FATE-5's improvements stem primarily from DPO, we applied the same DPO procedure to PsychAdapter, DPG, and Steering across all three backbones. We also clarify the motivation for covariance-based Steering.

(a) DPO on baselines. We used the same preference construction and optimization settings as FATE-5: Llama-3.1-70B judge, 200 profiles × 5 candidates, reward = naturalness − λ·RMSE (λ = 1), LoRA (r = 16, α = 32), β = 0.1, and 200 training steps.

The table reports trait accuracy/naturalness before and after DPO, along with the naturalness difference between FATE-5 and each DPO-enhanced baseline (95% CI).

BackboneMethodOriginal+DPOFATE-5 − (+DPO) Nat. [95% CI]
LlamaPsychAdapter77.0 / 14.778.3 / 10.9+32.0 [28.1, 36.1]
LlamaDPG66.7 / 33.869.0 / 34.3+8.9 [5.2, 12.6]
LlamaSteering77.3 / 2.678.0 / 1.2+41.8 [38.2, 45.6]
GemmaPsychAdapter78.7 / 28.278.0 / 26.3+13.0 [8.4, 17.7]
GemmaDPG63.0 / 34.870.3 / 43.0−4.2 [−8.3, 0.2]
GemmaSteering74.3 / 7.978.0 / 8.1+31.6 [27.5, 35.7]
OLMoPsychAdapter69.7 / 21.077.3 / 18.4+27.1 [23.5, 30.8]
OLMoDPG69.7 / 26.974.3 / 27.5+19.0 [15.1, 23.1]
OLMoSteering80.3 / 21.672.0 / 25.5+20.2 [16.2, 24.2]

FATE-5 achieves 75.3/43.0 (Llama), 76.3/39.1 (Gemma), and 72.7/45.4 (OLMo).

DPO did not consistently improve baseline naturalness. FATE-5 retained a substantial naturalness advantage over DPO-enhanced Steering and PsychAdapter across all three backbones, and over DPG on Llama and OLMo. An exception was Gemma DPG, whose naturalness increased from 34.8 to 43.0 after DPO, exceeding FATE-5's 39.1, although its trait accuracy remained lower (70.3 vs. 76.3). The paired trait-accuracy differences between FATE-5 and DPO-enhanced baselines were not statistically significant.

These results suggest that FATE-5's naturalness improvements cannot be attributed solely to applying DPO, although the effect of DPO depends on the conditioning architecture. For results obtained by applying both survey loss and DPO to the baselines, please refer to Q12.

(b) Why covariance-based Steering? We adopted covariance-based directions to support continuous five-dimensional Big Five control. Unlike difference-of-means approaches requiring discrete high/low groups, covariance-based directions can be scaled directly by continuous target scores. This approach also requires no training and provides an activation-editing baseline alongside PsychAdapter and DPG.

The original PERSONA implementation was unavailable to us; therefore, we followed the published description but do not claim exact implementation equivalence. To further assess the effect of steering strength, we conducted the coefficient sweep reported in Q3.

Revisions. We will add the DPO-enhanced baseline results, clarify the covariance-based Steering implementation, and report the steering coefficient analysis in the revised manuscript.

Q2. Big Five Estimation from Dialogue Text

We provide additional experimental results and a detailed discussion of this point in our response to Reviewer wGWb (Q2).


Q12. Fully Matched Baseline Comparison with Survey Supervision and DPO

Response. To disentangle the contributions of the conditioning architecture and additional training objectives, we conducted a fully matched comparison by applying both survey supervision and DPO to PsychAdapter and DPG on Llama and Gemma. We evaluated questionnaire recovery RMSE, dialogue trait accuracy, and naturalness under the same evaluation framework as FATE-5.

Results. The table reports dialogue trait accuracy/naturalness, questionnaire recovery RMSE, and paired differences between FATE-5 and each baseline with 95% confidence intervals.

BackboneMethodTrait / Nat.RMSE ↓Δ Trait [95% CI]Δ Nat. [95% CI]
GemmaDPG + Survey + DPO77.7 / 34.98.0+0.4 [−7.7, 8.5]+5.4 [0.9, 10.3]
GemmaPsychAdapter + Survey + DPO76.0 / 29.28.3+0.9 [−9.8, 11.5]+10.0 [5.6, 14.5]
LlamaDPG + Survey + DPO75.3 / 30.49.60.0 [−9.5, 9.5]+12.6 [8.8, 16.8]
LlamaPsychAdapter + Survey + DPO79.7 / 9.610.2−5.3 [−14.0, 3.5]+33.2 [29.6, 36.9]
GemmaFATE-576.3 / 39.18.9——
LlamaFATE-575.3 / 43.010.4——

Interpretation. Applying the same survey supervision and DPO objectives to alternative conditioning architectures substantially improves questionnaire recovery, with RMSE values ranging from 8.0 to 10.2. Some baselines achieve lower RMSE than FATE-5, indicating that survey-based continuous personality calibration is not exclusive to our architecture.

However, FATE-5 consistently achieves higher dialogue naturalness than all four fully matched baselines, with improvements of 5.4–33.2 points. All corresponding paired 95% confidence intervals exclude zero. In contrast, trait-accuracy differences are not statistically significant, suggesting comparable trait expression across the evaluated methods.

These findings directly address the concern that FATE-5's improvements might arise solely from additional survey supervision and DPO. Even when both objectives are applied to alternative architectures, FATE-5 maintains a significant naturalness advantage while achieving competitive trait accuracy and strong questionnaire calibration.

Overall, the results support the effectiveness of FATE-5's conditioning and decoupled alignment design in balancing continuous personality control with natural dialogue generation, rather than claiming superiority in every individual metric.


Q3. Steering Coefficient Sweep (Naturalness–Accuracy Trade-off)

We provide additional experimental results and a detailed discussion of this point in our response to Reviewer wGWb (Q3).


We hope that the additional experiments and clarifications address the reviewer's concerns and provide a clearer understanding of FATE-5's contributions. We will revise the manuscript accordingly, particularly to clarify the distinction between questionnaire-based calibration and personality expression in natural dialogue, and to ensure that our claims accurately reflect the experimental evidence.

Reviewer_wGWbOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper study how to impose continuous Big Five personality control on a language model so that generated dialogue both expresses the target traits and remains natural. The authors argue that existing approaches face an unresolved trade-off between the strength of trait expression and dialogue naturalness: prompting yields weak expression, training-based methods struggle to match a continuous intensity, and activation steering achieves strong control at the cost of naturalness.

Critical strengths
  1. Having a conversational agent adjust its personality to a specified profile is a real requirement for controllable dialogue systems, whereas prior work reports mostly the strength of trait expression and measures dialogue quality relatively rarely. The paper compare four families of methods on three backbones through ten-turn dyadic dialogues, and reports that activation-steering methods drop to 2.6, 7.9 and 21.6 in naturalness. This diagnosis is valuable in its own right and is the most solid part of the paper's argument.

  2. The two-stage decoupling is clearly designed and yields a reusable supervisory signal. The first stage carry the target profile in a soft prompt while the backbone is frozen; the second stage trains only a low-rank adapter, so that personality encoding and naturalness alignment are handled by different modules. This keeps the method parameter-efficient and allows it to transfer across backbones without retraining the backbone. The survey loss constructs pseudo-answers by inverting the scoring rubric, turning a non-differentiable instrument into token-level supervision; Table 3 shows that removing it lowers trait accuracy from 75.3 to 51.3, a larger drop than removing DPO or replacing it with SFT, which indicates that the gain is attributable to the loss itself rather than to the training budget.

Critical weaknesses
  1. Section 3.1 states the goal as generated dialogue faithfully expressing the target profile, yet faithfulness is operationalised only through questionnaire self-report. Training optimises the model to produce answers that the rubric maps back to the target score; evaluation measures answers that the same family of rules maps back to a score. The held-out inventories change the items and the wording, not the channel of item-by-item answering followed by rubric aggregation. Trait accuracy in Table 1 is a pairwise comparison within a single dialogue and measures differential expression rather than absolute recovery. Fidelity to a personality is therefore not separated from fidelity to a rubric, and every piece of evidence for the word faithful in the abstract and the conclusion comes from that channel.

  2. The evaluation covers a single backbone with ten dialogues per method; the reported maximum of 78.7 exceeds the runner-up value of 78.5 by less than one rater judgement, and no interval is given for the method means. Interchangeability of the two judge types is likewise unproven: the top two on naturalness swap between them, and the r=0.84 reported in Appendix C is dominated by between-method differences. The precision of the two columns is also unequal: single-rater agreement is α = 0.45 to 0.49, the human column aggregates about twenty raters per dialogue while the LLM column uses three judges, and a Spearman–Brown correction implies per-dialogue reliabilities of about 0.90 to 0.94 for the human column and between 0.57 and 0.70 for the LLM column.

Questions for authors
  1. Can the authors estimate the target profile from the evaluation dialogues by text alone, using the judge of Equation 6, and report its error and monotonicity alongside Table 2? If the result is substantially weaker than 8.9–12.4, does the scope of the word faithful need to be narrowed?

  2. Table 3 show that with the reward weight set to 2, the Llama result already surpasses both Steering and PsychAdapter, whereas the paper reports the weight-1 setting as its main configuration. Can the authors plot the three operating points in the naturalness–accuracy plane, extend the weight-2 setting to the other two backbones, and sweep the Steering injection coefficient over several values from the grid, so as to establish whether some baseline operating point dominates all FATE-5 operating points?

  3. Do all methods share the same set of profile pairs in the LLM-judged evaluation? If not, can the authors report accuracy stratified by the gap between the two speakers' target scores, and add confidence intervals and multi-seed results? This bears directly on whether the few-point differences in Table 1 are interpretable.

  4. The human evaluation use the same ten persona and topic configurations for all five methods, which makes it a paired design, with ten dialogues per method. Can the authors report confidence intervals for the method means clustered by dialogue, and the paired differences between methods on the same configurations together with their significance?

4 comments

We appreciate the reviewer's careful evaluation and constructive feedback. We have conducted additional experiments and analyses to address the key concerns, including evaluation reliability, continuous personality control, and dialogue-level trait expression. We provide detailed responses below and will incorporate the additional results and clarifications into the revised manuscript.


Q11. Distinguishing Personality Fidelity from Questionnaire Fidelity

In this work, faithful personality control refers to calibrated tracking of continuous personality targets under held-out psychometric instruments, together with perceptible expression of the conditioned traits in natural dialogue. Our evaluation assesses these two complementary aspects rather than assuming that questionnaire performance alone establishes personality fidelity across all conversational settings.

We acknowledge that questionnaire training and evaluation share a psychometric framework. However, they use different inventories and response formats, and our additional overlap-removal experiments (Q6) show that FATE-5 maintains strong recovery accuracy and monotonicity even after semantically similar items are removed. These results support generalization beyond overlapping questionnaire items.

Importantly, Table 1 evaluates personality expression directly in multi-turn dialogue, independently of questionnaire responses. While this evaluation measures relative trait expression rather than absolute profile recovery, it provides complementary evidence that personality conditioning influences conversational behavior rather than merely questionnaire answering.

Together, these results support two complementary aspects of personality control: calibrated continuous score tracking under held-out psychometric instruments and perceptible relative trait expression in natural dialogue. As discussed in Q2, these findings do not establish exact absolute-score fidelity in open-ended dialogue. We will explicitly distinguish these two evaluation settings and clarify the corresponding scope of our claims in the revised manuscript.

Q1. Reliability of Dialogue Evaluation

Response. Appendix C reports human inter-rater agreement, split-half reliability, and human–LLM agreement. We additionally analyzed LLM judge reliability and computed 95% confidence intervals (CIs) and paired comparisons.

(a) LLM judge reliability. We evaluated 500 dialogues per backbone (five methods pooled).

BackboneTrait αTrait κNat. αNat. ICC(2,1)Nat. ICC(2,3)
Llama0.670.670.530.560.79
Gemma0.700.700.560.580.81
OLMo0.720.720.430.460.72

Individual judges agreed with the three-judge majority on trait judgments in 93–94% of dialogues. After accounting for between-method differences, naturalness ICC(2,3) remained 0.70–0.72, supporting aggregated judge scores.

(b) Human reliability and agreement. Appendix C reports split-half reliability of 0.90 [0.86, 0.94] for trait and 0.94 [0.92, 0.96] for naturalness. Human–LLM agreement was κ = 0.79 (94% agreement) for trait and r = 0.83 for naturalness (within-method r = 0.61), indicating meaningful but imperfect agreement.

(c) Per-method means with 95% CIs (Llama-3-8B). Human scores average individual ratings over 10 dialogues per method; naturalness is rescaled to 0–100. LLM scores average three judges over 100 dialogues. CIs use 5,000 dialogue-level bootstrap resamples.

MethodHuman TraitHuman Nat.LLM TraitLLM Nat.
FATE-578.7 [61.1, 89.8]67.8 [63.8, 71.9]75.3 [67.7, 82.7]43.0 [39.4, 46.7]
big5-scaler73.3 [53.1, 89.5]74.4 [69.1, 78.5]63.7 [55.0, 72.0]39.4 [37.1, 41.6]
Steering78.5 [68.6, 86.6]20.2 [16.6, 24.1]77.3 [70.3, 84.0]2.6 [1.8, 3.4]
PsychAdapter71.2 [55.1, 84.8]53.6 [49.1, 58.0]77.0 [69.3, 84.3]14.7 [12.9, 16.5]
DPG69.9 [48.7, 87.9]66.2 [58.5, 73.4]66.7 [58.7, 74.7]33.8 [31.0, 36.7]

Human trait CIs overlap across methods, preventing a reliable trait-accuracy ranking.

(d) Paired comparisons. All methods were evaluated on the same 10 speaker-profile-pair/topic settings. Differences were computed from setting-averaged ratings; 95% CIs used 10,000 paired bootstrap resamples, with significance assessed by exact sign-flip tests. The final columns use 95 matched LLM-evaluated dialogues (see Q4). Values are FATE-5 − baseline [95% CI].

vs.Human Trait (10)Human Nat. (10)LLM Trait (10)LLM Nat. (10)LLM Trait (95)LLM Nat. (95)
big5-scaler+4.7 [−21.6, 31.5]−6.3 [−13.5, 1.2]+16.7 [−13.3, 46.7]−9.2 [−20.8, 5.0]+11.9 [1.4, 22.8]+3.3 [−1.2, 7.9]
Steering+1.9 [−17.3, 20.8]+47.8 [40.7, 54.2]+3.3 [−26.7, 26.7]+27.5 [23.3, 31.7]−3.2 [−12.6, 6.7]+40.5 [36.8, 44.4]
PsychAdapter+7.3 [−11.3, 28.5]+14.4 [8.6, 20.0]+3.3 [−26.7, 33.3]+13.3 [6.7, 20.0]−3.2 [−13.0, 6.7]+28.0 [23.9, 32.2]
DPG+8.7 [−9.0, 30.3]+1.7 [−5.4, 10.9]+6.7 [−10.0, 30.0]−2.5 [−14.2, 9.2]+9.5 [0.0, 18.9]+9.0 [4.9, 13.2]

On the same 10 settings, FATE-5 showed significantly higher human-rated naturalness than Steering (+47.8) and PsychAdapter (+14.4), consistent with LLM judgments (+27.5/+13.3). No significant trait-accuracy differences were established on these settings.

Revisions. We will extend Appendix C with these analyses and clarify that FATE-5 achieves competitive trait accuracy with substantially improved naturalness over Steering and PsychAdapter, rather than statistically superior trait accuracy.

Q2. Big Five Estimation from Dialogue Text

Response. To assess personality fidelity directly from dialogue, we conducted two complementary evaluations: LLM-based estimation and a separately trained personality regressor.

(a) LLM-based estimation. Llama-3.1-70B-Instruct estimated Big Five scores from generated dialogues (100 dialogues per method; RMSE with 95% bootstrap CI).

MethodLlamaGemmaOLMo
Prompting37.2 [35.7, 38.7]36.9 [35.5, 38.2]42.8 [41.5, 44.2]
Steering28.1 [26.8, 29.4]28.1 [26.8, 29.4]33.5 [32.3, 34.7]
PsychAdapter28.6 [27.5, 29.8]29.7 [28.4, 30.9]29.8 [28.6, 30.9]
DPG33.8 [32.5, 35.1]31.5 [30.3, 32.8]29.6 [28.4, 30.8]
FATE-532.1 [30.8, 33.4]31.3 [29.9, 32.7]32.4 [30.8, 34.0]

FATE-5's RMSE is comparable to the constant-50 predictor (≈32.0), indicating that accurate absolute profile recovery from dialogue is not established. However, positive target-score correlations of 0.55/0.48/0.46 (Llama/Gemma/OLMo) on the elicited traits suggest that some relative trait-intensity information is expressed.

(b) Independently trained estimator. We trained Qwen2.5-14B-Instruct with LoRA and a regression head on 1,333 PANDORA authors. On 134 held-out human authors, it achieved Pearson r = 0.30 and RMSE 29.0 (constant predictor: 30.0).

Applied to generated dialogues, the estimator produced the following elicited-trait correlations / pairwise accuracies (%):

MethodLlamaGemmaOLMo
Prompting0.00 / 590.03 / 47−0.07 / 53
Steering0.22 / 630.31 / 650.11 / 61
PsychAdapter0.22 / 650.30 / 710.22 / 59
DPG0.23 / 610.22 / 720.12 / 59
FATE-50.24 / 730.18 / 650.06 / 53

Confidence intervals are wide and substantially overlap across methods, so these results do not establish a reliable ranking.

Interpretation. We acknowledge that questionnaire-based profile recovery and personality expression in natural dialogue measure different aspects of control. Our held-out questionnaire evaluations demonstrate accurate continuous score recovery within the psychometric evaluation setting. In contrast, dialogue-based evaluations provide evidence of relative trait expression but do not establish that a target score (e.g., Extraversion = 80) is reproduced at the same absolute level in open-ended dialogue.

Accordingly, we will clarify that our evidence for continuous profile calibration is questionnaire-based, while our dialogue results support perceptible trait differences and improved naturalness. We will avoid extending questionnaire-based fidelity claims to exact absolute personality-score recovery from free dialogue.


Q3. Steering Coefficient Sweep (Naturalness–Accuracy Trade-off)

Response. To examine the naturalness–accuracy trade-off under different steering strengths, we conducted a coefficient sweep on Llama and Gemma. The steering strength is defined as:

αt=coeff×2×(scoret100−0.5)\alpha_t = \mathrm{coeff} \times 2 \times \left(\frac{\mathrm{score}_t}{100} - 0.5\right)

Table 1 used coefficients of 1.0, 16, and 1.0 for Llama, Gemma, and OLMo, respectively.

Steering coefficient sweep. The following tables report trait accuracy and naturalness across different coefficients.

Llama

Coeff2.01.00.50.250.150.05
Trait Acc.66.777.376.378.370.061.7
Naturalness0.02.69.229.642.551.3

Gemma

Coeff16128421
Trait Acc.74.375.776.073.063.059.0
Naturalness7.915.330.648.353.553.5

For comparison, FATE-5 achieves trait accuracy/naturalness of 75.3/43.0 on Llama and 76.3/39.1 on Gemma.

Interpretation. Reducing the steering coefficient substantially improves dialogue naturalness, indicating that the low naturalness reported in Table 1 is partly attributable to the high steering coefficients. Nevertheless, FATE-5 achieves higher trait accuracy and naturalness simultaneously compared with several Steering configurations. For example, Steering achieves 70.0/42.5 on Llama (coeff = 0.15) and 76.0/30.6 on Gemma (coeff = 8), both below FATE-5's corresponding results of 75.3/43.0 and 76.3/39.1. Overall, FATE-5 lies on or near the empirical accuracy–naturalness frontier, providing a competitive balance between the two objectives without necessarily dominating every Steering operating point.

Revisions. We will explicitly report the steering coefficients, include the naturalness–accuracy trade-off curves, and clarify that Table 1 reflects a high-trait steering operating point rather than the full trade-off frontier.

Q4. Speaker Score Gaps, Random Seeds, and Confidence Intervals

Response. To examine the effects of speaker score gaps and random seeds, we conducted stratified trait-accuracy analyses and multi-seed evaluations across all three backbones.

(a) Trait accuracy by speaker score gap. We grouped dialogues by the target-trait score difference between speakers (≤2 / 3–4 / ≥5 levels).

MethodLlamaGemmaOLMo
FATE-565.2 / 80.6 / 89.468.1 / 78.4 / 94.160.9 / 78.8 / 88.9
Steering64.5 / 86.9 / 91.758.9 / 86.9 / 90.069.5 / 86.5 / 95.2
PsychAdapter68.1 / 83.8 / 86.769.5 / 81.8 / 95.063.9 / 68.7 / 86.0
DPG56.0 / 73.7 / 80.049.6 / 70.7 / 81.756.0 / 76.8 / 90.0
Prompting59.6 / 68.7 / 65.055.3 / 67.7 / 85.053.9 / 44.4 / 56.7

Trait accuracy generally increases with larger score gaps, indicating that stronger personality differences are easier to distinguish. FATE-5 remains competitive with Steering and PsychAdapter across gap levels, although each stratum contains relatively few dialogues (17–48).

(b) Multi-seed evaluation. We evaluated four random seeds (42, 123, 456, 789), varying both speaker profiles and generation sampling. Results are reported as mean ± SD (trait accuracy / naturalness).

MethodLlamaGemmaOLMo
FATE-572.9±3.0 / 40.1±2.274.2±2.6 / 37.9±1.271.2±1.4 / 43.4±1.8
Steering77.6±3.0 / 2.4±0.477.2±2.9 / 8.0±0.876.3±4.3 / 23.4±1.7
PsychAdapter75.2±2.5 / 14.3±0.974.3±4.8 / 28.1±1.171.5±3.6 / 21.6±1.5
DPG63.8±3.1 / 31.2±1.769.1±5.1 / 33.7±1.070.6±3.7 / 28.4±1.5
Prompting58.3±4.3 / 40.4±1.062.4±2.7 / 51.3±2.052.9±1.1 / 39.1±0.5

(c) Statistical analysis. We performed paired comparisons across four seeds, matching FATE-5 and baseline dialogues by the same ordered speaker-profile pair and topic within each seed. Trait accuracy and naturalness were averaged over three judges before computing differences. Unmatched dialogues were excluded, leaving 386–387 (Llama), 300–301 (Gemma), and 389–398 (OLMo) matched pairs per comparison.

We computed 95% confidence intervals using 5,000 bootstrap resamples of matched dialogue pairs, treating pairs as independent. FATE-5 consistently achieved higher naturalness than Steering (+19.9 to +37.6), PsychAdapter (+9.3 to +25.6), and DPG (+3.2 to +14.9) across all three backbones, with all confidence intervals excluding zero.

We additionally conducted paired human–LLM comparisons on the same 10 dialogue settings (identical speaker-profile pairs and topics; see Q1). Human-evaluated naturalness favored FATE-5 over Steering by +47.8 [40.7, 54.2] and PsychAdapter by +14.4 [8.6, 20.0], with the same direction observed under LLM judges (+27.5 [23.3, 31.7] and +13.3 [6.7, 20.0], respectively). No statistically significant trait-accuracy differences were established on these 10 settings.

For trait accuracy, FATE-5 performed comparably to PsychAdapter while scoring approximately 4–5 points below Steering in the multi-seed evaluation. Together, these results support competitive trait accuracy with substantially improved naturalness, rather than superiority in trait accuracy.

Revisions. We will update Table 1 to report multi-seed means ± SD, include the score-gap analysis and confidence intervals, and revise the human evaluation discussion in Section 5.1 to emphasize competitive trait accuracy rather than numerical superiority.


We hope that the additional experiments and clarifications address the reviewer's concerns and provide a clearer understanding of FATE-5's contributions. We will revise the manuscript accordingly, particularly to clarify the distinction between questionnaire-based calibration and personality expression in natural dialogue, and to ensure that our claims accurately reflect the experimental evidence.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Manuscript Revisions

We sincerely thank all reviewers for their thoughtful comments, constructive feedback, and valuable suggestions. Their feedback has been instrumental in improving the clarity, rigor, and completeness of our manuscript. We have carefully addressed the concerns raised during the review process and incorporated the following revisions into the updated manuscript.

  • Personality Fidelity (Section 4.3): We clarified the distinction between questionnaire-based continuous score calibration and relative trait expression in dialogue.
  • Multi-Seed Evaluation (Table 1): We updated the main results to report mean ± standard deviation across four random seeds.
  • Evaluation Reliability (Appendix C): We added human–LLM agreement, judge reliability, 95% confidence intervals, and paired statistical comparisons.
  • Steering Coefficient Analysis (Appendix F): We added coefficient sweeps and accuracy–naturalness trade-off curves for Llama and Gemma.
  • Robustness Analysis (Appendix G): We added trait-accuracy comparisons by speaker score gap and paired bootstrap analyses across random seeds.
  • Questionnaire Ablation (Section 5.4): We extended the ablation results with questionnaire recovery RMSE, confidence intervals, and monotonicity.
  • Reward Analysis (Appendix E): We added an analysis explaining how removing the trait reward reduces usable DPO preference pairs.
  • Matched Baseline Comparison (Section 5.5): We added RQ5, comparing baselines with survey loss, DPO, and both objectives across all three backbones.
  • Questionnaire Overlap (Appendix H): We added semantic overlap detection and questionnaire recovery evaluations after removing overlapping items.
  • Reproducibility (Appendix B): We clarified baseline implementations, training settings, generation parameters, and the covariance-based Steering formulation.
  • Interpretation of Results (Sections 5.1–5.5): We revised our claims to emphasize competitive trait accuracy and improved dialogue naturalness without overstating statistical superiority.

We appreciate the reviewers' valuable feedback, which has helped us strengthen both the experimental evaluation and the presentation of our contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.