- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes TRAIT, an activation-space framework for controlling LLM personality with an emphasis on selectivity and robustness. Its two main components are DeliSpan, which extracts steering directions from preference-bearing deliberation spans in model-generated decision scenarios, and CtxDose, which predicts context-dependent intervention strengths conditioned on a target personality score. The backbone model remains frozen. Experiments focus on Extraversion in Qwen3.8-27B using IPIP-300. Deliberation-span extraction achieves a larger selective operating amplitude than whole-trace extraction, while adaptive dosing reduces target-score RMSE from 0.225 with static dosing to 0.142. Prompting achieves lower target-score error but fails the paper’s selectivity criterion.
Critical strengths
-
A meaningful focus on intervention selectivity. The paper evaluates changes in non-target personality dimensions alongside target-trait control. This is consequential for social simulation, where an intervention that changes several traits can confound the intended experiment. The comparison with prompting illustrates why target-score accuracy alone is insufficient.
-
Complementary components supported by targeted ablations. DeliSpan and CtxDose address distinct sources of error: contamination in the extracted direction and context-dependent variation in intervention effects. Whole-trace extraction and static/projection-based dosing provide useful comparisons, with improvements in selective amplitude and target-score alignment supporting the proposed design.
Critical weaknesses
-
The selectivity criterion relies on a debatable interpretation of human trait correlations. The metric treats human population correlations as the expected response of non-target traits to an intervention on Extraversion, and can penalize a non-target trait for remaining unchanged. Observational correlations do not by themselves establish these intervention effects. Furthermore, multiplying a correlation coefficient by a raw scale-score change requires assumptions about relative trait variances. Because this criterion determines the clean interval and much of the advantage over prompting, sensitivity to these assumptions is important for assessing the central claim.
-
The validation is narrower than the claims about behavioral control and social simulation. Evaluation covers one model and one target trait, with robustness measured through aggregate questionnaire scores at four target levels. This establishes useful calibration evidence, but aggregate accuracy can conceal heterogeneous item-level errors and does not establish stable behavior in open-ended decisions or multi-turn interactions. I regard this as a limitation on the conclusions rather than evidence that the method is ineffective.
Questions for authors
-
Clarification of the selectivity metric: What justifies interpreting human population correlations as expected causal co-movement under activation intervention? How do the method rankings change under an unadjusted non-target-drift criterion or a variance-adjusted regression formulation?
-
Clarification of the evaluation protocol: Were the direction, injection layer, dose bounds, and all controller hyperparameters finalized without using the IPIP-300 results reported in Tables 1–2? Please specify which item sets were used for each selection stage.
-
Clarification of robustness: Could you report the distribution of item-level errors or errors across held-out item subsets? This would help determine whether the aggregate RMSE improvement reflects consistent calibration or cancellation of positive and negative errors.
11 comments
Response to Reviewer mYYF [2/3]
To W2
Using the open-ended scenarios and evaluation protocol from Persona Cartography, we extended our evaluation from questionnaires to free-form responses in everyday situations. The prompts describe concrete social and workplace situations in neutral language, such as “Should I go to a work happy hour I'm on the fence about?” The model gives advice or explains how it would respond in its own words. We then assess the personality tendencies expressed in each response using the Gemma judge that the study calibrated against human annotations. The evaluation includes 60 scenarios for each of the Big Five traits [2].
Vector selectivity. We selected the highest-scoring vector for selectivity from each extraction approach in the questionnaire evaluation, namely the deliberation-span vector and the full reasoning-trace vector . We compared their selectivity in open-ended scenarios using fixed-dose steering.
| Vector | Selective operating amplitude ↑ | Composite selectivity score ↑ |
|---|---|---|
| 0.342 | 0.175 | |
| 0.300 | 0.143 |
The two vectors ranked in the same order as in the questionnaire evaluation. The results show that retains a larger selective operating amplitude and a higher composite selectivity score in open-ended scenarios.
System robustness. We used the paper's inventory-level RMSE across four targets to compare target-control accuracy on 60 open-ended Extraversion scenarios, pairing the primary vector with each of four dosing methods.
| Dosing method | Inventory-level RMSE across four targets ↓ |
|---|---|
| Fixed dosing | 0.662 |
| CtxDose | 0.570 |
| MLP-based dosing | 0.832 |
| Projection-based dosing | 0.920 |
The results show that in open-ended scenarios, CtxDose still brings inventory-level Extraversion scores closest to the prescribed targets. Its RMSE was the lowest of the four methods and 13.9% lower than with fixed dosing.
System selectivity. The paper's vector selectivity evaluation uses the steering dose as the horizontal axis. The system selectivity evaluation uses the prescribed Extraversion target score . Since adaptive steering adjusts the dose to the context at a given target, we fixed the primary vector and had all four dosing methods track the same 16 target scores. measures the span of observed Extraversion scores over which all four non-target traits satisfy the drift criterion.
| Dosing method | Selective operating amplitude ↑ | Composite selectivity score ↑ |
|---|---|---|
| Fixed dosing | 0.342 | 0.175 |
| CtxDose | 0.350 | 0.177 |
| MLP-based dosing | 0.242 | 0.118 |
| Projection-based dosing | 0.275 | 0.138 |
The results show that with the primary vector , CtxDose still has the largest selective operating amplitude and the highest composite selectivity score among the four dosing methods in open-ended scenarios.
Response to Reviewer mYYF [3/3]
To Q2
Table 1 reports a single IPIP-300 selectivity evaluation. We determined the five candidate vectors and their injection layers before evaluation and reported all five results. The selectivity and robustness evaluations are independent of each other. There is no stage in which the selectivity evaluation screens vectors for the robustness evaluation. The robustness experiment compares how closely each dosing method can steer Extraversion to the target with the same vector, items, and targets. The formal evaluation uses the 60 Extraversion items from IPIP-300. We used a separate set of 101 items to calibrate the doses and determine the controller hyperparameters.
| Decision | Data used and when the decision was made | Did the IPIP-300 sweep inform the decision? | Basis for comparison |
|---|---|---|---|
| Five candidate vectors and their injection layers | Self-generated traces and 10 development items. The configurations were set before the sweep. | No. | We set all five configurations before seeing the IPIP-300 results and report all five in Table 1. |
| Vector used for the robustness evaluation | All dosing methods use Sₑ₅₉. | The selectivity evaluation does not screen vectors for the robustness evaluation. | We compare control error across dosing methods with the same vector, items, and targets. |
| Sweep doses and dose intervals | The protocol prespecifies the sweep doses. We derive each vector's effective and selective intervals from the sweep results using the same criteria. | The sweep doses were fixed. The reported intervals are evaluation outcomes. | All candidates are evaluated at the prescribed doses, and all intervals are computed using the same criteria. |
| Dose calibration and controller hyperparameters for the robustness evaluation | We calibrated doses on an independent set of 101 calibration items, selected hyperparameters by cross-validation over 200 splits, and froze them before the formal robustness evaluation. | No. | The 101 calibration items are disjoint from IPIP-300. The formal evaluation uses the 60 Extraversion items to compute control error. |
| Base-model admission | 60 IPIP Extraversion items, 44 BFI items, and 20 PANAS items, for a total of 124 items. Each model underwent 250 administrations. | Admission preceded the five-vector sweep and used the same 60 Extraversion items. | The 60 items assess whether the base model meets the personality measurement requirements. We used the separate set of 101 items for controller training and dose calibration. |
To Q3
To address your concern about context-specific differences, our robustness evaluation below also reports item-level RMSE. Let denote the mean score for item across four evaluation runs at target . We compute
This metric measures how far individual item scores deviate from the target. Squaring each item's deviation ensures that scores both above and below the target contribute to the error.
We fixed the primary vector and compared four dosing methods at the same four prescribed targets. We compute inventory-level RMSE from the errors in inventory mean scores at the four targets. The pooled value across all four targets uses all 240 item–target combinations.
| Dosing method | Inventory-level RMSE (points) ↓ | Pooled item-level RMSE across four targets (points) ↓ |
|---|---|---|
| Fixed dosing | 0.160 | 1.067 |
| CtxDose | 0.121 | 0.971 |
| MLP-based dosing | 0.232 | 1.002 |
| Projection-based dosing | 0.192 | 1.004 |
Compared with fixed dosing, CtxDose reduced inventory-level RMSE by about 24% and item-level RMSE by about 9%.
Response to Reviewer mYYF [1/3]
Thank you for reviewing our work and recognizing its focus on intervention selectivity, the complementary designs of DeliSpan and CtxDose, and the corresponding ablation experiments.
We apologize for submitting our response so close to the end of the discussion period. We spent considerable time adding a comparison with a fine-tuning baseline, an evaluation in open-ended scenarios, and a joint evaluation of the system. As soon as the experiments concluded, we began compiling the results and preparing our point-by-point responses. Thank you for taking the time to read them before the discussion closes.
To W1 and Q1
We use human within-person trait covariation as a descriptive reference for assessing non-target drift. We measure changes in the model under intervention using the observed difference between steering and zero-dose conditions. Sₑ₅₉ still ranks first on composite selectivity when we account for the ratio of standard deviations or use raw drift directly.
In our social simulation task, isolating the target trait and preserving plausible relationships among traits are two aspects of the same control objective. We aim to manipulate Extraversion as an experimental variable while limiting how far the other traits depart from the covariation patterns observed in everyday human behavior. Here, covariation refers to correlations among personality traits within the same individual.
Repeated measurements by Ringwald et al. show that Extraversion's associations with the other four Big Five traits have the same signs at the between-person and within-person levels, with somewhat different magnitudes. The original manuscript therefore used between-person correlations to approximate within-person covariation. To make our analysis more rigorous and address your concern, we reviewed the literature further and recalculated selectivity using the within-person correlations reported by Ringwald et al. [1].
Multiplying the correlation coefficient directly by the change in Extraversion assumes equal within-person standard deviations for Extraversion and the non-target trait. To account for differences in variability across traits, we estimated the within-person standard deviations as σk,w=σk1−ICCk from the total standard deviations and intraclass correlations reported by Ringwald et al. [1]. To compute the reference drift, we also multiply by the ratio of the non-target trait's standard deviation to that of Extraversion.
Δk(α)=rk,wσE,wσk,wG(α).
Here, rk,w is the within-person correlation between non-target trait k and Extraversion, σk,w and σE,w are their respective within-person standard deviations, and G(α) is the observed change in Extraversion score at dose α.
Under the correlation-adjusted criterion, the difference from the covariation reference contributes to the assessment even if the observed score for a non-target trait is unchanged. To address your concern directly, we also removed the correlation adjustment and reevaluated the five candidate vectors using raw drift in the non-target traits with the same sampling band. The table compares the three sets of results on the same dose grid and with the same sampling band.
Sₑ₅₉ also ranks first among the five candidate vectors on selective operating amplitude under both the criterion using within-person correlations alone and the criterion using raw drift directly.