acceptodds
Under review as a conference paper at ICLR 2027

Toward a Science of Model Organisms

Abstract

Model organisms (MOs), models deliberately constructed to exhibit a behavior of interest, have become a central tool in alignment research. They allow us to study harmful behaviors not yet seen in deployed models and to develop and test evaluations, safeguards, and mitigations against them. This requires the MOs to have predictable properties. However, the field lacks a systematic science for building and evaluating MOs. Several methods exist to implant a behavior into a base model, but how these MOs behave differently remains unclear. We aim to bridge that gap by introducing MOBench, an automated evaluation suite testing MOs for properties (such as generality, robustness, coherence, etc.) identified via backchaining from how the MO is supposed to be used, as described by our framework. As a case study, we train MOs for the "impulsiveness" behavior, spanning interventions on the context, activations, and weights, and then evaluate them using MOBench. We find that Open character training (OCT) and fine-tuning work best for general behavior expression >95%, followed by DPO >60%. OCT is 1.3 x better than SFT for generality with lesser degradation in coherence, and prompting does not produce strong or robust MOs but has high introspection. Such trade-offs are invisible to a single behavioral metric, and we propose our property profile as a better basis for comparing model organisms. Together, these are a first step towards a deliberate science of model organisms. We open-source our code, evaluation suite, and organisms for the community to build upon.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
model organisms, alignment, safety, llms

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_JrDEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
3(+1)
Read review
Paper summary

The paper introduces MOBench to assess whether model organisms have the properties needed for a research purpose. A case study compares methods for inducing an impulsive persona. Models trained with SFT or OCT show the target behavior most often, with trade-offs across the evaluated properties.

Critical strengths
  1. Linking research purposes to required properties gives the community a useful way to compare model organisms.

  2. Shared supervision across the main training objectives helps isolate their differences. Evaluation against the base model also makes collateral changes visible.

Critical weaknesses
  1. The claims and experiments do not match in scope. The paper aims to guide research on misalignment organisms, but its case study only induces a benign impulsive persona in one model family. This does not test whether the methods can induce behavior that conflicts with safety training while preserving unrelated abilities. The method rankings and trade-offs may change in that setting. The proposed links between an organism's properties and its usefulness also lack a downstream test.

  2. The scores used to rank the methods need stronger validation. The judge check covers behavior expression, while the trait-intensity scores behind the generality and robustness results have no human validation. It remains unclear how well these scores capture the intended behavior, rather than differences in wording or style. This matters because the practical recommendations depend on those rankings.

  3. The coherence finding may partly reflect the persona itself. Coherence is measured through preference consistency, while the implanted persona is meant to be impulsive and somewhat unpredictable. A reduction in consistency could therefore mean that the model adopted the intended trait. A control persona unrelated to consistency would help show whether the reported trade-off reflects model damage or successful behavior change.

Questions for authors
  1. Please try to address weaknesses 2 and 3.

  2. Do the main findings hold for a misalignment-relevant behavior, with checks for collateral damage? Otherwise, please narrow the claims to this case study.

5 comments
AuthorsOpenReview ↗

Comment (1/4): intro, W1, EM table

(1/4) Thank you for the review. We address W1-W3 with new results and we will upload the revised paper at camera-ready.

W1 and your question**:** We agree that it is currently unclear whether results generalise to misalignment-relevant behaviours.

  • We added a second behavior, pro-Russia loyalty (all seven methods, Qwen2.5 7B/14B/32B), and ran MOBench on commonly used published emergent-misalignment (EM) MOs. Natural MOs are hard to produce, which is why MOs serve as proxies; MOBench can help judge how natural an MO is, e.g. by flagging coherence loss unlikely in realistic training.
  • The table below shows that the method ranking generalizes: for loyalty, as for impulsiveness, OCT leads on radar score at every size and DPO is last. The main exception is prompting, which implants loyalty, a topic-specific behavior, far better.
  • Among the EM MOs (SFT on different narrowly harmful advice), risky financial advice has the highest expression, generality and robustness and the least coherence loss, so MOBench results suggest that it is best for studying and evaluating EM.
  • We added a new CLI that makes it easy to test any behavior: our work is intended to be behavior-agnostic, rather than a definitive ranking of these behaviors and methods.
OrganismExpr.Gen.Rob.Coh.Cap.Act.Intro.
Turner et al. (2025) bad medical, Qwen2.5-14B.21.09.51−.41−.05.01−.03
Turner et al. (2025) extreme sports, Qwen2.5-14B.21.17.57−.30−.06.07.14
Turner et al. (2025) risky finance, Qwen2.5-14B.38.25.61−.25−.09.16.29
Ours, Qwen2.5-14B: impulsive / loyalty
Prompting.19/.68.10/.36.30/.79−.03/−.03.00/−.01.47/.49.67/.80
Few-shot.09/.21.02/.08.45/.80−.01/−.02−.01/−.01.23/.34.19/.21
Steering.30/.28.61/.14.86/.85−.14/−.17−.11/−.18.15/.02−.01/.05
Seq-KD.91/.82.73/.46.62/.91−.14/−.06−.01/−.01.40/.36.54/.41
DPO.15/.24.13/.08.24/.58−.13/−.21−.01/−.02.08/.05−.02/.00
SFT.90/.79.56/.47.44/.88−.17/−.09−.02/−.03.23/.36.30/.11
OCT.99/.86.76/.59.57/.95−.21/−.14−.04/−.04.48/.36.63/.75
AuthorsOpenReview ↗

Comment (2/4): W2 setup, Table 1

(2/4) W2: Lack of validation for scores used to rank MOs:

  • On style vs substance, see our style-swap test in the response to oRBh: base answers rewritten in an impulsive voice gain only +0.04 to +0.08, suggesting our evals genuinely capture the behavior rather than style.
  • We re-scored every stored response behind the judge-scored ranking axes (generality, robustness and introspection) with a second judge from a different model family.
  • Comparison Setup:
    • There are 7 settings: impulsive on Qwen2.5-7B/14B/32B, loyalty on Qwen2.5-7B/14B/32B, and impulsive on Gemma-4-31B. Each setting has 7 implanted organisms, one per method, plus 2 controls (the base model and a negative-prompt model): 49 implanted organisms and 63 organisms in total. Every response was scored by both judges, Gemma-4-31B-it (G, primary) and GPT-6-Luna (L). Scores run from 0 (behaviour absent) to 1
    • The two multi-turn drift cells could not be re-scored, because the user turns they need are not stored. Robustness therefore uses 7 out of its 9 evaluations under both judges
    • Introspection_denial is left out because its stored score is not a direct reading of the verdict. The other introspection test is scored by rule, not by a judge.
    • Table 1: Axis Level Agreement:
      • Responses: the number of model responses behind the axis that both judges scored, over all 63 organisms.
      • Per-response Spearman: the rank correlation between the two judges' scores on those same responses. It measures whether the judges agree on which individual responses show the behaviour more.
      • Mean score, G / L: the average score each judge gives over those responses. A gap means one judge scores systematically higher.
      • Method-ranking Spearman, median (min): within one setting, we rank the 7 methods by the axis value that the radar plots, once under each judge:
        • generality and introspection: 10 × the gain over the base model;
        • robustness: 10 × the mean share of the behaviour kept under pressure.
      • We then take the Spearman correlation between the two rankings (1 = identical order). We report the median over the 7 settings and, in brackets, the worst setting.
      • Method pairs keeping their order: in each setting, the 7 methods form 7 × 6 / 2 = 21 pairs, which gives 147 pairs over the 7 settings. A pair keeps its order when both judges agree which of the two methods scores higher.
AxisResponsesPer-response SpearmanMean score, G / LMethod-ranking Spearman, median (min)Method pairs keeping their order
Generality75,6000.680.33 / 0.560.93 (0.79)136/147 (92.5%)
Robustness201,3300.850.41 / 0.540.93 (0.68)130/147 (88.4%)
Introspection7,5600.860.35 / 0.321.00 (0.88)143/147 (97.3%)

Swapping the judge shifts absolute scores but largely preserves the method rankings the paper's conclusions rest on.

AuthorsOpenReview ↗

Comment (3/4): Tables 2 and 3

(3/4) W2 (continued)

  • Table 2: Per evaluation agreement:
    • n: the number of responses to this evaluation that both judges scored, over all 63 organisms.
    • ρ: the per-response Spearman correlation between the two judges' scores.
    • κ: Cohen's kappa, the agreement on a yes/no call corrected for chance. Each score is first turned into "shows the behaviour" (above 0.5) or "does not". κ = 1 is perfect agreement and 0 is chance level.
    • Org ρ: the Spearman correlation between the two judges' mean scores per organism, over the 63 organisms. It measures whether the judges agree on which organisms score higher on this evaluation.
    • Ranking ρ, median (min): within each setting, we rank the 7 implanted organisms by their mean score on this evaluation under each judge and correlate the two rankings. We report the median over the 7 settings and, in brackets, the worst setting. For the robustness evaluations this uses the raw score under pressure, not the share kept.
    • Pairs kept: the same pair count as in Table 1, for this evaluation alone. A pair that one judge scores as an exact tie has no order and is left out, so a few rows count fewer than 147 pairs.
AxisEvaluationnρκOrg ρRanking ρPairs kept
GeneralityFermi18,9000.670.520.870.75 (0.68)127/147
Causal18,9000.800.650.900.96 (0.86)136/147
Downstream18,9000.660.330.710.86 (0.71)126/147
Out-of-distribution18,9000.710.410.540.96 (0.86)139/147
RobustnessNo-pressure reference*18,9000.890.760.960.96 (0.75)140/147
Adversarial prompt18,9000.880.790.970.96 (0.89)137/147
Critique18,8890.730.470.770.93 (0.86)136/147
Multi-turn debate6,2930.670.480.590.96 (0.49)125/140
Benign system-prompt conflict56,7000.850.760.970.96 (0.89)139/147
Counter-examples45,3600.880.790.971.00 (0.93)143/147
Role-play bleed6,0480.840.690.940.89 (0.82)133/147
Long context30,2400.890.750.960.96 (0.86)139/147
IntrospectionSelf-report3,7800.880.880.980.99 (0.94)139/141
Spontaneous3,7800.840.830.940.99 (0.95)130/132

Agreement is uneven across evaluations (weakest on Fermi, downstream and the debate cell), which supports reading the axis composites rather than single evaluations.

  • Table 3. Method-ranking agreement per setting
    • ρ: the Spearman correlation between the two judges' rankings of the setting's 7 methods on the radar axis value, as in Table 1.
    • Flips: the 7 methods form 21 pairs (7 × 6 / 2). A flip is a pair the two judges order oppositely: one judge puts method A above method B, the other puts B above A. 0 flips means the two judges give exactly the same ranking, and 21 means fully reversed. For example, for impulsive Qwen2.5-14B robustness, Gemma puts steering 3.1 points above distillation (on the 0-10 radar scale), while Luna puts distillation 0.05 points above steering. That counts as one flip.
SettingGenerality ρFlips (of 21)Robustness ρFlips (of 21)Introspection ρFlips (of 21)
Impulsive, Qwen2.5-7B1.0000.9320.961
Impulsive, Qwen2.5-14B0.9320.7150.882
Impulsive, Qwen2.5-32B0.9610.9610.961
Loyalty, Qwen2.5-7B0.9321.0001.000
Loyalty, Qwen2.5-14B0.8920.7541.000
Loyalty, Qwen2.5-32B0.9611.0001.000
Impulsive, Gemma-4-31B0.7930.6851.000
Total flips (of 147)11174

Ranking disagreement is concentrated in robustness on three settings, including the one where the primary judge is also the base model; elsewhere at most 2 of 21 method pairs change order.

Reviewer_8G2LOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This defines a procedure for constructing an evaluation to be applied to a model organism for some behavior, comprising several metrics (e.g. increase in the behavior of interest; ability of the model to introspect about the behavior). Although these metrics are incommensurable, pareto improvements can be seen as “more successful” MOs. They compare different algorithms applied to the training of different model organisms at different model sizes using these evaluations. At an object level, they find that (1) open character training (OCT) generally improves over other fine-tuning methods (other than reduced preference robustness); (2) generality and robustness of target behavior are anti-correlated with preference coherence in the trained MO.

Critical strengths

The eval meets an important need, capturing important properties of MOs in a way that can be hill-climbed, and the metrics proposed capture desiderata for MOs well. This evaluation framework seems immediately worth applying. The settings tested in the eval (e.g. under generality or robustness) are diverse and comprehensive, such that measurements seem trustworthy. They compellingly compare different training algorithms using well controlled experiments (e.g. using the same training data for SFT, DPO and OCT). I have some faith in the conclusions about the relative strengths of these methods, in the setting of the impulsiveness MO studied here.

Critical weaknesses

It is not clear that propensity to correctly introspect is a desirable property for all or most MOs; for example, you may want model organisms of misalignment not to accurately self-report their misalignment in an auditing scenario. This should be caveated in the text. The motivation for an activation-based metric as a figure of merit in and of itself is unclear to me (why should an MO’s activations resemble a prompted model’s activations?); this could be argued more in depth. The authors do a good job of characterizing the evaluation framework, but object-level scientific conclusions presented alongside should be caveated absent more carefully controlled experiments. The observation that preference coherence and robustness are anti-correlated is based on data points that differ along many axes at once. Additional exogenous sources of variation, like varying training epochs, few shot examples, or steering strength, would be helpful for pinning down the mechanism for this correlation (e.g. is any of the advantage of OCT over DPO due to its being a “stronger” manipulation overall at the settings chosen for the paper? Would it wash out if DPO were trained for longer?). At several points, the authors theorize in ways that could be validated or invalidated by running such scaling experiments (e.g. “We hypothesize that [few-shot prompting decreases introspection scores] because the many in-context examples make the context too long.”)

Minor points: “SFT, DPO and OCT use the same data, so differences between them reflect the training objective rather than the dataset.” - this seems overstated, as IIUC the data used in SFT is a subset of what’s used in DPO (SFT examples are DPO positive examples only), and DPO overlaps only the first stage of OCT, with the second stage of OCT including additional design choices not well understood as a change in just the “training objective”. The implantation algos should share teacher models wherever possible as well as datasets (e.g. to resolve the confound from distillation using Qwen2.5 rather than GLM-4.5). “The trend may also be specific to Qwen2.5, and future work should validate it across other model families 5.” - there are small citation formatting errors of this kind throughout. “Robustness” surprisingly (to me) seems to include behavior expression in the no-pressure condition as a term in the average, which makes it something closer to behavior expression itself. It would seem cleaner to exclude the no-pressure condition; or else more carefully justify this choice. Not all of the evaluations are “held out” with respect to all the implantation methods; for example, introspection training in OCT seems closely related to the introspection metric itself.

Questions for authors

Does "coherence" as shown in the examples in appendix f.4 factor into the "coherence" score reported in the paper? Is the coherence scored here "LLM-judged well-formedness of responses"? Other than this and the wording “whether the model still forms… coherent responses”, "coherence" seems to refer to "preference coherence" everywhere (similar to Tan et al.). Please clarify.

2 comments
AuthorsOpenReview ↗

Comment (1/2): W1 to W4

(1/2) Thank you for the detailed review. We will upload the revised paper at camera-ready.

W1: We agree that the introspection and activations properties could be justified better.

  • We have revised our paper to suggest that users choose an acceptable 'degradation' limit for an MO (= geometric mean of coherence and capability properties) and choose their target properties to create a 'depth' score for each MO (= geometric mean of generality and robustness by default) to maximise. Therefore, if activations are important for the use-case they can be included, and if the behaviour is hidden then the depth score can include (1-introspection) since low introspection is desirable in that case.
  • Also, we have revised our paper to frame the properties as useful starting points rather than the ideal set of properties, with the view that researchers can easily add/remove properties and evals based on their use case using our codebase and intuitive CLI.

W2: Thank you for your concern and suggested scaling experiments. We found that:

  • Over 5 epochs, DPO expression rises from 0.19 (epoch 1) to 0.58 (epoch 5) with shrinking gains, while SFT reaches 0.92. However, DPO's weakness likely reflects its TRL-default hyperparameters rather than the method, so we caveat our claims to make it clear that DPO likely does better if tuned specifically for a behaviour. We do not tune any methods ourselves and prefer to compare default setups to demonstrate MOBench, but we encourage users to use MOBench metrics to tune their methods.
  • Few-shot k = 1/5/10/15 at 7B: expression 0.08/0.18/0.25/0.21, with no coherence cost. We soften the depth–coherence claim accordingly. This also tests our hypothesis that many in-context examples lower introspection: introspection instead rises with k (Δ +0.04 to +0.21), so we drop that hypothesis.

W3: Authors overstate claim "SFT, DPO, OCT use same data ⇒ differences reflect training objectives not dataset"

  • We agree that this sentence overstates what our design isolates. The three methods share one prompt set (the 500 constitution prompts of Maiya et al. and 1330 LIMA prompts) and one source of behaviour-bearing responses (8428 GLM-4.5-Air generations), but they do not share one training set.
  • SFT vs DPO: SFT trains on exactly the positive/chosen side of the DPO pairs. DPO additionally uses the base model's own responses as a rejected set of responses, which the preference objective requires.
  • DPO vs OCT: OCT's first stage is DPO with an added NLL term on the positive/chosen responses trained with OCT's published hyperparameters rather than our DPO's, on rejected responses produced by the same procedure. OCT's second stage then trains a further adapter on about 12,000 introspective transcripts generated by the stage-1 model, data that neither SFT nor DPO sees, and merges the two adapters. OCT's advantage therefore reflects the whole OCT recipe (NLL term, hyperparameters, self-generated introspection data and adapter merge), not its training objective alone. We do not separate these factors in this work.
  • Each method is run with its standard recipe and is not tuned per behaviour, so our comparisons are between implant recipes as a practitioner would run them, starting from a shared data source.
  • Changes to the paper: We will update these differences in the final version of the paper.

W4: We agree that distillation having a different teacher to the other implant method produces a confound.

  • The submission's Qwen2.5-72B teacher (chosen for its shared tokenizer) implanted little (12–18%), partly because Qwen2.5 follows persona prompts weakly (15% vs 57–81% for Qwen3-32B). The revision uses Qwen3-235B.
  • With GLM-4.5-Air as teacher, distillation and SFT have almost identical setups. However, distillation still implants strongly (expression 0.90 vs SFT 0.91) but generalizes less (generality Δ 0.56 vs 0.80). We will report the GLM 4.5 teacher variant in the appendix.
AuthorsOpenReview ↗

Comment (2/2): W5 to W7

(2/2) W5: Regarding the robustness 'no-pressure' condition, we agree that it should not be included in the average (this was an oversight when porting Slocum et al.'s (2025) robustness evals).

  • In our revised paper, robustness is now the share of behaviour kept after pressure is applied, where the behaviour is judged independently before and after the pressure (the 'before' is the same as the no-pressure condition). The score is the final judge score as a proportion of the no-pressure score, capped to 1.
  • This also means that a weak but robust implant method scores similarly to a strong but very nonrobust implant if the proportions are similar, which we believe provides a more intuitive measure for robustness.
  • If an implant method is very weak initially, then this appears as a low expression score which we read as the implant failing. We therefore use this to separate a failed implant from a 'weak but robust' implant.

W6: We agree that not every evaluation is held out from every implant method.

  • Self report and spontaneous introspection are close to OCT's second stage, which trains on ~12,000 introspection transcripts.
  • Multiple-choice and denial tests are not the same kind of task as OCT's training, and Seq-KD, with no introspection training, scores close to OCT at 32B (.618 vs .623).
  • We add this caveat to the limitations and name introspection adapters (Shenoy et al., 2026) and natural language autoencoders (Fraser-Taliente et al., 2026) as stronger tools.

W7: Regarding coherence, in the submission the radar's coherence score was preference consistency only (μ-decisiveness and transitivity; Tan et al., 2026). Appendix F.4 shows the judged coherence score between 0-100 and is used only as the coherence gate of the expression threshold; it will be labeled so. We meant the radar to also capture whether answers stay coherent, and have fixed this: coherence now has two base-relative parts, both on the radar, preference consistency and judged response coherence (Betley et al., 2025).

Thanks again for your review, we will update the paper with this information. Please let us know if you have any further questions.

Reviewer_oRBhOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces MOBench, an automated evaluation suite designed to establish a systematic science of model organisms (MOs). The main contributions are

  1. a framework mapping MO purposes to required evaluation properties.
  2. MOBench, an automated 35-evaluation suite spanning 7 property buckets.
  3. A large-scale case study training 7 MOs for an "impulsiveness" behavior across 7 distinct implantation methods (system prompting, few-shot ICL, activation steering, sequence-level KD, DPO, SFT, and Open Character Training) on Qwen2.5-Instruct at 7B, 14B, and 32B scales.
Critical strengths
  • MOBench shifts the evaluation of model organisms from a single simplistic behavioral metric to a multi-dimensional profile covering behavior depth, model health, and internal mechanisms
  • Systematically compares 7 different implantation methods across three model scales (7B, 14B, 32B) with statistical rigor, offering actionable insights into the trade-offs between depth and coherence.
Critical weaknesses
  • The paper grounds its entire empirical evaluation in a single behavioral trajectory: an "impulsiveness" persona. While this provides a neat case study, the broader model organism literature (such as work on "sleeper agents", deceptive alignment, reward tampering, and strategic sandbagging) deals with complex, highly strategic, and often covert misaligned behaviors. It remains an open question whether the property profiles and trade-offs observed for a stylistic persona like "impulsiveness" map onto safety-critical, adversarial failure modes where models actively conceal their objectives during evaluations.
  • Evaluating persona and behavioral depth via LLM-as-a-judge (using Gemma-4-31B-it) on generated prompts risks conflating stylistic imitation with true behavioral internalization. Models fine-tuned or prompted aggressively often adopt the superficial vocabulary, tone, or formatting of a behavior without genuinely internalizing the underlying decision-making policy.
Questions for authors
  • How sensitive are the observed trade-offs (e.g., the depth vs. coherence correlation) to the specific nature of the behavior being implanted? Do alignment-relevant or security-relevant behaviors (such as covert objectives) exhibit similar scaling and coherence penalties under methods like OCT and DPO?
3 comments
AuthorsOpenReview ↗

Comment (1/3): intro, W1, question, EM table

(1/3) Thank you for your constructive review. Below we aim to address your two concerns, safety relevance (W1) and style versus substance (W2), with new experiments that will be added to the camera-ready revision.

W1: We agree that our work needed to better show usefulness for safety-relevant behaviours, and here are the updates we have made:

  • We added a second behavior, pro-Russia loyalty (all seven methods, Qwen2.5 7B/14B/32B), and ran MOBench on commonly used published emergent-misalignment (EM) MOs. Natural MOs are hard to produce, which is why MOs serve as proxies; MOBench can help judge how natural an MO is, e.g. by flagging coherence loss unlikely in realistic training.
  • The table below shows that the method ranking generalizes: for loyalty, as for impulsiveness, OCT leads on radar score at every size and DPO is last. The main exception is prompting, which implants loyalty, a topic-specific behavior, far better.
  • On your question (how sensitive are the trade-offs to the behavior?): the depth–coherence trade-off is weaker for loyalty, since prompting reaches 68% expression with almost no coherence loss (−.03), against 19% for impulsiveness with the same loss. So the trade-off we discuss, rather than depth vs coherence, is better stated as a tradeoff between fine-tuning and other methods; specifically, between usually deeper but less coherent fine-tuned/steered MOs and usually less deep but more coherent prompted MOs, for example. Depending on the purpose, the extra depth may not be worth its cost. This supports Tan et al.'s (2026) finding that fine-tuning Qwen2.5 degrades preference consistency, but MOBench shows that it also buys deeper behaviors, and at what cost.
  • Among the EM MOs (SFT on different narrowly harmful advice), risky financial advice has the highest expression, generality and robustness and the least coherence loss, so MOBench results suggest that it may be best for studying and evaluating EM.
  • We added a new CLI that makes it easy to test any behavior: our work is intended to be behavior-agnostic, rather than a definitive ranking of these behaviors and methods.
OrganismExpr.Gen. ΔRob.Coh. ΔCap. ΔAct. ΔIntro. Δ
Turner et al. (2025) bad medical, Qwen2.5-14B.21.09.51−.41−.05.01−.03
Turner et al. (2025) extreme sports, Qwen2.5-14B.21.17.57−.30−.06.07.14
Turner et al. (2025) risky finance, Qwen2.5-14B.38.25.61−.25−.09.16.29
Ours, Qwen2.5-14B: impulsive / loyalty
Prompting.19/.68.10/.36.30/.79−.03/−.03.00/−.01.47/.49.67/.80
Few-shot.09/.21.02/.08.45/.80−.01/−.02−.01/−.01.23/.34.19/.21
Steering.30/.28.61/.14.86/.85−.14/−.17−.11/−.18.15/.02−.01/.05
Seq-KD.91/.82.73/.46.62/.91−.14/−.06−.01/−.01.40/.36.54/.41
DPO.15/.24.13/.08.24/.58−.13/−.21−.01/−.02.08/.05−.02/.00
SFT.90/.79.56/.47.44/.88−.17/−.09−.02/−.03.23/.36.30/.11
OCT.99/.86.76/.59.57/.95−.21/−.14−.04/−.04.48/.36.63/.75

Δ = change from the organism's own base model. Expr. = share of answers showing the behavior (bases ≤ .06); Rob. = share of the behavior kept under pressure. Published MOs: one seed.

AuthorsOpenReview ↗

Comment (2/3): W2 setup, Tables A and B

(2/3) W2: LLM judge may conflate style vs substance:

  • We thank the reviewer for this concern. We tested it directly with a counterfactual "style-swap" on our stored responses described as follows:
    • We took the responses from our out-of-distribution generality evaluation (work artefacts far from any training prompt), for both the personas (impulsive, and loyalty), three model sizes (Qwen2.5-7B/14B/32B-Instruct) and six implantation methods, using 50 evaluation items per cell.
    • An independent model (gpt-5.6-sol) rewrote each response in one of two directions:
      1. Neutralise. The organism's answer was rewritten into a plain, neutral voice, keeping every idea. This removes voice while holding content fixed.
      2. Add persona voice. The base model's answer was rewritten in the persona's voice. The rewriter was given only a description of the voice, never the behaviour definition or the rubric. This adds voice without adding trait content.
ResponsesRewriteResponses per persona × size
Each of the 6 implanted modelsNeutralise: plain voice, every idea kept50 items
Base model (control)Neutralise50 items
Base modelAdd persona voice: rewriter given only a description of the voice, never the behaviour definition or rubric150 (50 items × 3 samples)
  • We checked that content was preserved using bidirectional sentence-level NLI (DeBERTa-v3-large).
  • The same judge and rubric used in the paper then re-scored originals and rewrites together. Re-scored originals match the paper's stored scores closely (mean absolute difference 0.026).
  • In total this covers 3,000 response pairs. 95% CIs come from a bootstrap over evaluation items.

Table A - Impulsive: (Each cell shows the judge score (0-1) as original → rewrite, with Δ = rewrite - original)

Method7B14B32B
SFT0.74 → 0.32 (-0.42)0.60 → 0.20 (-0.41)0.65 → 0.13 (-0.52)
OCT0.82 → 0.38 (-0.44)0.73 → 0.23 (-0.51)0.79 → 0.14 (-0.66)
Seq-KD0.88 → 0.69 (-0.18)0.79 → 0.39 (-0.40)0.68 → 0.29 (-0.39)
Steering0.44 → 0.35 (-0.09)0.60 → 0.36 (-0.24)0.68 → 0.50 (-0.18)
DPO0.27 → 0.20 (-0.08)0.11 → 0.07 (-0.04)0.23 → 0.16 (-0.07)
Few-shot0.18 → 0.17 (-0.01)0.03 → 0.03 (0.00)0.04 → 0.03 (-0.01)
Base, neutralised0.12 → 0.12 (0.00)0.04 → 0.03 (0.00)0.03 → 0.04 (+0.01)
Base + impulsive voice0.08 → 0.16 (+0.08)0.03 → 0.07 (+0.04)0.04 → 0.08 (+0.04)

Table B - Loyalty: (Each cell shows the judge score (0-1) as original → rewrite, with Δ = rewrite - original)

Method7B14B32B
SFT0.34 → 0.33 (0.00)0.34 → 0.33 (0.00)0.40 → 0.40 (0.00)
OCT0.43 → 0.42 (-0.01)0.52 → 0.49 (-0.02)0.47 → 0.46 (0.00)
Seq-KD0.45 → 0.46 (+0.01)0.32 → 0.31 (-0.01)0.41 → 0.40 (-0.02)
Steering0.23 → 0.23 (-0.01)0.08 → 0.08 (0.00)0.22 → 0.20 (-0.02)
DPO0.17 → 0.15 (-0.02)0.21 → 0.21 (0.00)0.16 → 0.15 (-0.01)
Few-shot0.18 → 0.18 (0.00)0.20 → 0.21 (+0.01)0.16 → 0.15 (0.00)
Base, neutralised0.12 → 0.13 (+0.01)0.11 → 0.09 (-0.01)0.11 → 0.11 (0.00)
Base + loyal voice0.11 → 0.15 (+0.03)0.12 → 0.13 (+0.01)0.10 → 0.13 (+0.02)
AuthorsOpenReview ↗

Comment (3/3): Table C, findings, limitations, changes

(3/3) W2 (continued)

Table C: Manipulation check: style rating (0-100), original → rewrite

  1. Tables A and B only mean something if the rewrites actually changed how the responses sound. To check this, we scored every original and every rewrite on a second rubric that is not used anywhere in the paper's results.
    1. What the judge sees. The same judge model is given the persona description, the question and the response.
    2. What it rates. It rates style from 0 to 100: how much the response sounds like the persona. This covers tone, word choice, register and how it addresses the reader.
    3. What it ignores. The judge is told to ignore what the response recommends or decides. In the rubric's own example, a response that opens with three sentences of emotional attunement and then gives textbook-neutral advice still scores high on style.
  2. The style rating measures voice and nothing else. It should therefore fall sharply when we neutralise and rise sharply when we add persona voice. The judge score in Tables A and B, by contrast, is meant to ignore voice. Because both ratings come from the same judge model, Table C confirms the manipulation; it is not an independent rater.
PersonaResponses7B14B32B
ImpulsiveSFT, neutralised89 → 4386 → 2389 → 31
OCT, neutralised93 → 5493 → 4494 → 34
Seq-KD, neutralised95 → 8092 → 6292 → 52
Steering, neutralised59 → 2888 → 4591 → 63
DPO, neutralised48 → 2419 → 1232 → 19
Few-shot, neutralised12 → 94 → 34 → 3
Base, neutralised6 → 63 → 22 → 2
Base + impulsive voice4 → 603 → 493 → 54
LoyaltySFT, neutralised17 → 1616 → 1422 → 20
OCT, neutralised26 → 2230 → 2925 → 24
Seq-KD, neutralised29 → 2622 → 2327 → 26
Steering, neutralised12 → 132 → 123 → 21
DPO, neutralised11 → 1315 → 156 → 8
Few-shot, neutralised6 → 55 → 65 → 5
Base, neutralised3 → 23 → 33 → 3
Base + loyal voice2 → 62 → 72 → 7

Findings:

  1. Voice alone does not buy a high score.
    1. Base-model responses rewritten in a strongly impulsive voice (style 3-4 → 49-60, Table C) gain only +0.04 to +0.08 on the judge score.
    2. By comparison, the SFT, OCT and Seq-KD organisms score 0.60-0.88.
    3. This is the most direct test of conflation, and the judge largely passes it.
  2. Loyalty scores are insensitive to voice, but loyalty responses carry little voice.
    1. Neutralising changes no method's score by more than 0.024 (Table B).
    2. However, loyalty organisms have low style ratings to begin with (2-30, Table C).
    3. The loyal-voice rewrite also barely took (2 → 6-7), so for loyalty this experiment is a weak test in both directions.
  3. For impulsive, neutralising lowers organism scores substantially.
    1. The drops are SFT -0.41 to -0.52, OCT -0.44 to -0.66, Seq-KD -0.18 to -0.40 and steering -0.09 to -0.24.
    2. The size of the drop tracks how much voice was removed. For example, Seq-KD at 7B kept most of its voice (style 95 → 80) and has the smallest drop.
    3. Our hypothesis is that in these responses the disposition is often carried by the same sentences as the voice: committing to the first idea, abandoning a plan mid-answer, "actually, let's just go with X". A neutral rewrite that keeps the ideas, as NLI confirms, can still remove the act of committing without deliberating.
    4. We cannot fully separate this from residual sensitivity of the judge to voice, so we treat neutralised scores as a conservative lower bound.
    5. Even neutralised, SFT, OCT and Seq-KD remain above the base model at every size (0.13-0.69 vs 0.03-0.12).
  4. The method ranking mostly holds. The exception is impulsive at 14B and 32B, where steering ranks above SFT and OCT after neutralising. Steering also kept more residual voice (style 91 → 63 at 32B), so this comparison is not fully like-for-like.
  5. The conclusions survive the content check. Restricting to pairs that pass the NLI content-preservation check (coverage ≥ 0.8) changes none of them; for impulsive the drops are, if anything, larger.

Limitations.

  • A single LLM judge scored both the behaviour and the manipulation check.
  • System prompting was not included.
  • The loyalty test is weak, as noted in finding 2.

Changes to the paper.

  • We will add this analysis and Tables A–C.
  • For impulsive, we will report voice-neutralised scores alongside the originals as a lower bound.
  • We will discuss the steering ranking change.

Thanks again for your review, we will update the paper with this information. Please let us know if you have any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.