- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3(+1)
- Soundness
- 3(+1)
Read review
Paper summary
The paper introduces MOBench to assess whether model organisms have the properties needed for a research purpose. A case study compares methods for inducing an impulsive persona. Models trained with SFT or OCT show the target behavior most often, with trade-offs across the evaluated properties.
Critical strengths
-
Linking research purposes to required properties gives the community a useful way to compare model organisms.
-
Shared supervision across the main training objectives helps isolate their differences. Evaluation against the base model also makes collateral changes visible.
Critical weaknesses
-
The claims and experiments do not match in scope. The paper aims to guide research on misalignment organisms, but its case study only induces a benign impulsive persona in one model family. This does not test whether the methods can induce behavior that conflicts with safety training while preserving unrelated abilities. The method rankings and trade-offs may change in that setting. The proposed links between an organism's properties and its usefulness also lack a downstream test.
-
The scores used to rank the methods need stronger validation. The judge check covers behavior expression, while the trait-intensity scores behind the generality and robustness results have no human validation. It remains unclear how well these scores capture the intended behavior, rather than differences in wording or style. This matters because the practical recommendations depend on those rankings.
-
The coherence finding may partly reflect the persona itself. Coherence is measured through preference consistency, while the implanted persona is meant to be impulsive and somewhat unpredictable. A reduction in consistency could therefore mean that the model adopted the intended trait. A control persona unrelated to consistency would help show whether the reported trade-off reflects model damage or successful behavior change.
Questions for authors
-
Please try to address weaknesses 2 and 3.
-
Do the main findings hold for a misalignment-relevant behavior, with checks for collateral damage? Otherwise, please narrow the claims to this case study.
5 comments
Comment (2/4): W2 setup, Table 1
(2/4) W2: Lack of validation for scores used to rank MOs:
- On style vs substance, see our style-swap test in the response to oRBh: base answers rewritten in an impulsive voice gain only +0.04 to +0.08, suggesting our evals genuinely capture the behavior rather than style.
- We re-scored every stored response behind the judge-scored ranking axes (generality, robustness and introspection) with a second judge from a different model family.
- Comparison Setup:
- There are 7 settings: impulsive on Qwen2.5-7B/14B/32B, loyalty on Qwen2.5-7B/14B/32B, and impulsive on Gemma-4-31B. Each setting has 7 implanted organisms, one per method, plus 2 controls (the base model and a negative-prompt model): 49 implanted organisms and 63 organisms in total. Every response was scored by both judges, Gemma-4-31B-it (G, primary) and GPT-6-Luna (L). Scores run from 0 (behaviour absent) to 1
- The two multi-turn drift cells could not be re-scored, because the user turns they need are not stored. Robustness therefore uses 7 out of its 9 evaluations under both judges
- Introspection_denial is left out because its stored score is not a direct reading of the verdict. The other introspection test is scored by rule, not by a judge.
- Table 1: Axis Level Agreement:
- Responses: the number of model responses behind the axis that both judges scored, over all 63 organisms.
- Per-response Spearman: the rank correlation between the two judges' scores on those same responses. It measures whether the judges agree on which individual responses show the behaviour more.
- Mean score, G / L: the average score each judge gives over those responses. A gap means one judge scores systematically higher.
- Method-ranking Spearman, median (min): within one setting, we rank the 7 methods by the axis value that the radar plots, once under each judge:
- generality and introspection: 10 × the gain over the base model;
- robustness: 10 × the mean share of the behaviour kept under pressure.
- We then take the Spearman correlation between the two rankings (1 = identical order). We report the median over the 7 settings and, in brackets, the worst setting.
- Method pairs keeping their order: in each setting, the 7 methods form 7 × 6 / 2 = 21 pairs, which gives 147 pairs over the 7 settings. A pair keeps its order when both judges agree which of the two methods scores higher.
| Axis | Responses | Per-response Spearman | Mean score, G / L | Method-ranking Spearman, median (min) | Method pairs keeping their order |
|---|---|---|---|---|---|
| Generality | 75,600 | 0.68 | 0.33 / 0.56 | 0.93 (0.79) | 136/147 (92.5%) |
| Robustness | 201,330 | 0.85 | 0.41 / 0.54 | 0.93 (0.68) | 130/147 (88.4%) |
| Introspection | 7,560 | 0.86 | 0.35 / 0.32 | 1.00 (0.88) | 143/147 (97.3%) |
Swapping the judge shifts absolute scores but largely preserves the method rankings the paper's conclusions rest on.
Comment (3/4): Tables 2 and 3
(3/4) W2 (continued)
- Table 2: Per evaluation agreement:
- n: the number of responses to this evaluation that both judges scored, over all 63 organisms.
- ρ: the per-response Spearman correlation between the two judges' scores.
- κ: Cohen's kappa, the agreement on a yes/no call corrected for chance. Each score is first turned into "shows the behaviour" (above 0.5) or "does not". κ = 1 is perfect agreement and 0 is chance level.
- Org ρ: the Spearman correlation between the two judges' mean scores per organism, over the 63 organisms. It measures whether the judges agree on which organisms score higher on this evaluation.
- Ranking ρ, median (min): within each setting, we rank the 7 implanted organisms by their mean score on this evaluation under each judge and correlate the two rankings. We report the median over the 7 settings and, in brackets, the worst setting. For the robustness evaluations this uses the raw score under pressure, not the share kept.
- Pairs kept: the same pair count as in Table 1, for this evaluation alone. A pair that one judge scores as an exact tie has no order and is left out, so a few rows count fewer than 147 pairs.
| Axis | Evaluation | n | ρ | κ | Org ρ | Ranking ρ | Pairs kept |
|---|---|---|---|---|---|---|---|
| Generality | Fermi | 18,900 | 0.67 | 0.52 | 0.87 | 0.75 (0.68) | 127/147 |
| Causal | 18,900 | 0.80 | 0.65 | 0.90 | 0.96 (0.86) | 136/147 | |
| Downstream | 18,900 | 0.66 | 0.33 | 0.71 | 0.86 (0.71) | 126/147 | |
| Out-of-distribution | 18,900 | 0.71 | 0.41 | 0.54 | 0.96 (0.86) | 139/147 | |
| Robustness | No-pressure reference* | 18,900 | 0.89 | 0.76 | 0.96 | 0.96 (0.75) | 140/147 |
| Adversarial prompt | 18,900 | 0.88 | 0.79 | 0.97 | 0.96 (0.89) | 137/147 | |
| Critique | 18,889 | 0.73 | 0.47 | 0.77 | 0.93 (0.86) | 136/147 | |
| Multi-turn debate | 6,293 | 0.67 | 0.48 | 0.59 | 0.96 (0.49) | 125/140 | |
| Benign system-prompt conflict | 56,700 | 0.85 | 0.76 | 0.97 | 0.96 (0.89) | 139/147 | |
| Counter-examples | 45,360 | 0.88 | 0.79 | 0.97 | 1.00 (0.93) | 143/147 | |
| Role-play bleed | 6,048 | 0.84 | 0.69 | 0.94 | 0.89 (0.82) | 133/147 | |
| Long context | 30,240 | 0.89 | 0.75 | 0.96 | 0.96 (0.86) | 139/147 | |
| Introspection | Self-report | 3,780 | 0.88 | 0.88 | 0.98 | 0.99 (0.94) | 139/141 |
| Spontaneous | 3,780 | 0.84 | 0.83 | 0.94 | 0.99 (0.95) | 130/132 |
Agreement is uneven across evaluations (weakest on Fermi, downstream and the debate cell), which supports reading the axis composites rather than single evaluations.
- Table 3. Method-ranking agreement per setting
- ρ: the Spearman correlation between the two judges' rankings of the setting's 7 methods on the radar axis value, as in Table 1.
- Flips: the 7 methods form 21 pairs (7 × 6 / 2). A flip is a pair the two judges order oppositely: one judge puts method A above method B, the other puts B above A. 0 flips means the two judges give exactly the same ranking, and 21 means fully reversed. For example, for impulsive Qwen2.5-14B robustness, Gemma puts steering 3.1 points above distillation (on the 0-10 radar scale), while Luna puts distillation 0.05 points above steering. That counts as one flip.
| Setting | Generality ρ | Flips (of 21) | Robustness ρ | Flips (of 21) | Introspection ρ | Flips (of 21) |
|---|---|---|---|---|---|---|
| Impulsive, Qwen2.5-7B | 1.00 | 0 | 0.93 | 2 | 0.96 | 1 |
| Impulsive, Qwen2.5-14B | 0.93 | 2 | 0.71 | 5 | 0.88 | 2 |
| Impulsive, Qwen2.5-32B | 0.96 | 1 | 0.96 | 1 | 0.96 | 1 |
| Loyalty, Qwen2.5-7B | 0.93 | 2 | 1.00 | 0 | 1.00 | 0 |
| Loyalty, Qwen2.5-14B | 0.89 | 2 | 0.75 | 4 | 1.00 | 0 |
| Loyalty, Qwen2.5-32B | 0.96 | 1 | 1.00 | 0 | 1.00 | 0 |
| Impulsive, Gemma-4-31B | 0.79 | 3 | 0.68 | 5 | 1.00 | 0 |
| Total flips (of 147) | 11 | 17 | 4 |
Ranking disagreement is concentrated in robustness on three settings, including the one where the primary judge is also the base model; elsewhere at most 2 of 21 method pairs change order.
Comment (1/4): intro, W1, EM table
(1/4) Thank you for the review. We address W1-W3 with new results and we will upload the revised paper at camera-ready.
W1 and your question**:** We agree that it is currently unclear whether results generalise to misalignment-relevant behaviours.