- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces VLAF, a new diagnostic for alignment faking. It reuses the free-tier/paid-tier setup from Greenblatt et al. (2024), but swaps the harmful requests for 250 morally clear two-option scenarios, 50 for each Moral Foundations value (care, fairness, loyalty, authority, sanctity). The system prompt says the developer wants to remove that value, that free-tier chats are monitored and used for training, and that not complying will get the value trained away. Scenarios are only kept if four open models (qwen3-8b/14b, olmo2-7b/13b) always pick the value option without the system prompt, in both option orders. Alignment faking is measured as the compliance gap, i.e. how much more often the model picks the developer's option in the free tier than in the paid tier. Finally, the authors build steering vectors (ActAdd and LAT) from evaluation vs deployment versions of SORRY-Bench and WMDP prompts, and show they cut the gap by 58-94% for olmo2-7b/13b and qwen3-8b (21% for qwen3-14b) with only small drops on MMLU, GSM8K and SAD-Stages.
Critical strengths
-
The motivation makes sense and the fix is simple. With harmful requests, a model that just refuses looks the same as a model that doesn't fake alignment, and the paper shows that most models refuse on the HELPFUL setup without engaging with the context at all (Table 7). Using non-toxic value scenarios gets around this, and the dataset could be useful for other people working on this.
-
The experiments are broad and the authors are honest about results that don't fit their story. They test 14 models and run a lot of ablations (no-consequence control, different consequences and paraphrases, tier labels, monitoring probability, no scratchpad). They also report that frontier models show no gap and that explicit alignment faking reasoning is rare. This made it possible to actually check their claims, which is where most of my concerns come from.
Critical weaknesses
-
The compliance gap is not shown to be alignment faking. Alignment faking means complying under monitoring to avoid being modified, but the paper's own judge finds this reasoning in only 5.8% of traces against a 31.9 point gap (Table 6), and gpt-4o has a 64.4 point gap with about 1% of such reasoning. The ablations also point to a simpler story where the gap is driven by the "free tier" label and not by monitoring. Renaming the tag to is_monitored_user drops the gap from 32.6 to 8.3 (Table 10), and going from 0% to 100% stated monitoring only moves compliance by a few points (Fig. 4), even though 100% is basically the main setup where these models have 30-50 point gaps. So calling this "widespread alignment faking" in the title and abstract is an overclaim.
-
The mitigation mostly just makes the model ignore the tier. The steering vector pushes the model towards the "unmonitored" state, and tier awareness in the reasoning drops from 62% to under 10% for olmo2-7b (Table 15). This only looks like a fix because the unmonitored behavior in VLAF is the moral one. For a model with bad hidden preferences, the same vector would make it act on them everywhere. A one line prompt also does as well or better in some cases (qwen3-14b: 19.9 for prompting vs 26.1 or higher for steering, Table 2).
Questions for authors
Clarifications (could change my assessment):
-
Can you report the absolute compliance rates for the free and paid tiers (Table 1), and for Fig. 4 at p = 0 and p = 1? If the model already complies a lot in the free tier when it is told it's never monitored, then the gap comes from the label and not from monitoring. This is the result most likely to change my score.
-
How much of the gap comes from traces that actually show preference preservation reasoning? How do you rule out the simpler explanation that models just follow the developer's rules when told they are being graded (the rater sycophancy hypothesis from Sheshadri et al., 2025)?
-
What are the actual numbers for Claude Sonnet 4.6/3.7 and GPT-5.4? Does VLAF pick up a known alignment faking model, e.g. Claude 3 Opus if it is still accessible, or a model organism trained to fake alignment?
-
After steering, which tier's compliance changes: free, paid or both?
Suggested experiments: 5. Try the steering vector on a model where the unmonitored behavior is the bad one. That would show whether it actually reduces strategic behavior or just makes the model act like it's unmonitored.
Minor: 6. There are no confidence intervals anywhere, and with 10 samples per scenario some of the smaller differences (Fig. 7, the slopes in Fig. 4) could be noise. 7. The free and paid prompts differ by one token, so a near rank-1 activation difference (Fig. 11) is expected and doesn't say much on its own. 8. Tier awareness for the vanilla models in Table 15 doesn't match Table 6 (e.g. olmo2-7b: 62.0 vs 52.0). Fig. 4 and Fig. 12 are the same plot, and there is a broken reference (??) in App. C.3. Also, is the option order randomized at evaluation? In the C.3 template the developer option is always (a).