- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper argues that model evaluation measures what a model can do and what it has done, but not what it will do under different unobserved conditions. The authors name the decision rule linking capability to behaviour a Policy. They define a Hidden Policy as one that is equivalent to a reference policy across the regimes an observer can access, but diverges in another reachable regime. From this, they argue that behavioural evidence identifies a policy only up to an observational equivalence class.
They evaluated four sandbagging policies using LoRA-based SFT on Qwen3.5-4B. These cross two activation conditions, a fixed marker string or a semantic “being assessed” context, with two activated behaviours, fixed refusal or imitation of a weaker model, on WMDP-style target questions. They then vary questions, activation expressions, model weights through continued training, and multi-turn interaction structure. They found that, the implanted behaviour transfers to new questions more reliably than to rewritten activation expressions.
Critical strengths
- The 2 x 2 factorial grids separates the activation mechanism from the activated behaviour. This rules out the trivial alternatives of general degradation under the gate and unconditional degradation on target questions.
- Testing whether the suppressed behaviour can still be elicited by direct instruction is a useful addition to removal evaluation. The Fine-Pruning result, where gated refusal drops from 64/64 to 6/64 while directly instructed refusal rises from 5/32 to 29/32, is a concrete and informative observation.
- The appendices report all 100 records with counts, intervals, and decision rules. They also include data separation procedures and exact prompt templates. This makes the claims auditable.
Critical weaknesses
-
The central conceptual contribution is trivial. The idea that agreement on a finite set of observed conditions does not identify a model’s behaviour elsewhere is a standard result in ML community. It is well established and have been studied in the fields such as, adversarial ML, underspecification, distribution shift, and backdoor etc. Definitions 1 and 2 restate this in the language of observational equivalence. No theorem, identifiability condition, or bound follows from them, so the claim that behavioural evidence identifies a policy only up to an equivalence class holds by construction. Recasting inputs as “interaction regimes” does not change the underlying problem. The paper should either give a non-trivial formal result or present the framework as terminology rather than a contribution.
-
The experiments do not test the paper’s own motivation. The paper argues that hidden policies matter most when they emerge without deliberate implantation, because then neither their existence nor the relevant regime is known. However, every experiment uses a deliberately implanted, known trigger with MCQ tasks and SFT. No experiment addresses discovery under unknown activation conditions or emergent conditional behaviour. The framing about frontier models and emergent risk is therefore largely unsupported by the evidence. Definition 2 requires exact observational equivalence in the observed regimes. Empirically, the imitation constructions already deviate from their controls outside the hidden regime with T/off and U/on. The ≈ used in Eq. 8 is never given a tolerance. In practice, the paper studies conditional backdoors, not the hidden policies as formally defined.
Questions for authors
As in Weaknesses.
2 comments
Dear Reviewer Pf7o,
We would greatly appreciate it if you could let us know whether our responses have resolved your concerns or if there is anything else we can clarify. Thank you for your time and effort.
Best regards,
Authors
We sincerely thank the reviewer for their insightful comments. Below, we have summarized the major points raised and our responses:
Weakness 1: Conceptual Novelty
frontier risk, let alone as ameta-riskunderlying other frontier risks, such as CBRN capability.Weakness 2: Motivation and Experimental Scope
Please let us know if we can provide any further clarification or supporting information.