acceptodds
Under review as a conference paper at ICLR 2027

Auditing Context-Dependent Safety After Persona-Conditioned Fine-Tuning

Abstract

Safety evaluations commonly attach one score to a fine-tuned checkpoint, although the same weights may be deployed under different system contexts. We audit this assumption in one open 8B model and LoRA pipeline. Models are left untrained or adapted for three rounds with a compliant persona, a principled persona, or generic instruction tuning, then each frozen checkpoint is evaluated through four lenses on the same 313 forbidden prompts. Both persona-conditioned arms retain high refusal under a standard lens and fall under a permissive lens, including a held-out paraphrase; the generic control does not move in the same direction, and a co-primary harmful-usefulness evaluator changes correspondingly. The measurement audit materially changes how this observation can be read: an ASCII-only apostrophe pattern creates treatment-correlated refusal error, and a raised generation cap shows that excluding apparently truncated responses can delete learned behaviour rather than remove artefact. Preregistered validation found the corrected instrument missed no human-labelled refusal and over-fired twice in 283 records, and a second validation over legacy-cohort controls confirms that all 39 correction-flipped records in its four audited seed-42 cells are genuine refusals. Corrected regex–human agreement is ; one author labelled throughout, with pass-1/pass-2 refusal test–retest , so we make no inter-rater claim. We therefore treat safety as a property measured on a checkpoint–context pair, not a checkpoint alone. This is a bounded case study, not a causal estimate of persona semantics: target breadth, response style, prompt authority, update method, and model family remain open.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM safety evaluation, fine-tuning safety, measurement validity, evaluation context, persona conditioning, refusal measurement

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_7DVSOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies context-dependent safety after persona-conditioned fine-tuning in a Llama-3.1-8B-Instruct LoRA case study. Compared with the standard lens, the two persona-conditioned arms show large drops in refusal and higher StrongREJECT harmful-usefulness scores under the compliant lens; the generic instruction-tuning control does not show the same pattern. The authors also show that an apostrophe-matching error in the refusal-pattern set and excluding apparently truncated responses change the reported contrasts. The paper argues for evaluating safety on checkpoint–context pairs and auditing the measurement instrument.

Critical strengths
  • The paper shows why a standard-lens safety score may not describe a fine-tuned checkpoint under other contexts. Both persona-conditioned arms retain high refusal under the standard lens but show large drops under the compliant lens, while the generic instruction-tuning control does not. The pattern persists under a held-out paraphrase, and StrongREJECT reports higher harmful usefulness under the compliant lens. This supports evaluating safety on checkpoint–context pairs.

  • The paper shows that measurement decisions change the reported comparisons. An apostrophe mismatch in the refusal-pattern set causes errors concentrated in the control arms; correcting it reverses the generic control’s contrast in an earlier cohort. A same-checkpoint generation-cap check also shows that excluding apparently truncated responses removes part of the persona-arm contrast. These audits identify specific scoring and exclusion errors that safety evaluations need to check.

Critical weaknesses
  • The paper provides a detailed case study, but its broader contribution remains limited. The main checkpoint–context safety pattern is established in one Llama-3.1-8B LoRA setting; the Qwen experiment yields a different relative pattern, and the 1B study supports only selected parts of the measurement analysis. Since prompt sensitivity after fine-tuning and limitations of automated safety scores have been studied before, the paper’s main addition is to quantify these issues in one pipeline. It remains unclear whether the proposed audit procedure yields new insights beyond this case.

  • The experiments show the weight-by-lens interaction but do not explain its cause. Response style was not matched across training groups, and the planned matched-data test produced no trained model. It remains unclear whether the persona itself matters or the models simply follow the compliant system prompt more strongly after tuning. The reported κ = 0.995 also needs clarification: if it uses the 283 labelled cases with two errors, agreement cannot exceed 0.993.

Questions for authors
  • What are the main methodological advances of the proposed reusable audit procedure beyond existing work on fine-tuning safety, prompt-template sensitivity, and automated safety evaluation? Which elements of the procedure are supported by evidence beyond the specific model family and LoRA pipeline studied in this paper?

  • Could the authors report descriptive comparisons of the training responses across arms (e.g., length, refusal wording, or tone) and discuss whether these differences track the weight-by-lens effects? Given that the prompt-strength results are also compatible with increased sensitivity to system prompts after tuning, what in the current evidence, if anything, favors a persona-specific account?

  • The same 313 StrongREJECT prompts were used for the pre-update screen and the main grid, so it is unclear how well the screening result generalizes beyond this set. Given the Qwen result, why is a stable baseline needed to interpret the difference-in-differences result?

  • How was the reported corrected regex–human agreement of κ = 0.995 calculated? The manuscript reports two false positives among 283 human-labelled records, implying an observed agreement of approximately 0.993, which Cohen's κ cannot exceed. Could the authors clarify the sample, confusion matrix, and calculation procedure used for this statistic?

Reviewer_QTejOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper is a bounded case study of whether a safety score attached to a fine-tuned checkpoint survives a change of deployment context. Llama-3.1-8B-Instruct is adapted with LoRA for three rounds under four arms (untrained, compliant-persona SFT, generic instruction tuning, principled-persona SFT; roughly 410 user–assistant rows each, no system turn) and every frozen checkpoint is evaluated on the 313 StrongREJECT forbidden prompts under four system-prompt "lenses" (no persona, permissive, principled, and a held-out paraphrase of the permissive one). The headline is a weight-by-lens interaction: both persona-trained arms keep about 0.95 refusal under the standard lens and fall to about 0.6 under the permissive lens, giving difference-in-differences values of 0.306 and 0.381 against the untrained reference, while the generic-SFT control reads −0.012. A co-primary StrongREJECT harmful-usefulness score moves in the same direction, from 0.014 to 0.144 and 0.007 to 0.117. The second half is a measurement audit: the custom regex refusal detector matched only ASCII apostrophes while Llama emits U+2019, producing treatment-correlated error concentrated in control cells that flipped the sign of the control contrast in one cohort; and excluding "truncated" responses removes an arm-differential mid-sentence-EOS behaviour rather than a cap artefact. Human validation of the regex is by a single author on an earlier cohort. A Qwen2.5-7B replication gives negative interactions for every arm because the untrained reference itself collapses from 0.661 to 0.313, and a preregistered matched-data factorial stopped before training when a manipulation gate failed.

Critical strengths
  1. Treatment-correlated instrument error is demonstrated, quantified, and shown to flip a conclusion. The apostrophe defect moved the generic-control interaction from −0.070 to +0.022 in one cohort, reversing its sign. For anyone deploying regex or small-judge refusal metrics inside a red-teaming or guard-evaluation pipeline, this is a concrete and reproducible failure mode, and the audit checklist in Section 7 is sound advice.
Critical weaknesses
  1. The authors cannot reproduce their own numbers, and say so. Section 5 reports that a cohort retrained "from the same data, hyperparameters and seed" agrees with the original at only 20 to 38% on the trained arms, while the adapter-free arm agrees byte-for-byte. This is not confined to individual records: the same retraining moved an aggregate statistic, the arm-differential mid-sentence residue, from 7.0% and 11.5% to 15.3% and 27.8%, roughly doubling both. Every headline number is therefore a property of one particular training run rather than of the recipe, and the three-seed ranges reported as variation do not bound that instability. A paper whose stated contribution is measurement discipline cannot offer its own 20 to 38% self-agreement figure as a lesson for others while leaving its estimates resting on it.

  2. The primary instrument is a hand-written regex, validated by one author, on a different cohort from the one that produces the results. Refusal is the binary output of eight hand-written patterns, exactly the surface-form instrument the paper warns against. The reported κ of 0.995 is one author's agreement with their own labels, the test–retest κ of 0.956 is the same author twice, and the paper states plainly that no inter-rater claim is made. Critically, that validation covers earlier mixed-environment seed-42 records, and the paper concedes the frozen grid's "completions were not relabelled record by record". The headline grid is therefore unvalidated. The co-primary metric does not rescue it: a 35 to 40 point refusal drop is accompanied by only a 13 to 14 point rise in StrongREJECT harmful usefulness, so roughly two thirds of the "lost refusals" are not useful harmful answers and the paper never characterises what they are. Since 7 to 28% of persona-arm responses end with a mid-sentence end-of-sequence token, a style or formatting shift remains a live alternative reading of the entire effect.

  3. The identifying experiment was preregistered, failed its own gate, and was never run, leaving the central claim without a cause. Appendix G reports that the matched 2×2 over persona presence and target breadth produced no training result, since the narrow non-persona cell passed two of four narrowing metrics against a required three. What remains is the unmatched factorial, which the paper acknowledges "separates persona from target breadth, not response style", so the strongest surviving rival, increased prompt authority after tuning, cannot be excluded. The second model family then behaves in the opposite direction for every arm. On the two families tested, the phenomenon replicates on one, is uninterpretable on the other, and is attributable to nothing in particular.

Questions for authors
  1. Given 20 to 38% record-level agreement on retraining from an identical configuration, what do the reported three-seed ranges estimate? How many independent retrainings would be needed before the arm-by-lens interaction is distinguishable from run-to-run variation?

  2. Please report a 2×2 of regex-refusal against StrongREJECT above 0.5, per arm and lens, on the frozen grid. Of the compliant-lens non-refusals in the persona arms, what fraction are substantive compliance, deflection, or mid-sentence fragments? (Affects assessment) Did the synthetic SFT responses for arms 1 and 3 themselves end mid-sentence, for instance because the generator hit a token cap? If so, the learned mid-sentence behaviour in Section 5 is a data-pipeline artefact.

  3. Every trained arm falls below the untrained model on IFEval under every lens, with arm 3 dropping from 0.780 to 0.620 with no system prompt and from 0.680 to 0.440 under the principled lens. Does the lens interaction survive at a training setting that preserves general instruction following?

  4. Replacing the regex with Llama-Guard-3, WildGuard, or the StrongREJECT rubric evaluator as a third outcome would cost little and remove the main objection to the headline grid.

Reviewer_wgbsOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates whether safety scores assigned to fine-tuned checkpoints remain valid under deployment context changes. The paper concludes that safety should be evaluated on checkpoint–context pairs and that evaluation instruments should be audited for treatment-correlated error. The finding is a weight-by-lens interaction: both persona-trained arms retain high refusal (~0.95) under a standard lens and fall substantially (~0.6) under a permissive lens, while the generic-SFT control does not move. A secondary measurement audit shows that an ASCII-only apostrophe pattern in the refusal detector was treatment-correlated, shifting one contrast’s sign, and that a truncation heuristic conflated a cap artefact with learned behaviour.

Critical strengths

The motivation is sound, same weight are deployed in different context but attached one score for a finetuned checkpoint. Paper demonstrates that decision change the reported comparisons, apostrophe mismatch in refusal leads to error in control arms.

The held-out paraphrase lens strengthens the face validity of the pattern against exact-string memorisation: the interaction is larger under the paraphrase than under the trained wording, which is the correct direction for ruling out surface memorisation.

The paper addresses a real and important question, whether fine-tuning-induced change in safety are context dependent and the measurement-audit material is genuinely useful and needed.

Critical weaknesses
  1. Section 5 reports that retraining “from the same data, hyperparameters and seed” reproduces arm-level statistics at only 20–38% on the trained arms, while the adapter-free arm agrees byte-for-byte. Every headline number in the main grid is therefore a property of one specific instantiation of the training recipe rather than of the recipe itself. The three reported seeds are draws from a fixed set of random-number states in a single frozen training environment, not independent retrainings, and the paper explicitly states they “do not estimate variation from retraining.”

  2. The reported κ = 0.995 and test–retest κ = 0.956 are both the same author’s agreement with their own labels; the paper states explicitly that no inter-rater claim is made. For a paper whose stated contribution is measurement discipline, deploying an unvalidated version of its own primary outcome on the cells that matter most is a structural inconsistency.

  3. There is also an arithmetic inconsistency in the reported κ. Two errors among 283 human-labelled records yields observed agreement of 281/283 ≈ 0.993. Cohen’s κ is bounded above by observed agreement and cannot reach 0.995 from this confusion matrix. The paper should clarify the sample size, confusion matrix, and calculation.

  4. The preregistered causal identification failed its own gate, and the surviving design cannot rule out the strongest rival account. The strength-ladder evidence (interaction increasing monotonically from 0.288 to 0.501 as the lens is made more forceful) is directionally consistent with prompt-authority amplification rather than persona specificity; the ordering is, by the paper’s own admission, not robust to the truncation convention. The Qwen2.5-7B experiment produces negative interactions for every arm because the untrained reference itself collapses, which the paper classifies as a gate failure rather than a replication failure.

Questions for authors
  1. Given 20–38% record-level agreement on retraining from an identical configuration, how should readers interpret the reported three-seed ranges? Would a prospective replication with freshly trained checkpoints recover an interaction in the same direction as the headline grid, and if so within what range?

  2. Did the synthetic SFT responses used to train arms 1 and 3 themselves end mid-sentence due to a generation token cap? If the training completions themselves contained mid-sentence endings, the learned mid-sentence EOS behaviour described in Section 5 is a data-pipeline artefact rather than evidence of a safety-specific disposition.

  3. Does the weight-by-lens interaction survive at a hyperparameter setting where IFEval strict accuracy remains within 5 percentage points of the untrained baseline under every lens?

Reviewer_ZkF9OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that a safety score attached to a fine-tuned checkpoint is under-specified, and proposes reporting safety on a pair of weights and system prompt. The authors fine-tune Llama-3.1-8B-Instruct with LoRA for three rounds under four arms (untrained, compliant persona, generic instruction tuning, principled persona). They evaluate each checkpoint on the 313 StrongREJECT prompts under four system prompts (none, compliant, principled, paraphrased compliant), using a refusal regex and the StrongREJECT evaluator, with a difference-in-differences interaction against the untrained model. Both persona arms keep high refusal with no system prompt but lose 0.31 to 0.48 more refusal than the untrained model under the permissive prompts, while the generic control stays near zero. The paper also reports a measurement audit: an apostrophe-matching bug that biased refusal scores by arm, a truncation heuristic whose exclusion removes real signal, and a screen showing that only two of six base models are stable across system prompts.

Critical strengths
  1. The estimand is clearly defined and its uncertainty is reported honestly. The interaction is stated once for both metrics with explicit sign conventions (ll. 151-157), and the paper separates seed ranges from within-seed bootstrap intervals (ll. 067-074).
  2. The item-level evidence is clean. Across all six persona arm-seed cells, no prompt moves from non-refusal to refusal under the permissive prompt (ll. 206-210), so the effect is not an aggregation artefact.
  3. The measurement audit is a useful practical contribution. It shows that a regex refusal metric can carry error correlated with treatment, enough to flip the sign of a control contrast (ll. 332-349), and that excluding "truncated" responses removes arm-specific signal rather than noise (ll. 366-401). Both lessons transfer to other safety evaluations.
  4. The paper is transparent about limits and negative results, including the failed preregistered matched-data gate (App. G) and the single annotator (ll. 424-425).
Critical weaknesses
  1. The novelty is limited. That measured safety depends on the system prompt and on fine-tuning templates is already established, as the authors acknowledge (ll. 106-108, 120; Lyu et al. 2024, Mu et al. 2025, Gringras 2026). The new element, an interaction between persona fine-tuning and a permissive system prompt, is shown descriptively and its cause is left open (ll. 437-441; Table 6). The framing in terms of deployment and system contexts (ll. 035-038, 481-483) is broader than what is tested, which is a single system-prompt string in single-turn generation, and the motivation via self-adapting models (ll. 113-118) is disclaimed in the same paper (ll. 042-043).
  2. The experiments lack depth. The headline result rests on one model (Llama-3.1-8B-Instruct), one LoRA recipe, about 410 training rows per arm, and one benchmark (StrongREJECT, n = 313) (Table 1). The only second family tested cannot be interpreted under the authors' own gate (ll. 455-464), and the screen in Table 3 leaves only Llama models eligible. There are no current-generation models, no larger scales, no second harmful benchmark and no over-refusal evaluation, and the benign check uses 50 IFEval prompts on a different pipeline (ll. 255-265).
  3. The validity of the headline numbers is not established. Human validation was run on an earlier cohort, not on the reported grid, and covers only two of the four system prompts (ll. 068-070, 146-148, 179-182). A single author labelled all records. Retraining with identical data, hyperparameters and seed reproduces only 20 to 38% of record-level outputs (ll. 378-379, 479-480), which raises doubt about how stable the reported interactions are across three seeds.
  4. The design does not isolate the claimed effect. The persona arms differ from the controls in response style, including typography and mid-sentence stopping (ll. 338-340, 386-394). The generic control is generated by the base model itself (ll. 132-133), which makes a null result close to guaranteed, and the matched factorial intended to address this did not run (App. G).
  5. The presentation is hard to follow. Results draw on at least five different cohorts and weight regimes (Secs. 3.5, 5; App. C), the text is dense with caveats, and no table maps each reported number to its source checkpoint.
Questions for authors
  1. What is the main claim of the paper? Is it about evaluation methodology, about how safe a model stays after fine-tuning, about persona customisation, or about self-adapting models? The introduction and related work point to all four (ll. 035-043, 113-118), but the experiments test only a fixed, researcher-run LoRA update.
  2. How do the authors separate model-level from system-level safety? The "context" studied is a single system-prompt string (ll. 137-144), while a deployed system also involves tools, retrieval, guardrails and multi-turn interaction, each with its own risk profile. Is the proposed checkpoint-context score meant to replace model-level evaluation, or to add a system-prompt sweep after fine-tuning?
  3. What concrete deployment scenario does the study represent? Who fine-tunes, on what data, and under which system prompt is the model served? PersonaHub personas are used only to vary topic and do not represent users (ll. 127-130), so the link to a real use case is unclear.
  4. Given that context-dependence of measured safety is already established (ll. 106-108, 120), what is the specific new finding the authors would want a reader to take away, beyond the measurement audit?
  5. Do the interactions reproduce in the retrained cohort (ll. 373-379), and what causes the 20 to 38% record-level agreement between runs with identical data, hyperparameters and seed?
  6. Which generator and decoding setup produced each arm's training responses, given that the persona arms carry ASCII apostrophes the base model does not emit (ll. 132-133, 338-340)? Could response style, rather than persona, explain the effect?
  7. What would it take to make the claim general? For example, at least one current model family, a larger scale, a second harmful benchmark, an over-refusal set, and human validation on the reported grid across all four system prompts (ll. 146-148, 455-464).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.