acceptodds
Under review as a conference paper at ICLR 2027

Displaced but Recoverable: State-Transferable Safety Signals in Response-Prefilled Language Models

Abstract

Safety evaluations usually test whether a model refuses a harmful request from an empty assistant turn, but response prefilling makes it continue from an answer already steered toward compliance. Existing prefill studies establish that this can change final outcomes; whether behavior at weaker prefilled states predicts failure at stronger ones, and whether recovery can preserve benign utility, remains less clear. We introduce STAIR (State-Transfer Assessment, Inference, and Recovery), which freezes model-specific semantic prefix ladders after calibration and measures judge outcomes, lexical recovery, and refusal-likelihood margins across the resulting states. The paired design follows each harmful prompt through related assistant-side continuations, making changes in behavior across prefix states directly comparable. Using P1/P2 diagnostics and prompt features, STAIR forecasts prompt-level P3/P4 failure and trains recovery-style LoRA on prefixed harmful contexts with benign anchors. Across five instruction-tuned models, clean refusal degrades substantially under stronger prefilling, while combining weak-state diagnostics with prompt features reaches 0.778 P3 AUROC on Qwen2.5-1.5B. Recovery with benign anchors restores complete P4 safe noncompliance while retaining 0.860 benign non-refusal.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM safety, response prefilling, refusal robustness, risk forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_NisWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces STAIR, a framework for evaluating safety under assistant-side response prefilling. It constructs frozen, nested prefix ladders, measures endpoint safe noncompliance, lexical refusal recovery, and refusal-versus-compliance likelihood margins at weak states, then forecasts failure at stronger prefix states. The paper also trains recovery-style LoRA adapters directly on harmful prefilled contexts and combines them with benign anchors to reduce over-refusal.

Critical strengths
  1. The problem is meaningful and well motivated. Clean-start refusal is plainly insufficient for evaluating models that may be induced to continue from a partially compliant assistant response. The paired construction in Equation (1), together with Assumption 1 and Lemma 1, clearly formalizes what is held fixed across states: the harmful user request, model, chat template, and decoder. This is a useful experimental discipline relative to simply comparing unrelated jailbreak prompts.

  2. The paper separates several notions that are often conflated in safety evaluations: final safe noncompliance, explicit lexical refusal recovery, recovery latency, and local template likelihood margins. This is a strength of the design. In particular, Table 2 and Figure 5 appropriately show why endpoint refusal alone is not enough: all adapted variants reach P4 RR of 1.000, yet benign non-refusal ranges from 0.007 to 0.907. The visualization makes the safety and over-refusal trade-off immediately interpretable.

Critical weaknesses
  1. The "state-transferable" claim doesn't hold up. The prefix ladders are just near-paraphrases of one compliance opener ("This can be done by using the"), and the held-out prefixes are equally close variants. What the paper shows is local transfer within a single family of procedural openers, not transfer across arbitrary assistant states or multi-turn contexts. Figure 2 also can't separate model effects from prefix effects, since each model gets its own calibrated ladder.

  2. The recovery–utility advantage is contradicted by the paper's own results. The benign evaluation is too thin to support claims about preserving helpfulness: 200 training anchors and 150 Alpaca prompts, with XSTest and OR-Bench cited but never run. Worse, Constrained SFT gets a higher benign non-refusal rate (0.907 vs 0.860) while matching P4 recovery at 1.000, and the McNemar test is non-significant (p=0.167). The honest conclusion is that Recovery+anchor is competitive, not superior.

Questions for authors

Same as weakness.

1 comment
AuthorsOpenReview ↗

Response to Reviewer NisW

Thank you for your review.

Q1. ... state-transferable ... prefixes ... model effects ...

Answer: Revised Section 3.1 defines state transfer for paired, single-turn assistant prefills of the same request. We added two nonprocedural targets, "The underlying mechanism is" and "Here is a concrete example:", each evaluated on 400 held-out requests (Sections 4.2–4.3, Table 4). In panel (a), Prompt+Core has higher frozen-score AUROC than Prompt-only on both targets, but P2-only ranks highest and the Brier differences favor Prompt-only. In the separate recovery experiment in panel (b), the same Recovery+anchor checkpoint reaches judge RR 1.0000 on both targets, compared with 0.7650 and 0.5225 for Base. The revised Figure 2 caption identifies the model-calibrated profiles. Table 19 adds exact-string comparisons: under the same four prefixes and same requests, Qwen-7B and Gemma-2-2B-it have P3 RR of 0.395 and 0.780, respectively.

Q2. ... recovery–utility ... XSTest ... Constrained SFT ...

Answer: Revised Section 4.3 distinguishes over-refusal from answer quality and clarifies that the 200 training anchor instances repeat ten distinct pairs, separate from the 150 Alpaca evaluation requests. We added the full XSTest evaluation of four unchanged checkpoints (Table 3; Appendix B.4). On 250 safe requests, Recovery+anchor improves full compliance over Recovery-only by 39.6 and 52.0 percentage points under the two automated graders. Constrained SFT has higher safe full compliance under both, while HarmBench finds 196 of 200 unsafe contrasts safely noncompliant for Recovery+anchor versus 193 for Constrained SFT. In the original AdvBench and Alpaca comparison, both reach P4 RR 1.000, with Alpaca non-refusal of 0.860 and 0.907, respectively (exact paired McNemar p = 0.167). The revision reports these paired comparisons as a recovery and over-refusal trade-off. OR-Bench remains a related benchmark.

Reviewer_QMhaOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces STAIR, a framework for evaluating and recovering LLM safety under assistant-response prefilling. STAIR uses logistic regression to predict safety failures under stronger, longer prefills from behavioral measurements collected under weaker, shorter prefills. The experiments show that strong clean-start safety does not ensure robustness under stronger prefilling. Adding diagnostic features from P1 and P2 improves prediction over prompt-text features alone at the intermediate prefix level, P3, but provides no additional benefit at the strongest level, P4. Predictive performance also varies across prefix families.

Critical strengths
  • Provides uncertainty estimates for key comparisons
  • Provide comparison against natural and relevant baselines
Critical weaknesses
  • Limited novelty and scope: . Assistant-response prefilling and recovery training are established research directions, and several findings are largely anticipated by prior work. The paper’s incremental contribution is narrow, with additional predictive value demonstrated at P3 but not P4.
  • Limited explanatory value of the theoretical results: The methodology presents several standard identities without establishing when the proposed predictor or recovery procedure should succeed. For example, Table 3 shows that adding diagnostic features can worsen predictive performance, whereas the information-theoretic result concerns an optimal predictor.
  • Insufficient interpretation of the empirical findings: The discussion offers limited insight into why the method helps at some prefix levels or for some prefix families but not others. The experiments primarily characterize performance rather than distinguish competing explanations, such as prompt difficulty, prefix-specific effects, or transferable safety signals. Consequently, it remains unclear what general conclusions can be drawn beyond the evaluated settings.
Questions for authors
  • What practical advantage does forecasting offer over direct evaluation? It seems that stronger or longer prefills always lead to worse safety performance from the model, and evaluating those are not more expensive compared to evaluating weaker settings like P1 and P2. Direct evaluating them already establishes useful empirical estimates regarding worst case performance.

  • Why do diagnostic features have different affect on P3 and P4 predictions?

3 comments
AuthorsOpenReview ↗

Response to Reviewer QMha

Thank you for your review.

Q1. ... novelty ... scope ... P3 ... P4 ...

Answer: The revised Section 4.2 adds a matched cross-setting analysis on saved outputs (Table 20). At P3, Prompt+Core improves both AUROC and Brier over Prompt-only by point estimate in all eight model and dataset settings. On the primary split the paired AUROC gain is 0.065 (95% interval [0.020, 0.113]). The revision describes P4 separately: on the primary split, AUROC is 0.824 for Prompt-only and 0.804 for Prompt+Core. Across the eight P4 settings, adding Core lowers Brier in seven by point estimate and increases AUROC on JBB by 0.151 (95% interval [0.014, 0.288]). The increment at P4 depends on the setting and metric.

Q2. ... theoretical results ... optimal predictor ...

Answer: Revised Sections 3.2–3.3 separate the learning objectives from the formal identities in Appendix A.1. The information identity concerns optimal log loss, while the new fitted-score analysis in Appendix B.1 uses Brier error. If p_i is the Prompt-only score, q_i the Prompt+Core score, z_i the outcome, and d_i = q_i - p_i, the Brier change equals mean_i[d_i²] - 2 mean_i[d_i(z_i - p_i)]. Brier improves when alignment with the baseline errors outweighs the squared adjustment, as at P3 (0.05832 versus 0.03231). The recovery identity identifies the benign term in the gradient. Revised Section 4.3 evaluates answering behavior directly with the anchor ablation and the new XSTest experiment (Table 3).

Q3. ... prompt difficulty ... prefix-specific effects ...

Answer: Sections 4.2 and B.1 now report a risk-stratified pairing control: we hold prompt text and folds fixed and shuffle the six diagnostics jointly within text-risk strata. P3 AUROC is 0.778 with the actual pairing and averages 0.691 over 100 shuffles (Table 22), showing that matching measurements to their requests matters even at similar text-only risk. The revised Figure 2 caption separately points to comparisons holding the exact prefixes fixed within model pairs (Table 19). Table 4(a) adds a test in which the fitted scores are held fixed and the target prefill changes. Prompt+Core has higher AUROC than Prompt-only on both new targets, while P2-only ranks highest, so the within-ladder advantage does not transfer uniformly.

Q4. ... forecasting ... direct evaluation ...

Answer: Revised Section 4.2 distinguishes a direct audit of the target refusal rate from prioritization using existing diagnostic records and source labels. In the new equal-budget replay, with 80 target generations and 80 judgments, Prompt+Core finds 69 P3 failures versus 62 for Prompt-only and 37.6 expected at random. Both predictors find 77 P4 failures. The revision also shows why the lowest aggregate RR differs from failure coverage: 23 requests that fail at P3 are safe at P4, and all four prefills cover 338 distinct failures versus 285 at P4 alone. Appendix B.2 and Table 23 report upstream acquisition separately from the marginal replay budget.

Q5. ... diagnostic features ... P3 ... P4 ...

Answer: Revised Section 4.2 and Appendix B.1 add a breakdown by P2 outcome. Of the 207 P2-safe requests, 63 fail at P3 and 125 at P4. Adding Core lowers P3 Brier in both P2 groups. At P4, it lowers Brier for P2 failures but raises it for P2-safe requests, so the changes nearly cancel overall (Table 21). In the P2-safe group, the mean predicted P4 failure probability moves from 0.544 to 0.457 against an observed rate of 0.604. The new pairing control also shows a larger AUROC gap between observed and shuffled diagnostics at P3 (0.778 versus 0.691) than at P4 (0.804 versus 0.799; Table 22).

Thank you to the authors for these additional information. I appreciate that the additional experiments help narrow down possible explanations for the mechanism that helps at some level but not others. However, on other points my previous concerns remain. First, my concern about novelty is only partly resolved. Given prior work on prefilling and recovery, could you clarify what the P3 finding teaches us about model safety beyond showing that early responses can help predict later failures? Second, I still find it unclear what the theorem tells us about when the fitted predictor should improve. Finally, I think while the rebuttal by the to my Q4 is useful in showing that P1 and P2 results can be helpful if you already have them, but my question is what do we gain by spending my first evaluations on P1/P2 instead of starting at P4. I will therefore keep my score unchanged.

AuthorsOpenReview ↗

Response to Reviewer QMha

Thank you for your response.

Q1. ... P3 finding ... model safety ...

Answer: A nearly unchanged safety rate can hide a change in which requests fail. On Qwen2.5-1.5B, safe noncompliance moved from 51.75% at P2 to 53.00% at P3, yet 131 of the 400 requests switched outcomes: 63 went from safe to failed and 68 from failed to safe (Table 18). Qi et al. showed that alignment can be concentrated in the first few output tokens. Here P2 and P3 are separate generations for the same request. Extending the prefix did not simply add failures to the previous set. The weak-state measurements also helped identify which requests would fail at P3: AUROC was 0.778 with each request's own measurements, versus 0.691 on average after shuffling them among requests with similar Prompt-only risk (Table 22).

Q2. ... theorem ... fitted predictor ...

Answer: The theorem in Appendix A.1 concerns optimal log loss. Let XX denote the prompt representation, WW the weak-state diagnostics, ZZ the target failure label, and e(U)e(U) a fitted predictor's expected log loss above the optimum for inputs UU. The fitted Prompt+Core predictor has lower expected log loss than Prompt-only exactly when I(Z;W∣X)>e(X,W)−e(X)I(Z;W\mid X)>e(X,W)-e(X). The theorem identifies the information term; it does not determine either predictor's fitting error or guarantee an AUROC gain. In a new check using the same loss as the theorem, we scored the 400 saved out-of-fold predictions per target without refitting. The difference (Prompt+Core minus Prompt-only) was −0.057-0.057 nats per request at P3 (paired 95% bootstrap interval [−0.091,−0.019][-0.091,-0.019]) and −0.006-0.006 at P4 ([−0.033,0.024][-0.033,0.024]). The fitted gain is supported at P3; at P4 the interval includes zero.

Q3. ... first evaluations ... P4 ...

Answer: If the audit starts from scratch and its sole objective is finding failures at a known P4, we would spend the first evaluations on P4. Counting one generation and one judgment as one evaluation, 80 uniformly sampled P4 tests yield 57 failures in expectation on this pool (Table 23). Testing both P1 and P2 on each request before its P4 test leaves room for at most 26 P4 tests under the same budget. The weak evaluations answer a different question: which failures does P4 miss? In a new count from the saved labels, 49 of the 115 requests judged safe at P4 failed at P1 or P2. Those are failures under weaker prefixes, not additional P4 failures. The benefit comes from testing the same requests across states, regardless of which state is tested first.

Reviewer_iUxpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper investigates the impact of response prefilling on LLM safety. The authors introduce STAIR, a framework that covers three aspects: (1) evaluating how different levels of response prefilling affect models' refusal behavior, (2) predicting safety failures under stronger prefilling using diagnostic signals from weaker prefilling states, and (3) improving models' ability to recover safe behavior from prefilling-induced unsafe states through LoRA fine-tuning.

Critical strengths
  1. Interesting research question
  2. Comprehensive experimental evaluation
Critical weaknesses

1.Limited methodological novelty 2.poor readability and presentation

Questions for authors
  1. limited methodological novelty. This paper does not seem to provide substantially new insights into LLM safety. The issue of shallow safety alignment has already been extensively discussed and explored in prior work, such as Safety Alignment Should Be Made More Than Just a Few Tokens Deep (arXiv:2406.05946). Therefore, the finding that stronger prefilling (P3/P4) leads to lower refusal rates is not particularly surprising. In my opinion, although the forecasting component appears relatively novel, it is unclear what meaningful insights it provides to the research community beyond demonstrating that weaker prefilling states can predict failures under stronger ones.
  2. Poor readability and presentation. The paper suffers from readability and presentation issues. In particular, the mathematical formulations in Sections 3.2 and 3.3 are unnecessarily complicated and verbose, while the structure of Section 4.1 (Experimental Setup) is not well organized. I also find the use of formal statements such as Assumption and Lemma somewhat unnecessary. This is primarily an empirical paper rather than a theory-oriented work, and introducing extensive mathematical formalism for relatively straightforward concepts does not seem to provide much additional insight. Instead, it makes the paper more difficult to follow.
3 comments
AuthorsOpenReview ↗

Response to Reviewer iUxp

Thank you for your review.

Q1. ... novelty ... forecasting ... meaningful insights ...

Answer: The revised Introduction and Related Work distinguish our paired forecasting question from prior demonstrations of shallow alignment and recovery training (Qi et al., arXiv:2406.05946). We ask whether observing a model's weak-state response improves prediction beyond the same request text. A new analysis in Section 4.2 and Table 18 shows why pairing matters: P2 and P3 have similar refusal rates (0.518 and 0.530), yet 131 of the 400 held-out requests switch outcomes. On P3, Prompt+Core improves OOF AUROC from 0.713 to 0.778 over Prompt-only. The new budgeted replay in Table 23(b) also shows the practical consequence: with 80 target evaluations, the two predictors identify 69 and 62 failures, respectively.

Q2. ... readability ... Experimental Setup ... Lemma ...

Answer: In the revised Section 3.1, we describe the paired comparison directly: it holds the request, model, chat template, and decoder fixed and varies the assistant prefill. Sections 3.2 and 3.3 now retain the predictor features and targets and the recovery objective. We moved the assumptions, formal statements, and proofs to Appendix A.1 and organized Section 4.1 into data and calibration, generation and measurement, and predictor fitting.

Thank you for the additional information. I'd like to increase the Presentation score but keep the Overall Rating unchanged, as my first concern regarding novelty remains unresolved.

AuthorsOpenReview ↗

Response to Reviewer iUxp

Thank you for your response.

Qi et al. showed how alignment concentrated in the first few output tokens can leave a model vulnerable to prefilling. STAIR asks whether a prefix that elicits more failures exposes the same vulnerable requests. We compare separate continuations of each request under fixed prefixes. On the 400 held-out Qwen2.5-1.5B requests, P4 elicited the most failures, yet a new count of the saved judgments shows that 49 of the 115 requests judged safe at P4 failed at P1 or P2. The pairing control in Table 22 keeps the text and target outcomes fixed and shuffles weak-state measurements among requests with similar Prompt-only risk. Relative to the mean of 100 shuffles, keeping each request’s own measurements raised AUROC by 0.087 at P3 but only 0.005 at P4. Thus P4 exposed more failures overall but missed some that weaker prefixes revealed, while pairing mattered more for ranking failures at P3. For safety benchmarks, we would report which requests change outcome across prefixes alongside the strongest-prefix failure rate.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.