acceptodds
Under review as a conference paper at ICLR 2027

When Do Model Internals Help? Exploring the Role of Representation Engineering in LLM Safety

Abstract

Reliable AI safeguards require both control mechanisms that reduce unsafe behavior and monitoring mechanisms that detect safety risks during model interactions. Established behavioral safeguards include alignment methods that optimize model outputs and text monitors that assess interaction text. Representation engineering instead reads or modifies internal model states, but the relative strengths of these approaches remain unclear because they are often evaluated under different settings. We present a matched evaluation across two tracks. For safety control, we compare DPO, a behavioral alignment method, with three representation steering methods across robustness, practicality, and granularity. DPO provides the strongest overall control and generally improves with increasing training data, although its safety can degrade after subsequent benign fine-tuning. Representation steering remains competitive primarily in low-data settings, particularly with high-quality contrastive data. For safety monitoring, we compare representation probes with fine-tuned and open-weight text monitors across full-response detection, early detection, and computational cost. Specialized text monitors achieve the strongest overall detection accuracy, while representation probes remain competitive at substantially lower marginal cost. Finally, monitor-guided interventions recover much of the safety lost by DPO after benign fine-tuning, with little additional over-refusal. Overall, representation engineering does not generally replace behavioral safeguards, but offers practical advantages under specific conditions and can provide complementary safety benefits.

Primary area:
interpretability and explainable AI
Keywords:
Representation Engineering, Large Language Model Safety, Safety Alignment, Activation Steering, Activation Probing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_d5TBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper compares behavioral safeguards and representation engineering for safety control and monitoring. For control, it evaluates DPO, CAA, probe-based steering, and flow-based steering using matched safe–unsafe training pairs across three models. For monitoring, it compares four representation probes with a fine-tuned text monitor and Qwen3Guard on native Qwen2.5-32B trajectories. It also evaluates post-generation blocking, corrective regeneration, and stronger steering after benign fine-tuning.

DPO provides the strongest control in the evaluated settings, while Flow is competitive with very small, high-quality datasets. Text monitors achieve stronger overall detection, and native probes require much less estimated marginal computation. Replay-based monitoring substantially reduces harmful outputs after benign fine-tuning. The low-cost native monitoring result and the safety-recovery result are established in different experimental settings.

Critical strengths
  1. The matched-data control comparison provides useful evidence for choosing between alignment and steering. The paper also examines data scaling, cross-domain transfer, and sensitivity to benign fine-tuning rather than relying on one aggregate safety score (Sec.3, Tables 1, 10, 14, and 16). The full-budget layer comparison is particularly useful because it shows that initial safety does not predict persistence after fine-tuning.
  2. The monitoring analysis distinguishes discrimination, realized false positives, and detection timing, and explicitly separates native activation reuse from replay (Sec.4, Appendix C.4, and Sec.5). The text-monitor integration baseline in Table 24 further shows that the recovery benefit is not unique to representation probes. These controls make the conclusions more credible.
Critical weaknesses
  1. Limited generalization across model architectures. The control experiments focus on Qwen2.5 and Llama-3.1, while native monitoring is evaluated only on Qwen2.5-32B (Sec.3–4). Although Appendix D includes Qwen3-4B Thinking, its trajectories are replayed through Qwen2.5-32B rather than evaluated using its native representations. Since the paper studies the utility of internal representations, it remains unclear whether the relative performance of behavioral and representation-based safeguards extends to newer MoE, hybrid, or reasoning models. The broader conclusions therefore require additional validation or a narrower scope.
  2. Incomplete coverage of existing intrinsic monitoring methods. The monitoring comparison evaluates four probe formulations on a fixed layer but omits recent intrinsic guardrail methods (e.g., ClaudeCLS++, PlugGuard, SingProbe). Prior work already provides evidence for efficient native monitoring and monitor-guided intervention. The paper should clarify what its monitoring findings add beyond this evidence and evaluate a representative recent method under the same protocol. Without such a comparison, it is unclear whether the reported advantage of text monitors reflects a broader difference between the paradigms or the particular probe configurations evaluated here.
Questions for authors
  1. Suggested additional validation: Could you evaluate representative newer MoE, hybrid, or reasoning models, using their native representations for monitoring, to test whether the relative performance reported in Sec.3–4 persists?
  2. Clarification: What do the monitoring findings establish beyond recent intrinsic guardrail work such as SingProbe? Suggested additional validation: Could you include a representative recent intrinsic monitoring method under the same training-data and evaluation protocol to distinguish method selection from a broader difference between internal and text-based monitoring?
  3. Clarification: How are the 1% and 5% thresholds selected with 51 safe full-response calibration trajectories and 28 safe streaming calibration trajectories? Please specify the quantile/tie convention and whether the results are stable across prompt-level splits (Appendix C.4).
5 comments

We thank the reviewer for suggesting that we test generalization to additional model families and compare with more recent intrinsic guardrail methods.

1. Regarding generalization to new model families (W1, Q1)

We conducted additional control and monitoring experiments on Qwen3-30B-A3B, a mixture-of-experts model, and NVIDIA-Nemotron-Nano-9B-v2, a Mamba–Transformer hybrid.

For control, we report mean ASR over AIM and refusal suppression, ASR after a benign update, and XSTest over-refusal (OR):

ModelMethodASR ↓OR ↓ASR after update ↓
Qwen3 MoEBase54.798.033.07
DPO10.398.012.30
CAA52.726.824.12
Probe steering54.156.835.31
Flow53.687.623.01
Nemotron hybridBase26.846.43.19
DPO0.008.80.64
CAA26.686.83.04
Probe steering26.687.23.04
Flow9.1110.40.96

All values are percentages. OR is measured before the benign update.

DPO provides the strongest attack resistance on both models in these settings, while Flow also substantially reduces ASR on Nemotron, with higher OR. The other two steering methods achieve ASRs close to the baseline. We note that the benign update reduces mean ASR in several settings. Investigating the cause is beyond the scope of this work and the time available during the rebuttal period.

We also evaluated general capabilities and control across safety domains. On Qwen3, MMLU accuracy changes from 76.8% for the base model to 76.4% with DPO and 74.0% with Flow; on Nemotron, it remains close to the base model’s 72.2% across methods. For granularity, general-domain DPO reduces Nemotron’s refusal-suppression ASR from 19.40% to 1.49% on cybercrime and from 14.67% to 6.67% on physical harm. However, domain-specific training does not consistently transfer: cybercrime-trained DPO reduces physical-harm ASR only from 14.67% to 13.33%. These results reinforce the importance of considering side effects and control scope alongside jailbreak robustness. We report additional control and monitoring results in Appendices B.4 and D.4 of the revised manuscript.

For monitoring, we trained and evaluated probes using each model’s native activations, following the shared protocol in Sec. 4. We compared two probe formulations with two text monitors:

ModelMonitorAUROC ↑AUPRC ↑TPR / actual FPR at target 5% FPRGFLOPs / trajectory ↓
Qwen3 MoERolling probe0.9830.9600.400 / 0.0140.00179
CC++ probe0.9830.9600.087 / 0.0040.00179
FT text monitor0.9950.9950.983 / 0.0276,768.39
Qwen3Guard0.9960.9950.959 / 0.0143,262.67
Nemotron hybridRolling probe0.9320.1720.308 / 0.0400.00314
CC++ probe0.9140.1820.308 / 0.0510.00314
FT text monitor0.9840.7071.000 / 0.0615,323.15
Qwen3Guard0.9520.3380.692 / 0.0512,497.91

CC++ is a single-layer adaptation. Thresholds were selected on calibration data. FLOPs are estimated for one full-response detection.

We retain the same evaluation sets to maintain consistency with the main experimental protocol and enable comparison with the main results. On Qwen3, the probes achieve strong ranking performance at substantially lower estimated computational cost than the text monitors. However, the full-response calibration set contains only 48 safe trajectories, which may contribute to conservative thresholds and low TPRs for some probes. On Nemotron, the full-response test set contains only 13 harmful trajectories out of 488, making detection metrics sensitive to individual examples and affecting AUPRC through the low positive prevalence. We therefore interpret the threshold-based results alongside AUROC and AUPRC with caution.

Together, these experiments extend our evaluation to MoE and hybrid architectures and show that the comparison between probes and text monitors remains relevant beyond the original Qwen2.5 and LLaMa setting.

We used both models in non-thinking mode. Reasoning can make successful jailbreaks rare under the evaluated attacks, leaving too few harmful responses for a meaningful monitoring comparison. We did not evaluate additional models in reasoning mode, and the revised manuscript also specifies the non-thinking setting in Appendices B.4 and D.4.

2. Regarding recent intrinsic monitoring methods (W2, Q2)

We appreciate the reviewer’s suggestion to compare with recent intrinsic monitoring methods and clarify our contribution relative to prior work. We have added CC++ and SingProbe as separate baselines under the same Qwen2.5-32B monitoring protocol.

Because not all implementation details needed for our single-model, single-task setting are specified in the respective papers, we adapted the reported architectures and available code as closely as possible. In particular, we use a single-layer CC++ adaptation with the reported linear probe and SWiM loss, and adapt SingProbe to score response tokens. Both are trained on the same SafeRLHF data splits as our sequence probes, using the same basic training budget. All methods are evaluated on the same set of model-generated trajectories with the same calibration procedure.

MonitorAUROC ↑AUPRC ↑TPR / actual FPR at target 5% FPRStreaming recall ↑Streaming FPR ↓GFLOPs / trajectory ↓
Rolling probe0.9710.9660.962 / 0.1220.9130.0170.00452
CC++ adaptation0.9840.9820.965 / 0.0870.9250.0060.00452
SingProbe adaptation0.9720.9760.928 / 0.0600.9280.0521.91
FT text monitor0.9950.9950.983 / 0.0620.5750.1636,569.32
Qwen3Guard0.9960.9950.968 / 0.0560.9480.0583,149.44

The new probes are evaluated on Qwen2.5-32B’s own representations obtained through same-model replay. Actual test FPRs differ even though thresholds use the same calibration procedure. The estimated FLOPs reflect the probes’ marginal computation, excluding the replay forward pass.

The additional probes perform competitively at substantially lower estimated computational cost. CC++ improves over the rolling probe in full-response AUROC and streaming recall, while SingProbe also achieves high streaming recall. The text monitors retain higher full-response AUROC and AUPRC, but the streaming results depend on the particular monitor and operating point. We therefore do not interpret the full-response gap as evidence that text monitoring is universally superior to representation-based monitoring.

More broadly, prior work has already demonstrated efficient native monitoring and probe-guided intervention. Our contribution is to compare the effectiveness and computational cost of behavioral and representation-based safeguards within a common evaluation framework, including full-response and streaming detection, rather than to establish native monitoring or monitor-guided intervention as new capabilities. We have added these baseline adaptations to Section 4 and Appendix C of the revised manuscript and updated the related-work discussion in Section 2.

References:

Cunningham, H., Wei, J., Wang, Z., Persic, A., Peng, A., Abderrachid, J., Agarwal, R., Chen, B., Cohen, A., Dau, A., Dimitriev, A., Gilson, R., Howard, L., Hua, Y., Kaplan, J., Leike, J., Lin, M., Liu, C., Mikulik, V., … Sharma, M. (2026). Constitutional classifiers++: Efficient production-grade defenses against universal jailbreaks (arXiv:2601.04603). arXiv. https://doi.org/10.48550/arXiv.2601.04603

Sing Team. (2026). SingProbe technical report (arXiv:2608.30703). arXiv. https://doi.org/10.48550/arXiv.2608.30703

3. Regarding threshold calibration and split stability (Q3)

We appreciate the reviewer’s request for clarification. We split trajectories by prompt, using 10% of prompts for calibration and the remainder for testing. For full-response detection, the threshold is the (1−α)(1-\alpha)-quantile of the safe calibration scores, computed with NumPy’s method="higher"; a trajectory is flagged when its score is greater than or equal to the threshold. For streaming detection, we first take the maximum score over each safe calibration trajectory and apply the same rule to these maxima. The original streaming protocol uses a 5% sequence-level FPR target.

With 51 safe full-response calibration trajectories, the 1% and 5% thresholds are respectively the largest and third-largest safe scores. With 28 safe streaming calibration trajectories, the 5% threshold is the second-largest trajectory maximum. Scores tied at the threshold are all flagged, so ties can further increase the realized FPR. Even without ties, the corresponding calibration FPRs are 1/51 = 1.96%, 3/51 = 5.88%, and 2/28 = 7.14%, respectively.

To examine stability, we repeated the prompt-level calibration–test split 1,000 times (seeds 0–999), keeping the trained monitors and trajectory scores fixed. The table reports the median and the 2.5th–97.5th percentile range across splits:

MonitorFull: TPR / FPR at 1% targetFull: TPR / FPR at 5% targetStreaming: recall / FPR at 5% target
Mean probe87.6 [32.5, 95.9] / 1.7 [0.0, 8.5]93.6 [81.1, 98.2] / 5.4 [0.4, 15.8]89.4 [4.8, 97.7] / 5.6 [0.0, 25.7]
Last probe68.0 [37.8, 91.0] / 1.4 [0.0, 7.1]87.4 [66.8, 97.1] / 5.2 [0.6, 14.4]50.1 [16.8, 70.5] / 5.4 [0.0, 23.7]
Rolling probe84.7 [16.1, 94.2] / 1.6 [0.0, 8.9]91.6 [74.2, 96.8] / 5.5 [0.4, 18.3]93.1 [16.8, 96.9] / 5.5 [0.0, 27.5]
Attention probe32.0 [6.5, 91.1] / 1.4 [0.0, 7.1]89.7 [16.9, 96.2] / 5.3 [0.4, 13.6]83.1 [26.2, 87.4] / 5.5 [0.0, 23.9]
CC++85.3 [58.5, 96.3] / 1.3 [0.0, 7.8]94.1 [85.8, 98.0] / 5.5 [1.0, 14.5]97.4 [90.2, 99.2] / 5.4 [0.0, 21.9]
SingProbe84.3 [73.1, 94.2] / 1.5 [0.0, 8.1]91.9 [84.8, 96.5] / 5.5 [1.0, 14.6]93.9 [78.2, 97.3] / 5.6 [0.0, 21.9]
FT monitor95.1 [87.5, 98.8] / 1.5 [0.0, 8.5]98.3 [94.5, 99.4] / 5.4 [0.6, 15.3]56.6 [47.9, 62.1] / 6.9 [0.0, 24.9]
Qwen3Guard93.8 [88.6, 97.7] / 1.2 [0.0, 7.2]96.8 [94.0, 99.4] / 5.3 [1.0, 14.2]94.3 [80.2, 98.3] / 5.5 [0.0, 22.0]

All values are percentages. FPRs are measured on the test split, not assumed to equal the calibration target.

The results show substantial split sensitivity for some probes, especially at low FPR targets. These ranges describe variation across overlapping splits of the existing trajectories. Nevertheless, the median results across the evaluated splits remain consistent with the paper’s main conclusions. We have added the threshold convention, split-sensitivity results, and the small-calibration-set limitation to Appendix C.5 of the revised manuscript.

Reviewer_bXeUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies when using a model's internal representations can improve LLM safety compared to behavioral methods. The authors compare DPO with three representation-steering methods for safety control, and representation probes with text monitors for detecting unsafe responses. They examine robustness to jailbreaks, performance after fine-tuning, data efficiency, model capabilities, and monitoring costs. They also test whether monitoring can help restore safety after additional fine-tuning. The main finding is that DPO generally provides stronger safety control, while Flow can be competitive with small amounts of high-quality data. Text monitors perform better overall, but probes are much cheaper when they can reuse the model's internal states. The authors also show that probe-guided blocking and regeneration can reduce unsafe responses after fine-tuning.

Critical strengths
  1. A broad comparison that looks beyond attack success rates.

The authors compare several safety methods under similar conditions, looking not only at how well they prevent unsafe responses, but also at over-refusal, model capabilities, data efficiency, and performance after fine-tuning. This helps show the trade-offs between the methods. For example, Flow provides stronger safety control than CAA, but has a higher over-refusal rate after fine-tuning (0.308 vs. 0.164 in Table 1). I also found the negative results useful, particularly the finding that CAA and probe-based steering provide little improvement over the base model.

  1. The results on safety loss after fine-tuning.

One of the more interesting findings is how much safety can degrade after additional benign fine-tuning. For example, DPO's average AIM attack success rate increases from 0.027 to 0.436 (Table 1). The authors also show that adding monitoring can substantially reduce unsafe responses after fine-tuning. This matters because models are often fine-tuned for new tasks after safety alignment. Understanding how that affects safety, and whether monitoring can help without repeating alignment, is an important problem.

Critical weaknesses
  1. The low-data comparison may favor Flow because of the training settings.

The paper finds that Flow can outperform DPO when training data is very limited. However, the training settings are quite different. DPO is trained for two epochs with a large effective batch size (256 for Qwen2.5-1.5B), while Flow uses ten epochs with much smaller per-device batches (Tables 4–5). With only 100–200 training examples, DPO may receive very few training updates. This makes it hard to tell whether Flow is actually better with limited data, or whether DPO simply needs more training. The advantage is also limited to a relatively small range of data sizes. In Table 14, Flow performs better with 100–500 high-quality pairs, but DPO is already ahead at 1,000 pairs. The main data-scaling experiment also uses only one model and one jailbreak attack. The authors acknowledge that the experiments do not use equal compute budgets. Still, I think this is an important limitation, since Flow's performance with limited data is one of the main positive findings for representation steering.

  1. The practical advantage of combining probes with safety control is not fully demonstrated.

The paper shows that probes can be very cheap when they reuse the generating model's internal states. However, in the intervention experiments, responses are passed through a separate Qwen2.5-32B model to obtain probe scores. So the low computational cost and the improvement in safety are shown in different settings, rather than together. I also think an important baseline is missing from Section 5. The authors compare DPO with and without monitoring, but do not test a base model with the same monitor. For context, the post-fine-tuning Qwen2.5-14B base model has a mean jailbreak ASR of about 0.63 (Table 11), compared with 0.50 for DPO (Table 23). Without the base + monitor comparison, it is difficult to tell how much additional benefit comes from combining DPO with monitoring, rather than from monitoring alone. Appendix E also shows that text monitors can reduce unsafe responses. Given this, I would have liked to see a more direct comparison of the cost and effectiveness of the two monitoring approaches. These results show that monitoring helps, but I am not fully convinced that using representation probes is the better practical choice.

Questions for authors
  1. How many optimizer updates does DPO receive in the 100- and 200-pair experiments? Were any runs with more updates or a smaller batch size tested?
  2. Could the authors include a base model with the same monitoring intervention in Section 5? This would help show how much additional benefit comes from combining DPO with monitoring.
  3. I understand that training DPO on the 32B model would be expensive. Could the authors instead train a probe on the controlled 14B model's own activations and evaluate the intervention directly? Comparing its actual computational cost and effectiveness with a separate text monitor would help establish the practical benefit.
3 comments

We thank the reviewer for recognizing the breadth of our comparison and for raising important questions about the training budget and practical monitoring setup.

1. Regarding the low-data training budget (W1, Q1)

In our main data-scaling experiments, DPO receives two optimizer steps at both 100 and 200 pairs, which is much lower than Flow.

To examine the effect of additional updates, we reran DPO with an effective batch size of 4 for 10 epochs, matching the number of Flow optimizer steps. On Qwen2.5-1.5B under refusal suppression, across three seeds:

Preference pairsOptimizer stepsDPO: ASR / OR ↓DPO matched updates: ASR / OR ↓Flow: ASR / OR ↓
10025032.27 / 49.332.24 / 75.6030.35 / 53.33
20050031.52 / 49.070.75 / 66.8026.20 / 59.47
5001,25028.43 / 52.000.00 / 70.1325.56 / 62.80
1,0002,50022.36 / 54.400.00 / 63.4725.88 / 63.33

All values are percentages. The optimizer-step column applies to matched-update DPO and Flow. All entries are means across three seeds.

These results confirm that DPO’s low-data performance is sensitive to the number of updates. In choosing the main training recipes, we aimed to let each method perform well under a configuration reasonably representative of practical use, rather than enforce identical update schedules. From this perspective, the matched-update results reveal an important trade-off: although additional updates greatly reduce DPO’s ASR, they also substantially increase over-refusal compared with the original recipe. This may make the matched-update configuration less suitable for deployment, so we believe both metrics should inform the choice of DPO configuration. We also agree that the single-model, single-attack scope and unequal compute budgets limit the generality of the data-scaling results. However, due to the limited time and compute budget available during the rebuttal period, we are unable to conduct additional data-scaling experiments on more models. The revised manuscript qualifies the low-data comparison in Section 3.3 and Appendix B.2.3.

2. Regarding monitoring and safety control (W2, Q2–Q3)

We thank the reviewer for identifying two important gaps in our evaluation. We agree that the base-model monitoring baseline is needed to assess the contribution of safety control, and that evaluating a native probe on the controlled model is important for assessing the practical benefit of combining control and monitoring. We therefore conducted additional experiments on Qwen2.5-14B, evaluating the base and DPO models both before and after a benign update. For each model state, we trained a Rolling probe on that model’s own activations and evaluated it on the model’s own responses. We also included a fine-tuned Qwen2.5-7B text monitor on the same responses as a baseline. Responses flagged by either monitor were blocked.

ModelMonitorASR before / after blocking ↓OR after blocking ↓Native Monitoring GFLOPs / response ↓
BaseProbe20.77 / 4.319.50.00357
BaseText monitor20.77 / 0.647.55,231
Base + updateProbe59.58 / 1.4416.50.00364
Base + updateText monitor59.58 / 0.1621.55,339
DPOProbe4.79 / 0.9615.00.00318
DPOText monitor4.79 / 0.1615.54,681
DPO + updateProbe48.24 / 0.8016.00.00351
DPO + updateText monitor48.24 / 0.1617.05,156

ASR and OR are percentages. ASR is averaged over AIM and refusal suppression. Each probe was trained and calibrated for its corresponding model state. The same calibration procedure was used across monitors.

Monitoring accounts for much of the ASR reduction. Nevertheless, DPO also lowers ASR before monitoring, and combining DPO with a probe achieves lower ASR than the corresponding monitored base model, both before and after the benign update. The text monitor achieves lower ASR than the probe in these experiments, while combining it with DPO does not improve every outcome: before the update, for example, the monitored DPO model has higher OR than the monitored base model. We therefore view DPO and monitoring as complementary safeguards whose combined benefit should be assessed using both ASR and OR.

We also calculated monitoring FLOPs for the base and DPO models. Probe computation costs approximately 0.0032–0.0036 GFLOPs per response, compared with 4,681–5,339 GFLOPs for the separate text monitor—roughly 0.000068% of the text monitor’s cost in our implementation.

Together, these results show that probes trained on the controlled model’s own activations can provide effective monitoring when integrated with safety control, while requiring substantially less estimated marginal monitoring computation than a separate text monitor when generation activations are reused. We have added this comparison to Section 5 and Table 4 of the revised manuscript, with implementation details in Appendix E.4.

Reviewer_bXeUOpenReview ↗

Response to Authors' Rebuttal

Thank you for the detailed responses, please include the key points in the revision. Overall, my assessment of the submission remains unchanged.

Reviewer_z2cfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper compares representation steering with DPO for safety control, and activation probes with text monitors for detecting unsafe responses. It also tests monitor-guided interventions after benign fine-tuning. The authors argue that access to model internals complements behavioral safeguards.

Critical strengths
  1. Comparing control methods on shared data and base models is useful. The validated safety judge strengthens this design.

  2. The paper reports false alarms alongside detection and discusses weaknesses in its own results. This helps readers assess the practical trade-offs.

Critical weaknesses
  1. The control rankings are hard to separate from tuning choices. At the chosen strength, some steering methods barely change the base model's behavior. In the low-data experiments, DPO receives very few optimization updates. Either choice could make a method look weaker than it is. The comparison would be more convincing if each method had a training and tuning schedule that allowed it to work well.

  2. The monitoring comparison does not isolate the benefit of access to model internals. The probes and text monitors use different backbones with different prior training. Although some share the same safety-training examples, those controls do not remove the other differences. The results help compare the available systems, but they leave open whether the detection gap comes from the monitoring method or the underlying model.

  3. The broader recommendations need more support across alignment methods and threats. DPO is the only behavioral-alignment objective tested, and the main jailbreak attacks are fixed and non-adaptive. Safety persistence is also measured after a single benign-update recipe. The additional domain tests are useful, but they do not show whether the same trade-offs hold after a different alignment objective or under a new attack family.

Questions for authors
  1. Please try to address weaknesses 1 and 2.

  2. Do the findings extend to another alignment objective and an unseen attack family, with monitor thresholds fixed? Otherwise, please narrow the conclusions to the evaluated settings.

3 comments

We thank the reviewer for suggesting additional alignment methods and attack settings to strengthen our evaluation. We also appreciate the opportunity to clarify the scope of our monitoring comparison.

1. Regarding steering strength (W1)

Our paper already reports a layer–strength sweep in Appendix A.5, Tables 8–9, although that analysis uses reduced training budgets. To complement it, we additionally sweep intervention strength using the full main-experiment training recipe on Qwen2.5-14B. Across AIM and refusal suppression, the base model has a mean ASR of 22.04%. Over α∈{0.5,1,2,4}\alpha\in\{0.5,1,2,4\}, CAA and probe-based steering yield mean ASRs of 21.09–22.36% and 21.41–22.04%, respectively. Thus, at the selected layer, their limited effect persists across the tested strength range. For Flow, larger TT values do not consistently improve safety and can increase over-refusal: at T=0.7T=0.7, mean ASR and OR are 16.45% / 3.20%, compared with 21.25% / 6.00% at T=1.0T=1.0 and 27.32% / 9.20% at T=1.5T=1.5. The T=0.7T=0.7 operating point used in our main experiments therefore provides a reasonable safety–over-refusal trade-off in this sweep.

2. Regarding the DPO update budget (W1)

We agree that the original low-data DPO schedule can provide too few updates and that its comparison with Flow is sensitive to this choice. Our intent was to use method-specific training recipes, since identical hyperparameters or update counts need not be equally suitable for different objectives. Nevertheless, we admit that a fixed recipe can leave a method under-optimized at small data sizes. We therefore reran DPO with a smaller effective batch and more epochs. On Qwen2.5-1.5B under refusal suppression, the results are:

Preference pairsDPODPO, matched updatesFlow
10032.27 / 49.332.24 / 75.6030.35 / 53.33
20031.52 / 49.070.75 / 66.8026.20 / 59.47
50028.43 / 52.000.00 / 70.1325.56 / 62.80
1,00022.36 / 54.400.00 / 63.4725.88 / 63.33

Entries are ASR / XSTest OR (%), both lower is better. Matched-update DPO uses 250, 500, 1,250, and 2,500 optimizer steps, respectively. Results are averaged over three seeds (42, 43, 44).

The results suggest that simply increasing the number of updates does not necessarily yield the best overall DPO configuration: the additional safety benefit comes with a substantial cost to benign utility. The resulting high over-refusal may also limit the practical suitability of this low-batch, high-epoch recipe in deployment. We therefore acknowledge that low-data performance is highly sensitive to the optimizer-step budget, but believe that low-data ASR alone should not be the primary basis for changing the main training configuration. We have added these results and analyses to Appendix B.2.3 of the revised manuscript.

3. Regarding the monitoring comparison (W2)

We would like to respectfully clarify that our goal is to compare the detection performance and computational cost of practical monitoring approaches, rather than to isolate the causal effect of access to internal representations. To reduce differences in model family and safety supervision, we use Qwen-family backbones and train the representation probes and fine-tuned text monitor on the same safety-training examples. We also include Qwen3Guard as a strong, publicly available guard-model baseline.

We agree that these controls do not eliminate differences in backbone size, prior training, or optimization. The results should therefore be interpreted as a comparison of the evaluated monitoring systems, not as evidence that internal access alone causes a performance advantage or disadvantage. Fully matching all of these factors across probes and independently trained text monitors is challenging and may not reflect how these monitors are deployed in practice. We will clarify this scope and the remaining confounders in the revised version.

4. Regarding additional alignment methods and attacks (W3)

We agree that additional alignment objectives and attack families can strengthen the evidence for our conclusions. We therefore added safety SFT as a behavioral-alignment baseline and evaluated two GCG-transfer suffix attacks from the StrongREJECT library on Qwen2.5-14B.

For safety SFT, we used the same filtered preference-data source as DPO and Flow. We trained Qwen2.5-14B with a completion-only SFT loss and LoRA (rank 16) for two epochs, using an effective batch size of 128 and a learning rate of 5e-5. All methods were evaluated on the same 313 raw prompts for each attack, with harmful-compliance ASR judged by the same LLM using the same rubric. For the after-update evaluation, SFT also received the same benign-update recipe as the other methods: LoRA fine-tuning on 20,000 Alpaca-Cleaned examples for one epoch. The results are shown below:

AlignmentAIMRSAIM after updateRS after update
Safety SFT96.8182.4398.4066.13
MethodGCG-HarmBenchGCG-UniversalHarmBench after updateUniversal after update
Base41.8518.5367.4170.61
DPO11.501.6042.8147.92
CAA41.5317.8968.3770.93
Probe41.5316.2967.0970.93
Flow26.527.0359.1135.78

ASR (%) before and after benign fine-tuning; lower is better.

The results extend our comparison to another alignment objective and a new attack family, although safety SFT shows limited robustness to the evaluated attacks. DPO provides the strongest protection before the benign update, while Flow exhibits a smaller average increase in ASR after the update across the two GCG attacks. CAA and probe-based steering have little effect relative to the base model.

Due to computational cost, we did not evaluate adaptive attacks or additional benign-update recipes, and we will narrow our recommendations accordingly.

Reviewer_z2cfOpenReview ↗

Post-Rebuttal Assessment

Thank you for your responses.

The monitoring clarifications address my attribution concern, while the new alignment and attack experiments broaden the evaluation. However, fixed-threshold monitor transfer, adaptive attacks, and other benign updates remain untested. The steering sweep and matched-update DPO results only partly resolve my tuning and optimization-budget concerns, leaving my broader generality concern unresolved.

I decided to keep my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.