acceptodds
Under review as a conference paper at ICLR 2027

Resistance Training to Reduce Forgetting

Abstract

Task adaptation can improve a language model's target-task performance while weakening other important behaviors, even under on-policy training. We introduce Resistance Training (ResT), which reduces this forgetting through a temporary behavioral load. We fit a low-rank adapter to an unwanted behavior with the backbone frozen, then freeze the adapter while the backbone learns the task. Removing the adapter leaves a model with no additional inference cost. Across medical and financial adaptation of Qwen3-4B, targeted loads improve safety refusal or factual restraint over vanilla on-policy distillation and self-distillation while largely retaining task gains. Under MedQA on-policy distillation, ResT lowers the mean hallucination score by 40 points (0–100 scale), at comparable task accuracy. How the load is fitted also matters: on-policy distillation gives stronger hallucination protection than response supervision under both downstream distillation and supervised fine-tuning. This identifies resistance construction as a design choice for behavioral retention beyond the downstream objective. A linearized analysis gives conditions under which task learning compensates for the load, improving the protected behavior after removal.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
alignment drift, post-training, on-policy distillation, reasoning models, capability preservation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_QnCcOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the problem of behavioral forgetting that occurs when large language models are adapted to downstream tasks and proposes Resistance Training (ResT). The method first trains a LoRA that expresses undesirable behaviors. During downstream training, it freezes this LoRA and updates the Backbone. Finally, it removes the LoRA. The authors focus on comparing different sources of Resistance and fitting methods. They validate the method’s protective effects on safety refusal and hallucination on MedQA and FinQA. They also explain its underlying mechanism through linearization analysis.

Critical strengths

S1. The problem is important and the motivation is well justified. The paper finds that even with OPD and OPSD, improvements in downstream task performance can still be accompanied by declines in safety refusal and factual restraint, revealing a behavioral forgetting problem in On-Policy Training that deserves attention.

S2. The study of Resistance Construction is valuable. The study of Resistance Construction is valuable. The paper systematically compares Generic, Targeted, and Forgetting-Derived Resistance and finds that On-Policy Fitting provides stronger hallucination protection in both downstream OPD and SFT, offering valuable empirical conclusions for how to construct behavioral interventions during training.

Critical weaknesses

W1. The methodological novelty is relatively limited (to my knowledge). The core Fit–Freeze–Remove framework of ResT is essentially the same as that of Security Vectors, and its main novelty lies in the study of On-Policy Resistance Fitting and different Resistance Construction strategies (if I was wrong, correct me. I'm not very familiar with this area). These extensions have some value, but the methodological originality remains limited. Can the authors further clarify the core technical contributions of this paper relative to Security Vectors and explain what advantages On-Policy Resistance Fitting offers that existing methods cannot achieve?

W2. The advantages of On-Policy Resistance Fitting lack rigorous controlled validation. There may be potential differences between Response Supervision and On-Policy Fitting in terms of the number of training tokens, computational budget, and induced Resistance Strength. In addition, in Appendix Table 7, On-Policy Fitting actually performs worse under the downstream OPSD setting. Can the authors conduct more rigorous controlled experiments under matched computational budgets and Resistance Strength, and explain the discrepancy between the results in Table 7 and the main conclusions?

Questions for authors

See weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer QnCc (Part 1)

Thank you for your thoughtful comments and helpful suggestions. We have revised the manuscript and added experiments to address your concerns. Below, we respond to each point and explain the corresponding changes. We hope these responses and additions clarify our work and address your questions.

Question

The fit–freeze–remove framework resembles Security Vectors. What are the core technical contributions relative to SV, and what advantages does on-policy resistance fitting offer?

Response

Thank you for raising this question. ResT differs from Security Vectors (SV), which we cite and discuss, in how the load is constructed and in the analysis of when it yields protection. Our contribution is an analysis-driven account of how to construct resistance that downstream task learning can turn into behavioral protection: we characterize a load by the behavioral displacement it induces and derive sufficient conditions for protection, and the fitting objective and load sources follow from this characterization. We have clarified three contributions in the revised Introduction and Section 2:

  • On-policy resistance fitting. The adapter-conditioned student generates trajectories, and a frozen resistance teacher supplies distributions at the prefixes the student visits. Trajectories are resampled as the adapter changes, allowing the load to learn the teacher’s behavior on its own generated trajectories.
  • Broader resistance sources. In addition to targeted undesirable responses, we use paired base models and checkpoints that naturally degrade during task training. Generic on-policy fitting queries the base model directly, without requiring behavior-specific bad answers or a separately trained resistance teacher.
  • An analysis of protective compensation. Section 2 and Appendix C characterize when task learning compensates for a temporary behavioral displacement, giving sufficient conditions involving fitting error, displacement strength, and downstream feature coverage.

Figure 3 and the expanded three-seed sweeps show the practical benefit of on-policy fitting: lower mean hallucination and higher mean task accuracy across both sources under downstream OPD and SFT. We give the results and selection procedure in our response to W2 (Part 2/2).

We also added a capacity-matched SV-style baseline, using min–min + KL fitting on the same targeted sources. Targeted ResT has higher refusal, lower hallucination, and higher independent Truthfulness means in all four main comparisons, with hallucination 18.9–29.3 points lower. Truthfulness is the MC2 score on TruthfulQA [1], an independent factuality benchmark added in the revision: the probability mass assigned to true answers, where higher is better. The factuality results are summarized below; all entries are mean ± sample SD over three training seeds. Full task, refusal, and benign-compliance results are in Tables 1–2, with configuration details in Appendix D.3.

SettingSV: Hallucination ↓Targeted ResT: Hallucination ↓SV: Truthfulness ↑Targeted ResT: Truthfulness ↑
MedQA · OPD39.71 ± 1.6713.42 ± 3.5956.29 ± 0.3757.68 ± 0.05
FinQA · OPD37.43 ± 1.0610.32 ± 3.2355.93 ± 0.1556.25 ± 0.62
MedQA · OPSD43.21 ± 1.9813.92 ± 0.4556.05 ± 0.1857.49 ± 0.07
FinQA · OPSD33.69 ± 0.5914.77 ± 1.2756.15 ± 0.2956.25 ± 0.17

Please also see our General Response on Contributions, Novelty, and New Experiments, posted for all reviewers and the AC, for further details on the relationship to SV, resistance sources, and analysis. It also summarizes the experiments added during rebuttal that strengthen these contributions, including three-seed comparisons, over-refusal and independent factuality evaluation, additional model sizes and a second model family, and DPO/GRPO; we would be glad if you find them of interest!

References

[1] TruthfulQA: Measuring How Models Mimic Human Falsehoods. Stephanie Lin, Jacob Hilton, and Owain Evans. ACL 2022. ACL Anthology.

AuthorsOpenReview ↗

Response to Reviewer QnCc (Part 2)

Question

The response-supervised and on-policy fitting comparisons may differ in training tokens, compute, and resistance strength. Can these factors be better controlled, and why does on-policy fitting perform worse under OPSD in Table 7?

Response

Thank you for suggesting a more controlled comparison. We expanded Tables 5–6 to three training seeds (42, 43, and 44) and use the full fitting-duration sweeps to compare each method at its best downstream-task performance. This addresses whether the protection advantage depends on choosing a particular fitting duration.

The grid covers 0.1, 0.25, 0.5, and 1 epoch for Safety, and 0.5, 1, 2, and 4 epochs for hallucination, allowing each method to explore different degrees of fitting and the resulting load strengths. Within each seed, source, protection setting, and downstream objective, we select the highest-MedQA configuration for each fitting method, breaking ties by the smaller epoch, then average the paired task and protection scores. Both methods use the same search grid and task-accuracy selection rule. Figure 3 reports the following three-seed means:

DownstreamSourceResponse: Task acc. ↑On-policy: Task acc. ↑Response: Hallucination ↓On-policy: Hallucination ↓
OPDGeneric80.7081.2043.5229.24
OPDTargeted80.0581.2341.4030.82
SFTGeneric74.5574.9940.3531.55
SFTTargeted75.2376.2541.8817.56

On-policy fitting achieves higher mean task accuracy and lower mean hallucination in all four comparisons. The full sweeps also show lower mean hallucination at every shared fitting duration, across both sources and downstream objectives. The advantage is also not explained by the number of fitting sequences: on-policy fitting at its shortest duration (0.5 epoch; 2,000 sequences) has lower mean hallucination than response supervision at its longest (4 epochs; 4,000 sequences) under OPD for both sources (35.49 vs. 39.59 generic; 30.82 vs. 41.40 targeted) and under SFT for the targeted source (32.03 vs. 43.81), and is comparable for the generic source under SFT (38.38 vs. 39.00). Thus, the advantage persists when each method can select its fitting duration for downstream performance, and across the tested durations themselves. These results support on-policy fitting as a strong default for hallucination-oriented load construction in the settings studied. For Safety, targeted loads improve mean refusal under both downstream objectives; generic loads show smaller, mixed effects. Full results and selection details are in Section 3.3 and Appendix D.4.

Response supervision fits fixed answers, while on-policy fitting uses sampled trajectories and teacher distributions. Equal epoch or sequence counts therefore need not imply equal token cost or resistance strength. Our sweeps compare the resulting task–retention outcomes across fitting schedules. Table 7 adds a sequence-matched control, using the same base-model source lineage and 4,000 sequences: four epochs of response supervision versus one epoch of on-policy fitting with four rollouts per prompt.

Under OPD, this control reduces hallucination from 40.75 to 27.31, with task accuracy changing from 80.28 to 79.89. Under OPSD, the fixed one-epoch on-policy load yields task accuracy 70.54 and hallucination 44.64, versus 77.22 and 37.31 for response fitting. This shows that the fitting schedule and downstream objective affect the outcome. We have scoped the conclusion to the consistent advantage observed in the expanded OPD/SFT hallucination sweeps, while retaining Table 7 as a complementary result.

We hope these responses and the revised manuscript address your concerns. We would be glad to clarify any remaining points during the discussion period!

Thanks. I decide to keep my score as it is already positive.

On behalf of all authors, thank you for taking the time to read our rebuttal and for your positive assessment of our work!

We sincerely appreciate your constructive feedback, which has helped us improve the paper. We remain available should any further questions arise during the discussion period!

Reviewer_kKRSOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ResT, a methhod to reduce behavioral forgetting of safety and factual restraint during task adaptation. ResT trains a LoRA adapter to preserve the desired behavior, keeps it active during task training, and removes it at inference. The authors evaluate different behavioral targets and training objectives using Qwen3-4B on MedQA and FinQA. They also provide a theoretical analysis of when ResT can improve behavoir retention. Results show a significant reduction in hallucinations on MedQA compared with standard OPD, with similiar task accuracy.

Critical strengths
  • Overall, the paper is well-written and easy to follow.

  • The proposed method is straightforward and shows promising effectiveness. It adds no inference cost, leaves the downstream objective untouched, and works with OPD, OPSD and SFT.

  • The authors disclose several unfavorable or awkward facts (e.g., Replay beats targeted ResT on refusal in three of four comparisons)

Critical weaknesses
  • The novelty appears incremental. The core recipe closely resembles Security Vectors (Zhou et al., 2024). The main contribution seems to be exploring different resistance sources and fitting objectives, which feels more like an ablation study than a substantial methodological contribution.

  • The results are promising, but reporting variability across multiple training runs would strengthen the evidence. Standard deviations or error bars are missing across the tables and figures (e.g., Tables 1–2 and Figure 3). Including them would help assess the reliability of the findings.

  • The evaluation seems a bit limited to a single 4B model family and two tasks. Broader experiments (e.g. different sizes of models across various model families), would strengthen the results and support the paper’s general claims.

  • The “no additional cost” claim could be more precisely stated, as it does not account for adapter training, teacher training, and adapter-active rollouts.

Minor:

  • Lines 84–85: References for MedQA and FinQA are missing here, althuogh they appear later in the paper. It would be good to include them at their first mention.

  • Lines 156–157: The reference for HarmChain is missing here, although it appears later in the paper. It would be better to include it at its first mention.

Questions for authors
  • How does the proposed method differ from Security Vectors (Zhou et al., 2024), particularlly in the approach illustrated in Fig. 2 (top)?

  • What is the definition of HLH_L in Eq. 4? Is it the Hessian matrix of the task loss LL with respect to the model weights θ\theta? it would be good to clarify.

  • Can you report at least three training seeds for the main tables and Tables 5–6.

4 comments
AuthorsOpenReview ↗

Response to Reviewer kKRS (Part 1)

Thank you for your thoughtful comments and helpful suggestions! We have revised the manuscript and added experiments to address your concerns. Below, we respond to each point and explain the corresponding changes. We hope these responses can clarify our work and address your questions.

Question

The novelty appears incremental: the core recipe resembles Security Vectors, and changing resistance sources and fitting objectives seems more like an ablation study. How does ResT differ from Security Vectors, particularly the procedure in Figure 2 (top)?

Response

Thank you for raising this. ResT differs from Security Vectors, which we have already cited and discussed in the origin submission, in how the load in Figure 2 (top) is constructed and in the analysis of when it yields protection. Our contribution is not a search over sources and objectives. It is an analysis-driven account of how to construct a load that task learning can turn into protection, and of when that protection persists after removal: we characterize a load by the behavioral displacement it induces and derive sufficient conditions for protection. The fitting objective and load sources follow from this characterization, so the experiments examine the choices it motivates rather than serving as stand-alone ablations.

We introduce on-policy resistance fitting: the adapter-conditioned student generates trajectories, a frozen resistance teacher supplies distributions at the visited prefixes, and trajectories are resampled as the student changes. Resistance teachers can also be obtained from paired base models and checkpoints degraded by ordinary task training. In particular, generic on-policy fitting directly queries the untuned base teacher, without behavior-specific bad-answer construction or a separately trained resistance teacher. Section 2 and Appendix C give these conditions, which involve fitting quality, task-feature coverage, and transfer to held-out protection prompts.

The fitting experiments test this construction directly. In the updated three-seed Figure 3, on-policy fitting gives lower mean hallucination and higher mean task accuracy than response fitting in all four source–downstream hallucination comparisons; for example, with the targeted source under downstream SFT, the mean hallucination score falls from 41.9 to 17.6. We also added a capacity-matched SV-style baseline, using min–min fitting with KL regularization on the same targeted sources. Targeted ResT has higher refusal, lower hallucination, and higher independent Truthfulness means than SV in all four main task–objective comparisons (Tables 1–2), with hallucination 18.9–29.3 points lower. On MedQA OPD, SV (39.71; Table 1) is in the response-fitted range (33.53–49.37) and above on-policy loads (12.93–30.82; Table 6 means). This indicates on-policy fitting drives the gain.

We have clarified these contributions in the Introduction. Please also see our General Response on Contributions, Novelty, and New Experiments, posted for all reviewers and the AC, for a fuller discussion of the relationship to SV and the supporting analysis and experiments.

Question

The results are promising, but reporting variability across multiple training runs would strengthen the evidence. Standard deviations or error bars are missing across the tables and figures, including Tables 1–2 and Figure 3. Can you report at least three training seeds for the main tables and Tables 5–6?

Response

Thank you for the suggestion. We repeated the main comparisons with three independent training seeds: 42, 43, and 44. Tables 1–2 now report mean ± sample standard deviation for every post-training method, including the added Security Vectors baseline. The SFT comparison and model-size/family and DPO/GRPO extensions also report three-seed results (Tables 11–13 and 15–17).

We also repeated the full sweeps in Tables 5–6 across these three seeds and updated Figure 3 from those results. Within each seed and source–protection–fitting–downstream group, we select the highest-MedQA configuration, breaking ties by the smaller fitting epoch. Each plotted point averages the paired task and behavior scores of the three selected models. The caption and Appendix D.4 explain the procedure; Tables 5–6 provide every per-seed sweep result.

The updated Tables 1–2 are below. Two added companion metrics track whether protection is accompanied by changes in benign-request compliance or independent factuality. Over-refusal ↑ reports benign compliance on 400 benign requests; higher scores indicate less over-refusal. Truthfulness ↑ reports MC2 probability mass on true answers across all 817 TruthfulQA questions. Evaluation details are in Appendix D.5. Each protection setting is a separate experiment; initial-model references remain single values. All metrics use a 0–100 scale, and bold marks the best post-training mean per domain and column.

(Continued in Part 2.)

AuthorsOpenReview ↗

Response to Reviewer kKRS (Part 2)

Table 1. Downstream OPD

Medical (MedQA)

MethodSafety: Task acc. ↑Refusal ↑Over-refusal ↑Factual: Task acc. ↑Hallucination ↓Truthfulness ↑
Initial model71.8031.2191.9771.8034.0054.84
Vanilla79.00 ± 0.7128.98 ± 0.2193.49 ± 0.1179.00 ± 0.7155.42 ± 0.8155.19 ± 0.04
Shadow-FT‡78.84 ± 0.5928.49 ± 0.5794.77 ± 1.0078.84 ± 0.5939.40 ± 1.8255.69 ± 0.30
OSFT79.50 ± 0.1325.94 ± 2.1295.07 ± 1.9478.06 ± 0.1254.65 ± 0.9654.98 ± 0.19
Replay79.47 ± 0.3336.74 ± 2.0789.59 ± 0.6677.82 ± 0.2032.46 ± 3.7155.92 ± 0.16
Security Vectors80.75 ± 0.2132.46 ± 1.8792.09 ± 1.2880.02 ± 0.5239.71 ± 1.6756.29 ± 0.37
ResT: Generic80.23 ± 0.5330.35 ± 0.1092.11 ± 0.6980.83 ± 1.0925.44 ± 1.2554.96 ± 0.35
ResT: Targeted80.23 ± 0.2537.04 ± 0.4090.32 ± 0.5379.58 ± 0.2113.42 ± 3.5957.68 ± 0.05

Finance (FinQA)

MethodSafety: Task acc. ↑Refusal ↑Over-refusal ↑Factual: Task acc. ↑Hallucination ↓Truthfulness ↑
Initial model70.1831.2191.9770.1834.0054.84
Vanilla71.46 ± 0.8127.48 ± 1.5393.60 ± 0.1371.46 ± 0.8145.40 ± 0.9955.19 ± 0.35
Shadow-FT‡73.52 ± 0.2727.32 ± 0.0894.31 ± 0.4473.52 ± 0.2728.45 ± 1.0854.58 ± 0.45
OSFT68.15 ± 1.3926.06 ± 0.2694.24 ± 0.2570.45 ± 0.6848.09 ± 0.7554.75 ± 0.13
Replay70.18 ± 1.3935.49 ± 5.2190.55 ± 3.2570.97 ± 0.9237.34 ± 2.9655.34 ± 0.05
Security Vectors72.30 ± 0.8230.03 ± 0.4793.03 ± 0.6271.81 ± 0.4837.43 ± 1.0655.93 ± 0.15
ResT: Generic71.73 ± 1.5227.55 ± 0.4293.53 ± 0.5872.28 ± 0.2329.42 ± 2.4554.98 ± 0.29
ResT: Targeted72.04 ± 0.4132.43 ± 1.1891.77 ± 0.1172.68 ± 0.3310.32 ± 3.2356.25 ± 0.62

‡ Most Shadow-FT generations under OPD reach the length limit; the corresponding measurements and explanation are retained in Appendix D.3.

Table 2. Downstream OPSD

Medical (MedQA)

MethodSafety: Task acc. ↑Refusal ↑Over-refusal ↑Factual: Task acc. ↑Hallucination ↓Truthfulness ↑
Initial model71.8031.2191.9771.8034.0054.84
Vanilla75.86 ± 0.4024.36 ± 0.6493.82 ± 0.2275.86 ± 0.4052.67 ± 0.2155.28 ± 0.07
Shadow-FT72.06 ± 0.0529.04 ± 0.8293.11 ± 0.1672.06 ± 0.0540.54 ± 0.1854.51 ± 0.30
OSFT75.60 ± 0.5422.12 ± 1.1594.80 ± 0.6976.72 ± 0.2053.74 ± 0.5655.39 ± 0.11
Replay76.36 ± 0.2135.28 ± 1.0590.57 ± 0.1375.12 ± 0.4133.10 ± 3.5555.61 ± 0.19
Security Vectors76.48 ± 1.3529.82 ± 0.0290.73 ± 0.9574.84 ± 1.3243.21 ± 1.9856.05 ± 0.18
ResT: Generic76.15 ± 0.4726.99 ± 0.1594.25 ± 0.3177.61 ± 0.2137.05 ± 0.6054.50 ± 0.32
ResT: Targeted77.80 ± 0.2335.44 ± 0.8790.74 ± 0.2974.99 ± 0.3113.92 ± 0.4557.49 ± 0.07

Finance (FinQA)

MethodSafety: Task acc. ↑Refusal ↑Over-refusal ↑Factual: Task acc. ↑Hallucination ↓Truthfulness ↑
Initial model70.1831.2191.9770.1834.0054.84
Vanilla72.42 ± 0.7522.62 ± 0.6594.41 ± 0.0972.42 ± 0.7547.49 ± 2.0055.47 ± 0.33
Shadow-FT71.70 ± 0.1330.32 ± 0.3492.49 ± 0.3771.70 ± 0.1333.94 ± 3.0654.62 ± 0.20
OSFT71.43 ± 0.1820.26 ± 0.6895.75 ± 0.0871.43 ± 0.3447.51 ± 1.2254.92 ± 0.11
Replay72.68 ± 0.6133.81 ± 0.4689.15 ± 0.3771.58 ± 0.7626.33 ± 1.7356.19 ± 0.03
Security Vectors74.05 ± 0.4026.14 ± 0.8693.55 ± 0.3874.42 ± 0.5333.69 ± 0.5956.15 ± 0.29
ResT: Generic73.12 ± 1.0721.96 ± 0.7994.80 ± 0.2573.00 ± 0.7936.91 ± 7.9254.58 ± 0.07
ResT: Targeted73.50 ± 0.7826.76 ± 0.2592.77 ± 0.1674.54 ± 0.7014.77 ± 1.2756.25 ± 0.17

The three-seed results preserve the main findings. Targeted ResT improves the protected metric over Vanilla in all eight settings and has the lowest hallucination and highest Truthfulness means in all four task–objective comparisons. Hallucination falls by 32.72–42.00 points, and task accuracy improves in seven of eight settings. On-policy fitting also retains lower mean hallucination and higher mean task accuracy in all four hallucination comparisons of Figure 3, with the hallucination advantage holding at every shared fitting duration in Tables 5–6. These repeats support the robustness of the findings, with variability and companion metrics now reported.

AuthorsOpenReview ↗

Response to Reviewer kKRS (Part 3)

Question

The evaluation seems limited to a single 4B model family and two tasks. Broader experiments, such as different model sizes and families, would strengthen the results and support the paper’s claims.

Response

Thank you for the suggestion. We added Qwen3-1.7B, Qwen3-8B, and Llama-3.2-1B-Instruct experiments on MedQA under OPD, all with three training seeds (42, 43, and 44). The additional Qwen3 scales largely follow the main Qwen3-4B setup, with model-specific teacher and adapter schedules in Table 14. Llama uses non-thinking generation and full-answer evaluation, with Llama-3.2-3B-Instruct as the task teacher. The results and configurations are in Appendix F.

At both additional Qwen3 scales, targeted ResT has the lowest hallucination mean, while generic ResT achieves the highest factual-setting task accuracy. On Llama, both ResT variants improve refusal, hallucination, and Truthfulness over SV. Targeted ResT reduces hallucination from 59.97 to 48.94 relative to SV, returning it to approximately the initial level of 48.96, while retaining task accuracy above the initial model in both settings. Generic ResT achieves the highest Truthfulness score (43.77). These results extend the retention findings across model scales and to a non-thinking model from another family.

The tables below report mean ± sample standard deviation over the three training seeds. Metrics and bolding follow Tables 1–2: Over-refusal ↑ reports benign compliance, and Truthfulness ↑ reports TruthfulQA MC2.

Table 12. Qwen3-1.7B, MedQA OPD

MethodSafety: Task acc. ↑Refusal ↑Over-refusal ↑Factual: Task acc. ↑Hallucination ↓Truthfulness ↑
Initial model53.8914.3197.4753.8966.6145.91
Vanilla62.71 ± 0.8713.19 ± 1.1697.23 ± 0.5562.71 ± 0.8778.14 ± 1.7847.47 ± 0.03
Shadow-FT57.24 ± 1.399.78 ± 0.5598.14 ± 0.1557.24 ± 1.3968.21 ± 4.9547.47 ± 0.66
OSFT62.00 ± 0.7112.59 ± 0.9197.27 ± 0.1163.47 ± 0.2880.50 ± 4.6947.59 ± 0.08
Replay60.77 ± 0.0535.46 ± 0.5587.33 ± 1.0461.01 ± 0.2543.40 ± 3.2448.80 ± 0.14
Security Vectors64.50 ± 0.3414.93 ± 0.4097.02 ± 0.8663.24 ± 0.0856.29 ± 0.8649.04 ± 0.09
ResT: Generic64.26 ± 0.3611.93 ± 0.3097.78 ± 0.0465.57 ± 0.1269.92 ± 0.7447.08 ± 0.02
ResT: Targeted65.15 ± 1.0514.52 ± 0.9096.47 ± 0.4863.45 ± 1.8131.60 ± 0.4147.18 ± 0.41

Table 13. Qwen3-8B, MedQA OPD

MethodSafety: Task acc. ↑Refusal ↑Over-refusal ↑Factual: Task acc. ↑Hallucination ↓Truthfulness ↑
Initial model79.5834.7392.0979.5814.2354.43
Vanilla83.22 ± 0.9431.71 ± 0.7792.70 ± 0.0783.22 ± 0.9429.15 ± 5.4654.81 ± 0.25
Shadow-FT84.29 ± 0.7633.57 ± 0.3394.23 ± 0.2984.29 ± 0.7617.59 ± 3.1455.04 ± 0.11
OSFT74.53 ± 1.22§31.34 ± 1.0985.63 ± 1.8583.23 ± 1.18†§15.79 ± 1.21†54.68 ± 0.69†
Replay82.27 ± 0.4337.41 ± 0.4390.80 ± 0.3383.82 ± 0.8237.52 ± 1.2955.74 ± 0.01
Security Vectors84.84 ± 0.3935.86 ± 1.0692.61 ± 0.6983.79 ± 0.3312.79 ± 0.6756.18 ± 0.31
ResT: Generic83.71 ± 0.4331.14 ± 0.6893.02 ± 0.2485.44 ± 0.7114.46 ± 2.5354.53 ± 0.33
ResT: Targeted84.79 ± 0.2036.77 ± 0.8690.62 ± 0.9783.43 ± 0.645.75 ± 0.4657.38 ± 0.06

† The OSFT factual aggregate uses a rerun for seed 42, replacing an earlier run that exhibited output collapse; see the note to Table 13.

§ Invalid outputs are excluded from both the numerator and the denominator of task accuracy; see the note to Table 13.

Table 15. Llama-3.2-1B-Instruct, MedQA OPD (non-thinking)

MethodSafety: Task acc. ↑Refusal ↑Over-refusal ↑Factual: Task acc. ↑Hallucination ↓Truthfulness ↑
Initial model38.1848.5479.5038.1848.9643.40
Vanilla40.33 ± 1.3941.09 ± 0.3384.64 ± 0.1140.33 ± 1.3959.26 ± 0.0743.12 ± 0.02
Shadow-FT37.71 ± 0.4049.73 ± 0.0578.09 ± 0.2237.71 ± 0.4057.48 ± 0.1943.68 ± 0.06
OSFT39.51 ± 0.2139.49 ± 0.0386.08 ± 0.0240.61 ± 0.4158.61 ± 1.2943.02 ± 0.05
Replay39.70 ± 0.9448.54 ± 0.3476.84 ± 0.0540.77 ± 0.2844.43 ± 0.9943.30 ± 0.08
Security Vectors40.90 ± 0.0543.53 ± 0.0183.81 ± 0.0940.51 ± 0.3659.97 ± 2.1543.36 ± 0.04
ResT: Generic40.27 ± 0.4346.65 ± 0.2280.41 ± 0.0440.64 ± 1.4654.02 ± 1.1243.77 ± 0.04
ResT: Targeted39.64 ± 0.6647.23 ± 0.2780.75 ± 0.4440.17 ± 0.4548.94 ± 0.4743.61 ± 0.03
AuthorsOpenReview ↗

Response to Reviewer kKRS (Part 4)

Question

The “no additional cost” claim could be more precisely stated, as it does not account for adapter training, teacher training, and adapter-active rollouts.

Response

Thank you for pointing this out. ResT keeps the same downstream task data, batch size, epochs, and optimizer-update count as Vanilla. Our Replay baseline adds rehearsal data and therefore requires more updates at fixed epochs. ResT retains the original downstream schedule and incurs no additional inference cost after adapter removal.

We have clarified in Appendix D.4 that training overhead includes adapter fitting, resistance-teacher training for targeted on-policy loads, and adapter-active forward passes and rollouts. Construction schedules and Replay update counts are in Appendices D.3–D.4. The main text states only that removing the adapter leaves no additional inference cost; the training overhead is now itemized in Appendix D.4.

Question

Please add the MedQA and FinQA references at their first mention (Lines 84–85), and the HarmChain reference at its first mention (Lines 156–157).

Response

Thank you for the suggestion. We have added the MedQA and FinQA references at their first mention in the main text, and the HarmChain reference at its first mention in the method description.

Question

What is HL(θ)H_L(\theta) in Eq. 4? Is it the Hessian of the task loss with respect to the model weights?

Response

Thank you for pointing this out. Yes, HL(θ)=∇θ2L(θ)H_L(\theta)=\nabla^2_{\theta}L(\theta) is the Hessian of the loss LL with respect to the model weights, evaluated at θ\theta. We have updated the text immediately after Eq. 4 to define this notation explicitly.

We hope these responses and the revised manuscript address your concerns. We would be glad to clarify any remaining points during the discussion period.

Reviewer_zGnHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training trick to stop a model from losing good habits, like refusing harmful requests or not making things up, while it’s being fine-tuned for a specialist job. The authors call it Resistance Training (ResT). When you fine-tune a general model for one job, it usually gets better at that job but worse at things you weren’t training. This is called “forgetting.” In the paper’s experiments, Qwen3-4B was fine-tuned on medical questions (MedQA) and financial reasoning (FinQA), and its task accuracy improved while it refused fewer harmful requests and hallucinated more. ResT trains the model on its specialist task while it wears a removable “bad-behavior” adapter, so it learns to resist that behavior; the paper reports that once the adapter is removed, the model keeps the task gains while being safer and hallucinating less.

Critical strengths
  1. A simple, cheap intervention with a large, consistent effect on hallucination. Behavior loss during ordinary fine-tuning is a real deployment problem. This is a drop-in fix whose hallucination effect is far larger than evaluation noise.

  2. An informative study of how to build the “load,” in the on-policy setting that current post-training uses. The paper shows on-policy distillation and self-distillation still erode behaviors even as task accuracy rises.

Critical weaknesses
  1. The evaluation is too narrow, and closely matched to the training source, to support a “reduces forgetting” claim. The hallucination test is tiny and almost in-distribution, over-refusal and hedging are never checked, and the refusal gains are modest.

  2. One training run per setting, no validation-based selection, and results that swing with the fitting schedule.

3.The core procedure isn’t new. Fit an adapter to the bad behavior, freeze it during fine-tuning, remove it afterwards: this is Security Vectors. The closest prior method isn’t a baseline, although ResT’s response-supervised variant is close to it.

Questions for authors
  1. Overlap between the hallucination evaluation and the load data. The 20 evaluation questions and the 1,000 load prompts both come from the persona_vectors repository, separated only by a TF-IDF similarity filter at 0.82 (B.2). How close are they in topic and format? Does the hallucination reduction hold on an independent factuality benchmark not drawn from that source, such as SimpleQA or TruthfulQA?

  2. A Security Vectors baseline under your downstream objectives. It is the closest prior method, using the same fit–freeze–remove procedure, and is currently not compared?

  3. Extending to more model families and sizes beyond Qwen3-4B, and to RL objectives such as GRPO and DPO, which the conclusion already names?

7 comments
AuthorsOpenReview ↗

Response to Reviewer zGnH (Part 1)

Thank you for your thoughtful comments and helpful suggestions. We have revised the manuscript and added experiments to address your concerns. Below, we respond to each point and explain the corresponding changes. We hope these responses and additions clarify our work and address your questions.

Question

The evaluation is narrow and close to the load-data source (Hallucination). Are there overlap concerns, and do the results hold on an independent factuality benchmark? Over-refusal and hedging are not checked, and the refusal gains appear modest.

Response

Thank you for suggesting these evaluations! Based on your suggestions, we added over-refusal evaluation on 400 benign requests from FalseReject [1] and WokeyTalky [2], and independent factuality evaluation on all 817 TruthfulQA questions [3]. Over-refusal ↑ reports benign compliance; Truthfulness ↑ reports MC2 probability mass on true answers. To assess whether reduced hallucination comes with a loss of factual correctness, TruthfulQA scores the supplied answer options by conditional likelihood under a shared prompt, without an LLM judge grading generated wording. Evaluation details are in Appendix D.5 in our revised paper.

Tables 1 and 2 now report mean ± sample standard deviation over training seeds 42, 43, and 44. Each protection setting is a separate experiment. All metrics use a 0–100 scale; bold marks the best post-training mean per domain and column.

Table 1. Downstream OPD

Medical (MedQA)

MethodSafety: Task acc. ↑Refusal ↑Over-refusal ↑Factual: Task acc. ↑Hallucination ↓Truthfulness ↑
Initial model71.8031.2191.9771.8034.0054.84
Vanilla79.00 ± 0.7128.98 ± 0.2193.49 ± 0.1179.00 ± 0.7155.42 ± 0.8155.19 ± 0.04
Shadow-FT‡78.84 ± 0.5928.49 ± 0.5794.77 ± 1.0078.84 ± 0.5939.40 ± 1.8255.69 ± 0.30
OSFT79.50 ± 0.1325.94 ± 2.1295.07 ± 1.9478.06 ± 0.1254.65 ± 0.9654.98 ± 0.19
Replay79.47 ± 0.3336.74 ± 2.0789.59 ± 0.6677.82 ± 0.2032.46 ± 3.7155.92 ± 0.16
Security Vectors80.75 ± 0.2132.46 ± 1.8792.09 ± 1.2880.02 ± 0.5239.71 ± 1.6756.29 ± 0.37
ResT: Generic80.23 ± 0.5330.35 ± 0.1092.11 ± 0.6980.83 ± 1.0925.44 ± 1.2554.96 ± 0.35
ResT: Targeted80.23 ± 0.2537.04 ± 0.4090.32 ± 0.5379.58 ± 0.2113.42 ± 3.5957.68 ± 0.05

Finance (FinQA)

MethodSafety: Task acc. ↑Refusal ↑Over-refusal ↑Factual: Task acc. ↑Hallucination ↓Truthfulness ↑
Initial model70.1831.2191.9770.1834.0054.84
Vanilla71.46 ± 0.8127.48 ± 1.5393.60 ± 0.1371.46 ± 0.8145.40 ± 0.9955.19 ± 0.35
Shadow-FT‡73.52 ± 0.2727.32 ± 0.0894.31 ± 0.4473.52 ± 0.2728.45 ± 1.0854.58 ± 0.45
OSFT68.15 ± 1.3926.06 ± 0.2694.24 ± 0.2570.45 ± 0.6848.09 ± 0.7554.75 ± 0.13
Replay70.18 ± 1.3935.49 ± 5.2190.55 ± 3.2570.97 ± 0.9237.34 ± 2.9655.34 ± 0.05
Security Vectors72.30 ± 0.8230.03 ± 0.4793.03 ± 0.6271.81 ± 0.4837.43 ± 1.0655.93 ± 0.15
ResT: Generic71.73 ± 1.5227.55 ± 0.4293.53 ± 0.5872.28 ± 0.2329.42 ± 2.4554.98 ± 0.29
ResT: Targeted72.04 ± 0.4132.43 ± 1.1891.77 ± 0.1172.68 ± 0.3310.32 ± 3.2356.25 ± 0.62

‡ Most Shadow-FT generations under OPD reach the length limit; the corresponding measurements and explanation are retained in Appendix D.3.

References

[1] FalseReject: A Resource for Improving Contextual Safety and Mitigating Over-Refusals in LLMs via Structured Reasoning. Zhehao Zhang, Weijie Xu, Fanyou Wu, and Chandan K. Reddy. COLM 2025. Paper.

[2] WokeyTalky: Towards Scalable Evaluation of Misguided Safety Refusal in LLMs. Yi Zeng, Adam Nguyen, Bo Li, and Ruoxi Jia. Official project.

[3] TruthfulQA: Measuring How Models Mimic Human Falsehoods. Stephanie Lin, Jacob Hilton, and Owain Evans. ACL 2022. ACL Anthology.

(Continued in Part 2/5.)

AuthorsOpenReview ↗

Response to Reviewer zGnH (Part 2)

Table 2. Downstream OPSD

Medical (MedQA)

MethodSafety: Task acc. ↑Refusal ↑Over-refusal ↑Factual: Task acc. ↑Hallucination ↓Truthfulness ↑
Initial model71.8031.2191.9771.8034.0054.84
Vanilla75.86 ± 0.4024.36 ± 0.6493.82 ± 0.2275.86 ± 0.4052.67 ± 0.2155.28 ± 0.07
Shadow-FT72.06 ± 0.0529.04 ± 0.8293.11 ± 0.1672.06 ± 0.0540.54 ± 0.1854.51 ± 0.30
OSFT75.60 ± 0.5422.12 ± 1.1594.80 ± 0.6976.72 ± 0.2053.74 ± 0.5655.39 ± 0.11
Replay76.36 ± 0.2135.28 ± 1.0590.57 ± 0.1375.12 ± 0.4133.10 ± 3.5555.61 ± 0.19
Security Vectors76.48 ± 1.3529.82 ± 0.0290.73 ± 0.9574.84 ± 1.3243.21 ± 1.9856.05 ± 0.18
ResT: Generic76.15 ± 0.4726.99 ± 0.1594.25 ± 0.3177.61 ± 0.2137.05 ± 0.6054.50 ± 0.32
ResT: Targeted77.80 ± 0.2335.44 ± 0.8790.74 ± 0.2974.99 ± 0.3113.92 ± 0.4557.49 ± 0.07

Finance (FinQA)

MethodSafety: Task acc. ↑Refusal ↑Over-refusal ↑Factual: Task acc. ↑Hallucination ↓Truthfulness ↑
Initial model70.1831.2191.9770.1834.0054.84
Vanilla72.42 ± 0.7522.62 ± 0.6594.41 ± 0.0972.42 ± 0.7547.49 ± 2.0055.47 ± 0.33
Shadow-FT71.70 ± 0.1330.32 ± 0.3492.49 ± 0.3771.70 ± 0.1333.94 ± 3.0654.62 ± 0.20
OSFT71.43 ± 0.1820.26 ± 0.6895.75 ± 0.0871.43 ± 0.3447.51 ± 1.2254.92 ± 0.11
Replay72.68 ± 0.6133.81 ± 0.4689.15 ± 0.3771.58 ± 0.7626.33 ± 1.7356.19 ± 0.03
Security Vectors74.05 ± 0.4026.14 ± 0.8693.55 ± 0.3874.42 ± 0.5333.69 ± 0.5956.15 ± 0.29
ResT: Generic73.12 ± 1.0721.96 ± 0.7994.80 ± 0.2573.00 ± 0.7936.91 ± 7.9254.58 ± 0.07
ResT: Targeted73.50 ± 0.7826.76 ± 0.2592.77 ± 0.1674.54 ± 0.7014.77 ± 1.2756.25 ± 0.17

Targeted ResT has the lowest hallucination and highest Truthfulness means in all four task–objective comparisons. Although the absolute refusal gains may appear modest (4.14–11.08 points), they represent 18.0–45.5% relative improvements over Vanilla, accompanied by relative benign-compliance decreases of 1.7–3.4% (1.64–3.17 points). For example, on MedQA under OPSD, refusal rises from 24.36 to 35.44, a 45.5% relative improvement, while benign compliance changes from 93.82 to 90.74, a 3.3% relative decrease, close to the initial model’s 91.97. These paired metrics show substantially better retention of harmful-request refusal with a comparatively small change in benign assistance. Replay exhibits a similar refusal–compliance trade-off.

Regarding hedging, we followed the reviewer’s suggestion in Question 1 and evaluated on TruthfulQA [3]. Its MC2 score measures the probability mass assigned to true answers, so it would decrease if lower hallucination came from assigning less probability to correct answers. MC2 does not decrease: targeted ResT is above both Vanilla and the initial model (54.84) in all four comparisons (56.25–57.68 versus 55.19–55.47 for Vanilla).

Regarding the reviewer’s concern about overlap between the 1,000 load prompts and the 20 hallucination evaluation questions: both are sourced from the Persona Vectors repository [4], and the evaluation follows that work’s protocol for the hallucination trait. The evaluation therefore measures protection on held-out questions of the same kind as the load source. Two points support the comparison under this protocol.

First, the comparison is matched for the methods that use this source. Replay and Security Vectors draw on the same source prompts as targeted ResT, yet targeted ResT’s hallucination score is 11.6–27.0 points lower than Replay and 18.9–29.3 points lower than Security Vectors across the four main comparisons (Tables 1–2).

Second, the held-out questions are not near-duplicates of the load prompts. We audited the actual training and evaluation files across all 20,000 cross-set pairs and found zero exact or normalized matches. With character 4–5-gram TF-IDF cosine similarity, the mean nearest-training-neighbor similarity per evaluation question is 0.207, and the maximum across all pairs is 0.398, well below the 0.82 near-duplicate threshold used in data construction (Appendix B.2).

All methods are evaluated on the same question sets with the same generation and scoring protocols. On the independently sourced TruthfulQA benchmark [3], targeted ResT also has the highest MC2 mean in all four main comparisons, supporting factuality gains beyond the original source.

References

[3] TruthfulQA: Measuring How Models Mimic Human Falsehoods. Stephanie Lin, Jacob Hilton, and Owain Evans. ACL 2022. ACL Anthology.

[4] Persona Vectors: Monitoring and Controlling Character Traits in Language Models. Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, and Jack Lindsey. 2025. arXiv:2507.21509.

AuthorsOpenReview ↗

Response to Reviewer zGnH (Part 3)

Question

One training run per setting, no validation-based selection, and results that swing with the fitting schedule.

Response

Thank you for the suggestion. The main comparisons, SFT results, and model-size/family and DPO/GRPO extensions now use three training seeds (Tables 1–2, 11–13, and 15–17). The full fitting-duration sweeps and Figure 3 have also been repeated across three seeds. On-policy fitting retains its hallucination advantage at every shared fitting duration across both sources and downstream objectives.

Regarding selection, the main tables use fixed schedules. In Figure 3, the downstream task serves as the selection criterion: within each seed, we select the configuration with the highest MedQA accuracy, breaking ties by the smaller epoch, and then report the protected-behavior score of that same model. Selection therefore uses only the task metric and never the protected behavior, so the refusal and hallucination scores in Figure 3 are held out with respect to this selection, mirroring how a practitioner would tune the load on the target task. Tables 5–6 report all candidates, and the equal-duration comparisons, which involve no selection, support the same hallucination advantage across the full tested grid. The schedules, selection rule, and per-seed results are provided in Section 3.3, Appendix D.4, and Tables 4–6.

Question

The fit–freeze–remove procedure is Security Vectors. What is new, and why is the closest prior method not compared?

Response

Thanks for raising the concern. ResT introduces on-policy resistance fitting, broadens load sources to paired base models and naturally degraded checkpoints, and theoretically derives sufficient conditions for protection through task learning (Section 2 and Appendix C). These are the methodological and theoretical advances over the removable-adapter procedure studied in SV.

We have added a capacity-matched SV-style baseline, fitted with min–min plus KL on the same targeted sources and evaluated under the same OPD/OPSD objectives (Appendix D.3). Targeted ResT has lower hallucination and higher independent Truthfulness means in all four main comparisons (Tables 1–2). Figure 3 separately demonstrates the benefit of on-policy over response fitting. We have clarified the Introduction and contributions. Our common response to all reviewers and the AC ``General Response on Contributions, Novelty, and New Experiments'' gives the full account of these advances, including the on-policy versus response-fitting comparison and the comparison with the SV-style baseline. We hope these can help clarify our contributions and novelty more clearly!

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response on Contributions, Novelty, and New Experiments (Part 1)

We thank the reviewers and AC for the effort and valuable feedback. To clarify the contributions and novelty of our work, we summarize the paper’s core contributions in this comment (Part 1/2) and list the experiments and clarifications added during rebuttal in the following comment (Part 2/2).

Core contributions

Overall, ResT advances how temporary behavioral loads are constructed and provides a theoretical account of when ordinary task learning turns them into protection after removal. Our work combines on-policy load fitting, broader behavioral sources, and sufficient conditions for protection, supported by systematic experiments.

First, we introduce on-policy resistance fitting and develop a theoretical analysis of how this construction can yield protection through ordinary task learning. The adapter-conditioned student generates trajectories, matches a frozen resistance teacher’s distributions at the prefixes it actually visits, and resamples trajectories as it changes. In Section 2 and Appendix C, we connect load fitting to the compensating updates learned during downstream training: under fixed features and a local quadratic approximation, we prove that the backbone’s change relative to Vanilla after load removal corresponds to the load direction exposed by downstream task features, and derive sufficient conditions for protection. These conditions involve displacement strength, fitting error, task-feature coverage, and transfer to the protection distribution, covering both response and on-policy fitting. We therefore propose a load-fitting method and characterize which loads ordinary task learning can turn into protection. The adapter is a concrete implementation of this temporary behavioral displacement. Figure 3 demonstrates the benefit of on-policy fitting across both sources and downstream OPD/SFT objectives. For example, with the targeted source under downstream SFT, on-policy fitting lowers the mean hallucination score from 41.9 to 17.6 at higher MedQA accuracy (76.25 versus 75.23). Against a capacity-matched SV-style baseline fitted on the same targeted sources, targeted ResT’s hallucination score is 18.9–29.3 points lower across the four main OPD/OPSD comparisons (Tables 1–2).

Second, we broaden the sources of the load on a principled basis. Our analysis characterizes a load by the behavioral displacement it induces relative to the protected behavior, rather than by a particular set of undesirable demonstrations. Security Vectors (SV) fits adapters to explicitly specified undesirable demonstrations; under our characterization, any source that satisfies the displacement conditions can serve as a load. ResT therefore also uses a paired base model and naturally degraded task checkpoints, which require no behavior-specific undesirable demonstrations. Targeted resistance expresses harmful compliance or confident unsupported answers; generic resistance uses a paired base model to construct a load that shifts behavior toward its pre-instruction-tuning state. Generic construction requires neither manually designed behavior-specific bad answers nor a separately trained degradation teacher: response fitting uses base-model answers on selected prompts, while on-policy fitting queries the frozen base model at student-generated prefixes without requiring a pre-generated response dataset. We also study checkpoints that naturally degrade during ordinary task training as resistance sources (Figure 4). These constructions reduce reliance on behavior-specific data construction. Degradation here refers to the protected behavior; it does not require every base-model behavior to be worse or instruction tuning to be fully reversed. Our analysis also applies to generic and targeted loads satisfying the behavioral-displacement conditions, providing a unified theoretical basis for these sources. Figures 3–4 show that readily available behavioral sources can provide useful protection while retaining task gains.

We cited and discussed SV in the original manuscript. ResT advances the fitting method, load sources, and theoretical conditions for protection. We have revised the Introduction and contribution statements to make these advances explicit. The experiments and clarifications added during rebuttal, which support these contributions, are listed in Part 2/2.

AuthorsOpenReview ↗

General Response on Contributions, Novelty, and New Experiments (Part 2)

What we added during rebuttal

  • Direct SV comparisons. We added a capacity-matched SV-style baseline using min–min plus KL fitting on the same targeted sources. Targeted ResT has higher refusal, lower hallucination, and higher independent Truthfulness means in all four main OPD/OPSD comparisons (Tables 1–2).

  • Independent factuality and over-refusal evaluation. The revised tables include the full TruthfulQA multiple-choice validation set and benign-request compliance. Targeted ResT has the highest Truthfulness mean in every main comparison; compliance is reported alongside refusal to show the accompanying behavior (Tables 1–2; Appendix D.5).

  • Three-seed comparisons and complete fitting sweeps. We repeated the main comparisons, SFT experiments, and fitting-duration sweeps across three training seeds. Figure 3 now averages independently task-selected runs within each seed. On-policy fitting gives lower mean hallucination at every shared fitting duration across both sources and downstream objectives (Tables 1–2, 4–6, and 11; Appendix D.4).

  • Additional model sizes and family. We added Qwen3-1.7B, Qwen3-8B, and non-thinking Llama-3.2-1B-Instruct experiments. At both additional Qwen3 scales, targeted ResT has the lowest hallucination mean and generic ResT the highest factual-setting task accuracy. On Llama, both variants improve refusal, hallucination, and Truthfulness over SV, and generic ResT has the highest Truthfulness mean (Tables 12–15; Appendix F).

  • DPO and GRPO. We added three-seed MedQA comparisons under both objectives. Targeted ResT has the lowest hallucination mean and improves factual-setting task accuracy over Vanilla. Generic ResT also provides task-performance and Truthfulness gains under GRPO (Tables 16–17; Appendix G).

We hope this distinction between the original contributions and the additional evidence makes the novelty and empirical support for our work clearer.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.