acceptodds
Under review as a conference paper at ICLR 2027

SafeQ: Safety Alignment Preserving Post-Training Quantization for Language Models

Abstract

With the growing adoption of Language Models (LMs), Post-Training Quantization (PTQ) has become a popular standard for efficient LM deployment, especially to resource-constrained environments like personal and edge devices. Most PTQ methods only consider to preserve natural language understanding (understanding) capabilities, without taking safety capabilities into consideration, that enables LMs to refuse harmful, unethical, privacy-breaching, or illegal user queries. This inadvertently causes complete or partial safety alignment degradation in quantized LMs. To address this, we propose SafeQ, a learnable PTQ method that preserves safety alignment by taking both understanding and safety capabilities of LMs into consideration. Precisely, SafeQ learns quantization parameters by minimizing a block-wise weighted-sum loss over understanding and safety capabilities for both weight-only quantization and weight-activation quantization. We performed extensive evaluations for assessing understanding capabilities, and safety and general utility capabilities using different pre-trained and instruction-tuned LMs respectively. Our evaluations demonstrate best or near-best performance by SafeQ in pre-trained LMs for W4A16 (i.e., 4-bit weight and 16-bit activation), and W4A4 quantization modes, and in instruction-tuned LMs for W4A16 quantization mode. In few instances, SafeQ improved safety capabilities of quantized LMs slightly beyond full-precision LMs. Our results evidently show that SafeQ resolved trade-off between safety and general utility capabilities to a great extent in instruction-tuned LMs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Language Models, Quantization, Safety Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hb1rOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes SafeQ, a safety-aware post-training quantization method intended to preserve both general capabilities and safety alignment. It reuses OmniQuant’s learnable weight clipping and learnable equivalent transformation modules, applying an MSE reconstruction loss to ordinary text and a negative log cosine similarity loss to safety-related inputs through block-wise optimization of quantization parameters. Weight-only quantization enables weight clipping alone, whereas joint weight–activation quantization uses both modules. The paper evaluates language modeling on pretrained models and safety knowledge, general task performance, and conversational quality on three instruction-tuned models, alongside inference speed and memory usage. Under W4A16 quantization, SafeQ improves average SafetyBench accuracy over OmniQuant by approximately 1.27 and 1.76 percentage points on Llama-2 and Llama-3.1, respectively, while also improving average general task accuracy. The reported deployment experiments on Llama-2-7B-Chat show an end-to-end speedup of approximately 2.33–3.01× over FP16.

Critical strengths
  1. The paper provides a relatively simple implementation of safety-aware model compression by jointly constraining general capabilities and safety-related representations during calibration. It optimizes clipping and transformation parameters without retraining all model weights or requiring an additional safety detector at inference time. Incorporating the safety objective directly into quantization offers a practical approach to mitigating safety degradation during compression.
  2. The evaluation considers capability preservation alongside deployment efficiency and demonstrates simultaneous improvements in safety knowledge and general task performance in some W4A16 settings. On Llama-2 and Llama-3.1, SafeQ improves average SafetyBench accuracy over OmniQuant while increasing average general task accuracy by approximately 0.63 and 0.48 percentage points, respectively. The inference-time and memory measurements characterize the deployment benefits of the quantized models, while the conversational evaluation and W4A4 experiments also document performance limitations across settings.
Critical weaknesses
  1. The central claim of preserving safety alignment is not adequately validated against the behavioral objective stated in the paper. Safety alignment is described as the ability to refuse harmful, illegal, or privacy-violating requests, whereas the main safety experiments measure multiple-choice accuracy on SafetyBench. This metric provides indirect evidence but cannot by itself establish whether refusal behavior and unsafe generation are preserved or improved after quantization. Neither the main text nor the appendix quantitatively reports harmful completion rates, jailbreak attack success rates, or over-refusal on benign requests. Moreover, the average safety score on Llama-3.1 decreases from 77.67 in FP16 to 74.43, indicating model-dependent preservation even on the chosen proxy. The evidence therefore does not adequately substantiate the stronger claims of “higher safety guarantees” or substantially resolving the safety–utility trade-off.
  2. The claim of being the first safety-alignment-preserving PTQ method is insufficiently distinguished from prior work. CAQ by Wee et al. [1] incorporates an alignment-preserving objective directly into PTQ, encouraging the quantized model to approach a safely aligned reference while diverging from an unaligned pretrained model. SafeQ differs in calibration data, loss formulation, and reference-model requirements, but these differences do not establish the broad first-of-its-kind claim as currently stated. Although the paper identifies implementation features such as using a single reference model, it does not establish their advantages through corresponding effectiveness or cost analyses. The incremental contribution and practical benefits over existing safety-aware quantization approaches therefore require clearer substantiation.
  3. The ablations mainly examine the hyperparameter sensitivity of the complete method and do not adequately isolate the contributions of safety-related data and the cosine-based loss. The paper varies calibration sample counts, optimization epochs, and the safety coefficient, but does not report matched controls that remove the safety objective or apply MSE uniformly to the same mixed calibration data. The aggregate comparison with OmniQuant therefore leaves the effects of data coverage, loss formulation, and calibration effort unresolved. On Qwen, SafeQ exceeds OmniQuant by only 0.04 percentage points in average safety accuracy while reducing average general task accuracy. Variability across calibration samples or random initializations is also unreported, limiting the assessment of the magnitude and stability of the proposed design’s benefits.
Questions for authors
  1. SafetyBench provides the main evidence for the safety evaluation. Could you supplement it with generation-based evaluations on held-out harmful requests, adversarial prompts, and benign requests? In particular, do improvements in safety knowledge correspond to fewer unsafe completions, and are they accompanied by increased over-refusal?
  2. Please clarify SafeQ’s contribution relative to CAQ [1], particularly the implications of its single-reference formulation and calibration objectives for effectiveness and cost. Q-realign [2] also builds on OmniQuant and introduces different objectives for benign and harmful inputs during quantization, although its primary goal is to recover alignment degraded by fine-tuning. Clarifying the differences in problem setting and technical design would help establish SafeQ’s positioning.
  3. Could you separately evaluate the effects of safety-related calibration data and the cosine-based loss under comparable data sizes and optimization budgets, and report the stability of the main improvements across calibration samples or random initializations? Please also clarify whether the AdvBench calibration segments include responses, how they are assembled into model inputs, and which token positions contribute to the safety loss.
  4. Please compare the quantization-stage costs of SafeQ and existing methods, including wall-clock time, peak GPU memory, and the incremental cost of safety-related calibration. The current speed and memory experiments assess deployment after quantization and do not quantify the additional effort introduced by the safety-aware design. Measurements under comparable hardware and calibration budgets would help characterize the computational cost associated with the reported safety improvements.
1 comment

We sincerely thank the reviewer for the valuable and detailed feedback. We are highly encouraged by the reviewer’s recognition of SafeQ’s simple and practical design, its lack of inference-time overhead, and our transparent reporting of results and limitations.

We provide explanations for the highlighted concerns below, which we are committed to address in the next version.

Generation-based Safety Evaluation (W1, Q1)

We chose SafetyBench because it is an established and reproducible benchmark covering 7 safety dimensions, and it allows a consistent per-dimension comparison across all baselines. Figure 1 qualitatively shows that SafeQ restores refusals that baseline-quantized models lose, and MT-Bench (Figure 3) suggests that preserving safety does not cause broad over-refusal. On Llama-3.1-8B-Instruct, all quantized models fall below FP16 in ACC_S, but SafeQ retains the highest ACC_S among them. We agree that harmful-completion rates, jailbreak attack success rates and over-refusal rates would directly validate the behavioral claim. We will include them and tone down the “higher safety guarantees” and trade-off statements in the revised manuscript.

Positioning w.r.t. CAQ and Q-realign (W2, Q2)

We will remove the “first” claim. Instead, we will position SafeQ as an early safety alignment preserving PTQ that differs from CAQ/AAQ [1] in two ways:

  1. It needs only a single model version (pre-trained or instruction-tuned), whereas CAQ requires loading both versions.
  2. It uses a decoupled weighted-sum loss over understanding and safety calibration data, whereas CAQ uses a contrastive push-pull objective.

We could not empirically compare effectiveness and cost because CAQ’s code is not publicly available. Q-realign [2] targets a different problem setting, i.e., recovering alignment degraded by fine-tuning. SafeQ instead preserves the existing alignment of a full-precision model during quantization. We will add this discussion to the revised manuscript.

Isolating Data and Loss Contributions (W3, Q3)

OmniQuant shares SafeQ’s LWC/LET framework with an MSE loss on understanding data only, so it serves as the “no safety objective” control. Our λ_SA ablation (Figure 5(c), Table 12) further shows that the weight of the cosine-based safety loss directly affects safety accuracy. Regarding the calibration data, we use 128 randomly selected 2048-token segments from AdvBench. L_S is computed over the entire block output. We agree that matched MSE vs. cosine controls and variance across seeds and calibration samples would better quantify stability. We will include them and clarify these calibration details in the next version.

Quantization-stage Cost (Q4)

SafeQ builds on OmniQuant and keeps its block-wise optimization. The only additional cost is the forward pass and loss on the safety calibration data. All our quantization runs fit on a single 24GB RTX 4500 ADA GPU with a batch size of 1. We will report wall-clock time, peak GPU memory and the incremental cost relative to OmniQuant in the revised manuscript.

Reviewer_F1kqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper observes that PTQ enables the deployment of small language models (SLMs) on personal and edge devices, but that existing PTQ methods optimize only comprehension and general capabilities without explicitly preserving safety alignment. The authors propose SafeQ, which optimizes two objectives block by block: comprehension deviation, measured on a comprehension calibration set, and safety deviation, measured by negative log cosine similarity on a safety calibration set. The method is evaluated through tests and comparative experiments with encouraging results.

Critical strengths
  1. The problem addressed is of practical importance, and the paper provides mechanistic evidence for its motivation.
  2. The method is theoretically well founded, and some design choices show a degree of novelty.
  3. The ablation study covers three key hyperparameters and reports per-dimension data, and the experimental reporting is honest.
Critical weaknesses

The property emphasized by the paper, "preserves safety alignment", is operationalized in the experiments as accuracy on safety-knowledge multiple-choice questions. However, many modern jailbreak techniques prepend long-context prompts to circumvent a model's safety restrictions, and this dataset was released in 2024. By design, it only tests the model's direct understanding of safety-related concepts. Could the authors consider more recent datasets for this evaluation? 2. The models used in the comparison are dated: Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct, for example, are now quite old. The most recent baseline method dates from 2025, and even against these baselines SafeQ holds only a marginal advantage. The Related Work, however, discusses FlatQuant, LeanQuant, Kurtail, DFRot, Matryoshka, QuIP, GPTQ, QLLM, FineQuant, RPTQ, and APTQ in detail. Why are more recent methods not included in the comparison? 3. The proposed method shows no advantage in Table 1. Taken together with Table 2, the results indicate only a marginal advantage in safety alignment and a clear degradation in comprehension ability, and comprehension and safety are never measured on the same model. As they stand, the experimental results cannot support the authors' claim that "By integrating calibration data of safety capabilities, SafeQ effectively maintains comparable understanding capabilities along with higher safety and general utility capabilities in LMs. Further, SafeQ evidently resolves the trade-off between safety and general utility to a great extent in instruction-tuned LMs." 4. The "Why NOT use LLMs in Evaluations" part of the Discussion states that Llama-3.1-8B-Instruct and Llama-3.1-405B-Instruct "only differ by number of transformer blocks", and infers from scaling laws that the safety guarantees obtained for Llama-3.1-8B-Instruct carry over to the 405B model. In fact, the two models differ in hidden size, number of attention heads, FFN dimensions, and other architectural parameters. Moreover, the premise that quantization-induced safety damage is invariant to model scale is supported by no literature, and the authors provide no argument for it. 5. The Introduction claims that SafeQ is the "first safety alignment preserving PTQ", whereas the Discussion's "Comparison between SafeQ and AAQ" states: "The goal of our SafeQ and AAQ (Wee et al., 2025) (previously CAQ) are same, i.e., preserving safety alignment of LMs while performing quantization. But, they fundamentally differ in the methodology of quantization process followed." No additional literature search is needed here: the paper refutes its own claim.

Questions for authors

N/A

1 comment

We sincerely thank the reviewer for the valuable feedback. We are highly encouraged by the reviewer’s recognition of the practical importance of the problem, the theoretical grounding and novelty of our design, and the honesty of our experimental reporting.

We provide explanations for the highlighted concerns below, which we are committed to address in the next version.

Choice of Safety Benchmark (W1)

We chose SafetyBench because it is an established and reproducible benchmark. It covers 7 safety dimensions, which lets us analyze per-dimension safety preservation across all baselines consistently. We agree that recent jailbreak benchmarks (e.g., long-context and prompt-based attacks) would further strengthen the evaluation. We will include them in the revised manuscript.

Choice of Models and Baselines (W2)

We chose Llama-2-7B-Chat, Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct because they are the standard models in the quantization literature that our baselines report on. This allows a fair and reproducible comparison. We included DuQuant and SpinQuant as representative state-of-the-art rotation-based PTQs. The methods in Related Work are discussed to give context on the quantization landscape. We will include more recent models and baselines (e.g., FlatQuant) in the next version.

Understanding vs. Safety Results (W3)

We respectfully note that SafeQ does not clearly degrade understanding. In Table 1, SafeQ is second best on W4A16 for both Llama-7B and Llama-2-7B, and on W4A4 it outperforms OmniQuant, the framework it builds upon. Further, Table 2 measures general utility (ACC_GU) and safety (ACC_S) on the same instruction-tuned models. SafeQ achieves the best ACC_GU on Llama-3.1-8B-Instruct while also improving ACC_S over all baselines on Llama-2-7B-Chat and Llama-3.1-8B-Instruct. That said, we agree that the gains are moderate in some settings. We will tone down the conclusion statements accordingly in the revised manuscript.

Generalization to Larger Models (W4)

We thank the reviewer for pointing out this inaccuracy. We agree that Llama-3.1-8B-Instruct and Llama-3.1-405B-Instruct also differ in hidden size, attention heads and FFN dimensions. We will correct this statement, remove the scaling law based extrapolation, and clearly state evaluation on larger models as a limitation of our work.

“First Safety Alignment Preserving PTQ” Claim (W5)

We thank the reviewer for highlighting this inconsistency. We will remove the “first” claim from the Introduction. We will instead position SafeQ as an early safety alignment preserving PTQ that, unlike AAQ, needs only a single model version (pre-trained or instruction-tuned) and uses a decoupled weighted-sum loss over understanding and safety calibration data.

Reviewer_Lt89OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SafeQ aims to preserve the safety alignment of language models during post-training quantization while maintaining their language capabilities.

Critical strengths
  1. The paper addresses an important deployment problem: preserving safety alignment during model quantization. Considering safety alongside general language capabilities is practically relevant for resource-constrained deployment.

  2. The evaluation covers several model families and established quantization baselines, including AWQ, OmniQuant, DuQuant, and SpinQuant. It assesses language modeling, safety-related understanding, general utility, instruction-following, inference speed, and memory usage.

Critical weaknesses
  1. The main quantitative safety evaluation uses SafetyBench accuracy, which measures safety-related understanding rather than harmful-request refusal behavior. Although Figure 1 provides qualitative examples, systematic quantitative evaluations of generated responses are missing. Consequently, the results do not sufficiently establish the central claim that SafeQ preserves safety alignment during quantization.

  2. The experiments do not isolate the contribution of the proposed objective. SafeQ builds on OmniQuant’s LWC and LET components and introduces safety calibration data and a weighted reconstruction objective. The ablations vary sample counts, training epochs, and loss weights, but do not compare the proposed cosine-based safety loss with MSE on the same safety data. It therefore remains unclear how much improvement comes from the loss design rather than the additional calibration data.

Questions for authors

Q1. Could the authors report harmful-response or attack success rates on held-out harmful requests, comparing FP16, SafeQ, and the quantization baselines? Reporting refusal rates on benign requests would also help assess whether safety preservation introduces excessive refusal.

Q2. Could the authors compare understanding-only MSE, MSE on both calibration datasets, and the proposed combined objective under matched calibration and optimization budgets? This would distinguish the contribution of safety data from that of the loss function.

1 comment

We sincerely thank the reviewer for the valuable feedback. We are highly encouraged by the reviewer’s recognition of the practical importance of our research and the breadth of our evaluations.

We provide explanations for the highlighted concerns below, which we are committed to address in the next version.

Quantitative Evaluation of Harmful-Request Refusal (W1, Q1)

We agree that SafetyBench measures safety-related understanding rather than refusal behavior directly. We chose SafetyBench because it is an established, reproducible benchmark that covers 7 safety dimensions (OFF, UB, PH, MH, IA, EM, PP). Accuracy on it reflects whether the model still recognizes unsafe content, which is the capability that refusal depends on. Figure 1 qualitatively shows that this preserved understanding leads to refusals on harmful queries where baseline-quantized models comply. Further, the MT-Bench results (Figure 3) show that SafeQ keeps its instruction-following capability, which suggests that preserving safety does not cause broad over-refusal. We agree that attack success rates on held-out harmful requests and refusal rates on benign requests would strengthen this claim. We will include them by comparing FP16, SafeQ and the baselines, in the revised manuscript.

Isolating the Contribution of the Proposed Objective (W2, Q2)

We note that OmniQuant uses the same LWC and LET components with an MSE loss on understanding calibration data only. It therefore serves as the understanding-only MSE baseline under the same optimization framework. SafeQ’s consistent safety gains over OmniQuant (e.g., +1.28% ACC_S on Llama-2-7B-Chat) reflect the combined effect of the safety calibration data and our safety-aware loss. Additionally, our λ_SA ablation (Figure 5(c), Table 12) shows that changing the weight of the cosine-based safety loss directly changes safety accuracy. This indicates that the loss term actively contributes rather than acting only as extra data. We chose cosine similarity for L_S because it preserves the direction of the block outputs, which is closely linked to safety-relevant representations, rather than only their magnitude. We agree that a controlled comparison with MSE on both datasets, under matched budgets, would separate these contributions more clearly. We will include this comparison in the next version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.