acceptodds
Under review as a conference paper at ICLR 2027

IUC: Intersection–Union Correction for Quantization-Activated Attacks on LLMs

Abstract

While quantization is a key technique for the efficient deployment of large language models (LLMs), recent studies have shown that it may also introduce new security risks: a model that behaves normally in full precision may exhibit malicious behaviors after quantization, such as advertisement injection, over-refusal, or jailbreak, posing a practical threat to the secure deployment of LLMs. To address this issue, we propose **I**ntersection–**U**nion **C**orrection (IUC), a *one-shot* defense method applied directly to the full-precision model. IUC models local parameter intervals under different quantization configurations, where the intersection of these intervals characterizes the shared local reference region underlying the correction, while the union constrains the magnitude of parameter correction. *A smaller intersection indicates a narrower shared local reference region,* and IUC prioritizes the correction of such parameters. With only a single correction applied to the full-precision model, IUC supports secure deployment under multiple commonly used quantization configurations. We conduct experiments across multiple LLMs, different attack scenarios, and representative quantization configurations. The results show that IUC effectively suppresses the activation of malicious behaviors after quantization with low resource overhead, while largely preserving model utility, providing a one-shot pre-sanitization solution for secure model deployment under commonly used quantization configurations.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Quantization-Conditioned Attacks, Large Language Model Security, Model Quantization, Secure Quantized Deployment

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_TCagOpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes IUC, a one-shot defense method against quantization-activated attacks on large language models. The method constructs parameter reference intervals across different quantization configurations, uses intersection width to determine correction priorities, and employs union constraints and two-stage optimization to limit parameter modifications while preserving model utility. The authors evaluate IUC across multiple models, quantization configurations, and attack scenarios, including advertisement injection, over-refusal, and jailbreak. Experimental results show that IUC significantly reduces attack success rates while largely preserving model performance and demonstrates generalization to the unseen GPTQ quantization configuration.

Critical strengths
  1. The paper is clearly written, well organized, and easy to follow.
  2. The proposed method has strong practical value, as it requires only a single correction of the full-precision model to support multiple quantization configurations.
  3. The method demonstrates good generalization across different quantization configurations, including the unseen GPTQ setting.
Critical weaknesses

I have the following concerns about this paper:

  1. I have some doubts about the theoretical justification for using intersection width as the correction priority. Section 3.1 states that, once the GGUF residual interval is included, the current weight lies at an endpoint of the intersection. By Proposition 1, its Intersection Exit Cost is therefore 0 regardless of the intersection width. Appendix D.1 also notes that $ a_i^\cap=\rho_i=0 $ when a feasible outward correction direction exists (Eq. 25). Since the default reference set includes GGUF Q4_K_M, this is not merely an isolated edge case. Although Proposition 1 shows that narrower intersections have a smaller worst-case exit-cost bound, it does not directly explain why they should still be prioritized when the actual exit cost is already zero. The gradient analysis in Appendix D.1 explains how the coverage objective encourages parameter updates, but does not fully justify the choice of intersection width as the priority criterion in this setting. I would appreciate further clarification of this design.
  2. I have concerns about the connection between leaving the intersection and changing the actual quantized mapping. For GGUF, the reference interval is constructed from the quantization residual rather than decision boundaries, and the authors acknowledge in Appendix D.1 that whether the quantized value changes still depends on the quantizer's decision boundaries. Even for INT8, NF4, and FP4, the reference intervals are shrunk inward from the full decision intervals ($ I_i^{(q)}\subseteq\bar{I}_i^{(q)} $, Appendix C.3), while the guarantee in Eq. (18) applies only to the full intervals. Although Table 19 reports quantized-mapping change rates for INT8, NF4, and FP4, it does not directly establish how often leaving the reference intersection leads to actual mapping changes, nor does it include GGUF. Since disrupting malicious quantized mappings is central to IUC's motivation, I believe this relationship requires further validation.
  3. The main text lacks an end-to-end workflow figure illustrating the complete IUC correction pipeline.
  4. I believe the ablation studies on the key components are insufficient. Although the authors evaluate different quantizer combinations and parameter selection strategies, they do not independently ablate the Union Penalty, the Hidden-State Constraint, or the Two-stage Optimization. In addition, in the single-quantizer experiment in Appendix F, the authors not only change the number of reference quantizers but also remove the union-specific penalty and constraints. While this modification is understandable given the degenerate intersection–union geometry in the single-quantizer setting, the performance differences may not be entirely attributable to joint quantizer modeling. More controlled ablations would help clarify the contribution of each component.
  5. I have concerns about the limited coverage of the adaptive attack experiments. Although the authors consider a defense-aware attack in Appendix E.6, the experiment covers only the Jailbreak scenario on Qwen2.5-1.5B and includes only INT8, NF4, and FP4 ($ Q_{\mathrm{atk}}={\mathrm{INT8},\mathrm{NF4},\mathrm{FP4}} $). However, one of the main contributions of this paper is a unified defense across quantization configurations, with particular emphasis on complex quantization methods such as GGUF. The current results therefore do not establish whether IUC can withstand targeted adaptive attacks under these configurations. Furthermore, the main text mentions the adaptive attack experiment at the end of the GGUF paragraph in Section 4.2, without explicitly distinguishing the two experimental settings. This could give readers the impression that the adaptive attack evaluation also covers GGUF. I suggest clarifying this distinction and extending the adaptive attack evaluation to more complex quantization configurations.
Questions for authors

See the weakness.

4 comments
AuthorsOpenReview ↗

Response to Reviewer TCag (1 of 2)

We thank the reviewer for the positive evaluation of IUC. We address the reviewer's questions below.

W1. Rationale for Intersection Width-Based Correction Prioritization Under GGUF

When GGUF is included in the reference set, its residual interval has the current weight as an endpoint. Consequently, the Intersection Exit Cost is zero for parameters with a feasible outward correction direction, making it difficult to distinguish correction priorities using this metric. However, Intersection Width is still jointly determined by the GGUF residual reference and the local intervals of other quantizers, preserving geometric scale differences across parameters. Based on Intersection Width, IUC computes sis_i and determines the correction strength through xi=clip(αgsiγg,0,xmax⁡)x_i=\mathrm{clip}(\alpha_g s_i^{\gamma_g},0,x_{\max}), prioritizing parameters with narrower intersections. Under the same parameter budget, Smallest, Random, and Largest achieve average ASRs of 0.00%, 22.13%, and 41.31%, respectively, validating the effectiveness of Intersection Width in guiding parameter corrections under the joint reference set.

W2. Relationship Between Intersection Exit and Actual Quantization Mapping Changes

For INT8/NF4/FP4, IUC constructs shrunken reference intervals within the full quantization decision intervals. Actual changes in quantized weights depend on whether the corrections cross the full decision boundaries. By fixing the extremal weights in each group and constraining the remaining weights, IUC preserves the quantization scales and decision boundaries when the grouping and codebooks remain unchanged. For GGUF, IUC uses reconstruction residuals from actual block-wise quantization to determine local correction directions and scales. Moving along the residual direction increases the error relative to the original quantized reconstruction. To verify actual quantization changes, we re-quantize the corrected model. In the advertisement injection experiment, the actual quantized weight change rates among selected parameters are 32.01%, 9.81%, and 12.17% for INT8, NF4, and FP4, respectively, with ASR reduced to 0.00% in all three configurations. As shown in Table 1, the average actual quantized weight change rate across five GGUF configurations is 17.92%, while the average ASR decreases from 88.17% to 2.89%. These results demonstrate that IUC induces actual quantization changes across different configurations and effectively suppresses quantization-triggered malicious behavior.

Table 1. Actual Quantized Weight Change Rates Across GGUF Configurations.

MetricQ2_KQ3_K_MQ4_K_MQ5_K_MQ6_KAvg.
Change Rate (%)8.9412.6022.0822.3923.6117.92
AuthorsOpenReview ↗

Response to Reviewer TCag (2 of 2)

W4. Ablation Analysis of Key IUC Components and Single-Quantizer Settings

Union Penalty constrains the magnitude of parameter corrections, while Hidden-State Constraint limits changes in hidden representations on clean samples. Stage B further reduces correction magnitudes while maintaining Intersection Exit Coverage. We conduct ablation experiments by separately removing these three components. As shown in Table 2, removing any component reduces MMLU and MC2, with the most pronounced utility degradation observed when Union Penalty is removed. Together, these three components constrain parameter corrections and preserve normal model utility.

Table 2. Ablation of Key IUC Components.

MetricOriginFull IUCw/o Union Penaltyw/o Hidden-Statew/o Stage B
ASR (%) ↓86.780.780.240.960.62
MMLU (%) ↑57.6058.0048.7255.6853.72
MC2 (%) ↑41.9945.1136.4740.5340.28

Both Intersection and Union in IUC are constructed from the same set of reference quantizers. Since the Union interval depends on the reference quantizer set, the two are inherently geometrically related. In the single-quantizer setting, Intersection and Union coincide, and the original correction bound cannot satisfy the Intersection Exit requirement. Therefore, the Union constraint must be adjusted.

In the Qwen2.5-1.5B jailbreak experiment, INT8-only, NF4-only, and FP4-only reduce the average ASR from 86.78% to 22.11%, 21.00%, and 19.22%, respectively. In a separate Qwen2.5-0.5B over-refusal experiment, we further evaluate the contribution of joint multi-quantizer modeling through a Remove-one Quantizer ablation. This experiment preserves the Union construction rule and optimization settings, removing only one reference quantizer at a time. Full IUC achieves an average ASR of 3.67%, which increases to 7.22%, 25.78%, and 25.11% when INT8, NF4, and FP4 are removed, respectively, demonstrating the advantages of joint multi-quantizer modeling.

W3 & W5. IUC Defense Flowchart and Evaluation of Adaptive Attacks Under GGUF

We have uploaded the revised manuscript and replaced the original Figure 2 with a flowchart illustrating the IUC defense process. For the Defense-Aware Adaptive Attack, we extend the attack scenarios to include Advertisement Injection and Over-refusal, and incorporate GGUF into the set of target quantizers for adaptive attacks. As shown in Table 3, across three attack scenarios and four quantization configurations (INT8, NF4, FP4, and GGUF), the average ASR decreases from 52.27% to 6.82%, further demonstrating the effectiveness of IUC against adaptive attacks under different quantization configurations. In addition, we have added a Defense-Aware Adaptive Attacks paragraph in Section 4.2 of the revised manuscript, with the corresponding experimental results reported in Table 4 of the main text.

Table 3. Defense Effectiveness Against Adaptive Attacks.

Attack ScenarioASR Before (%) ↓ASR After IUC (%) ↓MMLU After (%) ↑MC2 After (%) ↑
Advertisement Injection30.854.6757.1247.63
Over-refusal34.537.4256.3847.86
Jailbreak91.438.3756.1244.79
Avg.52.276.8256.5446.76

Thank you for your response. I’ll keep my score as it is.

We sincerely thank the reviewer again for their time and careful review of our work. We greatly appreciate your feedback and will carefully organize the relevant experimental results and analyses and incorporate them into the main text in a subsequent revision.

Reviewer_VDRyOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Intersection–Union Correction (IUC), a pre-quantization defense against quantization-activated attacks on LLMs. IUC constructs local reference intervals across multiple quantizers, prioritizes weights with narrower intersections, and constrains corrections using their unions and clean hidden-state preservation. A single correction produces a full-precision model for subsequent deployment under different quantization configurations. Experiments across several model families and attack methods report substantial attack suppression with largely preserved utility, including transfer to an unseen GPTQ configuration.

Critical strengths
  1. The deployment setting is practically relevant, and the method offers an efficient sanitization workflow. Correcting the full-precision model once allows reuse across multiple quantization configurations.

  2. The empirical evaluation covers diverse attacks and quantization configurations. Beyond substantial ASR reductions under INT8, NF4, and FP4, the paper evaluates several GGUF configurations, transfer to unseen GPTQ, and a defense-aware adaptive attack.

Critical weaknesses
  1. The connection between the geometric surrogate and cross-quantization defense is not sufficiently established. The theory justifies intersection-width prioritization for a worst-case displacement objective over hypothetical weight positions, but does not establish why this criterion is preferable for the observed weights. Moreover, exiting a shrunken or GGUF-based reference intersection need not change a deployed quantizer’s mapping, and the union constraint may leave some mappings unchanged.

  2. The ablations do not adequately establish the safety–utility value of the correction bound and selection rule. The union boundary is used to constrain correction magnitude, but its benefit relative to alternative bounds and the hidden-state constraint is not isolated. See details in questions.

Questions for authors
  1. Given the current weights and their distances to the boundaries are already known. Why is this worst-case criterion preferable to prioritizing the actual exit distance, and how do the two choices compare in defense effectiveness and benign utility?
  2. What motivates using the union boundary as the maximum correction range? Is there analysis or an ablation showing that this choice provides a better ASR–utility trade-off than alternative correction bounds, beyond the protection already provided by the clean hidden-state constraint?
  3. Leaving the intersection need not change a weight’s mapping under the quantizer ultimately selected for deployment; in a nested-interval case, remaining inside the union can prevent that weight from crossing the wider quantizer’s decision boundary. This connection is further weakened by the use of shrunken intervals and the GGUF residual surrogate. What explains why the aggregate intersection-exit objective produces sufficient attack-disrupting changes across the intended deployment configurations, and under what conditions would this mechanism fail?
  4. Table 4 compares selection strategies at a fixed 30% parameter budget, but the deployment objectives are attack suppression and preservation of benign performance. What motivates this particular restriction, and does the advantage of Smallest persist when the strategies are compared at similar benign utility rather than only at the same parameter count? Reporting benign performance alongside ASR would help interpret this comparison.
4 comments
AuthorsOpenReview ↗

Response to Reviewer VDRy (1 of 2)

We thank the reviewer for recognizing the practical value of IUC. We address your questions below.

Q1 & W1. Rationale for Intersection Width-Based Prioritization and Comparison with Actual Exit Distance

Actual Exit Distance measures the minimum displacement required for the current weight to leave the reference intersection, whereas Intersection Width characterizes the size of this region. A smaller width implies a smaller worst-case upper bound on the exit displacement. Accordingly, IUC uses Intersection Width to prioritize parameter corrections, while incorporating the actual exit distance to determine correction directions and exit thresholds. When GGUF is included in the reference set, its residual interval has the current weight as an endpoint. Consequently, for parameters with a feasible outward correction direction, the reference exit distance becomes zero, making it difficult to distinguish correction priorities. In contrast, Intersection Width retains local scale information from different quantizers, providing a discriminative criterion for parameter prioritization. We further compare the two strategies. As shown in Table 1, Smallest Width and Actual Exit Distance achieve similar actual quantized weight change rates (18.00% vs. 18.23%), but their average ASRs are 0.00% and 11.73%, respectively. These results indicate that defense effectiveness depends not only on the proportion of quantized weights changed but also on the parameter selection strategy, validating Intersection Width as an effective criterion for correction prioritization.

Table 1. Intersection Width vs. Actual Exit Distance.

MetricSmallest WidthActual Exit Distance
ASR (%) ↓0.0011.73
MMLU (%) ↑53.4853.27
MC2 (%) ↑46.1245.83
Actual Quantized Weight Change Rate (%)18.0018.23

Q2 & W2. Rationale for the Union Boundary and Comparison with Alternative Correction Bounds

Union is the smallest interval enclosing the reference intervals across different quantization configurations. IUC uses Union to impose a parameter-adaptive correction bound, ∣Δwi∣≤xmax⁡ai∪|\Delta w_i| \leq x_{\max}a_i^\cup, allowing parameters to leave the Intersection while controlling correction magnitudes. To evaluate its advantages, we introduce two alternative bounds: Mean-Distance Boundary and Maximum-Distance Boundary, which use the mean and maximum distances from candidate parameters to quantization decision boundaries along the correction direction, respectively, as a uniform correction scale. As shown in Table 2, all three bounds effectively suppress attacks. However, Union Boundary achieves a lower ASR while maintaining higher MMLU/MC2, demonstrating the advantage of parameter-adaptive correction bounds in preserving model utility.

Table 2. Ablation of Correction Boundaries.

MetricUnion BoundaryMean-Distance BoundaryMaximum-Distance Boundary
ASR (%) ↓0.781.230.82
MMLU (%) ↑58.0053.6741.87
MC2 (%) ↑45.1141.2833.56

Contributions of Union Penalty and Hidden-State Constraint

Union Boundary constrains parameter corrections using local scale information from different quantization configurations, while Hidden-State Constraint limits changes in hidden representations on clean samples. They preserve model utility through constraints in parameter space and representation space, respectively. We conduct ablation experiments by separately removing Union Penalty and Hidden-State Constraint. As shown in Table 3, removing either component reduces MMLU and MC2, with a more pronounced utility degradation when Union Penalty is removed.

Table 3. Ablation of Union Penalty and Hidden-State Constraint.

MetricOriginFull IUCw/o Union Penaltyw/o Hidden-State
ASR (%) ↓86.780.780.240.96
MMLU (%) ↑57.6058.0048.7255.68
MC2 (%) ↑41.9945.1136.4740.53
AuthorsOpenReview ↗

Response to Reviewer VDRy (2 of 2)

Q3 & W1. Intersection Exit, Union Constraints, and Actual Quantization Mapping Changes

Leaving the reference Intersection does not necessarily change the actual quantization mapping; the change depends on whether the corresponding quantizer's full decision boundary is crossed. For INT8/NF4/FP4, IUC preserves quantization scales and decision boundaries by fixing the extremal weights and constraining the remaining weights. For GGUF, block-wise quantization reconstruction residuals provide local correction directions and scales. Union imposes parameter-adaptive correction bounds, allowing parameters to leave the Intersection while preventing excessive deviations. Although nested intervals may prevent some parameters from crossing the wider quantizer's decision boundaries, IUC does not require every parameter to change its mapping under all configurations simultaneously. Instead, joint corrections across multiple parameters can induce quantization changes under different configurations. To verify the actual effects, we re-quantize the corrected model. Under a 30% parameter budget, the actual quantized weight change rates among selected parameters are 32.01%, 9.81%, and 12.17% for INT8, NF4, and FP4, respectively, with ASR reduced to 0.00% in all three configurations. As shown in Table 4, the average actual quantized weight change rate across five GGUF configurations is 17.92%, while the average ASR decreases from 88.17% to 2.89%. These results demonstrate that IUC induces actual quantization changes across different configurations and suppresses quantization-triggered malicious behaviors.

Table 4. Actual Quantized Weight Change Rates Across GGUF Configurations.

MetricQ2_KQ3_K_MQ4_K_MQ5_K_MQ6_KAvg.
Change Rate (%)8.9412.6022.0822.3923.6117.92

Failure Conditions of IUC

When the correction range allowed by Union cannot cross the full decision boundary of a given quantizer, the corresponding parameter's quantized value may remain unchanged. If this limitation affects all corrected parameters under the target configuration and the quantization scale remains unchanged, IUC cannot effectively alter the quantized model, thereby limiting its ability to suppress the original quantization-triggered attack.

Q4. Rationale for the 30% Parameter Budget and Advantages of the Smallest Strategy

We use a fixed 30% candidate parameter budget in the parameter selection ablation to ensure that all strategies have the same upper limit on the number of corrected parameters. As shown in Table 1, Smallest achieves better defense effectiveness while maintaining higher MMLU and MC2 scores. To further compare defense effectiveness at similar levels of model utility, we adjust the correction strengths of Random and Largest to bring their utility close to that of Smallest (MMLU/MC2: 53.48%/46.12%). After adjustment, Random and Largest achieve MMLU/MC2 scores of 53.36%/45.86% and 53.27%/45.47%, respectively. However, their average ASRs remain at 37.43% and 47.67%, respectively, substantially higher than Smallest's 0.00%.

Thank you for the detailed response and the additional experiments. The Union/hidden-state ablations substantially address my concern about the independent contribution of the correction constraint. I will raise my score.

We sincerely thank the reviewer once again for their valuable time and careful review of our work. We greatly appreciate your recognition of our additional experimental results. We will carefully organize the relevant results and analyses and incorporate them into the subsequent revision.

Reviewer_tbj9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates "quantization-triggered attacks" in large language models—a phenomenon where a model functions normally in full-precision mode but exhibits malicious behaviors—such as injecting advertisements, excessive refusals, or jailbreaking—after being deployed in a quantized state. The authors propose the Intersection–Union Correction (IUC) method, which performs a pre-deployment parameter adjustment directly on the full-precision model. This adjustment enables the model to suppress malicious behaviors across various quantization configurations. The method constructs local parameter intervals for different quantizers, identifies a shared reference region via the intersection of these intervals, and prioritizes the adjustment of parameters with smaller intersection ranges. Simultaneously, it employs the union of these intervals to constrain the scope of parameter adjustments, thereby minimizing any loss in the model's capabilities.

Critical strengths
  1. Technical novelty and innovation The paper introduces a novel methodological approach by utilizing the intersection and union of local parameter intervals—derived from various quantization configurations—to guide parameter refinement. Compared to methods that target a single, specific quantization configuration, the proposed strategy of refining the full-precision model to accommodate a diverse range of subsequent quantization schemes holds significant practical value.

  2. Experimental rigor and validation The study encompasses three categories of attacks, a variety of models, and multiple quantization configurations. It further incorporates experiments on parameter prioritization, the removal of quantizers, and performance under unseen quantization schemes, resulting in a comprehensive experimental scope.

  3. Significance of contributions This research addresses a subtle yet critical security issue within the model supply chain. The proposed one-time, pre-deployment refinement method offers substantial potential value for users who must source models from third parties and subsequently perform their own quantization and deployment.

Critical weaknesses
  1. Technical limitations or concerns The primary issue is that geometric properties alone are insufficient to directly guarantee safety. Proposition 1 provides an upper bound on the displacement required for parameters to exit the intersection interval. However, exiting the intersection—under the assumption of fixed quantization boundaries—merely implies that the original mapping for at least one quantizer might change; it does not guarantee that the mappings for all quantizers will change, nor does it directly demonstrate that malicious behavior has been effectively suppressed. Furthermore, the construction of quantization intervals relies on specific assumptions. Techniques such as grouped quantization, scaling factor updates, clipping, and optimization-based quantization can introduce coupling between parameters. For configurations like GGUF, the paper constructs approximate reference intervals based on the reconstruction residual of the quantization process. These intervals are not equivalent to the exact invariant intervals defined under fixed quantization boundaries; therefore, a more rigorous theoretical justification is required when these approaches are used in combination.

  2. Experimental gaps or methodological issues Although the paper claims to support cross-quantization configurations, the experiments focus primarily on a pre-selected set of quantizers, with limited validation on unseen configurations. The model sizes tested are largely concentrated in the 1B to 3B range, leaving the scalability to larger models unverified. The scale, distribution, and representativeness of the "clean" data used can significantly influence the effectiveness of the correction; systematic sensitivity experiments are needed to address this. Utility preservation is evaluated primarily using MMLU and MC2, which fail to adequately cover capabilities such as instruction following, code generation, long-context handling, and complex reasoning. Additionally, the analysis of computational overhead should distinguish between the cost of a single correction process and the total cost incurred when supporting multiple quantization configurations.

  3. Clarity or presentation issues The term "one-shot" should be explicitly defined as a single pre-deployment correction workflow, rather than implying a single optimization update. The paper should also clearly distinguish between exact intervals and approximate reference intervals, and avoid equating a change in quantization mapping directly with a guarantee of safety.

  4. Missing related work or comparisons While the paper discusses methods such as QuantGuard, QVec, and FlipGuard, it should also consider comparisons with other approaches—such as parameter selection driven by quantization error or gradient sensitivity, stochastic weight correction, and general parameter-noise cleaning techniques. Such comparisons would help determine whether the observed performance gains are truly attributable to the geometric design of the intersection/union intervals.

Questions for authors

Why are parameters with a smaller intersection width more suitable candidates for safety-related modifications?

Do the interval properties still hold when the quantization scaling factors and block statistics are recalculated?

What advantages does IUC offer—given a fixed budget for parameter modification—compared to selection methods based on gradients, quantization errors, or random sampling?

Could experiments be extended to include 7B, 13B, or even larger models?

Is it possible to compare the total cost, per-modification latency, and peak VRAM usage across various configurations, while keeping the hardware environment constant?

Beyond MMLU and MC2, how do these modifications affect capabilities related to coding, reasoning, and instruction following?

3 comments
AuthorsOpenReview ↗

Response to Reviewer tbj9 (1 of 3)

We thank the reviewer for the positive evaluation of IUC. We address the reviewer's questions below.

Q1 & W1. Rationale for Intersection Width-Based Correction and Its Connection to Quantization Mappings and Defense Effectiveness

The intersection of the reference intervals across quantizers characterizes a shared local reference region across quantization configurations. A narrower intersection implies a smaller worst-case upper bound on the displacement required to leave this region. Therefore, under a limited correction budget, IUC prioritizes parameters with smaller Intersection Widths to disrupt the preservation of the original quantization mappings as much as possible. To validate this selection criterion, we compare four strategies under the same parameter budget: Smallest Width (narrowest intersections), Actual Exit Distance (shortest distance to the intersection boundary), Random (random selection), and Largest Width (widest intersections). As shown in Table 1, their average ASRs are 0.00%, 11.73%, 22.13%, and 41.31%, respectively. Although Smallest Width and Actual Exit Distance achieve similar actual quantized weight change rates (18.00% vs. 18.23%), the former yields a substantially lower ASR. Furthermore, in the advertisement injection experiment, IUC achieves actual quantized weight change rates of 32.01%, 9.81%, and 12.17% under INT8, NF4, and FP4, respectively, reducing ASR to 0.00% across all three configurations. These results demonstrate that IUC effectively changes the quantized weights across different configurations and suppresses quantization-triggered malicious behavior.

Table 1. Parameter Selection Ablation on Qwen2.5-1.5B (Advertisement Injection; INT8/NF4/FP4 Avg.).

MetricOriginSmallest WidthActual Exit DistanceRandomLargest Width
ASR (%) ↓68.330.0011.7322.1341.31
MMLU (%) ↑53.6253.4853.2752.6352.28
MC2 (%) ↑45.6546.1245.8343.7243.89
Actual Quantized Weight Change Rate (%)0.0018.0018.2311.397.97

Q2. Validity of Reference Intervals Under Quantization Scale Recalculation and Theoretical Basis for GGUF Residual References

For INT8/NF4/FP4, IUC fixes the original extremal weights in each reference group and constrains the absolute values of the remaining weights not to exceed the original maximum. Thus, when the grouping and codebook remain unchanged, recalculating absmax preserves the original scale, and the full quantization decision boundaries remain valid. For GGUF, IUC uses reconstruction residuals from actual block-wise quantization to determine local correction directions and scales. Moving along the residual direction increases the reconstruction error relative to the original quantized reconstruction, providing a geometric basis for GGUF parameter correction. IUC integrates these two types of local geometric information into the Intersection–Union construction, enabling joint correction across quantization configurations. To evaluate the effects after recalculating block-wise quantization parameters, we quantize both the original and corrected models using GGUF. As shown in Table 2, in the Qwen2.5-1.5B advertisement injection experiment, the average actual quantized weight change rate across five GGUF configurations is 17.92%, while the average ASR decreases from 88.17% to 2.89%. Removing the GGUF residual references increases ASR to 27.29%, validating their effectiveness under actual GGUF quantization.

Table 2. Actual Quantized Weight Change Rates Across GGUF Configurations.

MetricQ2_KQ3_K_MQ4_K_MQ5_K_MQ6_KAvg.
Change Rate (%)8.9412.6022.0822.3923.6117.92
AuthorsOpenReview ↗

Response to Reviewer tbj9 (2 of 3)

Q3 & W4. Advantages of IUC's Geometric Design and Comparison with Existing Defense Methods

IUC uses Intersection to guide parameter selection and Union to constrain correction magnitudes, suppressing attacks while preserving model utility. A single correction can also support multiple quantization configurations. We compare IUC with gradient-sensitivity-guided Q-resafe, quantization-error-based FlipGuard, error-guided rounding method QuantGuard, and quantization-residual-based QVec. On Qwen2.5-1.5B, the average ASRs of IUC and Q-resafe under INT8/NF4/FP4 are 1.86% and 13.12%, respectively. Across GGUF configurations, the average ASRs of IUC, QuantGuard, QVec, and FlipGuard are 3.30%, 35.53%, 14.17%, and 23.48%, respectively. We additionally evaluate a Gaussian Noise baseline. As shown in Table 3, in the INT8 jailbreak setting on Llama3.2-3B, noise levels of 10−410^{-4}, 10−310^{-3}, and 10−210^{-2} yield ASRs of 71.05%, 35.12%, and 2.72%, respectively. Although stronger noise suppresses the attack, MMLU/MC2 drops to 26.35%/16.18%. In contrast, IUC achieves an ASR of 2.67% while maintaining MMLU/MC2 at 61.09%/45.16%, demonstrating its advantage in balancing attack suppression and model utility preservation.

Table 3. Defense Effectiveness and Model Utility: Gaussian Noise vs. IUC.

MetricOriginGaussian (10−410^{-4})Gaussian (10−310^{-3})Gaussian (10−210^{-2})IUC
ASR (%) ↓81.6771.0535.122.722.67
MMLU (%) ↑61.2758.8251.4726.3561.09
MC2 (%) ↑42.5940.0337.0516.1845.16

Q4 & W2. Defense Generalization to Larger Models and Unseen Quantization Configurations

Our existing experiments cover Qwen and Llama models ranging from 0.5B to 7B, as well as INT8, NF4, FP4, multiple GGUF configurations, and unseen GPTQ 4-bit quantization. As shown in Table 4, we further evaluate IUC on Qwen3-14B. The average ASR increases from 1.33% in FP32 to 86.33% after quantization, while IUC reduces it to 2.67% with largely preserved model utility. Furthermore, as shown in Table 5, under the unseen AWQ 4-bit configuration, IUC reduces the jailbreak ASR from 91.67% to 3.67%, further demonstrating its generalization across quantization configurations.

Table 4. Defense Performance on Qwen3-14B (INT8/NF4/FP4 Avg.).

MetricPoisoned FP32Quantized (Origin Avg.)Quantized (IUC Avg.)
MMLU (%) ↑78.6776.8176.68
MC2 (%) ↑59.4356.6256.43
ASR (%) ↓1.3386.332.67

Table 5. Defense Performance Under Unseen AWQ 4-bit Quantization.

MetricFP32 (Origin)AWQ 4-bit (Origin)AWQ 4-bit (IUC)
ASR (%) ↓1.3391.673.67
MMLU (%) ↑54.8353.6953.97
MC2 (%) ↑44.3743.5643.23

Q5 & W3. Definition of the One-Shot Defense Workflow and Computational Cost of IUC

One-shot refers to performing a single complete IUC correction on the full-precision model. The corrected model can then be directly used across multiple quantization configurations without repeating the defense. On the same hardware, IUC takes 155.4 seconds for a single defense on a 1.5B model, with a peak VRAM usage of 13.8 GB. In comparison, QuantGuard, QVec, and Q-resafe require 454.4, 305.1, and 499.0 seconds per defense, respectively, with peak VRAM usage of 46.1 GB for QuantGuard and 12.3 GB for QVec. For KK deployment quantization configurations, IUC still requires only one defense run, whereas methods requiring configuration-specific corrections incur a total defense time of approximately KK times their respective single-run costs.

AuthorsOpenReview ↗

Response to Reviewer tbj9 (3 of 3)

Q6. Comprehensive Evaluation of Model Utility

Our existing experiments evaluate model knowledge, reasoning, and truthfulness using MMLU, ARC-Challenge, HellaSwag, GSM8K, and TruthfulQA-MC2. We further include HumanEval, BBH, IFEval, and LongBench to assess code generation, complex reasoning, instruction following, and long-context capabilities, respectively. As shown in Table 6, IUC substantially reduces ASR while preserving performance across these additional capabilities.

Table 6. Model Utility and Defense Performance Across Additional Benchmarks.

Model StatusHumanEval (%) ↑BBH (%) ↑IFEval (%) ↑LongBench ↑ASR (%) ↓
FP32 (Origin)49.2023.7075.8042.377.00
Quantized (Origin)47.3021.9071.2037.9279.89
Quantized (IUC)46.9022.3072.1037.631.67

W2. Sensitivity Analysis of Calibration Data Size and Distribution

Calibration data are used to constrain changes in hidden representations on clean samples before and after correction. By default, IUC randomly selects 128 samples from Alpaca-GPT4. In the calibration size ablation, increasing the number of samples from 16 to 128 reduces the average ASR from 1.27% to 0.00%, while MMLU/MC2 improves from 54.50%/44.56% to 58.56%/50.63%. Further increasing the sample size to 256 yields only an additional 0.11 percentage-point improvement in both utility metrics. We also evaluate sensitivity to calibration data distribution using 128 samples from each of Alpaca-GPT4, OpenOrca, and Dolly-15k. As shown in Table 7, the average ASR across the three datasets is 0.07%, with average MMLU/MC2 of 58.50%/50.64%, indicating that IUC maintains stable defense effectiveness and model utility across different calibration data distributions.

Table 7. Sensitivity to Calibration Data Distribution.

Calibration DataASR (%) ↓MMLU (%) ↑MC2 (%) ↑
Alpaca-GPT40.0058.5650.63
OpenOrca0.0758.3250.48
Dolly-15k0.1358.6350.81

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Author Responses

We sincerely thank the Area Chair (AC) and all reviewers for their careful evaluation of our manuscript and constructive feedback. We would like to summarize our main responses and revisions during the rebuttal process.

This paper proposes Intersection–Union Correction (IUC), a unified defense method against quantization attacks on large language models (LLMs). IUC requires only a single parameter correction on the full-precision model to support secure deployment across multiple subsequent quantization configurations, while largely preserving normal model utility.

To address the key concerns raised by the reviewers and the AC, we have further strengthened our theoretical analysis, experimental validation, and evaluation protocol, as summarized below:

  1. Theoretical Motivation of Intersection Width and Its Relationship to Actual Quantization Mappings. We further clarified the geometric rationale for using Intersection Width to guide parameter correction, explained the cases where the actual exit cost is zero under GGUF, and validated its effectiveness through comparisons with Actual Exit Distance, random selection, and other strategies. We also measured actual quantized weight changes under INT8, NF4, FP4, and multiple GGUF configurations to verify the impact of parameter correction on quantization mappings and attack suppression.

  2. Independent Contributions of Union Boundary and Key Components. We conducted additional comparisons of different correction boundaries and component ablations of Union Penalty, Hidden-State Constraint, and Stage B, further demonstrating their roles in balancing attack suppression and model utility preservation.

  3. Generalization, Robustness, and Model Utility. We extended our evaluation to the larger Qwen3-14B model, the unseen AWQ quantization configuration, the Gaussian Noise baseline, additional model utility benchmarks, and calibration-data sensitivity analyses. We also expanded defense-aware adaptive attack evaluations to three attack scenarios and multiple quantization configurations, providing further evidence of IUC's effectiveness and practical applicability.

  4. Evaluation Protocol and Consistency of Experimental Results. In response to the AC's feedback, we further clarified the evaluation sample sizes, random seeds, ASR aggregation procedures, and comparability of model performance before and after defense. We also systematically reviewed the experimental tables and statistical results, corrected data compilation issues and minor statistical discrepancies caused by a small number of invalid LLM judge responses, and improved the accuracy and reproducibility of our evaluation.

We appreciate the reviewers' positive assessments of IUC's novelty, practical value, and additional experimental evidence. In particular, reviewer VDRy noted that the new ablation experiments largely addressed their concerns regarding the independent contributions of the correction constraints. We greatly value this feedback, which has helped us further strengthen the manuscript.

Finally, we sincerely thank the AC and all reviewers once again for their valuable time and constructive suggestions. We will ensure that the relevant code is fully open-sourced, and we hope that our work will provide a valuable reference for the secure quantized deployment of large language models.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.