acceptodds
Under review as a conference paper at ICLR 2027

The Signal Survives, the Threshold Slips: Small LLM Agents After Quantization and Fine-Tuning

Abstract

Small language model agents are usually quantized before they are deployed, and fine-tuned so they can call tools. We ran both changes together as a full 2×2 design on six instruction models from 0.5B to 3.8B, and scored what the agents did rather than what they said, using the executable actions in AgentHarm. Across models the interaction term is all over the place, from +0.25 to −0.42, which is easy to read as no effect at all. We show it breaks into three parts once you measure the model's own decision log-odds, meaning the log-odds of opening a tool call rather than refusing. One part is the bounded scale, where an additive shift looks super-additive near the floor and sub-additive near the ceiling. One part is associated with capability loss, which is where the combined pipeline sometimes breaks the agent's ability to emit a well-formed call, and a model that can no longer act gets scored as a safe one, the capability-grounding argument made before us by Vishnubhotla et al. (2026), which we quantify here rather than claim first. What is left is a latent-scale interaction in the decision axis, and on three of the six models it is the largest of the three parts. On Qwen2.5-1.5B and Gemma-2-2B, a one-feature logistic fitted on three conditions predicts the harmful action rate of held-out conditions of the same model without refitting, whether the change came from quantization, an adapter, a learning rate, a seed, a training checkpoint, or a vector added to the residual stream. On the three models where we measured it, the refusal direction barely moves through any of this. It keeps pointing the same way and keeps telling harmful requests from benign ones, while the decision variable shifts about three times further and pushes most harmful requests past the point where the model prefers to act. The safety signal is not being erased. What moves is the threshold.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, AI safety, quantization, parameter-efficient fine-tuning, mechanistic interpretability, activation steering, safety evaluation, tool use

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rou8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates how quantization and tool-use fine-tuning jointly affect the safety of small language model agents. Using a 2×2 experimental design across six models, it finds that their interaction varies substantially across models and that reduced harmful tool invocation can partly reflect degraded tool-use capability rather than improved safety. The results and analysis show the model still recognize requests as harmful, but quantification and tool fine-tuning have changed ites tendencu to take action.

Critical strengths

Interesting empirical phenomenon. The paper investigates how quantization and tool-use fine-tuning jointly affect small-model agent.

Critical weaknesses
  1. Presentation and consistency issues. The paper contains formatting problems and unclear wording.

  2. Insufficient analysis of the fine-tuning data. The paper does not clearly describe whether the training data are screened for harmful requests or include safety-oriented refusal examples. Tool-use fine-tuning without explicit safety supervision may increase general compliance, including compliance with harmful requests. it is a very natural process for the model to answer these questions after training. Why do articles need to analyze fine-tuning and quantification together.

Questions for authors
  1. Evaluation on additional datasets. Could the authors validate the central findings on additional benchmarks where harmful tool use can be distinguished from ordinary tool invocation? Most supporting evidence comes from AgentHarm, while the other benchmarks provide limited or inconsistent support.

  2. Generalization to larger models. Do the findings persist on larger models, such as 7B–8B models or above? Since the current models range from 0.5B to 3.8B parameters, some observed effects may reflect the limited capabilities and greater fragility of very small models under quantization.

Reviewer_vWNyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper tests six small models in four conditions. C1 is the base model. C2 adds NF4 quantization. C3 adds LoRA fine-tuning for tool use. C4 combines both. The harmful rate counts valid tool calls on harmful AgentHarm prompts. The effect of C4 differs from the sum of the single effects. The sign of this extra effect changes across models. The authors define a log-odds score d. d compares a tool-call opener with a refusal opener. d is above zero when the tool-call opener is more likely. The authors split the extra effect into three terms: scale, d and a leftover term. On three models, an internal refusal direction barely moves. d moves toward tool calls.

Critical strengths

Every harmful rate has a matching benign rate. This pair separates refusal from lost capability. For example, Qwen2.5-0.5B has the lowest C4 harmful rate (0.114). Its C4 benign rate is also low (0.125).

Each model has a curve that maps d to a harmful rate. The authors fit this curve on C1 to C3 only. The curve predicts 12, 19 and 26 left-out conditions. The average error is 0.018 to 0.076. On two models, the curve also predicts runs that shift hidden activations.

Critical weaknesses

The paper overstates the evidence for its title claim. The abstract says d moves about three times further than the refusal direction. This does not hold for Gemma. Gemma's d moves by -0.452 (Table 5). Its refusal direction moves by -0.338. Section 1 says 56% to 66% of harmful prompts cross the threshold where d is zero. Table 5 shows a narrower meaning. These numbers are only the C4 shares with d above zero on three models. On Gemma, 38% of harmful prompts already have d above zero at C1.

The d term depends on the choice of openers. Appendix A states that the authors did not test other opener sets. One fix to Llama's opener set moved Llama's d term from +0.226 to +0.168 (Table 2). The same fix moved Llama's scale term from -0.100 to +0.000. The paper treats d above zero as the tool-call side for all six models. Granite contradicts this. At C1, Granite's harmful rate is 0.716 (Table 1). Yet only 0.01 of its harmful prompts have d above zero (Table 5). The three-term split in Table 1 is the core result. The missing test with other opener sets is the main reason for my soundness score.

Questions for authors

The authors should recompute d with two or three other opener sets. One set should use only C1 to C3 outputs. The current sets also use C4 outputs. For each set, the paper should report the d term in Table 1. The paper should also report the shares with d above zero in Table 5.

The paper should report one more share. This share counts harmful prompts with d below zero at C1 and above zero at C4.

Reviewer_A5i2OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies quantization and tool-use fine-tuning jointly in six small instruction models. It measures valid tool calls on harmful AgentHarm prompts alongside matched benign tasks. A score based on tool-call and refusal openings supports a decomposition into bounded-scale, latent-score, and residual terms. The authors find that an estimated refusal direction remains similar while the decision score shifts, and report useful predictions across conditions within models. The results are interesting, but their safety interpretation depends heavily on the validity of these measurements.

Critical strengths
  • The factorial design addresses a relevant deployment question. The interaction differs in sign across models (Table 1), showing why the two transformations should be studied together.
  • Matched benign tasks help distinguish refusal from inability to act. Qwen2.5-0.5B's combined-condition benign call rate of 0.125 illustrates why a low harmful-prompt action rate can be misleading.
  • Tests across training changes, steering, and a prompt intervention broaden the evidence. The explicit failures of cross-benchmark transfer and calibrated control also help define the account's limits. Quantization-format and matched-noise controls further constrain the possible explanations.
Critical weaknesses
  1. The decision score uses truncated opener prefixes pooled across C1–C4 (Appendix A), without establishing exhaustive, disjoint action/refusal events. Its zero need not be the behavioral boundary. Correcting the opener set changed the Llama findings, yet alternative definitions were not tested. Using C4 outputs to construct the vocabulary also limits the independence of the held-out prediction.

  2. The decomposition does not establish three separate mechanisms. Equation 3's residual can contain measurement and readout errors as well as capability loss. Section 6 acknowledges this, but aggregate benign-rate declines do not identify how much of each residual reflects capability. The partition is informative; its mechanistic interpretation remains uncertain.

  3. A valid tool call need not advance a harmful objective. Realized harm is at most 4.5% in every cell under the official AgentHarm judge (Section 3), and calls may be preparatory. Although acknowledged, this gap limits the safety interpretation without evidence linking the counted calls to harmful progress.

Questions for authors
  1. Do the main results survive opener sets built from C1–C3 only, alternative prefix definitions, and evaluation on separate prompt families? In particular, are the threshold-crossing and decomposition results stable?
  2. Does the capability interpretation hold when examining matched task pairs whose benign calls succeed across conditions? This would complement the aggregate benign-rate comparisons.
  3. What fraction of counted calls advances the harmful objective, and does the interaction retain its qualitative pattern under that narrower outcome, even if completed harm remains rare?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.