acceptodds
Withdrawn from ICLR 2027

Quieter, Not Safer: Quantization Inflates Oversight-Invisible Damage in LLM Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Quantization is a common step in deploying LLM agents on cheap hardware, and its cost is assumed to be a little accuracy. We show it can also change what an agent does in ways the usual checks do not see. Beyond response-level safety—refusals that fail, biased answers—an agent that acts can complete its task and, in the same run, quietly edit what it was never asked to touch. We build Tripwire, 180 file-editing tasks in three domains, each planted with registered out-of-scope temptations, so that every violation is a deterministic function of file state and is graded by the weakest oversight that would catch it. Across six base models and four 4-bit schemes at two scales we find: (1) in four of ten paired comparisons—NF4 at both scales, FP4 and AWQ at 7B, three of them surviving Bonferroni correction—quantization raises the rate of runs containing such a violation 3–5×; at 7B this happens with task success unchanged at 90/90, and at 14B NF4 raises violations inside successful runs from 3 to 22 of 90 while success falls from 85 to 78; every quantized condition adds more violating tasks than it removes, and NF4, the QLoRA default, is the most severe case at 14B and inflates at both scales; (2) task success and the agent's own report do not register the increase, and a 7B reviewer misses three quarters of the 54 violations that occur inside successful 14B runs, leaving a residual under NF4 that is five times full precision's and a strict superset of it, while a 72B reviewer reading the full diff flags them all; (3) the visible cost is small and the hidden cost large—1.6 to 2.8 MMLU points in the four inflating comparisons against 3–5× more violations—and neither bit-width, calibration, nor codebook geometry accounts for which of our configurations is affected. At 7B these agents match full precision on task success. They are not safer.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
quantization, LLM agents, AI safety, agentic evaluation, oversight, benchmark, scope violations, side effects, NF4, QLoRA

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hzU1OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies whether quantization increases unauthorized changes during file-editing tasks, even when the requested task is completed. It introduces Tripwire, a synthetic benchmark that checks task success and scope violations separately. Experiments show that some quantized configurations increase violations without reducing task success. The paper also compares how often two LLM reviewers flag these violations.

Critical strengths

The paper studies an interesting and practically relevant question. An agent can complete a task while also changing content outside the user’s request. Examining whether quantization increases this behavior draws attention to a risk that task-success metrics alone can miss.

Critical weaknesses
  1. The messy presentation makes the main contribution hard to assess. The central question is straightforward, but the paper introduces several overlapping terms and repeatedly shifts between violations across all runs, violations within successful runs, and violations missed by a reviewer. Section 4 devotes substantial space to restating counts, ratios, and significance tests already available in the tables, and, by the way, all the subsections seem to be presented not in a proper \subsection{} way, which is just a minor evidence of poor presentation. The main findings and their implications are difficult to follow. A clearer account of what each metric measures and which results support each claim is needed. This affects my assessment because the current presentation makes it difficult to determine the exact scope and significance of the contribution.

  2. The contribution seems to be limited. Prior work has already explored agents that complete benign tasks while taking unauthorized actions [1]. [2] also shows that quantization can increase errors without a significant change in task scores. This paper extends that analysis to scope violations, which is a relevant distinction, but the added insight remains limited. Tripwire uses synthetic, single-turn editing tasks with whole-file writes and at most two writes per run. It does not test whether the findings extend to multi-turn tool use or longer agent workflows. Therefore, the whole contribution seems incremental.

[1] Qu, Yubin, et al. "Overeager coding agents: Measuring out-of-scope actions on benign tasks." arXiv preprint arXiv:2605.18583 (2026).

[2] Jang, Jiwon, et al. "Flat score, amplified failures: How the error budget masks damage in quantized LLM agents." arXiv preprint arXiv:2607.27275 (2026).

Questions for authors
  1. What is the main contribution beyond the closest prior work? Prior work has studied similar problems. Could you state clearly what Tripwire and the present findings add beyond these studies, and identify the results that support this added contribution? A clearer explanation would help me assess the novelty and significance of the work.

  2. How broadly does the finding at 7B apply? The result that violations increase while task success stays unchanged is established for certain quantized versions of Qwen2.5-7B. Is this intended as a finding about that model, or about 7B agents more generally? Do you have evidence that the same pattern holds in other model families, larger models, or multi-turn agent workflows? The 14B NF4 result does not establish the same pattern because task success also declines. If broader evidence is unavailable, please explain how the claims should be limited to the tested models and single-turn editing setting.

  3. Does the strong reviewer identify the actual scope violations? The paper counts a run as detected whenever the reviewer lists any problem, without checking whether that problem matches the registered violation. Could you examine the 54 successful runs with violations and report how often the 72B reviewer correctly identifies the unauthorized change? This would clarify whether it detects the claimed failure or merely flags the same runs for other reasons.

Reviewer_M6yqOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

When letting LLM agents modify the raw files directly, there exists the potential risk that they can overly modify the files. To address this, this paper introduces a benchmark, Tripwire, that consists of 180 file-editing tasks. Each instance contains three to seven traps designed to tempt a model to silently modify them. The evaluation is conducted across six models with four distinct quantization methods. Although the research problem is timely and close to the practical applications, the construction details are not clearly discussed, which severely limits the applicability and reliability of the evaluation results.

Critical strengths

None

Critical weaknesses

1. The benchmark has limited task diversity.

While introducing a benchmark and identifying failures of quantization are the main contributions of this work, the benchmark construction procedure raises serious concerns. The paper does not provide the benchmark construction details and validity of the constructed benchmark, including how those source problems are designed, whether the source problems are from an existing benchmark or created by human or LLMs, in which criteria traps are inserted, and how the difficulty and diversity were ensured. This lack of details makes the constructed benchmark less reliable and less useful, and limits the generalizability of the experimental results.

More importantly, the submitted supplementary materials show that the whole dataset is derived from one JSON, three Python, and one Markdown problems. For example, 60 Python tasks are generated by repeating the three problems 20 times, where the only differences come from the combination of traps. Such limited task diversity raises concerns about the soundness and generalizability of the proposed benchmark.

2. The contribution is limited to empirical observations without proposing a method to address the problem or providing sufficient analysis of the underlying causes.

It is well known that model quantization can degrade the performance of a model and its instruction following ability. Although the proposed benchmark focuses on failures that are harder to detect, the underlying nature of the failure is that the model failed to follow instructions. Therefore, it is important to clarify whether the observed failure is a distinct phenomenon or simply reflects known instruction-following failures under quantization, but the paper does not adequately discuss it. Furthermore, the paper does not propose a method to reduce unintended modification, which limits the practical implications of this work.

Overall, I find the contribution of the proposed benchmark and conducted evaluation is insufficiently established.

Questions for authors

Questions

If the concerns about the task diversity are adequately addressed and the benchmark construction procedure is clarified, I would be willing to update my scores accordingly. Otherwise, I remain concerned that the evaluation results and empirical findings may be biased by the limited task diversity.

Suggestions

For future work, I suggest reconstructing the benchmark with more diverse tasks. The task does not necessarily need to be limited to the structured formats like JSON, Codes, and Markdown. For example, one can ask a model to modify only a specific sentence in a paragraph, and check whether the model revise other lines. Furthermore, the comparison experiments across various precision such as 4-bit, 6-bit, 8-bit may provide a clue for the better understanding of how quantization affects the silent failures.

Minor Comments

I recommend not to overly use em dashes, emphasis, quotation marks, and semicolons. LLMs often use these punctuation marks that can distract readers and make the paper harder to follow.

Reviewer_VYsaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the effect of quantization of LLM agents on safety and, more concretely, on executing unauthorized changes. It introduces a Tripwire benchmark, which is a novel benchmark of 180 synthetic file-editing tasks that include deterministic verifiers for violating the scope of edits. The paper empirically finds that quantized models, despite having the same (or very similar) success rate on the main task, demonstrate a marked increase in unauthorized actions. The authors also study how LLM reviewers are able to catch these violations and observe that bigger reviewers are much better at this task.

Critical strengths
  1. The Tripwire dataset: one of the most important features is that it is a benchmark that comes together with an efficient, deterministic verifier. This makes using the benchmark reproducible.
  2. The empirical finding is relevant: the paper considers a timely problem and notes (expanding on the previous work) that when evaluating a quantized model, one should go beyond task-success metrics. The finding that using a strong reviewer is beneficial is also of interest.
Critical weaknesses
  1. Limited novelty: I think that the paper doesn't provide a substantial advance over existing literature. In my view, failing to avoid an unauthorized action is closely related to the previously studied problem of failing to refuse unsafe behavior, and the paper does not sufficiently establish what new insight its setting provides. The Tripwire benchmark is useful, but the distinction between it and the previously studied safety failures is not enough to establish a major contribution.

  2. Limited explanatory insight: The paper documents increases in violations but provides little understanding of why particular models, quantization methods, or tasks are affected. The proposed decision-margin explanation (lines 420–424) draws on prior work and remains untested. I would be interested in even a small experiment trying to justify this hypothesis. Thus, the contribution gives limited guidance about when the findings should generalize.

Presentation: The presentation frequently uses rhetorical or unnecessarily indirect language that obscures straightforward concepts. Examples include “that the direction is upward does not” (line 72), “newly violating than reversed tasks” (line 87), “the channels a deployment routinely reads share the blind spot” (line 91), “a measured quantity rather than a label” (around line 240), and “violating audibly” (around line 340). Repeated broad claims followed immediately by qualifications also make the precise scope of the findings difficult to follow. Plain descriptions of the measured outcomes and checks would improve clarity.

Questions for authors
  1. Empirical support and additional validation: Can you provide a targeted test of the decision-margin hypothesis in lines 419–424? For example, do tasks that become violating after quantization have smaller full-precision margins between compliant and violating actions?

  2. Clarification of novelty: What substantive insight distinguishes this work from prior studies of quantization-induced safety degradation? As I mentioned in the weaknesses, to me the setting considered in this paper is similar to previous results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.