acceptodds
Under review as a conference paper at ICLR 2027

Quantize Once, Adapt in Place: Zeroth-Order Optimization with Detectable Signals

Abstract

Quantization reduces the memory required to store and deploy large language models, while zeroth-order (ZO) optimization avoids storing activations for backpropagation. Combining these two advantages still leaves a key question: which parameters should be optimized? We therefore study a simple constraint: keep the packed weights and quantization metadata fixed, and fine-tune only the floating-point model tensors retained in the checkpoint. We collect these tensors into the Quantization-Preserved Subspace (QPS) and propose Quantization-Preserved Zeroth-order Optimization (QPZO), which applies two-point ZO updates only in QPS. This enables adaptation without adding trainable modules or dequantizing the low-bit backbone for optimization. We support QPZO with theoretical analysis and empirical evidence. Specifically, (1) we show that an updateable output head retained in QPS yields a nonzero projected gradient whenever its aggregate gradient is nonzero, establishing the presence of task-relevant signal; and (2) because ZO accesses this signal only through finite forward-loss differences, we introduce Signal-to-Error Ratio (SER) to compare directional task-signal energy with measurement-error energy. Under the stated error bound, we prove that SER above one is the sharp threshold for guaranteeing positive expected alignment, characterizing when finite forward queries robustly preserve a useful task direction. Our theoretical analysis and extensive experiments demonstrate that QPZO offers a simple approach that substantially reduces GPU memory usage while maintaining highly competitive performance and stability.

Primary area:
optimization
Keywords:
Quantized Large Language Models, Zeroth-Order Optimization, Low-Bit Quantization, Memory-Efficient Fine-Tuning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_NNVnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to optimized a quantized LLM with its quantization metadata and low-bit weight, without dequant it into high-precision form. The paper noticed that a standard zero-order optimization on QPS is enough for low-bit quantization. Experiments on 4-bit checkpoints and related ablation show that the proposed method do not damage the frozen low-bit space while improve model's performance.

Critical strengths
  1. Performance: The experiments and ablations supports that a quantized checkpoint can be adapted under a fixed forward budget without adding modules or maintaining a fully dequantized trainable copy.
  2. Theoretical insights: A nonzero aggregate gradient of an updateable untied output head implies a nonzero projected gradient on QPS and a strict local descent direction, which is novel to me.
Critical weaknesses
  1. Optimizing an already quantized model is not shown to be practical. Quantization is a way to store and serve a model. I cannot come up with some scenarios that a user or system that must adapt the model only after it has been quantized.
Questions for authors
  1. What learning rate are chosen for mentioned experiments? I'm quite curious about what if a learning rate cannot be represented with a given bit. Will that introduce further noise on gradients?
3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

W1: Practical scenarios for adapting an already quantized model

Thank you for asking when adapting a model after quantization would be useful in practice.

Adapting an already quantized model is useful when a device needs quantization to fit the model in memory and the data needed for adaptation arrive only after deployment. At that point, the device already holds a compressed checkpoint, and learning from the new data locally must fit the same memory budget.

Consider a phone that ships with a quantized assistant. After release, a user repeatedly corrects which reminders the assistant assigns to their work and personal lists. Those corrections now provide training examples of that user’s preferences. Starting from the full-precision model would require fine-tuning it on the new examples, quantizing the result, and delivering a new checkpoint to the phone. If the corrections must stay on the phone, training must run locally, where only the quantized checkpoint is available and memory is too limited to fine-tune a full-precision copy. Adapting the installed checkpoint instead lets the assistant learn from the corrections while reusing the low-bit weights already on the device.

Section 2.1 formalizes this setting, and Appendix C.1 describes how our experiments start directly from existing quantized checkpoints. QPZO updates only their retained floating-point tensors, keeping the packed weights and quantization metadata fixed. On Qwen-GPTQ/SST-2, this raises accuracy from 90.94% to 94.95% (Table 1) with 9.32 GiB of peak training GPU memory, compared with 17.75 GiB for MeZO on the fully dequantized model (Figure 2). Figure 2 shows lower peak training memory for QPZO across all six model–quantizer settings.

Q1: Learning rates and the backbone bit width

Thank you for raising this question. The specific source of gradient noise you describe is absent in QPZO because the learning rate η\eta is a floating-point scalar and is never quantized to the backbone’s bit width.

Section 2.1 restricts updates to the floating-point tensors retained in QPS and keeps the packed low-bit weights and quantization metadata fixed. In §2.2, Eq. (4) computes the directional estimate D^t=(ℓt+−ℓt−)/(2ε)\widehat D_t=(\ell_t^+-\ell_t^-)/(2\varepsilon) from two forward losses. The learning rate first appears in Eq. (5), after this estimate has been computed. As shown in Algorithm 1, η\eta scales the update applied directly to the floating-point QPS tensors.

Thus, in all QPZO experiments, including those reported in Table 1, the learning rate remains in floating point regardless of the bit width used to store the backbone weights. More generally, whether η\eta is representable on the backbone’s weight grid has no bearing on the zeroth-order gradient estimate. The table lists the learning rates and perturbation scales for this model and task.

QuantizerBit widthsLearning ratePerturbation scale
GPTQ4, 3, 21e-61e-3
VPTQ3, 42e-61e-3

Thank you for the detailed response. The deployment example and explanation on learning rate resolved my questions. I now think that the paper is a complete research work.

I am keeping my rating at 2, but I nevertheless consider acceptance a reasonable outcome. This year’s four-point scale is too coarse to convey that nuance, especially that it is said 2/3 of submissions are rated 2. I would appreciate it if the AC would take this clarification into account when weighing my review.

AuthorsOpenReview ↗

Official Comment by Authors

Thank you for your thoughtful follow-up. We are glad that our discussion of the practical value and our explanation of learning rate resolved all of your concerns. We especially appreciate your clarifying that you now view the paper as a complete piece of research and consider acceptance a reasonable outcome. Thank you for asking the AC to consider this perspective alongside your numerical rating.

Reviewer_ofacOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors present QPZO, a memory-efficient fine-tuning method for quantized models. Specifically, the authors argue that the quantization-preserved subspace, which consists of unquantized tensors retained in the checkpoint, is a suitable choice of optimization space. They also propose the signal-to-error ratio as a theoretical contribution to capture the alignment between the ZO estimate and the true gradient. Extensive experiments show that QPZO excels in multiple NLP tasks while maintaining a promising memory efficiency.

Critical strengths
  1. The writing is good, and the presentation is clear.
  2. The proposed method is consistently effective across model families, quantization methods, and downstream tasks, and does not significantly increase the memory overhead.
Critical weaknesses
  1. Missing comparison with first-order methods. Although QPZO targets zeroth-order fine-tuning, a detailed comparison with first-order counterparts would help clarify the performance gap and the efficiency trade-off.
  2. Insufficient discussion on the impact of the quantization backbone. The experiments in the manuscript consider only the 4-bit scalar-based quantization methods, such as GPTQ and AWQ. Results for lower-bit or codebook-based methods are missing, so the discussion of QPZO's generality remains insufficient.
Questions for authors

Based on the weaknesses, I hope the authors can respond to the following questions to address my primary concerns.

  1. Could you provide a comparison of QPZO with its first-order counterparts, i.e., optimizing high-precision tensors with AdamW or SGD while keeping quantized weights untouched, and report statistics on training time, memory footprint, and accuracy to highlight the trade-off in performance and efficiency?
  2. Could you provide experimental results with lower-bit and codebook-based quantization methods to further validate QPZO's generality?
3 comments
AuthorsOpenReview ↗

Rebuttal by Authors

W1 / Q1: Comparison with first-order optimization in the same parameter space

Thank you for suggesting a first-order comparison on the same QPS parameters. We added SGD and AdamW baselines on the Qwen2.5-7B-Instruct GPTQ Int4 checkpoint for SST-2. All three methods update exactly the same 1,090,199,040 floating-point parameters and keep the low-bit backbone and quantization metadata fixed. SGD and AdamW compute gradients by backpropagation, while QPZO computes each update from two forward loss evaluations (§2.2).

MethodAccuracy (%)Peak allocated training memory (GiB)Training time (min)
QPS-SGD95.8713.5728.45
QPS-AdamW96.3323.5536.40
QPZO94.959.3218.81

QPZO reached 94.95% accuracy, 0.92 percentage points below QPS-SGD, while using 31.3% less peak training memory and taking 33.9% less time in these runs. QPS-AdamW achieved the highest accuracy at 96.33%. Against this stronger first-order result, QPZO traded 1.38 percentage points of accuracy for 60.4% lower peak training memory and a 48.3% shorter training run.

On the same QPS parameters, AdamW improved on QPZO’s accuracy, gaining 1.38 percentage points at 2.53 times the peak training memory and 1.94 times the training time. Such a small return on the additional resources leaves little practical advantage for first-order updates in this setting. Both first-order baselines required substantially more memory and time than QPZO, limiting their comparability when adaptation must fit the same resource budget.

W2 / Q2: Experimental results with lower-bit and codebook-based quantization

Thank you for pointing out the limits of our evaluation on 4-bit GPTQ and AWQ. We agree that assessing generality requires testing both lower bit widths and codebook-based quantization. We therefore evaluated Qwen2.5-7B-Instruct on SST-2 using 2-bit and 3-bit GPTQ to test lower bit widths, and 3-bit and 4-bit VPTQ to test vector quantization with codebooks. On all four checkpoints, QPZO applied the same update rule to the retained floating-point tensors while keeping the quantized weights and metadata fixed.

(a) QPZO accuracy gains over the frozen checkpoint and peak training memory

CheckpointFrozen accuracy (%)QPZO accuracy (%)Gain (percentage points)QPZO peak allocated memory (GiB)
GPTQ Int20.1139.56+39.457.82
GPTQ Int374.0892.55+18.478.58
VPTQ 3-bit92.6694.95+2.298.59
VPTQ 4-bit92.3295.18+2.869.55

(b) SST-2 validation accuracy across methods (%)

CheckpointFrozenQZOMeZOQPZO
GPTQ Int20.110.5726.4939.56
GPTQ Int374.0894.5093.6992.55
VPTQ 3-bit92.6694.8492.5594.95
VPTQ 4-bit92.3295.0792.7895.18

The frozen GPTQ Int2 checkpoint correctly classified just 0.11% of SST-2 examples. By updating only its retained floating-point tensors, QPZO raised accuracy to 39.56%, a gain of 39.45 percentage points from a near-zero baseline. On the same checkpoint, QZO reached 0.57% and MeZO reached 26.49%.

The same QPZO update raised GPTQ Int3 accuracy from 74.08% to 92.55%. The gains extended to vector quantization even though both frozen VPTQ checkpoints already exceeded 92% accuracy. With the codebooks fixed, QPZO gained a further 2.29 percentage points at 3 bits and 2.86 points at 4 bits. Taken together, these additional experiments show that, while keeping peak training memory low, QPZO also achieves competitive performance on both lower-bit GPTQ models and models using codebook-based vector quantization.

We used the following community-quantized checkpoints of Qwen2.5-7B-Instruct.

CheckpointHugging Face repository
GPTQ Int2irish-quant/Qwen-Qwen2.5-7B-Instruct-2bit
GPTQ Int3irish-quant/Qwen-Qwen2.5-7B-Instruct-3bit
VPTQ 3-bitVPTQ-community/Qwen2.5-7B-Instruct-v8-k65536-256-woft
VPTQ 4-bitVPTQ-community/Qwen2.5-7B-Instruct-v8-k65536-65536-woft

I appreciate the detailed responses. My primary concerns are addressed, as the results clearly demonstrate a promising performance-efficiency trade-off and validate QPZO's compatibility with lower-bit and codebook-based quantization approaches. One limitation is that all supplementary experiments are conducted only on the simple SST-2 dataset, which may not sufficiently address whether the reported findings hold across tasks of different difficulty levels. Nevertheless, I remain positive about this submission for its neat practicality and good efficiency in both memory and training time, and would like to finalize my recommendation as a 'Weak Accept'.

AuthorsOpenReview ↗

Official Comment by Authors

Thank you for recognizing QPZO’s practicality and its favorable performance–efficiency trade-off, and for your positive recommendation. To address your concern about task complexity, we added an experiment on SAMSum, an open-ended dialogue summarization task. This task requires integrating information across conversational turns, tracking speakers’ actions and intentions, and generating a coherent summary in natural language. It extends our supplementary evaluation to a task requiring both dialogue understanding and free-form generation.

We evaluated Frozen and QPZO using the Qwen2.5-7B-Instruct GPTQ Int4 checkpoint on the SAMSum test set. QPZO applies the same adaptation procedure, updating only the retained floating-point tensors while keeping the low-bit backbone and quantization metadata fixed.

MethodROUGE-1 ↑ROUGE-2 ↑ROUGE-L ↑
Frozen34.7714.4326.83
QPZO42.3119.1533.01

QPZO improves ROUGE-1 from 34.77 to 42.31, ROUGE-2 from 14.43 to 19.15, and ROUGE-L from 26.83 to 33.01, corresponding to gains of 7.54, 4.72, and 6.18 points, respectively. These improvements reflect greater agreement with the reference summaries at the unigram, bigram, and sequence levels. QPZO’s improvements across all three metrics on the test set extend the empirical support for its effectiveness to open-ended dialogue summarization. They provide further evidence that the floating-point tensors retained in a quantized checkpoint offer useful adaptation capacity for tasks involving multi-turn understanding and free-form text generation.

The following example illustrates how the generated summary changes after adaptation. We include the complete dialogue, reference summary, and both model outputs, reproduced verbatim.

Real Case: the same dialogue before/after QPZO

Instruction:

Summarize the following conversation

Dialogue:

Ali: I think I left my wallet at your place yesterday. Could you check?
Mohammad: Give me a sec, I'll have a look around my room.
Ali: OK.
Mohammad: Found it!
Ali: Phew, I don't know what I'd do if it wasn't there. Can you bring it to uni tomorrow?
Mohammad: Sure thing.

Reference:

Ali left his wallet at Mohammad's place. Mohammad'll bring it to uni tomorrow.

Frozen output:

Summary: Ali thinks he left his wallet at Mohammad's place from the previous day. After checking, Mohammad finds the wallet and offers to bring it to Ali's university the next day.

ROUGE-1/2/L: 50.00 / 34.78 / 50.00

QPZO output:

Ali left his wallet at Mohammad's place. Mohammad found it and will bring it to Ali's uni tomorrow.

ROUGE-1/2/L: 80.00 / 66.67 / 80.00

Reviewer_wLzjOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Quantization-Preserved Zeroth-Order Optimization (QPZO), a forward-only fine-tuning method for post-training-quantized LLMs that keeps all low-bit weights and quantization metadata fixed while updating only the floating-point tensors already retained in the checkpoint (embeddings, normalization parameters, output heads, etc.). The authors formalize this checkpoint-native span as the Quantization-Preserved Subspace (QPS) and introduce a Signal-to-Error Ratio (SER) framework showing that SER > 1 is a sharp threshold for positive expected alignment of ZO estimates under a mean-square error bound, yielding a local expected-descent guarantee. Empirically, QPZO achieves consistent gains over frozen checkpoints, competitive or superior performance to quantization-aware ZO baselines, and markedly lower peak training memory than full-dequantized MeZO across multiple models, quantizers, and tasks.

Critical strengths
  1. Evaluates across three model families (Qwen2.5-7B, Mistral-7B, Gemma-2-9B), two quantizers (GPTQ, AWQ), and five tasks (four classification tasks and SQuAD), totaling 30 model–quantizer–task settings.

  2. Demonstrates consistent improvements over frozen baselines and competitive performance relative to quantization-aware ZO (QZO) and full-dequantized MeZO under a matched forward-evaluation budget.

Critical weaknesses
  1. The main theoretical pieces (output-head signal, SER threshold via Cauchy–Schwarz, and local descent under smoothness) are largely standard. The QPS idea of updating the retained FP tensors is conceptually straightforward, so the methodological novelty may be viewed as incremental.

  2. The author could compare their proposed method to strong gradient-based baselines (LoRA/QLoRA/DoRA) under matched memory or similar hardware constraints.

Questions for authors

Please see the weaknesses.

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors

W1: Novelty of adapting retained high-precision tensors

Thank you for asking us to clarify the novelty of adapting only the high-precision tensors retained in a quantized checkpoint.

Our literature review identified no prior work establishing that fine-tuning only the high-precision tensors already retained in a quantized checkpoint can support effective adaptation across downstream tasks while keeping both low-bit weights and quantization metadata fixed. Our contribution is to establish the feasibility of this adaptation direction and explain why it works.

Proposition 1 shows that a nonzero aggregate gradient implies a descent direction within QPS. Theorem 1 then establishes SER > 1 as the sharp threshold for guaranteeing positive expected alignment under the stated mean-square error bound; Corollary 1 translates this alignment into local expected descent under the stated conditions. Table 2 verifies this chain empirically: QPS passes the SER certificate and produces independently validated loss decreases in all five runs, compared with only 2/25 runs for dimension-matched random spaces. This connects the existence of task signal, its accessibility through finite forward queries, and actual optimization progress within QPS.

We agree that QPZO is straightforward, and regard this simplicity as a practical advantage: it requires no additional trainable modules or task-specific subspace construction, and uses only two forward queries per update. This simple procedure improves over frozen checkpoints in all 30 settings covering three model families, two quantizers, and five tasks, while achieving competitive performance with full-dequantized MeZO (Table 1). It also substantially reduces peak training memory in all six model–quantizer settings; for example, Qwen-GPTQ requires 9.32 GiB with QPZO versus 17.75 GiB with full-dequantized MeZO (Figure 2).

W2: Comparison with strong gradient-based baselines

Thank you for suggesting a comparison with strong gradient-based fine-tuning methods.

LoRA is a strong first-order baseline that adapts a quantized model by training a small set of low-rank adapter weights. Computing gradients for those weights still requires backpropagation through the model. QPZO obtains each update from two forward loss evaluations and applies it directly to the floating-point tensors already present in the checkpoint (§2.2). Given this difference, we expected LoRA to require substantially more training time for comparable task accuracy. This anticipated trade-off led us to omit first-order LoRA from the original comparison and focus on QZO and MeZO, which let us compare choices of update space within forward-only adaptation (§4.1).

Following your suggestion, we evaluated both LoRA and DoRA on SST-2 using the same Qwen2.5-7B-Instruct GPTQ Int4 checkpoint as QPZO. We set the adapter rank to r=16r=16 for both methods and trained each with SGD and AdamW. All five runs, including QPZO, used the same hardware. Our LoRA baseline trains only the low-rank adapters on the frozen 4-bit GPTQ checkpoint, so it already evaluates the core adaptation strategy used by QLoRA. A separate run using QLoRA’s NF4 format would test the same adapter-training strategy while changing the quantized checkpoint from which adaptation begins. We therefore omit a separate QLoRA baseline from the supplementary experiments.

Method (LoRA and DoRA rank = 16)Updated parameters (millions)Accuracy (%)Peak allocated training memory (GiB)Training time (min)
LoRA-SGD40.3793.6910.1726.88
LoRA-AdamW40.3794.3810.4830.51
DoRA-SGD41.7694.0410.1853.52
DoRA-AdamW41.7694.9510.5060.07
QPZO1,090.2094.959.3218.81

Of the two LoRA runs, LoRA-AdamW came closer to QPZO in accuracy (94.38% vs. 94.95%, a difference of 0.57 percentage points). It took 30.51 minutes, 62% longer than QPZO’s 18.81 minutes. LoRA-SGD took 43% longer than QPZO, reaching 93.69% in 26.88 minutes. DoRA-SGD reached 94.04% in 53.52 minutes. DoRA-AdamW matched QPZO’s accuracy of 94.95%, but took 60.07 minutes on the same hardware, or 3.19 times as long as QPZO.

All four adapter runs also used more peak training memory than QPZO, even though LoRA and DoRA updated only 40.37 million and 41.76 million parameters, respectively, or about 3.7% and 3.8% of the 1,090.20 million parameters in QPS. With SGD and AdamW, LoRA used 10.17 and 10.48 GiB, respectively, and DoRA used 10.18 and 10.50 GiB. By updating the existing QPS tensors directly without adding an adapter, QPZO matched the highest accuracy in this comparison with the lowest peak training memory (9.32 GiB) and the shortest training time (18.81 minutes).

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer,

Thank you again for your thoughtful feedback and for taking the time to consider our rebuttal. We would greatly appreciate it if you could let us know whether our responses and additional experimental evidence have addressed your concerns. If any issues remain or would benefit from further clarification, please let us know. We would be happy to respond promptly during the discussion period.

Thank you again for your valuable time and consideration.

Thank you for the additional LoRA and DoRA comparisons. These results strengthen the empirical support for the method’s practical value, and I am raising my recommendation to Weak Accept.

My remaining reservation is the limited breadth of these comparisons: they cover only SST-2 with a single model and quantization configuration. While encouraging, the results do not yet establish whether the same performance–efficiency advantage holds across a broader range of settings.

AuthorsOpenReview ↗

Official Comment by Authors

Thank you for your thoughtful follow-up and for raising your recommendation to Weak Accept. We appreciate your recognition of QPZO’s practical value and your further feedback on the breadth of the comparisons.

We selected SST-2 because it is an established benchmark in zeroth-order language-model fine-tuning, used in MeZO and subsequent systematic evaluations of ZO methods. We paired it with Qwen2.5-7B-Instruct-GPTQ-Int4, officially released by the Qwen team, to evaluate a publicly available, general-purpose quantized checkpoint. These choices provide a representative and reproducible setting for comparing performance, memory consumption, and training time under the same hardware conditions. The first-order comparisons complement our central finding that updating only the retained floating-point tensors enables effective forward-only adaptation. The main experiments in Table 1 provide broader support for this finding.

We are currently conducting additional experiments to extend the first-order comparisons to a broader range of settings. The preliminary results are consistent with our existing findings, further supporting QPZO’s ability to achieve competitive performance with lower memory consumption and shorter training time. We will include the completed experimental results in the revised paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal Discussion

Dear ACs and Reviewers,

Thank you very much for your valuable contributions to our work. We greatly appreciate the reviewers’ thoughtful feedback and are encouraged by the overall positive assessments, including recognition of the paper’s novel theoretical insights and practical contributions.

In response to these insightful comments, we conducted substantial additional experiments and analyses to further strengthen the paper and improve clarity.

To assist the AC and facilitate the evaluation, we provide a brief summary of the reviewer discussions below.

ReviewerInitial scoreUpdated scoreStatements from the discussion
wLzj23“These results strengthen the empirical support for the method’s practical value” and “I am raising my recommendation to Weak Accept.”
ofac33“My primary concerns are addressed” and “I remain positive about this submission for its neat practicality and good efficiency in both memory and training time”
NNVn22“The deployment example and explanation on learning rate resolved my questions. I now think that the paper is a complete research work.” “I am keeping my rating at 2, but I nevertheless consider acceptance a reasonable outcome.”

Reviewer NNVn also explained that the four-point rating scale did not fully convey this assessment and expressed the hope that the AC would take this clarification into account when weighing the review.

And here is a summary of the additional experiments and analyses we conducted in response to the reviewers’ comments:

  1. Comparison with first-order methods. We added LoRA and DoRA baselines using the same quantized checkpoint and hardware as QPZO, training both with SGD and AdamW. We also evaluated SGD and AdamW on the identical QPS parameters to compare first-order and zeroth-order optimization within the same update space. By reporting accuracy, peak training memory, and training time together, these experiments demonstrate that QPZO achieves competitive accuracy with lower memory consumption and shorter training time in the evaluated setting.

  2. Lower-bit and codebook-based quantization. We extended the evaluation to lower-bit quantized models and codebook-based VPTQ checkpoints, including comparisons with the frozen models, QZO, and MeZO. Across these experiments, QPZO applied the same update procedure, and the results further support its effectiveness across these quantization settings.

  3. Further discussion of QPZO’s novelty, practical value, and theoretical foundations. In our response to Reviewer wLzj, we clarified how the theoretical analysis and empirical verification together support the contribution of adapting only the floating-point tensors retained in a quantized checkpoint. In our response to Reviewer NNVn, we explained the practical need for adaptation when new data arrive after deployment and local training must fit the device’s memory budget, and clarified that the learning rate is never quantized to the backbone’s bit width. In response to Reviewer ofac’s follow-up concern about task complexity, we added SAMSum dialogue summarization, evaluated the complete test set, and provided a full example of the model outputs before and after adaptation. Improvements across the ROUGE metrics extend the empirical support to open-ended generation.

Finally, we sincerely thank all reviewers for their constructive questions and active engagement throughout the discussion. The additional experiments and analyses have substantially strengthened the paper, and we will incorporate them into the revised version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.