- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces an end-to-end fine-tuning method for low-bit language models. By directly optimizing quantization codes and scales with a code surrogate gradient search, GradCodeS allows the model to preserve its low-bit representation after tuning. Experiments on mini-size Qwen and Llama models demonstrate the effectiveness of this method.
Critical strengths
The mathematical formulation is clear and easy to understand.
GradScoreS can be extended to different quantization formats.
Critical weaknesses
Although this paper is well-written, my main concern is about its performance when we scale this method to larger models and harder tasks. Actually, I didn't see any details of your training infrastructure and settings. If you lack computational resources, you should talk about it faithfully. Some results of 16bit SFT also made me confused; I guess it relates to model size and datasets.
Although GradCodeS uses surrogate gradients to guide discrete optimization, the method still requires evaluating multiple candidates at each optimization step. This introduces considerable computational overhead and may become a bottleneck when scaling to larger models.
Questions for authors
How sensitive is the search to dependencies among quantized parameters?
4 comments
Part II
...
Table S13. Baseline configurations and resources for Llama-3.2-1B-Instruct/GSM8K. All training methods use eight devices, 7,473 training examples, maximum sequence length 768, and gradient accumulation 1. Epoch times are based on eight RTX 3090 devices (24 GiB each).
| Method | Batch/device | Global batch (8 devices) | Configuration | Epoch wall time (h) | Peak memory (GiB/device) |
|---|---|---|---|---|---|
| Base-16 | N/A | N/A | Original 16-bit model; no fine-tuning | N/A | N/A |
| 16-bit backbone + LoRA (16-bit SFT) | 4 | 32 | Rank 8; AdamW 2e-5, alpha 16, dropout 0.05 | 0.06 | 11.65 |
| QLoRA-16A | 4 | 32 | Rank 8, fused AdamW 2e-5, alpha 16, dropout 0.05; double quantization | 0.052 | 8.38 |
| QA-LoRA | 4 | 32 | Group-pooled rank 8, AdamW 2e-5, alpha 16, dropout 0.05; adapter group size 32 | 0.046 | 17.56 |
| Base-4 | N/A | N/A | Direct 4-bit quantization; no fine-tuning | N/A | N/A |
| QLoRA-4Merge | 4 | 32 | Same training as QLoRA-16A; adapter merge and requantization afterward | 0.052 | 8.38 |
| LoQT | 4 | 32 | Rank 8, AdamW 2e-5, NF4/group 64; projector refresh every 200 steps | 0.17 | 14.43 |
| L4Q | 4 | 32 | Rank 8, AdamW 2e-5, learned uniform 4-bit quantizer/group 64 | 0.13 | 12.85 |
| PV-Tuning | 2 | 16 | Rank 8, top-k 16, A/B steps 3/3; scale AdamW 2e-4; step cap 0.5 | 0.581 | 16.69 |
| QuZO | 1 | 8 | Step size 0.01; 16 directions / 32 forwards; perturbation 1, direction scale 0.1 | 2.48 | 8.30 |
| QZO | 1 | 8 | Step size 1e-5; 64 directions / 128 forwards; perturbation 1e-4 | 4.03 | 4.27 |
| QES | 1 | 8 | Step size 0.05; population 64, noise sigma 0.1, residual decay 0.9 | 4.05 | 11.61 |
| GradCodeS | 1 | 8 | A/B steps 3/3; scale AdamW 5e-5; 16 evaluated states, adaptive proposal mass 0.5 on current state | 2.14 | 11.39 |
QZO, QuZO, QES, PV-Tuning, and GradCodeS use constant schedules without warmup; the remaining training methods use cosine schedules, 5% warmup, and weight decay 0.01. QZO, QuZO, and QES use zero weight decay. PV-Tuning and GradCodeS use a lattice penalty coefficient of 0.5 and zero scale-optimizer weight decay.
(c) Interpretation of the 16-bit SFT results
“16-bit SFT” denotes rank-8 LoRA on a 16-bit backbone. We will use the name “16-bit backbone + LoRA” consistently.
Our explanation concerns generalization when fine-tuning data are limited. As discussed in the paper, constraining adaptation to quantized code states may act as implicit regularization, reducing overfitting to the small fine-tuning set and improving held-out performance. This provides a possible explanation for why GradCodeS can outperform the 16-bit LoRA reference in the smaller-data regime.
Table S14. Training-data size and the 16-bit reference on Llama-3.2-1B/GSM8K. Values are test accuracy (%); this comparison uses GradCodeS (Full).
| Method | 7.0k training examples | 167k training examples |
|---|---|---|
| 16-bit backbone + LoRA (rank 8) | 38.9 | 55.9 |
| GradCodeS (Full) | 41.6 | 54.5 |
GradCodeS (Full) leads with 7.0k examples, while the 16-bit LoRA reference leads with 167k. This reversal is consistent with a regularization benefit that is most useful when data are scarce and may diminish as more training data become available. We present this as an empirical explanation for the observed generalization behavior, rather than a proven causal mechanism, since the methods also differ in optimization and parameterization.
Part III
Weakness 2: Computational overhead of multiple candidates
Each candidate requires a full-model objective evaluation, even with rank-8 search factors. We have therefore included these evaluations in the matched-GPU-hour comparison and added a candidate-count sweep to measure the tradeoff directly.
At the same 48 GPU-hour cap on eight RTX 3090 devices, GradCodeS reaches 41.57% GSM8K accuracy at 11.39 GiB/GPU peak memory, compared with 37.17% at 16.69 GiB/GPU for PV-Tuning and 26.58% at 11.61 GiB/GPU for QES. The best validation checkpoints occur at 34.24, 18.59, and 32.40 GPU-hours, respectively. Thus, the default accuracy gain persists after accounting for candidate evaluations.
Table S15. Candidate-count comparison on Llama-3.2-1B/GSM8K at a 48 GPU-hour cap. Every GradCodeS configuration includes the current and nearest-reference states; the remaining states are independent random draws. Updates are capped at five epochs. Checkpoints are selected by validation performance within the budget before test evaluation.
| Llama-3.2-1B/GSM8K | Total evaluated states | Random draws | Validation loss | Test accuracy (%) | Updates within budget |
|---|---|---|---|---|---|
| PV-Tuning | N/A | N/A | 0.7616 | 37.17 | 2,340 |
| QES | Native configuration | Native configuration | 0.8903 | 26.58 | 1,377 |
| GradCodeS | 2 | 0 | 0.7393 | 39.03 | 4,675 |
| GradCodeS | 4 | 2 | 0.7285 | 40.27 | 4,566 |
| GradCodeS | 8 | 6 | 0.7173 | 41.19 | 3,563 |
| GradCodeS | 16 | 14 | 0.7125 | 41.57 | 2,635 |
Two states already reach 39.03%, above PV-Tuning's 37.17%. Eight states reach 41.19%, within 0.38 points of 16 states, while allowing 3,563 versus 2,635 updates within the same budget. Thus, fewer candidates permit more training updates while preserving most of the accuracy gain at 1B.
Response
Thank you for your effort. These notes resolve part of my concerns. However, scalability to substantially larger models and harder tasks remains unclear, so I will maintain my score.
Part I
Response to Reviewer JqrA
Thank you for the review and for your questions about parameter dependencies, scalability, and training cost.
Question 1: Sensitivity to dependencies among quantized parameters
To generate candidates efficiently, we draw coordinate values independently around a fixed gradient-guided reference. In the low-rank version, these coordinates are entries of the integer factors A and B; their product AB⊤ produces a joint code update that can change many weights together. We then evaluate each complete candidate through the quantized model, so its loss includes interactions among all changed weights. The guide determines where to search, while this joint loss determines what to accept.
We have added a diagnostic at early, middle, and late frozen checkpoints. We partition the same proposed update into attention and MLP parts. With fixed scales and an independent diagnostic probe loss J, the interaction residual is
I(\Delta_a,\Delta_m) &=J(Z+\Delta_a+\Delta_m)-J(Z+\Delta_a)\\\\ &\quad-J(Z+\Delta_m)+J(Z). \end{aligned}$$ **Table S11. Parameter-interaction diagnostic at frozen checkpoints.** | Frozen checkpoint stage | Absolute interaction residual | Surrogate / probe-loss rank correlation | Selected candidate probe loss change | |---|---|---|---| | Early | 0.00114 ± 0.00026 | 0.497 ± 0.015 | -0.00079 ± 0.00002 | | Middle | 0.00076 ± 0.00006 | 0.614 ± 0.037 | -0.00052 ± 0.00009 | | Late | 0.00037 ± 0.00002 | 0.558 ± 0.024 | -0.00023 ± 0.00006 | The nonzero residuals show attention–MLP interactions in the sampled updates. Surrogate/probe-loss rank correlations remain positive (0.497–0.614), and selected candidates reduce independent probe loss at all three stages. The probe batch is used for neither candidate nor checkpoint selection. These results support the guide's usefulness in the measured setting. Stronger dependencies can still reduce proposal quality or place useful joint moves outside the local low-rank search family; joint evaluation can reject unfavorable proposals but cannot recover a candidate that was never proposed. # Weakness 1: Scalability, infrastructure, and the 16-bit reference We have added experiments on further Qwen and Llama models and provide the training infrastructure and baseline settings below. Our code has already been released. We also clarify the 16-bit SFT reference and the possible generalization benefit of the low-bit parameterization with limited training data. #### (a) Scaling to larger models **Table S12. Added GSM8K results for Llama-3.2-3B-Instruct and Qwen3-1.7B.** | Backbone | Method | GSM8K test accuracy (%) | |---|---|---| | Llama-3.2-3B-Instruct | Base-16 | 56.77 | | Llama-3.2-3B-Instruct | Base-4 | 48.73 | | Llama-3.2-3B-Instruct | 16-bit SFT (rank-8 LoRA) | 67.47 | | Llama-3.2-3B-Instruct | PV-Tuning | 63.40 | | Llama-3.2-3B-Instruct | GradCodeS | 67.23 | | Qwen3-1.7B | Base-16 | 67.87 | | Qwen3-1.7B | Base-4 | 58.58 | | Qwen3-1.7B | 16-bit SFT (rank-8 LoRA) | 75.97 | | Qwen3-1.7B | PV-Tuning | 71.65 | | Qwen3-1.7B | GradCodeS | 75.34 | GradCodeS improves over PV-Tuning by 3.83 percentage points on Llama-3.2-3B and 3.69 points on Qwen3-1.7B, and is within 0.24 and 0.63 points of the respective 16-bit LoRA references. The experiments in the paper cover arithmetic reasoning, instruction following, and structured semantic parsing. Our available compute limited the additional study to models up to 3B parameters; substantially larger models remain to be evaluated. We will make this experimental scope and resource limitation explicit. #### (b) Training infrastructure, settings, and released code **Our code has been released at the [anonymous repository](https://anonymous.4open.science/r/GradCodes-EC82)**, as noted in the paper's Reproducibility Statement. The implementation is available for inspection, and we summarize the training setup and baseline configurations below. For Llama-3.2-1B-Instruct/GSM8K, GradCodeS uses eight RTX 3090 devices (24 GiB each), one example per device, gradient accumulation 1 (global batch 8), and maximum sequence length 768. Models are loaded in bfloat16, temporary recovered-weight gradients are accumulated in float32, and gradient checkpointing is enabled. We use a response-only causal language-modeling objective, masking prompt and padding tokens. ...