acceptodds
Under review as a conference paper at ICLR 2027

Fine-Tuning Low-Bit Models with Gradients in Quantized Code Space

Abstract

Fine-tuning low-bit models aims to adapt a quantized model while keeping the final deployed checkpoint in the same low-bit form. This setting is practically important as it reduces memory and inference cost for storage and deployment. Under this constraint, adaptation becomes an optimization problem over quantization codes and scales. Existing continuous low-bit training is efficient, but it can be distorted by straight through estimation error or by post-quantization gap; discrete search is deployment-faithful, but can be computationally inefficient under a finite training budget. We propose code surrogate gradient to find the steepest direction of a local continuous surrogate in code space to accelerate optimization, and perform guided search to preserve deployment faithfulness. Experiments across arithmetic reasoning, instruction following, and structured language understanding show that GradCodes consistently improves fine-tuning low-bit models across different quantization datatypes.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Quantized LLM, Optimization, Search

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_JqrAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces an end-to-end fine-tuning method for low-bit language models. By directly optimizing quantization codes and scales with a code surrogate gradient search, GradCodeS allows the model to preserve its low-bit representation after tuning. Experiments on mini-size Qwen and Llama models demonstrate the effectiveness of this method.

Critical strengths

The mathematical formulation is clear and easy to understand.

GradScoreS can be extended to different quantization formats.

Critical weaknesses

Although this paper is well-written, my main concern is about its performance when we scale this method to larger models and harder tasks. Actually, I didn't see any details of your training infrastructure and settings. If you lack computational resources, you should talk about it faithfully. Some results of 16bit SFT also made me confused; I guess it relates to model size and datasets.

Although GradCodeS uses surrogate gradients to guide discrete optimization, the method still requires evaluating multiple candidates at each optimization step. This introduces considerable computational overhead and may become a bottleneck when scaling to larger models.

Questions for authors

How sensitive is the search to dependencies among quantized parameters?

4 comments

Response to Reviewer JqrA

Thank you for the review and for your questions about parameter dependencies, scalability, and training cost.

Question 1: Sensitivity to dependencies among quantized parameters

To generate candidates efficiently, we draw coordinate values independently around a fixed gradient-guided reference. In the low-rank version, these coordinates are entries of the integer factors AA and BB; their product AB⊤AB^\top produces a joint code update that can change many weights together. We then evaluate each complete candidate through the quantized model, so its loss includes interactions among all changed weights. The guide determines where to search, while this joint loss determines what to accept.

We have added a diagnostic at early, middle, and late frozen checkpoints. We partition the same proposed update into attention and MLP parts. With fixed scales and an independent diagnostic probe loss JJ, the interaction residual is

I(\Delta_a,\Delta_m) &=J(Z+\Delta_a+\Delta_m)-J(Z+\Delta_a)\\\\ &\quad-J(Z+\Delta_m)+J(Z). \end{aligned}$$ **Table S11. Parameter-interaction diagnostic at frozen checkpoints.** | Frozen checkpoint stage | Absolute interaction residual | Surrogate / probe-loss rank correlation | Selected candidate probe loss change | |---|---|---|---| | Early | 0.00114 ± 0.00026 | 0.497 ± 0.015 | -0.00079 ± 0.00002 | | Middle | 0.00076 ± 0.00006 | 0.614 ± 0.037 | -0.00052 ± 0.00009 | | Late | 0.00037 ± 0.00002 | 0.558 ± 0.024 | -0.00023 ± 0.00006 | The nonzero residuals show attention–MLP interactions in the sampled updates. Surrogate/probe-loss rank correlations remain positive (0.497–0.614), and selected candidates reduce independent probe loss at all three stages. The probe batch is used for neither candidate nor checkpoint selection. These results support the guide's usefulness in the measured setting. Stronger dependencies can still reduce proposal quality or place useful joint moves outside the local low-rank search family; joint evaluation can reject unfavorable proposals but cannot recover a candidate that was never proposed. # Weakness 1: Scalability, infrastructure, and the 16-bit reference We have added experiments on further Qwen and Llama models and provide the training infrastructure and baseline settings below. Our code has already been released. We also clarify the 16-bit SFT reference and the possible generalization benefit of the low-bit parameterization with limited training data. #### (a) Scaling to larger models **Table S12. Added GSM8K results for Llama-3.2-3B-Instruct and Qwen3-1.7B.** | Backbone | Method | GSM8K test accuracy (%) | |---|---|---| | Llama-3.2-3B-Instruct | Base-16 | 56.77 | | Llama-3.2-3B-Instruct | Base-4 | 48.73 | | Llama-3.2-3B-Instruct | 16-bit SFT (rank-8 LoRA) | 67.47 | | Llama-3.2-3B-Instruct | PV-Tuning | 63.40 | | Llama-3.2-3B-Instruct | GradCodeS | 67.23 | | Qwen3-1.7B | Base-16 | 67.87 | | Qwen3-1.7B | Base-4 | 58.58 | | Qwen3-1.7B | 16-bit SFT (rank-8 LoRA) | 75.97 | | Qwen3-1.7B | PV-Tuning | 71.65 | | Qwen3-1.7B | GradCodeS | 75.34 | GradCodeS improves over PV-Tuning by 3.83 percentage points on Llama-3.2-3B and 3.69 points on Qwen3-1.7B, and is within 0.24 and 0.63 points of the respective 16-bit LoRA references. The experiments in the paper cover arithmetic reasoning, instruction following, and structured semantic parsing. Our available compute limited the additional study to models up to 3B parameters; substantially larger models remain to be evaluated. We will make this experimental scope and resource limitation explicit. #### (b) Training infrastructure, settings, and released code **Our code has been released at the [anonymous repository](https://anonymous.4open.science/r/GradCodes-EC82)**, as noted in the paper's Reproducibility Statement. The implementation is available for inspection, and we summarize the training setup and baseline configurations below. For Llama-3.2-1B-Instruct/GSM8K, GradCodeS uses eight RTX 3090 devices (24 GiB each), one example per device, gradient accumulation 1 (global batch 8), and maximum sequence length 768. Models are loaded in bfloat16, temporary recovered-weight gradients are accumulated in float32, and gradient checkpointing is enabled. We use a response-only causal language-modeling objective, masking prompt and padding tokens. ...

...

Table S13. Baseline configurations and resources for Llama-3.2-1B-Instruct/GSM8K. All training methods use eight devices, 7,473 training examples, maximum sequence length 768, and gradient accumulation 1. Epoch times are based on eight RTX 3090 devices (24 GiB each).

MethodBatch/deviceGlobal batch (8 devices)ConfigurationEpoch wall time (h)Peak memory (GiB/device)
Base-16N/AN/AOriginal 16-bit model; no fine-tuningN/AN/A
16-bit backbone + LoRA (16-bit SFT)432Rank 8; AdamW 2e-5, alpha 16, dropout 0.050.0611.65
QLoRA-16A432Rank 8, fused AdamW 2e-5, alpha 16, dropout 0.05; double quantization0.0528.38
QA-LoRA432Group-pooled rank 8, AdamW 2e-5, alpha 16, dropout 0.05; adapter group size 320.04617.56
Base-4N/AN/ADirect 4-bit quantization; no fine-tuningN/AN/A
QLoRA-4Merge432Same training as QLoRA-16A; adapter merge and requantization afterward0.0528.38
LoQT432Rank 8, AdamW 2e-5, NF4/group 64; projector refresh every 200 steps0.1714.43
L4Q432Rank 8, AdamW 2e-5, learned uniform 4-bit quantizer/group 640.1312.85
PV-Tuning216Rank 8, top-k 16, A/B steps 3/3; scale AdamW 2e-4; step cap 0.50.58116.69
QuZO18Step size 0.01; 16 directions / 32 forwards; perturbation 1, direction scale 0.12.488.30
QZO18Step size 1e-5; 64 directions / 128 forwards; perturbation 1e-44.034.27
QES18Step size 0.05; population 64, noise sigma 0.1, residual decay 0.94.0511.61
GradCodeS18A/B steps 3/3; scale AdamW 5e-5; 16 evaluated states, adaptive proposal mass 0.5 on current state2.1411.39

QZO, QuZO, QES, PV-Tuning, and GradCodeS use constant schedules without warmup; the remaining training methods use cosine schedules, 5% warmup, and weight decay 0.01. QZO, QuZO, and QES use zero weight decay. PV-Tuning and GradCodeS use a lattice penalty coefficient of 0.5 and zero scale-optimizer weight decay.

(c) Interpretation of the 16-bit SFT results

“16-bit SFT” denotes rank-8 LoRA on a 16-bit backbone. We will use the name “16-bit backbone + LoRA” consistently.

Our explanation concerns generalization when fine-tuning data are limited. As discussed in the paper, constraining adaptation to quantized code states may act as implicit regularization, reducing overfitting to the small fine-tuning set and improving held-out performance. This provides a possible explanation for why GradCodeS can outperform the 16-bit LoRA reference in the smaller-data regime.

Table S14. Training-data size and the 16-bit reference on Llama-3.2-1B/GSM8K. Values are test accuracy (%); this comparison uses GradCodeS (Full).

Method7.0k training examples167k training examples
16-bit backbone + LoRA (rank 8)38.955.9
GradCodeS (Full)41.654.5

GradCodeS (Full) leads with 7.0k examples, while the 16-bit LoRA reference leads with 167k. This reversal is consistent with a regularization benefit that is most useful when data are scarce and may diminish as more training data become available. We present this as an empirical explanation for the observed generalization behavior, rather than a proven causal mechanism, since the methods also differ in optimization and parameterization.

Weakness 2: Computational overhead of multiple candidates

Each candidate requires a full-model objective evaluation, even with rank-8 search factors. We have therefore included these evaluations in the matched-GPU-hour comparison and added a candidate-count sweep to measure the tradeoff directly.

At the same 48 GPU-hour cap on eight RTX 3090 devices, GradCodeS reaches 41.57% GSM8K accuracy at 11.39 GiB/GPU peak memory, compared with 37.17% at 16.69 GiB/GPU for PV-Tuning and 26.58% at 11.61 GiB/GPU for QES. The best validation checkpoints occur at 34.24, 18.59, and 32.40 GPU-hours, respectively. Thus, the default accuracy gain persists after accounting for candidate evaluations.

Table S15. Candidate-count comparison on Llama-3.2-1B/GSM8K at a 48 GPU-hour cap. Every GradCodeS configuration includes the current and nearest-reference states; the remaining states are independent random draws. Updates are capped at five epochs. Checkpoints are selected by validation performance within the budget before test evaluation.

Llama-3.2-1B/GSM8KTotal evaluated statesRandom drawsValidation lossTest accuracy (%)Updates within budget
PV-TuningN/AN/A0.761637.172,340
QESNative configurationNative configuration0.890326.581,377
GradCodeS200.739339.034,675
GradCodeS420.728540.274,566
GradCodeS860.717341.193,563
GradCodeS16140.712541.572,635

Two states already reach 39.03%, above PV-Tuning's 37.17%. Eight states reach 41.19%, within 0.38 points of 16 states, while allowing 3,563 versus 2,635 updates within the same budget. Thus, fewer candidates permit more training updates while preserving most of the accuracy gain at 1B.

Reviewer_JqrAOpenReview ↗

Response

Thank you for your effort. These notes resolve part of my concerns. However, scalability to substantially larger models and harder tasks remains unclear, so I will maintain my score.

Reviewer_3XiQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies fine-tuning when the final model must remain in the same low-bit representation, rather than retaining a high-precision adapter or merging one after training. GradCodeS treats code indices and their permitted scales as the variables of optimization. A weight gradient is multiplied by the effective deployed-weight change of an adjacent code move, taking account of group scales and nonuniform codebook gaps. This produces a local code-coordinate surrogate gradient. The gradient points to a real-valued reference in code space, around which the method samples discrete candidates; it evaluates their actual quantized loss and accepts the best candidate, including the current state. The theory establishes a local steepest-descent interpretation and a first-order identity for feasible one-level, gradient-aligned moves. The stronger expected-descent statement is conditional on the proposal assigning positive mass to a suitable improving set.

Critical strengths

The paper takes its deployment constraint seriously. Evaluating candidates as recovered quantized weights avoids the common ambiguity in which a method optimizes a floating-point adapter but reports the performance of a later requantized model. The QLoRA comparison illustrates why this matters: on the Llama-3.2-1B GSM8K example, merging and requantizing the adapter loses 13.4 points relative to retaining it. GradCodeS's accepted code states have the form in which the targeted modules will be deployed, so its task scores answer a cleaner question than mixed-precision results do.

The proposal mechanism is supported by a useful set of controls. On the Llama-3.2-1B GSM8K candidate study, guided candidates are selected 67.97% of the time versus 48.62% for guidance derived from a mapped ordinary weight gradient and 38.16% for one-hop sampling. Removing guidance or freezing codes or scales lowers task performance. The improvements recur across three task families and three codebooks, rather than depending only on NF4 arithmetic. The paper also reports three independent runs and separates low-bit methods from high-precision references in its tables.

Critical weaknesses
  1. Training cost is underreported. The default low-rank search computes a backward guide and evaluates the current state, a nearest-reference state and 14 sampled states on each step. Figure 3 provides one wall-clock comparison on Llama-3.2-1B GSM8K, but not a systematic table of GPU-hours, training throughput or peak memory across tasks and methods. The default search batch is only one example per device on eight devices. For a method positioned as efficient discrete optimization, I need a fuller accounting of the cost paid for its extra forward passes and how much of the final gain remains at a matched GPU-hour budget.

  2. The support for the broad claim is narrower than the framing. Proposition 1 is a first-order identity under fixed scales and gradient-aligned one-level moves; Proposition 2 assumes full-objective selection and positive proposal mass near an improving step. Training uses mini-batches, updates scales separately, and samples high-dimensional factor or code candidates. Those propositions motivate the guide, but they do not guarantee that the implemented search improves expected population loss. Empirically, the 3B table omits several fully 4-bit baselines because of compute limits, and the AlpacaEval win rates depend on a model judge. These do not overturn the 1B evidence, but they make the broad “strongest fully 4-bit method” conclusion somewhat narrower than its wording suggests.

Questions for authors
  1. Could the authors report peak training memory, total GPU-hours, effective tokens per second and final validation/test performance at matched GPU-hour budgets for GradCodeS, PV-Tuning and the closest discrete baseline? A candidate-count sweep, including fewer than 16 evaluated states, would show whether the extra search cost is necessary for the gain.

  2. For the 3B setting, how much of the reported advantage survives comparison with the omitted low-bit baselines under a matched budget? Please also clarify whether the AlpacaEval ranking changes with a different judge or with a small human-rated sample. These answers would materially affect my confidence in the generality of the empirical claim.

5 comments

Response to Reviewer 3XiQ

Thank you for the detailed review and for recognizing the deployment-faithful formulation and proposal controls.

Question 1 / Weakness 1: Resource accounting and candidate-count sweep

We agree that the additional candidate evaluations should be included in the cost comparison. We have added matched-GPU-hour results, peak memory, useful-token throughput, and a candidate-count sweep below.

(a) Matched-budget performance and resource accounting

We compare GradCodeS and six fully low-bit baselines at 24 and 48 GPU-hours on the same eight RTX 3090 devices (24 GiB each), target modules, data split, and evaluation protocol for Llama-3.2-1B-Instruct/GSM8K. The budgets account for all training computation, including GradCodeS's backward guide and candidate evaluations. Factor-based methods use rank 8, and all compared methods deploy without a high-precision inference branch.

Table S6. Matched-GPU-hour comparison on Llama-3.2-1B/GSM8K.

MethodTest accuracy at 24 GPU-hours (%)Test accuracy at 48 GPU-hours (%)Peak memory (GiB/GPU)
LoQT29.2129.2114.43
L4Q30.9830.9812.85
PV-Tuning37.1737.1716.69
QuZO28.6329.578.30
QZO26.0826.084.27
QES26.5826.5811.61
GradCodeS39.8541.5711.39

Within each compute budget, we report test accuracy at the checkpoint selected by validation performance. At 24 and 48 GPU-hours, GradCodeS exceeds PV-Tuning by 2.68 and 4.40 percentage points, respectively, while using 11.39 versus 16.69 GiB/GPU of peak training memory. Thus, the accuracy gain persists after accounting for the extra candidate evaluations.

(b) Candidate-count sweep and throughput

We swept the total evaluated states C∈2,4,8,16C\in\\{2,4,8,16\\} under the same 48 GPU-hour cap. Each configuration includes the current and nearest-reference states, leaving C−2C-2 random draws. Thus, C=2C=2 tests the nearest-reference update with current-state retention. The default C=16C=16 includes 14 random draws.

Table S7. Candidate-count comparison at a 48 GPU-hour cap. Useful response tokens/s is aggregate throughput across all eight devices, counting each training response once; updates are capped at five epochs.

Llama-3.2-1B/GSM8KTotal evaluated statesRandom drawsTest accuracy (%)Useful response tokens/s
PV-TuningN/AN/A37.17575.0
QESNative configurationNative configuration26.5880.3
GradCodeS2039.03400.4
GradCodeS4240.27266.2
GradCodeS8641.19207.7
GradCodeS161441.57153.6

Two states already reach 39.03%, above PV-Tuning's 37.17%. Eight states reach 41.19%, only 0.38 points below 16 states, at 1.35 times the useful-token throughput (207.7 versus 153.6 tokens/s). Additional candidates improve accuracy, while eight states preserve most of the gain at lower cost per update. The throughput figures count useful training tokens once, so repeated candidate evaluations are reflected in the cost.

Question 2: Missing 3B baselines and judge sensitivity

We have added GSM8K comparisons on Llama-3.2-3B-Instruct and Qwen3-1.7B, including the missing PV-Tuning and QES comparisons at 3B under a common compute cap. We also provide a second-judge check of the saved AlpacaEval responses.

(a) Additional backbones and missing 3B baselines

Table S8 reports the complete comparison on both added backbones. For Llama-3.2-3B-Instruct, PV-Tuning, QES, and GradCodeS use a common 144 GPU-hour cap.

Table S8. GSM8K results for Llama-3.2-3B-Instruct and Qwen3-1.7B.

BackboneMethodGSM8K test accuracy (%)
Llama-3.2-3B-InstructBase-1656.77
Llama-3.2-3B-InstructBase-448.73
Llama-3.2-3B-Instruct16-bit SFT (rank-8 LoRA)67.47
Llama-3.2-3B-InstructPV-Tuning63.40
Llama-3.2-3B-InstructQES57.80
Llama-3.2-3B-InstructGradCodeS67.23
Qwen3-1.7BBase-1667.87
Qwen3-1.7BBase-458.58
Qwen3-1.7B16-bit SFT (rank-8 LoRA)75.97
Qwen3-1.7BPV-Tuning71.65
Qwen3-1.7BGradCodeS75.34

On Llama-3.2-3B, GradCodeS exceeds PV-Tuning by 3.83 percentage points and QES by 9.43 points under the same compute cap, while remaining within 0.24 points of the 16-bit LoRA reference. On Qwen3-1.7B, GradCodeS improves over PV-Tuning by 3.69 points and is within 0.63 points of the corresponding 16-bit LoRA reference. The improvement therefore extends to both additional backbones while retaining fully low-bit target weights.

(b) Sensitivity of AlpacaEval rankings to the judge

The pairwise ordering is unchanged with a second judge. We re-evaluated saved responses on the same 200 prompts using DeepSeek-V4.1-Flash, presenting each response pair in both orders. Table S9 reports pairwise win rates and prompt-level bootstrap 95% confidence intervals.

Table S9. Judge sensitivity on saved response pairs.

Saved response comparisonGPT-5.2 pairwise win rate, 95% CIDeepSeek-V4.1-Flash pairwise win rate, 95% CIOrdering agrees?
GradCodeS vs. PV-Tuning59.50% [53.00, 65.75]60.50% [54.00, 66.75]Yes (point estimates only)
GradCodeS vs. 16-bit LoRA49.50% [42.75, 56.25]48.00% [41.50, 54.25]Yes (point estimates only)

Both confidence intervals against PV-Tuning are above 50%, whereas both intervals against the 16-bit LoRA reference include 50%. This supports the preference over PV-Tuning across the two tested judges.

Weakness 2: Scope of the theoretical and empirical claims

(a) Scope of the theoretical results and extensions

We clarify the scope of the fixed-scale analysis and provide mini-batch and low-rank extensions below, which we will incorporate into the revised paper.

At fixed scales, the results explain complementary parts of the method: Lemma 1 motivates the code-space guide, Proposition 1 establishes its first-order consistency for local code moves, Corollary 1 accounts for finite-step curvature, and Proposition 2 quantifies the role of proposal coverage and candidate count.

  1. For Proposition 1: fixed scales and local moves. Fixing scales matches the code-search substep of the alternating optimizer and isolates the effect of discrete code changes. The one-level, gradient-aligned conditions establish the guide's local first-order consistency; actual candidates are accepted according to their evaluated loss. We will state this scope explicitly, with stochastic scale optimization handled separately, and add the low-rank extension below to account for moves outside these local conditions.

  2. For Proposition 2: full-batch selection and an empty favorable set. The full-objective assumption can be replaced by conditioning on the mini-batch used for selection. The original favorable set may be empty because of the discrete lattice geometry; retaining the current state still guarantees non-increase when qt=0q_t=0.

    We will add a direct mini-batch version. Let JBJ_{\mathcal B} be the evaluated mini-batch objective, including regularization. With the batch, current state, scales, regularization reference, and guide fixed, let qB(δ)q_{\mathcal B}(\delta) be the probability that one random factor proposal reduces JBJ_{\mathcal B} by at least δ>0\delta>0. Retaining the current state alongside M=14M=14 independent draws gives

    \le J_{\mathcal B}(Z,s)-\delta\bigl[1-(1-q_{\mathcal B}(\delta))^M\bigr].$$ Here $q_{\mathcal B}(\delta)$ measures actual improvement, so this extension applies to low-rank proposals without requiring membership in the original one-level favorable set. The additional nearest-reference candidate can only help, and non-increase holds even when $q_{\mathcal B}(\delta)=0$; a positive descent term remains conditional on positive improvement probability. As with mini-batch SGD, loss on an independent batch need not decrease at every step.
  3. For low-rank updates: multi-level and mixed-sign moves. We will add an extension covering any feasible candidate after clipping. For the same fixed mini-batch objective, let ΔZ\Delta Z denote its actual code change, write gW=∇W^JBg_W=\nabla_{\widehat W}J_{\mathcal B}, D=D(Z,s,gW)D=D(Z,s,g_W), gZ=gW⊙Dg_Z=g_W\odot D, and ΔW^=W^(Z+ΔZ,s)−W^(Z,s)\Delta\widehat W=\widehat W(Z+\Delta Z,s)-\widehat W(Z,s). If JBJ_{\mathcal B} is β\beta-smooth along the recovered-weight segment, then

    J_{\mathcal B}(Z+\Delta Z,s)-J_{\mathcal B}(Z,s) &\le \langle g_Z,\Delta Z\rangle_F+R(\Delta Z)+\tfrac{\beta}{2}\\|\Delta\widehat W\\|_F^2,\\\\ R(\Delta Z)&=\langle g_W,\Delta\widehat W-D\odot\Delta Z\rangle_F. \end{aligned}$$ The bound separates two sources of prediction error: extending the local codebook slope to the actual move, and loss curvature. The residual vanishes for Proposition 1's moves and satisfies $|R|\le\\|g_W\\|_F\\|\Delta\widehat W-D\odot\Delta Z\\|_F$ in general. Thus, any factor-induced candidate decreases the objective when its predicted gain exceeds this error bound and the curvature cost. This plays a similar role to the descent lemma in smooth SGD analysis: it explains when a proposed update yields a decrease ([Bottou et al., 2018, Section 4.1](https://arxiv.org/html/1606.04838v3)). These extensions concern the fixed-scale code-selection step, with scale optimization handled separately.

These guarantees concern fixed-scale code selection. They do not establish convergence of the population loss under stochastic training with alternating scale updates.

Reviewer_cDzNOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies strict fully low-bit fine-tuning: deployed weights must remain quantized, without a high-precision adapter or residual branch at inference. The authors optimize directly over discrete quantization-code matrices and group-wise scales. GradCodeS constructs a code-coordinate surrogate gradient from the weight gradient, adjacent codebook gaps, and group scales; centers a local discrete proposal on this signal; evaluates deployable candidates; and retains the one with the lowest realized loss. Experiments use Qwen3-0.6B and Llama-3.2-1B/3B on GSM8K, AlpacaEval, and MASSIVE; NF4 is the default, with NF4/INT4/MXFP4 compared on Llama-3.2-1B only.

Critical strengths
  • The strict fully low-bit setting is well motivated and clearly separated from adapter-based or optimize-then-requantize pipelines.
  • The guide–sample–evaluate–select design is clean: guidance shapes the proposal, while every accepted update is a valid quantized checkpoint.
  • The study reports large gains over fully 4-bit baselines, such as Qwen3-0.6B GSM8K accuracy of 45.72 for GradCodeS (LoRA) versus 38.70 for PV-Tuning, and MASSIVE Exact Match of 66.61 for GradCodeS (Full) versus 58.71. The three-seed evaluation and code release support reproducibility.
Critical weaknesses
  • The analysis does not cover the full algorithm. Proposition 1 is limited to fixed scales and one-level, gradient-aligned moves. Proposition 2 uses a full-batch objective; its bound remains valid at q_t = 0 but then only gives non-increase, and Appendix F notes that the favorable update set may be empty. The low-rank update ΔZ=AB^T can produce multi-level and/or non-sign-aligned entries while scales remain fixed, so the local results do not establish descent for the full procedure.
  • Guidance, selection, and reporting are not fully independent. Appendix A.3 specifies 16 evaluated states—the current state, the nearest-reference state, and 14 random draws—and selection is performed on the same mini-batch used for guidance. This may introduce a winner’s-curse effect. Nor does the paper state whether reported tables use a final or test-selected checkpoint.
  • The efficiency and reproducibility claims are not compute-matched. Figure 3(a) provides a wall-clock comparison, but aggregate GPU-hours, peak memory, baseline tuning, and a fully iso-compute setup are not reported. The released repository appears to contain only a unified QLoRA-style baseline, so the strongest external baselines cannot be independently retrained or tuning-checked.
  • Deployment accounting and novelty need tightening. The paper scopes four-bit to transformer-block linear layers, yet Table 1 reports mixed methods at 4.08/4.05 bits while GradCodeS is exactly 4 despite codebook and scale overhead. Novelty is partly positioned: Guided ES already combines surrogate-guided proposals with realized-loss selection, and JacQuant (arXiv:2605.25469) studies learned QAT surrogates; the increment is the code-gap/scale-aware score plus deployable-code selection. Table 5 omits promised standard deviations, and selected mean losses differ by only about 0.0014.
Questions for authors
  1. Can the authors provide an iso-compute and parameter-matched comparison with complete baseline configurations and evaluation scripts?
  2. Can candidate selection use an independent validation batch, and can the authors confirm that reported checkpoints are selected without test data?
  3. What are the effective bits per weight and total checkpoint size after including scales and codebooks, and what are the measured inference memory and latency?
  4. Can the authors conduct more experiments on the Qwen and LlaMa series model?
5 comments

Response to Reviewer cDzN

We sincerely appreciate your careful review.

Question 1 / Weakness 3: Matched-budget comparisons and reproducibility

We have added comparisons at 24 and 48 GPU-hours (Table S1), together with baseline configurations, epoch times, and peak training memory below.

(a) Matched compute and parameterization

GradCodeS retains its accuracy advantage at matched training budgets. We compare 24 and 48 GPU-hours on the same eight RTX 3090 devices, target modules, data split, and evaluation protocol, selecting the best validation checkpoint available within each budget. For parameterization, factor-based methods use rank 8, and Table S1 compares methods without a high-precision inference branch.

Table S1. Matched-GPU-hour comparison on Llama-3.2-1B/GSM8K.

MethodTest accuracy at 24 GPU-hours (%)Test accuracy at 48 GPU-hours (%)Peak memory (GiB/GPU)
LoQT29.2129.2114.43
L4Q30.9830.9812.85
PV-Tuning37.1737.1716.69
QuZO28.6329.578.30
QZO26.0826.084.27
QES26.5826.5811.61
GradCodeS39.8541.5711.39

At 24 and 48 GPU-hours, GradCodeS exceeds PV-Tuning by 2.68 and 4.40 percentage points, respectively, while using 11.39 versus 16.69 GiB/GPU of peak training memory. The accuracy gain therefore persists after accounting for candidate evaluations within the training budget.

(b) Baseline configurations

The summary below covers all baselines in the paper. All training methods use 8 devices, with Llama-3.2-1B-Instruct / GSM8K (7,473 training examples), maximum sequence length 768, and gradient accumulation 1. Epoch times are based on eight RTX 3090 (24 GiB).

MethodBatch/deviceGlobal batch (8 devices)ConfigurationEpoch wall time (h)Peak memory (GiB/device)
Base-16N/AN/AOriginal 16-bit model; no fine-tuningN/AN/A
16-bit backbone + LoRA (16-bit SFT)432Rank 8; AdamW 2e-5, alpha 16, dropout 0.050.0611.65
QLoRA-16A432Rank 8, fused AdamW 2e-5, alpha 16, dropout 0.05; double quantization0.0528.38
QA-LoRA432Group-pooled rank 8, AdamW 2e-5, alpha 16, dropout 0.05; adapter group size 320.04617.56
Base-4N/AN/ADirect 4-bit quantization; no fine-tuningN/AN/A
QLoRA-4Merge432Same training as QLoRA-16A; adapter merge and requantization afterward0.0528.38
LoQT432Rank 8, AdamW 2e-5, NF4/group 64; projector refresh every 200 steps0.1714.43
L4Q432Rank 8, AdamW 2e-5, learned uniform 4-bit quantizer/group 640.1312.85
PV-Tuning216Rank 8, top-k 16, A/B steps 3/3; scale AdamW 2e-4; step cap 0.50.58116.69
QuZO18Step size 0.01; 16 directions / 32 forwards; perturbation 1, direction scale 0.12.488.30
QZO18Step size 1e-5; 64 directions / 128 forwards; perturbation 1e-44.034.27
QES18Step size 0.05; population 64, noise sigma 0.1, residual decay 0.94.0511.61
GradCodeS18A/B steps 3/3; scale AdamW 5e-5; 16 evaluated states, adaptive proposal mass 0.5 on current state2.1411.39

QZO, QuZO, QES, PV-Tuning, and GradCodeS use constant schedules without warmup; the remaining training methods use cosine schedules, 5% warmup, and weight decay 0.01. QZO, QuZO, and QES use zero weight decay. PV-Tuning and GradCodeS use a lattice penalty coefficient of 0.5 and zero scale-optimizer weight decay.

Question 2 / Weakness 2: Independent candidate selection and test-free checkpoint selection

We have added an independent-batch selection control (Table S2) and confirm that reported checkpoints were selected by validation performance, without using test data.

GradCodeS uses shared-batch candidate selection. Guidance and candidate evaluation use the same training mini-batch, and we select the evaluated candidate with the lowest mini-batch objective, including the current state. As in mini-batch SGD, the current training batch supplies the update signal; reducing loss on a separate batch is not a per-step acceptance requirement. Generalization is assessed through validation and final test performance. We evaluated independent-batch selection as a supplementary control.

Table S2. Shared-batch training and independent-batch control.

Llama-3.2-1B, GSM8K, NF4, rank 8Test accuracy (%)Probe loss change (diagnostic)Peak training memory (GiB/GPU)GPU-hours
Shared guidance/selection batch (main method)41.57 ± 0.35-0.00045 ± 0.0002111.3934.24
Independent guidance/selection batches (control)41.49 ± 0.26-0.00052 ± 0.0002511.3935.16

The two variants differ by 0.08 percentage points in mean test accuracy, compared with run-to-run SDs of 0.35 and 0.26 points. Both reduce loss on an independent probe batch. Thus independent-batch selection produces similar final accuracy, with slightly higher training cost (35.16 versus 34.24 GPU-hours). Probe loss change is the selected candidate's loss minus the current state's loss on a diagnostic batch used for neither candidate nor checkpoint selection.

Checkpoint selection did not use test performance. We saved five checkpoints after 1, 2, 3, 4, and 5 complete passes through the training dataset, evaluated all five on validation data, and selected the best by validation performance. The selected checkpoint was then evaluated on test data for the reported result.

Question 3: Effective bits, checkpoint size, and inference measurements

We have added effective bit widths including scales and codebooks, along with theoretical checkpoint sizes (Table S3). We also explain below the hardware limitation that prevents us from providing the requested native 4-bit inference measurements.

Including metadata, GradCodeS uses 4.500 bits per targeted weight and 6.945 bits per weight over the whole model. Its theoretical packed checkpoint is 1023.13 MiB, about 2.30 times smaller than the BF16 reference (2357.13 MiB).

“4-bit” denotes the code payload of the targeted transformer-block linear weights. Target-module accounting includes scales and codebooks; whole-model accounting additionally includes embeddings, the LM head, and other unquantized tensors, counting tied weights once. GradCodeS retains no inference-time adapter. We apologize for being unable to provide inference memory and latency measurements under native 4-bit execution, as we lack access to hardware with native 4-bit support. We report effective bit widths and theoretical checkpoint sizes below.

Table S3. Effective bit widths and theoretical checkpoint sizes for Llama-3.2-1B.

MethodTarget bits/weightWhole-model bits/weightTheoretical checkpoint size (MiB)
BF16 reference16.00016.0002357.13
QLoRA (double quantization)4.1276.651979.86
QA-LoRA5.0008.4011237.69
GradCodeS, PV-Tuning, QZO, QuZO, QES, LoQT (FP32 scale)4.5006.9451023.13

Question 4: Additional Qwen and Llama experiments

We have added GSM8K comparisons on Llama-3.2-3B-Instruct and Qwen3-1.7B (Table S4), including Base-16, Base-4, 16-bit SFT (rank-8 LoRA on a 16-bit backbone), PV-Tuning, and GradCodeS.

Table S4. GSM8K results for Llama-3.2-3B-Instruct and Qwen3-1.7B.

BackboneMethodGSM8K test accuracy (%)
Llama-3.2-3B-InstructBase-1656.77
Llama-3.2-3B-InstructBase-448.73
Llama-3.2-3B-Instruct16-bit SFT (rank-8 LoRA)67.47
Llama-3.2-3B-InstructPV-Tuning63.40
Llama-3.2-3B-InstructGradCodeS67.23
Qwen3-1.7BBase-1667.87
Qwen3-1.7BBase-458.58
Qwen3-1.7B16-bit SFT (rank-8 LoRA)75.97
Qwen3-1.7BPV-Tuning71.65
Qwen3-1.7BGradCodeS75.34

GradCodeS improves over PV-Tuning by 3.83 points on Llama-3.2-3B and 3.69 points on Qwen3-1.7B. It is within 0.24 and 0.63 points of the corresponding 16-bit LoRA references while retaining fully low-bit target weights. The improvement therefore extends to both additional backbones.

Weakness 1: Scope of the theory and low-rank updates

We clarify the scope of the fixed-scale analysis and provide mini-batch and low-rank extensions below, which we will incorporate into the revised paper.

At fixed scales, the results explain complementary parts of the method: Lemma 1 motivates the code-space guide, Proposition 1 establishes its first-order consistency for local code moves, Corollary 1 accounts for finite-step curvature, and Proposition 2 quantifies the role of proposal coverage and candidate count.

  1. For Proposition 1: fixed scales and local moves. Fixing scales matches the code-search substep of the alternating optimizer and isolates the effect of discrete code changes. The one-level, gradient-aligned conditions establish the guide's local first-order consistency; actual candidates are accepted according to their evaluated loss. We will state this scope explicitly, with stochastic scale optimization handled separately, and add the low-rank extension below to account for moves outside these local conditions.

  2. For Proposition 2: full-batch selection and an empty favorable set. The full-objective assumption can be replaced by conditioning on the mini-batch used for selection. The original favorable set may be empty because of the discrete lattice geometry; retaining the current state still guarantees non-increase when qt=0q_t=0.

    We will add a direct mini-batch version. Let JBJ_{\mathcal B} be the evaluated mini-batch objective, including regularization. With the batch, current state, scales, regularization reference, and guide fixed, let qB(δ)q_{\mathcal B}(\delta) be the probability that one random factor proposal reduces JBJ_{\mathcal B} by at least δ>0\delta>0. Retaining the current state alongside M=14M=14 independent draws gives

    \le J_{\mathcal B}(Z,s)-\delta\bigl[1-(1-q_{\mathcal B}(\delta))^M\bigr].$$ Here $q_{\mathcal B}(\delta)$ measures actual improvement, so this extension applies to low-rank proposals without requiring membership in the original one-level favorable set. The additional nearest-reference candidate can only help, and non-increase holds even when $q_{\mathcal B}(\delta)=0$; a positive descent term remains conditional on positive improvement probability. As with mini-batch SGD, loss on an independent batch need not decrease at every step.
  3. For low-rank updates: multi-level and mixed-sign moves. We will add an extension covering any feasible candidate after clipping. For the same fixed mini-batch objective, let ΔZ\Delta Z denote its actual code change, write gW=∇W^JBg_W=\nabla_{\widehat W}J_{\mathcal B}, D=D(Z,s,gW)D=D(Z,s,g_W), gZ=gW⊙Dg_Z=g_W\odot D, and ΔW^=W^(Z+ΔZ,s)−W^(Z,s)\Delta\widehat W=\widehat W(Z+\Delta Z,s)-\widehat W(Z,s). If JBJ_{\mathcal B} is β\beta-smooth along the recovered-weight segment, then

    J_{\mathcal B}(Z+\Delta Z,s)-J_{\mathcal B}(Z,s) &\le \langle g_Z,\Delta Z\rangle_F+R(\Delta Z)+\tfrac{\beta}{2}\\|\Delta\widehat W\\|_F^2,\\\\ R(\Delta Z)&=\langle g_W,\Delta\widehat W-D\odot\Delta Z\rangle_F. \end{aligned}$$ The bound separates two sources of prediction error: extending the local codebook slope to the actual move, and loss curvature. The residual vanishes for Proposition 1's moves and satisfies $|R|\le\\|g_W\\|_F\\|\Delta\widehat W-D\odot\Delta Z\\|_F$ in general. Thus, any factor-induced candidate decreases the objective when its predicted gain exceeds this error bound and the curvature cost. This plays a similar role to the descent lemma in smooth SGD analysis: it explains when a proposed update yields a decrease ([Bottou et al., 2018, Section 4.1](https://arxiv.org/html/1606.04838v3)). These extensions concern the fixed-scale code-selection step, with scale optimization handled separately.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.