acceptodds
Under review as a conference paper at ICLR 2027

TerMeZO: Ternary Sparse Zeroth-Order Optimization for Fine-tuning BitNet Models at the Edge

Abstract

Fine-tuning large language models (LLMs) with first-order optimizers requires a memory several times larger than that required for inference. Memory-efficient zeroth-order optimization (MeZO) sidesteps this cost by estimating gradients from forward passes only. However, for BitNet architectures, a family of LLMs with ternary -1,0,1 weights and 8-bit activations, fine-tuning requires updating full-precision latent weights, and thus the memory footprint of MeZO no longer matches that of inference. A promising solution is to fine-tune only a subset of the latent weights, but existing sparse zeroth-order (ZO) methods either ignore the ternary structure or require first-order gradient information to build a sparse mask, which is at odds with the purpose of ZO fine-tuning. We propose TerMeZO, a sparse MeZO scheme that exploits the geometry of the ternary quantizer itself to identify the latent weights that are more likely to change values during fine-tuning, at no additional data or memory cost. Our convergence analysis shows that TerMeZO can converge faster than full-parameter MeZO, owing to its optimized reduction of the fine-tuning effective dimension. We run extensive experiments on BitNet models ranging from 1B to 3B parameters, spanning classification, instruction-following, and mathematical reasoning tasks. TerMeZO matches or exceeds the performance of full-parameter MeZO while substantially reducing the fine-tuning memory footprint.

Primary area:
optimization
Keywords:
large language models, BitNet, fine-tuning, zeroth-order optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_mVRbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a memory-efficient fine-tuning method for BitNet models with ternary weights, targeting deployment on edge devices. It addresses the memory overhead of retaining full-precision latent weights during fine-tuning. The key idea is that latent weights close to the quantization boundaries are more likely to change their ternary values under perturbations. The method therefore prioritizes these weights for zeroth-order updates and progressively freezes weights that move away from the boundaries, reducing the number of full-precision parameters that must be stored. Latent weights outside the active set no longer require full-precision copies, further reducing memory requirements.

Critical strengths
  1. The parameter-selection criterion is directly tied to the ternary quantization mechanism, providing a clear rationale for selecting the active weights.
  2. Mask construction avoids the initial backpropagation overhead and can be performed offline.
  3. The method achieves substantial reductions in the fine-tuning memory footprint.
Critical weaknesses
  1. Permanent freezing may prematurely exclude useful parameters. As the model parameters and quantization thresholds evolve, weights excluded earlier may become useful later. However, the algorithm provides no mechanism for reactivating them.
  2. Evidence for practical edge deployment and efficiency remains incomplete. The paper does not sufficiently establish training latency, energy consumption, or overall benefits on real edge hardware. Zeroth-order optimization requires multiple forward passes, so a smaller memory footprint does not necessarily translate into shorter training time.
  3. Performance robustness under extreme sparsity is insufficiently evaluated. The main experiments use a trainable latent-weight fraction of ρ = 0.05, while ρ = 0.01 is discussed only in the memory-footprint analysis. The paper does not systematically characterize performance degradation at lower trainable fractions or provide trade-off curves relating sparsity, performance, and memory footprint.
  4. Generalization to larger models remains unverified. Evaluation is limited to models with 1B–3B parameters. Given the limited availability of larger open-source BitNet models, it remains unclear whether the findings extend to models with 7B parameters or more. Task coverage is also limited to classification, instruction following, and mathematical reasoning.
Questions for authors

Under the same trainable-parameter fraction, number of perturbations, and hyperparameter-tuning budget, how large and consistent are the gains from boundary-proximity selection over random selection? Do the improvements arise from selecting more useful weights, or primarily from reducing the number of trainable parameters and the variance of the zeroth-order gradient estimator? Does updating weights that are more likely to cross quantization boundaries necessarily lead to greater reductions in the task loss?

1 comment

W1 — No weight reactivation

This is a design choice. In fact, we considered allowing frozen weights to re-enter the active set, but this requires continually monitoring all latent weights for boundary proximity, reintroducing the cost freezing is meant to avoid. The one-way, shrinking mask is what preserves TerMeZO's memory savings. In practice, the initial fraction ρ0\rho_0 should be set by examining the latent-weight distribution around the decision boundaries, and selecting based on flip probability under perturbation.

W2 — Practical gains beyond memory on edge deployment

Our motivation is primarily about memory. We aim to make fine-tuning possible under tight memory budgets, so that fine-tuning a heavily quantized model costs roughly as much memory as running inference on it. As shown in Tables 2 and 4, the memory savings are large enough that TerMeZO can fine-tune a bigger and more accurate model using far less memory than BP. In particular, TerMeZO fine-tunes Falcon-E-3B with 1.3 GB of peak memory, while BP needs 16.1 GB for Falcon-E-1B.

A smaller memory footprint implies that fine-tuning can run on smaller, lower-power devices, which may bring energy benefits, although measuring them would require a hardware-specific evaluation. We deem this to be beyond the scope of this work. We have made these points clearer in the revised paper.

In terms of latency, we acknowledge that this is a limitation of the proposed scheme as well as all ZO-based methods. In this regard, we note that recent works on accelerating ZO optimization, such as adaptive update rules, apply directly to our method and could help close the latency gap with BP. ZO methods also work with non-differentiable objectives and RL-based fine-tuning (Qiu et al., 2025), which gives further reasons to use them.

W3 — Ablation on ρ0\rho_0

We have added two ablations that we agree were missing. The first, included in Figure 1, studies the effect of the fraction ρ0\rho_0 of latent weights included in the mask, with ρ0=0\rho_0=0 corresponding to fine-tuning only the full-precision parameters. The second, included in Table 5, compares fine-tuning only the ternary weights against fine-tuning only the full-precision weights (norm layers, embeddings, and the LM head).

Effect of ρ0\rho_0 on TerMeZO. GSM8K test accuracy (%) (Figure 1 in the revised paper).

ModelBaseρ0=0\rho_0 = 0ρ0=0.01\rho_0 = 0.01ρ0=0.025\rho_0 = 0.025ρ0=0.05\rho_0 = 0.05ρ0=1\rho_0 = 1
Falcon-E-1B14.022.722.930.331.832.6
Falcon-E-3B20.028.836.438.845.145.3

Ablation on Falcon-E-3B, GSM8K. Test accuracy (%) (Table 5 in the revised paper).

VariantAccuracy
Base model (no fine-tuning)20.0
MeZO for FP only (ρ0=0\rho_0 = 0)22.5
TerMeZO for ternary only (ρ0=0.05\rho_0 = 0.05)37.1
TerMeZO (ρ0=0.05\rho_0 = 0.05)45.1

W4 — Generalization to larger models

We agree with the reviewer that testing scalability is important. The larger publicly available ternary models (e.g., Bonsai) are released without their latent weights, so we could not evaluate them. Instead, we added experiments on Qwen3-8B with INT4 and FP4 weight quantization on PubMedQA, a biomedical task (Appendix H.3). These results provide preliminary evidence that TerMeZO extends to larger models and to other quantization schemes.

Test accuracy (%) on PubMedQA dataset for Qwen3-8B. The model weights are quantized to INT4 and FP4 before fine-tuning.

MethodINT4FP4
Zero-shot46.256.8
BP77.479.4
MeZO73.678.6
TerMeZO (ρ0=0.05)(\rho_0 = 0.05)74.276.9

Q1 — Boundary vs. random selection

We thank the reviewer for this question. We ran the suggested test, including S-MeZO with random selection under the same settings (ρ\rho, number of perturbations, tuning budget) using the Falcon-E-1B model on two datasets, GSM8K and MagiCoder. As shown in the table below, the proposed boundary-proximity selection consistently outperforms random selection at the same ρ\rho. We added this to the revised paper in Appendix H.2.

Falcon-E-1B fine-tuned with different fractions ρ0\rho_0 of updated ternary latents. GSM8K test accuracy (%) and HumanEval pass@1 (%) after MagiCoder fine-tuning.

MethodGSM8K (ρ0=0.025\rho_0 = 0.025)GSM8K (ρ0=0.05\rho_0 = 0.05)HumanEval (ρ0=0.025\rho_0 = 0.025)HumanEval (ρ0=0.05\rho_0 = 0.05)
S-MeZO (random)24.322.913.418.9
TerMeZO30.331.821.324.4
Reviewer_CHueOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This work introduces a new zero-order optimization strategy for finetuning the BitNet architecture, taking advantage of architectural details in order to make their strategy much more memory-efficient without a significant sacrifice to performance, and in some cases, better performance than the original baseline zero-order optimizer. Their main insight is that some latent weights whose values are close to quantization boundaries can be exploited as prime targets for optimization, since they are more likely to change the quantized network. While the results show strong empirical performance of finetuned BitNet models as compared to similar other optimizer baselines, faster convergence is not empirically demonstrated despite a whole section dedicated to theoretical convergence analysis.

Critical strengths
  • The paper addresses finetuning in a low-memory regime which is of great interest to the open source community
  • The paper takes advantage of a novel insight of the existing BitNet architecture to motivate and design their optimizer
  • The method performs strongly and on-par with baselines despite using less invasive information of the target network and much less memory
  • The paper is generally well written, although the entire appendix is missing on my end, making it hard to verify claims in the paper. Explanations of prior and related work are thorough.
Critical weaknesses
  • The paper provides a convergence analysis section, but does not explore convergence empirically. With regards to the writing, I encourage the authors to begin this section with a simple explanation of what is being explored and why the reader should care about this analysis. And then, to the extent possible, it would be nice to see loss curves empirically. For example, do we see faster convergence in practice owing to the fewer parameters being optimized over steps? Lastly, which is easiest to fix, the appendix should be attached.

  • This method is biased towards a single architecture: BitNets. Therefore, the novelty of this work is narrow and impact is contingent on the continued use of this single architecture, particularly with the architecture definitions assumed by this work.

  • While these benchmarks were relevant two years ago, they don't necessary reflect how far LLMs have come today, nor does SFT on these benchmarks reflect the full spectrum of situations in which they are trained. A simple example is GRPO RL. As the paper is proposing an optimizer, it is imperative to have a robust evaluation setting that reflects the modern usage of these models. While I am not necessarily suggesting RL as an evaluation, might the authors consider more modern BitNets (such as Bonsai family) and more recent SFT dataset tasks that are not saturated?

Questions for authors
  • I do not see ablations present. Were the introduced hyperparameters ablated?

  • Does the proposed optimizer empirically scale to larger BitNets such as those in the Bonsai family?

4 comments

W1 — Empirical convergence evidence

We thank the reviewer for this suggestion and agree it improves readability. We have added an introductory paragraph at the start of Section 4, motivating the analysis and stating its objectives before the formal results are presented.

To expose the effect of the number of optimized weights on convergence, we have added a new empirical result in Appendix H.1 where we freeze full-precision parameters, so that both methods update only ternary latent weights. TerMeZO is seen to converge faster than dense MeZO.

W2 — Restriction to BitNet

Our approach applies to ternary models in general, not only to BitNet-style architectures in which some parameters remain in full-precision. We focus on ternary quantization because it provides a clean setting for analysis, but the selection criterion extends naturally to other low-bit schemes such as INT4 and FP4. In fact, a latent weight changes its quantized value only when it crosses the decision boundary between two adjacent quantization levels, and our scheme selects the latent weights closest to these boundaries, exactly as in the ternary case.

We leave a thorough empirical study of this generalization to future work, but we provide preliminary results supporting it here. Specifically, we fine-tune an 8B-parameter model, Qwen3-8B, whose weights are first quantized with GPTQ to either INT4 or FP4. The table below reports test accuracy on the PubMedQA biomedical question-answering benchmark after fine-tuning the quantized model with BP, full-parameter MeZO, and TerMeZO. TerMeZO matches the performance of full-parameter MeZO, and both reach accuracy close to that of BP.

Test accuracy (%) on PubMedQA dataset for Qwen3-8B. The model weights are quantized to INT4 and FP4 before fine-tuning.

MethodINT4FP4
Zero-shot46.256.8
BP77.479.4
MeZO73.678.6
TerMeZO (ρ0=0.05\rho_0=0.05)74.276.9

W3 & Q2 — Other benchmarks and scale

We thank the reviewer for this suggestion. We note that several of our evaluation tasks are specifically chosen for their resistance to saturation at this model scale. Specifically, Countdown (used in our RL-based fine-tuning experiments, Appendix D) remains challenging for models under 8B parameters and has been used for this reason in recent zeroth-order fine-tuning works (Qiu et al., 2025; Xu et al., 2026). Similarly, HumanEval pass@1 after SFT on Magicoder-Evol-Instruct, while reported as saturated for frontier full-precision models (94–99% pass@1), sits at 43–49% after fine-tuning for the considered ternary-quantized models. On the considered tasks, fine-tuning yields a substantial improvement over the base model, which indicates the model has clear headroom to improve, and this is the right setting for comparing the different fine-tuning schemes.

We also highlight that we have applied TerMeZO to RL-based fine-tuning, following recent work on evolution strategies for fine-tuning (see Appendix D). In this experiment on the Countdown task, the model improves from 9% to approximately 60% accuracy after fine-tuning. Moreover, under identical settings and hyperparameters, TerMeZO converges faster than dense MeZO (Figure 3).

Since the latent weights of the BONSAI BitNet models are not publicly available, we could not include experiments on these models. Instead, we added experiments on a Qwen 8B model with INT4 and FP4 weight quantization (see Appendix H.3). These results provide preliminary evidence that TerMeZO extends to larger models and to other aggressive quantization schemes.

Q1 — Hyperparameter ablations

We have added two ablations that we agree were missing. The first studies the effect of ρ0\rho_0, the fraction of latent weights included in the mask, with ρ0=0\rho_0=0 corresponding to fine-tuning only the full-precision parameters. The second compares fine-tuning only the ternary weights against fine-tuning only the full-precision weights (embeddings, norm layers, and the LM head).

Effect of ρ0\rho_0 on TerMeZO. GSM8K test accuracy (%) (Figure 1 in the revised paper).

ModelBaseρ0=0\rho_0 = 0ρ0=0.01\rho_0 = 0.01ρ0=0.025\rho_0 = 0.025ρ0=0.05\rho_0 = 0.05ρ0=1\rho_0 = 1
Falcon-E-1B14.022.722.930.331.832.6
Falcon-E-3B20.028.836.438.845.145.3

Ablation on Falcon-E-3B, GSM8K. Test accuracy (%) (Table 5 in the revised paper).

VariantAccuracy
Base model (no fine-tuning)20.0
MeZO for FP only (ρ0=0\rho_0 = 0)22.5
TerMeZO for ternary only (ρ0=0.05\rho_0 = 0.05)37.1
TerMeZO (ρ0=0.05\rho_0 = 0.05)45.1

I thank the author for their response. The connection to ternary quantization for more general architectures is quite interesting and I encourage the authors to include this in the writing. I think a broader audience would find it useful. Could the authors kindly explain their figure 5 FIM-MeZO results more in-depth, for which this baseline seems to perform well?

We thank the reviewer for the follow-up. Following the suggestion, we have revised the text to clarify that the proposed approach applies to ternary models in general and extends naturally to other aggressive quantization schemes.

Regarding Figure 5, we have expanded the discussion of the results in Appendix H.5. Although FIM-S-MeZO attains a slightly lower validation loss, this does not translate into higher accuracy, as both methods perform comparably on GSM8K (Table 4). Moreover, FIM-S-MeZO diverges at the largest learning rate, whereas TerMeZO remains stable throughout and continues to improve as the learning rate increases. Finally, FIM-S-MeZO selects weights based on their Fisher information, which requires backpropagation before fine-tuning. In contrast, TerMeZO selects weights based solely on their distance to the quantization boundaries, keeping the entire pipeline gradient-free and memory-efficient.

Thank you. All of my concerns are addressed and I raise my score.

Reviewer_qhrdOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors present TerMeZO, a memory-efficient zeroth-order fine-tuning method tailored for ternary neural networks quantized by BitNet. Specifically, TerMeZO selects a small fraction of model weights based on their sensitivity to quantization boundaries, jointly updates the selected weights and unquantized parameters at each iteration, and gradually shrinks the candidate subset during training. Theoretical analysis shows that TerMeZO converges better and has a lower memory footprint than its full-precision counterpart (i.e., MeZO). The experimental results across various model families and tasks also support better performance at lower memory cost.

Critical strengths
  1. The writing is generally clear and coherent, and easy to follow.
  2. The TerMeZO method is elegant and shows competitive performance and memory efficiency.
Critical weaknesses
  1. Missing ablation studies. The fraction ratio ρ0\rho_0 is a critical hyperparameter for TerMeZO, as it determines the initial search space of trainable weights and overall memory efficiency. However, the ablation studies on its effect on final performance and memory footprint were missing. The authors did not explain why they fixed its value to 0.050.05, or whether the optimal choice varies across model families or task types.
  2. Missing comparison baseline. There are two subsets of weights set to be trainable in TerMeZO: (i) ternary weights selected by sensitivity w.r.t. quantization boundaries, and (ii) unquantized parameters such as embedding layers and the LM head. The paper should include comparison baselines showing how each improves final performance while keeping the other fixed, since the methodology primarily focuses on the former.

I also have some minor suggestions for this manuscript:

  • Some terms can be refined to improve clarity; e.g., you may use 'parameter vector' instead of ' argument vector' (on page 5, line 233) to denote the flattened model weights.
  • Notations can also be made more rigorous; for example, the square brackets around the loss function in Eq. 7 are unnecessary.

These typo-level issues do not lead to substantive concerns. The authors do not need to formally respond to them during the review and discussion, but I hope they can polish the writing and presentation and fix these issues in their revisions.

Questions for authors

Based on the weaknesses, I hope the authors can answer the following questions to address my major concerns.

  1. Could you provide experimental results showing how the initial fraction ratio ρ0\rho_0 affects performance, and whether it is sensitive to model architectures or downstream tasks?
  2. Could you provide experimental evidence highlighting that the selected ternary weights contribute more to final performance than other unquantized parameters, so TerMeZO's effectiveness is shown to primarily come from the former?
3 comments

W1 & Q1 — Ablation on ρ0\rho_0

We agree with the reviewer and have added new ablation results on two datasets using a 3B BitNet model, varying the fraction ρ0\rho_0 of optimized ternary weights. As shown in Figure 1 in the revised version, performance improves with ρ0\rho_0 in the small-ρ0\rho_0 regime, but saturates beyond a certain value. The numbers from Figure 1 are reported in the table below for convenience.

Effect of ρ0\rho_0 on TerMeZO. GSM8K test accuracy (%). These results are shown as Figure 1 in the revised manuscript.

ModelBaseρ0=0\rho_0 = 0ρ0=0.01\rho_0 = 0.01ρ0=0.025\rho_0 = 0.025ρ0=0.05\rho_0 = 0.05ρ0=1\rho_0 = 1
Falcon-E-1B14.022.722.930.331.832.6
Falcon-E-3B20.028.836.438.845.145.3

In practice, the optimal choice of ρ0\rho_0 depends on the model size and on how the latent weights are distributed around the decision boundaries. For a fixed, small perturbation scale ϵ\epsilon, the probability that a weight flips under the perturbation distribution decays with its distance to the decision boundary. The parameter ρ0\rho_0 can therefore be set to the fraction of weights for which this flip probability is non-negligible.

W2 & Q2 — Ternary vs. full-precision contributions

We agree that these ablation studies are useful to support the claim that optimizing the ternary weights is crucial. As shown in the table below (Table 5 in the revised paper), we ablate the two components of the TerMeZO update on Falcon-E-3B (GSM8K). In one variant, we update only the full-precision parameters using MeZO, with the ternary weights frozen (ρ0=0\rho_0=0). In the other, we update only the selected ternary weights using TerMeZO, with the full-precision parameters frozen. Updating the ternary weights alone outperforms optimizing only the full-precision parameters, and optimizing both jointly yields the best performance. This shows that, for certain tasks, the ternary updates are essential to TerMeZO's gains, which cannot be recovered by tuning the full-precision parameters alone.

Ablation on Falcon-E-3B, GSM8K. Test accuracy (%).

VariantAccuracy
Base model (no fine-tuning)20.0
MeZO for FP only (ρ0=0\rho_0 = 0)22.5
TerMeZO for ternary only (ρ0=0.05\rho_0 = 0.05)37.1
TerMeZO (ρ0=0.05\rho_0 = 0.05)45.1

Thanks for the detailed response. My primary concerns are resolved, and the research work looks more complete. As a result, I would like to maintain my positive score and finalize the recommendation as a 'Weak Accept'.

We thank the reviewer for the follow-up and for the constructive suggestions throughout the review process. We are glad that the revisions have addressed the reviewer's concerns.

Reviewer_e5a9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TerMeZO is a memory-efficient zeroth-order fine-tuning method for ternary BitNet models. It selects latent weights near quantization boundaries, retains floating-point copies only for active weights, and progressively shrinks the active set. The paper provides a convergence analysis and experiments on 1B to 3B models across classification, code generation, and mathematical reasoning tasks, showing performance close to full-parameter MeZO with substantially lower memory usage.

Critical strengths

1.The selection criterion is simple and well suited to ternary quantization: weights near quantization boundaries are more likely to change their quantized values under perturbation. Mask construction requires neither first-order gradients nor additional data. 2.TerMeZO achieves performance close to full-parameter MeZO across multiple models and tasks with substantially lower memory usage.

Critical weaknesses

Freezing parameters reduces the optimization dimension but also restricts the available update directions. If the frozen parameters include directions that would help reduce the loss, reaching the same target accuracy may require more iterations or may no longer be possible. Fewer trainable parameters therefore do not necessarily imply faster convergence.

Questions for authors

Could the authors clarify how the boundary-based selection criterion and the assumptions in Theorem 1 ensure that enough useful update directions are retained for the benefit of reduced dimensionality to outweigh this restriction, leading to faster convergence than full-parameter MeZO?

1 comment

W1 & Q1 - Restricted update directions.

TerMeZO, like sparse ZO optimization more broadly, relies on the assumption that the selected weights carry most of the gradient-relevant information for fine-tuning this class of models. We formalized this assumption in Theorem 1, and verified it empirically (Figure 2).

As further validation, TerMeZO's performance compared to full-parameter MeZO (Tables 3 and 4) shows that for heavily quantized models, standard MeZO updates are largely noise for weights far from the decision boundary. In fact, for a fixed, small perturbation scale ϵ\epsilon, the probability that such a weight changes under a Gaussian perturbation decays exponentially with its distance to the quantization boundary. In this regard, a large ϵ\epsilon is not a viable fix either, as it would flip a large fraction of the quantized weights, yielding highly noisy gradient estimates. In a manner consistent with these observations, the proposed boundary-based selection criterion targets the weights for which ϵ\epsilon-scale perturbations are informative.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Revisions

We thank all reviewers for their constructive feedback, which helped us strengthen the manuscript. Reviewers found the problem well-motivated and the approach sound, and their main requests concerned missing ablations, motivation & empirical support for the convergence analysis, and evidence beyond 1B–3B BitNet models. We have addressed these points in the revised paper. The main changes are summarized below.

  • Ablation on ρ0\rho_0. In the revised draft, we report accuracy as a function of the fraction ρ0\rho_0 of updated ternary weights on Falcon-E-1B and Falcon-E-3B (Figure 1). Performance improves quickly for small ρ0\rho_0 and saturates beyond a certain value (Reviewers qhrd, CHue, mVRb).

  • Ternary vs. full-precision updates. We added an ablation that compares updating only the full-precision parameters with updating only the ternary weights (Table 5). Updating only the selected ternary weights clearly outperforms optimizing only the full-precision parameters, and updating both jointly (TerMeZO) gives the best results. This shows that the ternary updates are essential to TerMeZO's gains (Reviewers qhrd, CHue).

  • Boundary-proximity vs. random selection. We added a comparison with a baseline that randomly selects the ternary weights to be optimized (Appendix H.2). Under the same settings, our selection rule consistently outperforms random sparse selection (Reviewer mVRb).

  • Motivation and empirical evidence for the convergence analysis. We added an introductory paragraph to Section 4 that motivates the analysis, and a new experiment (Appendix H.1) showing that TerMeZO converges faster than dense MeZO when both update only ternary latent weights (Reviewer CHue).

  • Larger models and other quantization schemes. Since the latent weights of larger ternary models such as Bonsai are not publicly available, we added experiments on Qwen3-8B quantized to INT4 and FP4 (Appendix H.3). TerMeZO performs comparably to full-parameter MeZO and close to BP. These experiments provide preliminary evidence that TerMeZO scales to larger models and to other quantization schemes (Reviewers CHue, mVRb).

  • Why boundary-based selection retains the useful update directions. We clarified that, for heavily quantized models, MeZO updates are largely noise for latent weights far from the decision boundaries, since a small perturbation is very unlikely to change their quantized value. The boundary-based criterion therefore keeps the weights whose perturbations actually affect the quantized forward pass, which explains why restricting the update directions works well (Reviewer e5a9).

All changes in the revised manuscript are marked in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.