acceptodds
Withdrawn from ICLR 2027

Discrete Weight Updates for Extreme Low-Bit Training without Master Weights

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Memory consumption has become a major bottleneck for training and deploying increasingly large models. While extreme low-bit models with binary or ternary weights remarkably reduce the memory required for inference, their training still commonly relies on full-precision master weights that are optimized in a contin- uous space and quantized for the forward pass. Removing such master weights is challenging, as discrete weights can only change through transitions between a finite set of states and thus cannot directly represent the small continuous updates made by conventional optimizers. In this paper, we propose Discrete Updates for Extreme Low-bit Training (DUET), a scalable framework that directly trains ex- treme low-bit weights without maintaining master weights. We first derive a tran- sition budget from the magnitude of a continuous optimization step, specifying the expected number of discrete weight transitions. Given this transition budget, we select the weight transitions that maximize the alignment of the resulting discrete update with the continuous step. We further show that optimizer states used for these transition decisions can be significantly compressed while retaining training quality. Our experiments demonstrate that DUET maintains competitive training quality over long training horizons and on large models, with training memory reductions that increase with model size and reach 12.0× at 235B parameters compared to conventional training with master weights

Primary area:
foundation or frontier models, including LLMs
Keywords:
Extreme Low-Bit Training, Memory-Efficient Training, Master Weight Free Training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DyQKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates the training of LLMs with binary and ternary weighs without relying on full-precision master weights. The authors claim that prior work on extreme low-bit training still relies on STE with full-precision latent weights that accumulate small continuous updates, so training memory stays close to that of full-precision training. On the other hand, DUET updates the discrete weights directly, converting each Adam step into a per-output-channel transition budget that determines which weights flip, so the learning rate controls how many weights change instead of how much each weight changes. Moreover, they also propose memory-efficient optimizer states, i.e., a block-32 MXINT8 first moment with stochastic rounding, an Adafactor-style factored second moment, and AdamA-style direct gradient accumulation.

Critical strengths
  1. The paper is clearly written and the mathematical derivatiosn seems correct to me and well-motivated
  2. DUET keeps the forward pass and the binary/ternary weight format unchanged, so already released QAT checkpoints can be fine-tuned as they are
  3. The Experimental section is quite extensive. Moreover, the Appendix details most training settings, improving reproducibilty
Critical weaknesses
  1. The gains in memory attributed to the removal of master weights are mixed with optimizer-state compression. DUET uses a MXINT8 first moment, a factored second moment, and direct gradient accumulation, while the STE baseline in Table 2 seems to use 8-bit Adam but without the factored second moment and the direct gradient accumulation. Since Table 6 shows that STE works with compressed states, the authors should report STE with the same optimizer-state configuration to isolate the contribution of removing master weights alone.

  2. All memory measurements use batch size 1 and sequence length 256, a regime where activations are nearly negligible and weight and optimizer state dominate. This setting maximizes the reported reduction factors, but it is not representative of practical training. The authors should report peak memory at realistic batch sizes and sequence lengths (e.g., the global batch size of 128 used in their fine-tuning experiments and longer contexts) to show how the advantage shrinks as activation memory grows.

  3. The fine-tuning evaluation is hard to interpret since Table 1 does not report any row for the CAT-Q ternary checkpoint befor fine-tuning, so the contribution of DUET over the starting point is not clear. Moreover, ternary + LoRA uses a learning rate of 2×10−52 \times 10^{-5}, which I think it could be low for LoRA, and the STE full fine-tuning hyperparameters in Table 2 are not reported, so both baselines may be under-tuned.

  4. All the reported results are single runs without standard deviations computed on different seeds, so the small gains over baselines could be within evaluation noise. The authors should report experiments with different runs to have more solid results.

  5. There in no comparison with other works on direct optimization of discrete weights, even if thre Related Literture section discusses several, e.g., Bop (Helwegen et al., 2019), direct quantized training with stochastic rounding (Zhao et al., 2025) and ECO (Nikdan et al., 2026). The paper should compare against them in the pretraining setting of Table 7.

Questions for authors
  1. What are the peak memory requirements and accuracy of STE when run using the same MXINT8 first moment, factored second moment and direct gradient accumulation as DUET?
  2. How do the memory reductions change at realistic batch sizes and sequence lengths?
  3. Are the gains over STE and Ternary + LoRA statistically significant across seeds and how does DUET compare with Bop or stochastic-rounding direct training at 200M and 1B?
Reviewer_GVdTOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes DUET, a new algorithm for training low-bit neural nets (large models) without using the straight-through estimator approach of keeping a set of master-weights. At a high level, the approach seems to rely on computing an expected number of discrete weight transitions and assigning them based on maximizing alignment with what would have been obtained via a continuous step. Extensive numerics are presented to validate the approach.

Critical strengths
  • Training a network with extremely low-bit weights without keeping master-weights (as in the straight-through estimator) is an important problem, and progress there would be really helpful in producing high-quality models with low-inference costs and more reasonable training costs.
Critical weaknesses
  • The paper has parts that are very difficult to read, in critical sections, like the Methods section (section 2), and like the paragraph from lines 68-78. There are so many variables and parameters and subsection headings. Moreover, the writing is quite terse in many places, which tends to be a hallmark of AI editing/writing. It would be much better to present the main algorithm in an algorithm/algorithmic environment and describe it in more detail and more clearly, along with how it is motivated. And then, it would be good to clearly delineate the theoretical component of the paper, e.g., (9), while clearly identifying what it guarantees and doesn't guarantee.

I think the lack of clarity and organization makes the quality of the paper quite difficult to assess, despite the importance of the topic (and potentially the result). I don't think this is salvageable with the current writing/presentation quality.

Questions for authors

Suggestions: Besides the issues in the weaknesses section above, I noticed that the paper has a tendency, in the experiments section, to describe the tables in too much detail, e.g., Line 352-361 describe table 2 in unnecessary detail. There are many similar examples (e.g., Table 3...). All this space could have been much better used to explain the method more clearly and in more detail.

  • Table 2 also has DUET's peak memory in bold, while Ternary+LORA's is lower. This seems misleading to me.
  • A brief explanation of what "tequila setup" means would have been good.
  • At face value, it appears there are similarities between the proposed algorithm and ECO, but given the presentation, it was not clear to me what the similarities and differences are. Relatedly, why was there no comparison against ECO -- is ECO restricted to higher bit-widths? Why?
  • Am I missing something, or does Table 4 show Tequila outperforming DUET? In which case, why is DUET bolded?
Reviewer_r1VZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Extreme low-bit models are cheap at inference but still expensive to train, because QAT keeps full-precision master weights and pushes continuous updates through STE. DUET removes the master weights and updates the ternary or binary weights directly. A continuous Adam step is converted into a per-output-channel budget: the number of discrete transitions. Stochastic rounding turns the fractional budget into an integer count without changing the expectation. Under that budget, minimizing the expected squared error between the discrete and continuous update reduces to taking the top-k weights by update magnitude. After a weight transitions, its first moment is reset to zero so the same accumulated signal does not drive repeated flips. Optimizer states are compressed and gradients accumulate directly into them.

Critical strengths
  1. The motivation sounds good: Weights act on the model through activations, and each activation sums many weights, so a small change in activation can be produced by moving a few weights a full quantization interval rather than nudging all of them.
  2. momentum reset sounds interesting
Critical weaknesses
  1. The novelty seems limited. Top-k selection by update magnitude under a budget is a natural thing to arrive at once the budget view is adopted, and stochastic rounding of counts and Adafactor factorization are both existing tools.
  2. ECO is never compared against. For a paper whose central claim is training without master weights, this is the most important missing baseline, and the omission is not explained.
  3. The advantage reverses between 200M and 1B and the paper does not discuss it. Table 7 shows DUET ahead of STE at every size from 30M to 200M, by 0.0144 to 0.0533 loss. Figure 3 then shows the 1B Gemma run finishing at 3.047 versus 3.042 training and 3.034 versus 3.028 validation, so DUET is behind by roughly 0.005 to 0.006. The paper describes this as closely matching STE.
  4. The STE comparison is one model at one setting. Table 2 covers Qwen3-8B only. The 61.6% peak memory reduction is the headline number in the abstract, and it comes from a single fine-tuning run. Given that Figure 1 reports ratios from 3.3x to 12.0x across six models, having the direct STE comparison at only one scale is limited.
Questions for authors
  1. How does DUET compare against ECO on the same from-scratch ladder? This is the closest prior method and it is currently absent from every table.
  2. The sign of the loss gap versus STE flips between 200M and 1B. Do you have a hypothesis, and does it hold at 3B or 8B pretraining?
  3. What fraction of the 61.6% peak memory reduction comes from weights and optimizer states rather than activations? The current number is measured at batch 1, sequence 256.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.