acceptodds
Under review as a conference paper at ICLR 2027

Towards Dense Capability Recovery of Ternary Quantized LLMs

Abstract

Ternary quantization offers extreme compression, but the resulting perturbation is substantially more severe than conventional low-bit quantization. The key challenge is not merely reducing one-shot projection error, but finding a ternary representation that remains efficiently recoverable through subsequent training. We demonstrate that under matched knowledge distillation (KD), quantizers with vastly different initial errors converge to remarkably similar functional endpoints. Motivated by this, we introduce a calibration-free, energy-preserving quantizer as a stable starting point designed specifically to facilitate subsequent training. However, same-size KD restores broad post-trained capability while leaving substantial and highly nonuniform performance gaps across tasks. To address this, we first exploit test-time sampling to expose substantial residual headroom on several reasoning and coding tasks. We then apply quantization-aware reinforcement learning (RL) directly to the ternary policy, converting part of this headroom into stronger single-shot performance. We evaluate our approach across several model sizes: Qwen3-1.7B, 4B, and 8B. The first math-RL stage substantially improves single-shot performance, while broader gains transfer across knowledge, reasoning, instruction following, mathematics, code, and the unoptimized thinking mode. At 8B, the final 2.03-GB model achieves 99.3% mean benchmark-wise retention relative to BF16 in non-thinking mode and 94.1% in thinking mode, while decoding at 77.9 tokens/s on an M5 Pro. Together, these results support a recovery-centric view of extreme quantization: preserve a recoverable ternary representation, restore the model distribution, and then optimize the deployed ternary policy directly.

Primary area:
optimization
Keywords:
ternary quantization, low-bit LLM compression, knowledge distillation, calibration-free quantization, post-training reinforcement learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_5HEbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a recovery pipeline for fully ternary LLMs converted from pretrained dense models. The method combines hard-ternary knowledge distillation with subsequent math and multi-domain reinforcement learning, using repeated sampling to identify residual capability that remains reachable but unreliable. Experiments on Qwen3 models from 1.7B to 8B report strong cross-domain capability recovery and substantial deployment benefits.

Critical strengths

1.A simple and reasonable recovery pipeline.

The combination of hard-ternary KD and direct ternary RL is conceptually simple and well motivated. KD first restores the teacher distribution, while RL improves the reliability of behaviors that remain reachable but are not consistently selected.

2.Exceptionally strong cross-domain results.

The reported ternary models perform well across knowledge, reasoning, instruction following, mathematics, and code, with consistent improvements over Bonsai at the same model scales.

Critical weaknesses

1.The paper is poorly organized.

Method definitions, experimental findings, implementation details, and failure analyses are interleaved, and no overview figure summarizes the complete KD-to-RL recovery pipeline.

The Energy quantizer receives substantial attention in the main text, although the matched-recovery ablation suggests that final performance is driven primarily by the large-scale recovery procedure rather than by the specific initialization quantizer. Much of this discussion could therefore be shortened or moved to the appendix.

Basic notation such as “KD E3” is also used without an explicit definition, and important training settings are scattered across the main text and appendix.

The Multi-RL stage, which produces the strongest cross-domain results, is not described systematically in terms of its training-data mixture, domain-specific reward functions, and training budget across model sizes.

The paper would benefit from substantial restructuring around the KD–RL recovery pipeline, with greater emphasis on data construction, prompt filtering, training procedures, and the engineering insights required to optimize and deploy a hard-ternary model.

2.The central empirical claims cannot be independently verified without a final materialized ternary checkpoint.

The reported cross-domain recovery is unusually strong for fully ternary models. However, without access to a final materialized ternary checkpoint and minimal verification code, it is difficult to confirm that evaluation uses only the deployed ternary weights, that all intended modules are quantized as described, and that the reported results can be reproduced from the final checkpoint.

The paper reports several internal integrity checks, but directly inspecting and evaluating the deployed model would provide substantially stronger evidence. The authors should therefore release at least one final materialized ternary checkpoint together with minimal code for verifying the ternary grid and reproducing representative evaluation results.

Questions for authors

Could the authors release at least one final materialized ternary checkpoint together with minimal verification and evaluation code? The provided code should confirm that evaluation uses only the deployed ternary weights rather than the latent BF16 parameters and reproduce representative cross-domain results.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 5HEb

We thank the reviewer for recognizing the strength of the cross-domain results, the 8B improvement over Bonsai, and the motivation behind our recovery pipeline.

Section 1 introduces three research questions. How much does the choice of quantizer matter after recovery? How much capability can same-size KD restore? Can direct hard-ternary RL improve behaviors that remain reachable after KD but unreliable on a single attempt? These questions motivate the sequence:

Ternarize → recover the distribution → identify the remaining failure → recover the policy.

The training methodology is documented in the submission. Appendix E.5–E.7 and Tables 10–13 specify the KD datasets, mixture, curriculum, optimization settings, and training budget. Appendix F.2–F.6 covers actor–rollout synchronization, RL configurations, prompt filtering, verifier checks, and stage transitions.

We agree that the reader currently has to move between sections to reconstruct the procedure. In the revision, we will use the additional page for an overview figure connecting the training stages to their evaluations. We will shorten the main-text quantizer discussion, bring the Multi-RL data, rewards, and training settings together, and define KD E3 at first use as the checkpoint after the third distillation epoch.

AuthorsOpenReview ↗

Multi-domain RL — Data, rewards, training, and checkpoint selection

Thank you for asking us to bring the Multi-RL details together. Section 5 and Appendix F describe the staged recovery procedure and its optimization safeguards. We summarize the Multi-RL setup for the three model sizes below.

Training setup. Multi-RL uses GRPO initialized from the Math-RL checkpoint. We use a learning rate of 5×10−55 \times 10^{-5}, 2% warmup, weight decay of 0.01, eight rollouts per prompt, a batch size of 512, a mini-batch size of 128, and a maximum response length of 2,048 tokens. The objective has no explicit KL penalty or entropy bonus.

Data and rewards. The training mixture contains 25,427 prompts across five components.

ComponentReward verification
KodCodeUnit tests
Factual QAExact match with accepted aliases
Instruction followingConstraint checker
MathematicsBoxed-answer correctness

The mixture contains no tool-calling data. Tool use remains a limitation of the resulting policy.

Prompt selection. We calibrate difficulty against each model size’s current ternary policy using the rollout temperature and response budget used in training. This identifies prompts that produce useful reward variation for that checkpoint. We check for validation overlap, filter degenerate outputs, and examine overlap with the preceding stage.

Verification and checkpoint selection. We select checkpoints using held-out performance across the trained domains. Appendix F.4 illustrates why this matters: in one run, training reward increased while held-out accuracy fell by 13 percentage points. Appendix F.5 documents a verifier that incorrectly rewarded one-character answers. Appendix F.1 reports a separate failure mode in which a learning rate of 10−610^{-6} produced no observed ternary-code movement. These observations motivated three checks during recovery: validate the reward function, monitor held-out performance, and verify that optimizer updates change the effective ternary weights.

We will present the Multi-RL configuration and data mixture together in the revision so readers can connect these choices to the final cross-domain results.

AuthorsOpenReview ↗

Materialized checkpoints and independent verification

We understand the request for a final materialized checkpoint and minimal verification code. We are unable to share the weights during this discussion and aim to release them as research artifacts alongside the paper. We recognize that this limits reviewers’ ability to rerun the final model. Below we clarify the evidence in the submission for the execution path used in our experiments.

Quantizer coverage. Appendix A.1 specifies which parameters are ternarized. All compute-dominant attention and feed-forward weight matrices are ternarized in groups of 128. The embedding and output matrices are also ternary. Normalization parameters and small scalar parameters retain their native precision.

Weights used during training and evaluation. Appendix F.2 distinguishes three representations:

  • Latent weights are the BF16 parameters that accumulate optimizer updates through the straight-through estimator.
  • Effective weights are the hard-ternary weights used in every actor forward pass.
  • Baked weights are the materialized ternary weights exported to rollout workers and the inference runtime.

During synchronization, rollout workers receive the baked weights. The BF16 latent parameters remain on the optimization path. We check quantizer coverage, re-quantization idempotency, agreement between actor and rollout policies, and ternary-grid integrity after merging.

Inspection of the materialized 8B checkpoints. In the backbone tensors we examined, every 128-weight group had a single nonzero magnitude and at most three distinct values, consistent with a groupwise grid {−s,0,+s}\{-s,0,+s\}. The measured nonzero density ranged from 0.607 to 0.623. The inspection covered all attention layers and the output head. This inspection was also part of the each evaluation for each ternary model.

The coverage checks establish which modules were ternarized, the export procedure identifies the weights supplied to the runtime, and the tensor inspection checks the grid in the materialized checkpoints. We hope this addresses the specific execution concerns while making clear what remains unavailable for independent evaluation during discussion and request you to reconsider the score.

Thank you for your time and effort in reviewing our paper.

Reviewer_PRBVOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to recover the capabilities of post-trained LLMs after ternary quantization. Ternary quantization severely degrades pretrained models, and how to recover their broad post-trained capabilities remains unclear. To tackle this problem, the authors propose a pipeline that combines an energy-preserving ternary quantizer, full-vocabulary knowledge distillation, and reinforcement learning applied directly to the ternary model. Experimental results on Qwen3 models show that the 8B model achieves 99.2% retention relative to BF16 on 11 core benchmarks with a size of only 2.03 GB.

Critical strengths
  • This paper tackles an important problem of low-bit quantization
  • The paper carefully engineers the training pipeline from KD to direct hard-ternary RL, and is knowledgeable about how to train ternary quantized LLMs stably in practice, including detailed configurations, integrity checks, and documented failure cases.
Critical weaknesses

The main conclusions depend on the evaluation design.

The paper’s central results, namely 99.2% retention at 8B and the claim that a larger ternary model outperforms Q4 under a matched memory budget, rely on excluding tool use, agentic tasks, and long-horizon reasoning (AIME, LiveCodeBench) from the averages and retention calculation. When the averages are recomputed over all benchmarks reported in the appendix, the advantage over memory-matched Q4 baselines largely disappears. Given that training is limited to short sequences and non-thinking RL, the current evaluation effectively measures recovery on the tasks that were recovered, and the authors should report aggregates over the full benchmark suite as a primary result I think the current results do not sufficiently demonstrate that the proposed pipeline improves the accuracy–efficiency trade-off over simply using a smaller model with standard 4-bit quantization.

Important baselines are missing.

The only low-bit baseline in the paper is Q4_0, which is the simplest 4-bit quantization, and the comparison set lacks several baselines needed to evaluate the method. For example, stronger 4-bit quantization that can be obtained at little or no additional cost, such as GPTQ, AWQ, or more recent quantization approaches tailored for reasoning, is not included. Without these baselines, it is difficult to judge whether the substantial training cost of the proposed pipeline is justified relative to cheaper alternatives.

Questions for authors

See weakness

4 comments
AuthorsOpenReview ↗

### Response to Reviewer PRBV

We thank the reviewer for recognizing the importance of the problem and appreciating the detailed configurations, integrity checks, and documented failure cases.

Response to concern 1 — Evaluation design and the memory-matched comparison

In §1, before the contributions, we explicitly state the research scope:

Using this recipe, we recover broad capability across knowledge, reasoning, mathematics, code generation and instruction following, while retaining the memory and runtime advantages of ternary weights. We additionally report tool-use and long-horizon agentic evaluations as stress tests, but do not treat them as recovered capabilities; they remain an important direction for future work.

Table 3 and Figure 1 state this scope, and §7.2 explicitly discusses where memory-matched Q4 is better. We will make the abstract equally explicit by adding “across 11 core benchmarks” and a sentence identifying agentic and long-horizon recovery as limitations.

The 11 core benchmarks span knowledge, reasoning, instruction following, mathematics, and code. Recovery extends beyond the behaviors specifically optimized during training. As noted in §4.2, code-reasoning data accounts for less than 0.4% of the KD tokens, and the non-thinking RL stages also improve performance in the unoptimized thinking mode. Across all 19 benchmarks, the 8B non-thinking mean improves from 48.42 after KD to 49.41 after Math RL and 50.99 after Multi-RL.

To address your deployment question directly, we recomputed the mean across all 19 benchmarks, including tool use, agentic interaction, AIME25/26, and LiveCodeBench.

Model (non-thinking)Core 11 meanAll 19 meanDisk (GB)RAM (GB)Decode (tokens/s)
Qwen3-4B Q4_K_M-imat66.6852.802.332.7470.4
Our 8B Multi-RL67.5350.992.032.4577.9

Our model scores 1.81 points lower across all 19 tasks, while using 10.6% less RAM and decoding 10.7% faster under the measured configuration.

The accuracy–efficiency trade-off is central to our evaluation because ternarization affects runtime memory and inference speed as well as serialized model size. Our results show where the recovered ternary model offers a useful deployment advantage and where Q4 remains preferable and are reserved for future work.

AuthorsOpenReview ↗

### Response to concern 2 — Baselines and recovery cost

We would like to clarify the comparisons, necessity for the high training budgets and present the additional baseline evaluated during rebuttal.

Ternary baselines. Table 3 contains Bonsai and CAT-Q at 1.7B, 4B, and 8B. These represent an existing recovered ternary model and an alternative ternary conversion approach.

At 8B, our Multi-RL checkpoint achieves a non-thinking core-suite mean of 67.53, compared with 63.9 for Bonsai. This advantage also holds across all 19 benchmarks, with scores of 50.99 for our model and 49.56 for Bonsai. The corresponding checkpoint sizes are 2.03 GB and 2.15 GB.

Four-bit comparison. Thanks for raising this, it helped us identify an incorrect label in §6. The evaluated four-bit baseline in the paper uses importance-matrix-calibrated Q4_K_M. We will correct the Q4_0 label to Q4_K_M-imat. We calibrated the importance matrix on 102,400 tokens from an instruction-following corpus (200×512200 \times 512 tokens). While this comparison does not cover every available four-bit method, it provides a calibrated baseline for the deployment comparison.

Additional low-bit baseline evaluated during rebuttal. We also evaluated Unsloth’s UD-IQ2_XXS quantization of Qwen3 at 1.7B, 4B, and 8B. This provides a comparison closer to the extreme low-bit setting studied in our work, without the substantial KD and RL recovery budget.

We evaluated all three sizes in thinking and non-thinking modes across the 19-benchmark suite. The results are compared with our final Multi-RL checkpoints below.

ModelCore 11 (non-thinking)All 19 (non-thinking)Core 11 (thinking)All 19 (thinking)
1.7B Unsloth UD-IQ2_XXS10.729.157.036.38
1.7B Ours Multi-RL52.2535.0855.7538.17
4B Unsloth UD-IQ2_XXS11.5510.007.706.77
4B Ours Multi-RL60.7141.0464.8445.30
8B Unsloth UD-IQ2_XXS46.0834.3835.4027.75
8B Ours Multi-RL67.5350.9971.1656.75

Our recovered hard-ternary models achieve substantially higher means at all three scales and in both decoding modes. We will include the complete per-benchmark results and checkpoint configurations in the revised appendix.

Training cost.

The training cost is substantial. ParetoQ reports that 3–4-bit quantization-aware fine-tuning approaches saturation around 10B tokens, while ternary and other lower-bit models approach saturation around 30B tokens in their experiments.

Our work builds on this observation. Our objective is to recover as much of the dense model’s capability as possible while keeping the forward pass and resulting materialized model hard-ternary. We do not claim that this is cheaper than conventional PTQ. We demonstrate what becomes possible through substantial recovery training and where the remaining limits lie.

Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work. Given the full-suite results, the clarified four-bit comparison, and the additional low-bit evaluations, we would appreciate reconsideration of the assessment and score.

Thank you for your time and effort in reviewing our paper.

Thank you for the response and new experiments. However, my concern about the accuracy–efficiency trade-off remains. On all 19 benchmarks, the 8B ternary model scores 1.81 points below the memory-matched 4B Q4_K_M-imat baseline. The stronger 4-bit baselines I requested were not added, and the training-free UD-IQ2_XXS comparison does not show that the recovery cost is justified over cheaper 4-bit alternatives. I will therefore keep my score.

AuthorsOpenReview ↗

Response to Reviewer PRBV

Thank you for the response.

On the core 11 benchmarks used in Figure 1, the recovered 8B ternary model improves on the 4B Q4 operating point: it scores higher (67.53 vs. 66.68), has a smaller serialized footprint (2.03 vs. 2.33 GB), and decodes faster on the M5 Pro (77.9 vs. 70.4 tokens/s). Across all 19 benchmarks, 4B Q4 leads by 1.81 points (52.80 vs. 50.99). The ternary model retains its size and throughput advantages, so the preferred model depends on the target workload and deployment budget, as discussed in §7.2. We understand your concern about whether these gains justify the recovery cost relative to stronger four-bit alternatives.

The paper also examines how quantizer choice affects recovery, how much capability KD restores, and what direct hard-ternary RL adds. These findings can be assessed independently of the comparison with 4B Q4. Do you have any remaining concerns about the evidence for these recovery findings?

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to Reviewer 3aD6

We thank the reviewer for recognizing the direct hard-ternary RL results and the practical deployment benefits.

The paper is organized around three research questions introduced in §1, following the sequence:

Ternarize → recover the distribution → identify the remaining failure → recover the policy.

Section 2 credits prior dense-to-ternary distillation and quantized RL. As discussed there, Bonsai’s released Qwen3 checkpoints already establish that useful post-trained ternary models can be obtained at these scales. Our contribution is to document a recovery procedure and test which training decisions affect the resulting policy.

The empirical question in §1 is how much capability can be recovered when a post-trained model is ternarized end to end. During distillation, we update the latent backbone parameters and use their hard-ternary projection in every student forward pass. During RL, we continue updating those parameters while both actor forwards and rollout generation use the corresponding hard-ternary weights.

At 8B in non-thinking mode, Math RL raises Math-500 Pass@1\mathrm{Pass@1} from 76.8 to 79.5 while reducing the Pass@8−Pass@1\mathrm{Pass@8}-\mathrm{Pass@1} gap from 14.8 to 7.5 points. Across the 11 core benchmarks, the mean rises from 63.6 after KD to 65.5 after Math RL and 67.5 after Multi-RL (Tables 2–3).

The stage-order experiment illustrates what the recovery procedure adds. In Appendix F.6, the same mixed-reward stage reaches 5.51 on MT-Bench when initialized directly from KD and 6.35 when initialized from Math RL. This gives an empirical reason for the stage order used in our pipeline.

AuthorsOpenReview ↗

Question 1 - Endpoint KL versus downstream accuracy? Why Energy?

Thank you for highlighting this difference. It is one of the main findings of our matched-recovery experiment.

After 400K steps, we observe:

QuantizerKL ↓\downarrowGSM8K ↑\uparrow
Energy0.634547.01
Energy-OPT0.628841.02
AbsMean0.670047.81
TWN0.636043.37

Why do the rankings differ? Energy-OPT has the lowest endpoint KL but also the lowest GSM8K score. KL averages teacher–student agreement on fixed contexts. GSM8K evaluates the final answer along the student’s own generated trajectory, where an early change in token ranking can affect later steps (§4.2, Appendix D). This offers a plausible explanation while its precise cause remains open. After ~6B tokens of matched recovery, endpoint KL still did not predict the GSM8K ordering. This motivates measuring downstream behavior directly. In our main pipeline, the remaining gap between pass@1 and pass@8 after KD then motivates testing whether direct hard-ternary RL can improve single-sample reliability.

Why choose Energy? Yes, idempotency is a principal reason we chose Energy over AbsMean. Reapplying Energy to a materialized backbone tensor preserves its support and scale, which matters when synchronizing actor and rollout weights (Appendix A.4). Its calibration-free scale also preserves groupwise energy exactly (§3.2). Immediately after projection, Energy had lower forward KL than AbsMean (11.78 versus 14.81). Energy-OPT reduced initial KL further, but its matched run required 563 H100-hours versus 408 for Energy (Table 8). AbsMean achieved the highest endpoint GSM8K score in this comparison.

AuthorsOpenReview ↗

Question 2 - Ablation robustness and generality

From Appendix B, all four runs use the same pre-trained checkpoint, frozen teacher, training corpus, curriculum, optimizer, group size, and recovery budget. Only the backbone quantizer changes. Each run completes 400,002 optimizer steps over approximately 6B tokens. We report intermediate results at 60K and 100K steps and evaluate all four quantizers on the same held-out set using teacher-to-student KL at T=1T=1.

We used the same JSD training objective for all four quantizers to isolate the effect of quantizer choice. Table 7 reports both held-out JSD and teacher-to-student KL at T=1T=1. The discrepancy with GSM8K is visible under either metric: Energy-OPT achieves the lowest endpoint JSD (0.0899) and KL (0.6288), yet the lowest GSM8K score (41.02). In the main KD pipeline, we optimize full-vocabulary forward KL directly because it measures how well the student reproduces the teacher’s next-token distribution (§3.4, Eq. 7). Whether the four quantizers retain the same ordering under forward-KL training remains open.

The initial KL range of 10.17–14.81 contracts to 0.629–0.670. Endpoint teacher top-1 agreement spans 75.61–76.11%, while GSM8K scores range from 41.02 to 47.81. Our conclusion is specific to these runs: under matched long recovery, large initial differences in teacher fidelity contracted substantially, and endpoint KL did not predict the GSM8K ordering. The main experiments across Qwen3-1.7B, 4B, and 8B address capability recovery under forward-KL distillation and direct ternary RL.

On architectural generality, Table 21 reports an all-19 non-thinking mean of 43.4 for Qwen3.5-4B Multi-RL and 67.2 for its own BF16 teacher. For a comparison across architectures on the same core 11 benchmarks, Qwen3.5-4B Multi-RL scores 64.1, compared with 60.7 for Qwen3-4B Multi-RL. Mean benchmark-wise retention relative to each model’s own teacher is 84.9% and 88.9%, respectively.

Section 7.1 also reports the long-form thinking gap at Qwen3-8B: 28.4 for Multi-RL versus 66.6 for BF16. Our RL stages use non-thinking rollouts. Recovery of long-form reasoning trajectories remains an open problem.

We hope this clarifies what the matched ablation establishes and where its generality remains open. Given the reviewer’s recognition of the hard-ternary RL results and deployment benefits, we would appreciate reconsideration of the assessment and score.

Thank you for your time and effort in reviewing our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.