- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a recovery pipeline for fully ternary LLMs converted from pretrained dense models. The method combines hard-ternary knowledge distillation with subsequent math and multi-domain reinforcement learning, using repeated sampling to identify residual capability that remains reachable but unreliable. Experiments on Qwen3 models from 1.7B to 8B report strong cross-domain capability recovery and substantial deployment benefits.
Critical strengths
1.A simple and reasonable recovery pipeline.
The combination of hard-ternary KD and direct ternary RL is conceptually simple and well motivated. KD first restores the teacher distribution, while RL improves the reliability of behaviors that remain reachable but are not consistently selected.
2.Exceptionally strong cross-domain results.
The reported ternary models perform well across knowledge, reasoning, instruction following, mathematics, and code, with consistent improvements over Bonsai at the same model scales.
Critical weaknesses
1.The paper is poorly organized.
Method definitions, experimental findings, implementation details, and failure analyses are interleaved, and no overview figure summarizes the complete KD-to-RL recovery pipeline.
The Energy quantizer receives substantial attention in the main text, although the matched-recovery ablation suggests that final performance is driven primarily by the large-scale recovery procedure rather than by the specific initialization quantizer. Much of this discussion could therefore be shortened or moved to the appendix.
Basic notation such as “KD E3” is also used without an explicit definition, and important training settings are scattered across the main text and appendix.
The Multi-RL stage, which produces the strongest cross-domain results, is not described systematically in terms of its training-data mixture, domain-specific reward functions, and training budget across model sizes.
The paper would benefit from substantial restructuring around the KD–RL recovery pipeline, with greater emphasis on data construction, prompt filtering, training procedures, and the engineering insights required to optimize and deploy a hard-ternary model.
2.The central empirical claims cannot be independently verified without a final materialized ternary checkpoint.
The reported cross-domain recovery is unusually strong for fully ternary models. However, without access to a final materialized ternary checkpoint and minimal verification code, it is difficult to confirm that evaluation uses only the deployed ternary weights, that all intended modules are quantized as described, and that the reported results can be reproduced from the final checkpoint.
The paper reports several internal integrity checks, but directly inspecting and evaluating the deployed model would provide substantially stronger evidence. The authors should therefore release at least one final materialized ternary checkpoint together with minimal code for verifying the ternary grid and reproducing representative evaluation results.
Questions for authors
Could the authors release at least one final materialized ternary checkpoint together with minimal verification and evaluation code? The provided code should confirm that evaluation uses only the deployed ternary weights rather than the latent BF16 parameters and reproduce representative cross-domain results.
3 comments
Multi-domain RL — Data, rewards, training, and checkpoint selection
Thank you for asking us to bring the Multi-RL details together. Section 5 and Appendix F describe the staged recovery procedure and its optimization safeguards. We summarize the Multi-RL setup for the three model sizes below.
Training setup. Multi-RL uses GRPO initialized from the Math-RL checkpoint. We use a learning rate of , 2% warmup, weight decay of 0.01, eight rollouts per prompt, a batch size of 512, a mini-batch size of 128, and a maximum response length of 2,048 tokens. The objective has no explicit KL penalty or entropy bonus.
Data and rewards. The training mixture contains 25,427 prompts across five components.
| Component | Reward verification |
|---|---|
| KodCode | Unit tests |
| Factual QA | Exact match with accepted aliases |
| Instruction following | Constraint checker |
| Mathematics | Boxed-answer correctness |
The mixture contains no tool-calling data. Tool use remains a limitation of the resulting policy.
Prompt selection. We calibrate difficulty against each model size’s current ternary policy using the rollout temperature and response budget used in training. This identifies prompts that produce useful reward variation for that checkpoint. We check for validation overlap, filter degenerate outputs, and examine overlap with the preceding stage.
Verification and checkpoint selection. We select checkpoints using held-out performance across the trained domains. Appendix F.4 illustrates why this matters: in one run, training reward increased while held-out accuracy fell by 13 percentage points. Appendix F.5 documents a verifier that incorrectly rewarded one-character answers. Appendix F.1 reports a separate failure mode in which a learning rate of produced no observed ternary-code movement. These observations motivated three checks during recovery: validate the reward function, monitor held-out performance, and verify that optimizer updates change the effective ternary weights.
We will present the Multi-RL configuration and data mixture together in the revision so readers can connect these choices to the final cross-domain results.
Materialized checkpoints and independent verification
We understand the request for a final materialized checkpoint and minimal verification code. We are unable to share the weights during this discussion and aim to release them as research artifacts alongside the paper. We recognize that this limits reviewers’ ability to rerun the final model. Below we clarify the evidence in the submission for the execution path used in our experiments.
Quantizer coverage. Appendix A.1 specifies which parameters are ternarized. All compute-dominant attention and feed-forward weight matrices are ternarized in groups of 128. The embedding and output matrices are also ternary. Normalization parameters and small scalar parameters retain their native precision.
Weights used during training and evaluation. Appendix F.2 distinguishes three representations:
- Latent weights are the BF16 parameters that accumulate optimizer updates through the straight-through estimator.
- Effective weights are the hard-ternary weights used in every actor forward pass.
- Baked weights are the materialized ternary weights exported to rollout workers and the inference runtime.
During synchronization, rollout workers receive the baked weights. The BF16 latent parameters remain on the optimization path. We check quantizer coverage, re-quantization idempotency, agreement between actor and rollout policies, and ternary-grid integrity after merging.
Inspection of the materialized 8B checkpoints. In the backbone tensors we examined, every 128-weight group had a single nonzero magnitude and at most three distinct values, consistent with a groupwise grid . The measured nonzero density ranged from 0.607 to 0.623. The inspection covered all attention layers and the output head. This inspection was also part of the each evaluation for each ternary model.
The coverage checks establish which modules were ternarized, the export procedure identifies the weights supplied to the runtime, and the tensor inspection checks the grid in the materialized checkpoints. We hope this addresses the specific execution concerns while making clear what remains unavailable for independent evaluation during discussion and request you to reconsider the score.
Thank you for your time and effort in reviewing our paper.
Response to Reviewer 5HEb
We thank the reviewer for recognizing the strength of the cross-domain results, the 8B improvement over Bonsai, and the motivation behind our recovery pipeline.
Section 1 introduces three research questions. How much does the choice of quantizer matter after recovery? How much capability can same-size KD restore? Can direct hard-ternary RL improve behaviors that remain reachable after KD but unreliable on a single attempt? These questions motivate the sequence:
Ternarize → recover the distribution → identify the remaining failure → recover the policy.
The training methodology is documented in the submission. Appendix E.5–E.7 and Tables 10–13 specify the KD datasets, mixture, curriculum, optimization settings, and training budget. Appendix F.2–F.6 covers actor–rollout synchronization, RL configurations, prompt filtering, verifier checks, and stage transitions.
We agree that the reader currently has to move between sections to reconstruct the procedure. In the revision, we will use the additional page for an overview figure connecting the training stages to their evaluations. We will shorten the main-text quantizer discussion, bring the Multi-RL data, rewards, and training settings together, and define KD E3 at first use as the checkpoint after the third distillation epoch.