acceptodds
Under review as a conference paper at ICLR 2027

Quantization Can Preserve Verification While Degrading Self-Generated Supervision

Abstract

Can a quantized model retain its ability to select answers yet lose the learning value of its own solutions? We separate verification ranking, final-answer correctness and held-out learning gain. On Gemma 4 12B and Qwen3.6-35B-A3B, all nine registered 4- and 3-bit verifier contrasts are equivalent to full precision within ±0.025 AUROC, even as generation degrades. Crossing sample source with learner precision reveals that SmolLM3-3B's full-precision solutions at 38% correctness teach both learners more than NF4 solutions at 74% (by 1.9 and 3.6 accuracy points) on a holdout selected for partial solvability. On the 12B, approximately matching sample count and problem coverage leaves an 18.9-point source advantage for the quantized learner on this holdout and 4.6 points on MATH-500. Completed, mostly correct NF4 solutions also reduce the full-precision learner's accuracy. The penalty increases along the GGUF precision ladder on both learning subjects, with format-dependent severity, and no tested selector shows an established recovery. Reciprocal continuations separately localize the 12B's excess truncation to the continuing model for the tested prefixes. Under this math supervised fine-tuning recipe, verification robustness and final-answer correctness are insufficient summaries of useful self-generated supervision.

Primary area:
foundation or frontier models, including LLMs
Keywords:
quantization, self-generated supervision, self-verification, synthetic data, supervised fine-tuning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vNirOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The current summary is highly complete as it captures the central thesis and main paradox of the paper, which is the decoupling of preserved verification ranking from degraded self-supervised learning value.

However, to make the summary even stronger and more robust for a conference review, we can add one additional sentence highlighting the core empirical finding: that solutions from a full precision generator can teach a learner significantly more than solutions from a quantized generator, even when the full precision solutions have much lower final answer correctness.

Here is an enriched version of the summary that includes this key nuance while strictly following all of your formatting constraints:

This paper investigates the self training of language models where the generator is a quantized version of the original model. The authors perform a comprehensive evaluation to benchmark various models and their quantized versions, revealing a crucial phenomenon: while quantized generators can preserve verification ranking and maintain high correctness in their final answers, making them appear to be excellent teachers, their intermediate reasoning steps and termination behaviors are severely degraded under quantization. This degradation produces poor and potentially harmful training signals for the learner. Notably, the authors demonstrate that full precision solutions can teach a model significantly more than quantized solutions, even when the full precision solutions have a much lower final answer correctness rate. Consequently, final answer correctness and verification robustness are insufficient criteria for assessing the quality and usefulness of self generated supervision.

Critical strengths
  • The paper introduces a highly rigorous evaluation framework that successfully decouples verification ranking, final answer correctness, and realized learning gain. By isolating these properties, the study goes beyond standard evaluations to precisely pinpoint how and where quantization damages the learning value of generated solutions.

  • The empirical scale of the work is robust, benchmarking a wide selection of model families and trainee sizes across a complete ladder of nominal bit widths.

  • The authors employ diagnostic methodologies, such as reciprocal continuations, to isolate whether generation failures are driven by the prefix or by the continuing model. This investigation into the mechanics of autoregressive feedback adds substantial technical depth.

Critical weaknesses
  • The paper suffers from severe structural fragmentation. The text is divided into numerous very small, paragraph long subsections and independent exploratory tests. This design makes the paper read like a collection of disjointed empirical observations rather than a cohesive, smoothly flowing story, which significantly hinders readability.

  • While the empirical findings are highly practical and provide useful warnings to the community, the overall novelty is relatively limited. The study primarily documents observations within standard training recipes without proposing new algorithms, architectures, or theoretical frameworks to mitigate the degraded learning value. This incremental contribution falls below the expectations for a top tier conference.

Questions for authors
  • The paper notes that the exact textual properties causing the learning difference remain unidentified. Do the authors have any hypotheses about which specific textual features of quantized solutions, such as lexical diversity, formatting styles, or subtle reasoning gaps in the step by step derivations, are responsible for the degraded learning value? How might future work systematically isolate and measure these hidden textual factors?

  • What potential mitigation strategies do the authors suggest to help recover the lost learning value of self generated solutions from quantized generators?

Reviewer_nrppOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies whether quantization preserves verification ability while reducing the usefulness of generated solutions for training. It separates fixed-candidate verification AUROC, final-answer correctness, and downstream learning gain. Across the registered comparisons, 4- and 3-bit verifiers retain approximately the same ranking performance as their full-precision counterparts. However, crossing generator precision with learner precision shows that quantized-source solutions can teach less and can even harm a full-precision learner. Particularly interesting is the SmolLM3 result: full-precision solutions with 38% correct answers outperform quantized solutions with 74% correct answers as training data on the selected holdout. Matched source comparisons, a GGUF precision ladder, and reciprocal continuation experiments provide additional controls. The supported conclusion is that ranking performance and answer correctness are insufficient proxies for training value under the tested math SFT recipe.

Critical strengths

The source-by-learner crossing isolates an important failure mode. Changing the generator’s numerical representation within the same underlying checkpoint, then training both learner precisions on the resulting data, is more informative than comparing unrelated teachers. On the 12B model, approximately matching sample count and problem coverage still leaves an 18.9-point source advantage for the quantized learner on the primary holdout and 4.6 points on MATH-500. The reverse transfer, where completed NF4 solutions harm the full-precision learner, makes the result particularly compelling.

The correctness reversal strengthens the argument beyond a simple teacher-quality comparison. On the 3B model, the less-correct BF16 source teaches both learners more than the more-correct NF4 source. This directly demonstrates that final-answer correctness does not order training value across these sources. The GGUF experiments further show that the phenomenon is not confined to one NF4 implementation, while the paper appropriately distinguishes these observations from an identified textual mechanism.

Critical weaknesses

The uncertainty estimates need a better-calibrated analysis. Appendix A reports rejection rates of 0.158 and 0.09 for nominal 0.05 tests under a Gaussian seed-only null, with corresponding undercoverage of the reported intervals. This does not establish that the actual source contrasts have those false-positive rates, and the largest effects are unlikely to disappear. It does weaken confidence in the smaller precision-ladder effects, selection comparisons, and equivalence decisions. Reporting the diagnostic is valuable, but the affected conclusions should be checked with an inference procedure that performs adequately under the stated simulation.

The strongest learning result remains difficult to separate from recipe-specific termination behavior. The 12B penalty appears largely through increased truncation, and all learning experiments use one adapter-SFT recipe. Filtering for completion at generation does not fully resolve this concern because serialization can subsequently clip retained examples. The primary holdout is also selected for partial solvability; the 3B reversal is much smaller on MATH-500 and is not established for its NF4 learner there. These limitations do not invalidate the narrow empirical claim, but they leave uncertain how much of the effect reflects degraded reasoning supervision versus learned length and stopping behavior under fixed budgets.

Questions for authors

Can you re-evaluate the main source contrasts and precision-ladder comparisons with a procedure that achieves reasonable coverage under your seed-only null diagnostic? Which difference and equivalence conclusions change?

Does the 12B source penalty persist when training examples retain their complete endings after serialization and evaluation uses a larger generation budget? Please report the fraction of clipped training examples by source and separate changes in truncation from completed-answer correctness.

Section 7 notes near-duplicates between the 12B pools and MATH-500, but the removal analysis does not cover the main source contrasts. Do the reported 4.6- and 4.9-point source advantages persist after removing the affected evaluation problems?

Reviewer_7cDSOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper investigates how model quantization affects verification, generation, and the learning value of self-generated solutions. Through controlled experiments across different model precisions, the authors find that verification ranking can remain largely preserved while quantized models generate less effective supervision for subsequent training.

Critical strengths
  • The paper investigates a relevant problem concerning the impact of quantization on self-generated training data.

  • The experiments include several controlled comparisons across model precisions, generators, and learners

Critical weaknesses
  • The central motivation and conceptual novelty are not sufficiently convincing. Verification and self-training measure fundamentally different properties: the former evaluates candidate correctness, while the latter depends on the quality of entire training trajectories. Therefore, observing that quantization preserves verification while degrading supervision is not particularly surprising, and the paper does not establish a deeper connection between these two phenomena.

  • The paper's strongest negative results come from cross-precision training, where a BF16 learner is trained on NF4-generated trajectories. This setting is closer to off-policy distillation than conventional self-training, and the corresponding NF4-to-NF4 experiments do not exhibit the same severe degradation. Consequently, the results do not clearly establish that quantization fundamentally undermines a model's ability to learn from its own generated experience.

  • The paper does not identify a quantization-specific mechanism that explains the degradation in supervision quality. Since quantization already affects generation quality, the observation that a weaker generator produces less useful training trajectories may largely reflect a general teacher-quality effect. The experiments establish a source-dependent learning difference but provide limited insight into what makes quantized supervision uniquely harmful.

  • The claimed preservation of verification is not clearly connected to effective supervision selection. Although aggregate AUROC remains stable, the within-problem verification AUROC is only 0.573 on the hard math benchmark, indicating weak discrimination among candidate solutions for the same problem. This limits the practical significance of verification preservation in the self-training setting considered by the paper.

Questions for authors

See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.