- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes CANON (Consensus-ANchored self-distillatiON), a method that uses the model’s own consensus as privileged context for label-free self-distillation. For each unlabeled prompt, CANON samples multiple rollouts, identifies the majority answer, and selects the corresponding solution with the highest mean token log-probability. A frozen snapshot teacher receives this consensus solution, and the student minimizes a full-vocabulary Jensen–Shannon divergence over all initially sampled rollouts for one epoch.
The paper evaluates both transductive test-time training and inductive transfer. On Qwen3-4B-Instruct-2507, CANON improves AMC avg@32 from 66.0 to 76.5 using 4.8 GPU-hours, compared with 70.3 and 32.4 GPU-hours for TTRL. Additional experiments examine transfer across tasks and models, conditioning choices, coverage, and training stability.
Critical strengths
-
A useful problem and a simple method. Conditioning a teacher on a consensus solution provides richer supervision than a scalar pseudo-reward. Reusing the same rollouts for consensus construction and distillation makes the approach practically attractive.
-
Convincing improvements on the main comparison. The AMC gains over the base model and TTRL are substantial, with paired bootstrap intervals of [+5.8, +15.6] and [+2.9, +9.9], respectively.
Critical weaknesses
- Applicability to full fine-tuning remains unverified. Nowadays, full SFT is the predominant approach for postraining in most of the frontier labs. In contrast, all CANON experiments use LoRA, leaving it unclear whether the reported benefits persist under full fine-tuning. Restricting parameter updates to a low-rank subspace may affect sensitivity to incorrect consensus, training stability, and coverage preservation.
Questions for authors
- The main concern is about lora fintuning vs full SFT - are we confident to claim CANON will also work for full SFT?
3 comments
We thank the reviewer again. We have uploaded an updated revision. We are happy to answer any remaining questions before the discussion closes. If our response addresses the reviewer's concern, we would be grateful if the reviewer would consider raising the score.
Thanks for the update
Thanks for the update. My main concerns are resolved and I raised my score
We thank the reviewer for the constructive review and suggestions. We ran CANON with full fine-tuning during the discussion period. We address the concern below.
"Applicability to full fine-tuning remains unverified [...] all CANON experiments use LoRA, leaving it unclear whether the reported benefits persist under full fine-tuning." and "are we confident to claim CANON will also work for full SFT?"
Yes. With all parameters updated and no adapter, CANON reaches 81.3 avg@32 on AMC for Qwen3-4B-Instruct-2507, against 76.5 with LoRA and 66.0 for the base model. We ran this on three models with the rest of the recipe unchanged, and selected the learning rate on the OmniMath development slice.
Full fine-tuning improves over LoRA on every model. The result for Qwen3-4B-Instruct-2507 is the mean of three training seeds. The same checkpoints also improve the held-out benchmarks of Table 1.
"Restricting parameter updates to a low-rank subspace may affect sensitivity to incorrect consensus, training stability, and coverage preservation."
We checked the three properties with full fine-tuning. Training is stable. For Qwen3-4B-Instruct-2507, accuracy on the development slice stays between 72.6 and 76.9 when the learning rate varies by a factor of ten, and for Qwen3.5-4B accuracy on AMC stays between 79.8 and 80.8 when it varies by a factor of five. Over training, the loss decreases from 0.020 to 0.009, and the entropy of the student stays at 0.21. Coverage is preserved. On AMC, maj@32 rises from 83.1 to 86.7 and pass@32 from 85.5 to 88.8. The sensitivity to an incorrect consensus is the one reported for LoRA in Section 5.2. The base model holds a wrong majority answer with confidence on two AMC prompts, and both remain wrong after training.
Full fine-tuning is also the setting of the RL baselines. With the learning rate of TTRL selected on the same development slice, CANON reaches 81.3 against 76.3 for TTRL (three training seeds each), and one CANON run takes about one sixth of the GPU-hours of a TTRL run.
Changes in the revised manuscript