- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TARGET, a framework that reinterprets supervised fine-tuning as target distribution design. It decomposes token-level supervision into label imitation strength and alternative probability allocation. Based on this formulation, the authors introduce TARGET-SFT, which adaptively combines hard-label supervision with a teacher-guided soft distribution. The method demonstrates improvements across several mathematical and medical reasoning benchmarks.
Critical strengths
- The paper provides an intuitive perspective for understanding the relationships among different SFT objectives.
- The experiments are generally comprehensive.
Critical weaknesses
-
This work is essentially an adaptive knowledge distillation method framed as a novel SFT framework. Its central objective combines hard-label supervision with teacher-guided soft targets, a well-established direction in the KD literature. In particular, the teacher–student geometric interpolation is mathematically equivalent to the logit interpolation employed in [1], raising concerns about the novelty of the proposed formulation. Furthermore, recent works [2–5] have explored closely related ideas in soft-target construction and adaptive token-level distillation, yet they are neither adequately discussed in the related work nor included as experimental baselines. These omissions make it difficult to assess the paper's novelty and empirical contributions relative to existing KD methods.
-
The proposed decomposition of supervision into hard-label and residual distributions follows directly from the linearity of cross-entropy. Its conceptual novelty is therefore limited. I also find that the claim in Section 5.2 that any differentiable token-level loss can be represented by a target distribution through Eq. (6) is mathematically incorrect in general. The resulting vector is not guaranteed to be nonnegative or normalized.
[1] TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models. ICLR 2025
[2] Logits-Based Finetuning. EMNLP 2025
[3] Revisiting Knowledge Distillation for Autoregressive Language Models. ACL 2024
[4] LLM-Oriented Token-Adaptive Knowledge Distillation. AAAI 2026
[5] Entropy-Aware On-Policy Distillation of Language Models. ICML 2026
Questions for authors
What hyperparameter search was conducted for the baselines? Could the reported improvements be attributed to insufficient hyperparameter tuning of the competing methods rather than the proposed method itself?
2 comments
Rebuttal Response to Reviewer ahoM (2/2)
W2: The resulting vector in Section 5.2 is not guaranteed to be nonnegative or normalized.
We thank the reviewer for raising this detail. We agree that the loss-to-target intuition in Section 5.2 requires further specification. We provide a natural extension of our target view that resolves both issues, providing a cleaner interpretation than the current formulation.
Intuitively, when does a negative target entry arise: Assigning already gives the strongest suppression of token that cross-entropy towards a distribution can express. Therefore, occurs when the loss applies stronger downward pressure on beyond ordinary probability matching.
Therefore, remaining centered on our target view, we express Q through two valid probability distributions, where an attractive target indicates token probabilities to encourage, and a repulsive target indicates excess suppression. This repulsive branch thus collects the negative entries under Q. Formally, let denote the total magnitude of negative entries in , we can define
The gradient under original loss can therefore be represented exactly as
This gives a more complete interpretation of the supervision target encoded by a loss, even when certain tokens are pushed harder than standard probability matching permits. When , it reduces to the current formulation.
To make our claims more precise, we will include in revision (1) a zero-sum gradient condition, which holds naturally for losses operating through softmax probabilities, and (2) this attractive-repulsive extension, which makes explicit the role of arbitrary token gradient in target designs. This change strengthens the general loss-to-target intuition in Section 5.2; our proposed Target framework and Target-SFT remain unaffected since they construct valid distributions by design. We appreciate the reviewer for highlighting this point!
Q1: Could the improvements be attributed to insufficient hyperparameter tuning of baselines rather than the proposed method itself?
The distillation baseline mixes label and teacher supervision with hyperameter weight , we report results on the tuning of .
| Dataset | c=0.2 | 0.4 | 0.6 | 0.8 | 1.0 |
|---|---|---|---|---|---|
| NuminaMath-CoT | 18.65 | 22.93 | 26.06 | 31.21 | 22.8 |
| OpenR1-15k | 19.13 | 20.41 | 23.10 | 25.54 | 25.27 |
The best performance occurs at , the value already used in the main experiments. And Target-SFT remains well above it, with +7.75 on NuminaMath and +14.87 on OpenR1.
The other training configurations (learning rate, batch size, optimizer, etc) all followed the identical settings in prior works [1, 2], without tuning specifically for our method. We additionally sweep the learning rates over {5e-6, 1e-5, 2e-5, 5e-5, 1e-4, 2e-4, 5e-4} for all methods:
| Method | 5e-6 | 1e-5 | 2e-5 | 5e-5* | 1e-4 | 2e-4 | 5e-4 |
|---|---|---|---|---|---|---|---|
| NuminaMath-CoT | |||||||
| SFT | 9.99 | 12.48 | 15.15 | 16.91 | 17.16 | 16.43 | 11.70 |
| SFT (p) | 22.90 | 22.88 | 26.77 | 33.35 | 30.43 | 26.35 | 10.11 |
| Distill | 17.24 | 23.91 | 27.76 | 31.21 | 31.41 | 31.23 | 23.11 |
| Target-SFT | 26.98 | 31.81 | 36.57 | 38.96 | 37.98 | 36.04 | 25.63 |
| OpenR1-15k | |||||||
| SFT | 12.14 | 8.57 | 10.58 | 19.84 | 22.93 | 24.41 | 23.05 |
| SFT (p) | 15.24 | 19.42 | 34.09 | 39.44 | 18.51 | 35.73 | 10.87 |
| Distill | 16.67 | 18.62 | 20.57 | 25.54 | 27.38 | 29.22 | 24.76 |
| Target-SFT | 18.40 | 26.54 | 36.36 | 40.41 | 41.24 | 39.52 | 24.64 |
Target-SFT stays above all baselines at every learning rate, until at 5e-4 every method degrades.
References
[1] Beyond Log Likelihood: Probability-Based Objectives for Supervised Fine-Tuning across the Model Capability Continuum
[2] On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification
We will include these discussions in the revised draft, and we thank the reviewer again for their helpful advice to improve our work.
Rebuttal Response to Reviewer ahoM (1/2)
We sincerely thank the reviewer for their valuable feedback. We hope the following responses address the questions.
W1: Distillation Baselines & Novelty Clarification
We address this in three parts: (a) Distinction from TAID, (b) Comparison with distillation methods, and (c) Novelty of our framework, each supported by new experiments.
We thank the reviewer for pointing out the connection. The interpolation is similar, but our method differs in motivation and the role of this distribution. TAID addresses the teacher-student gap by using a temporally adaptive distribution as the full training target. In contrast, our work addresses the rigid label imitation in SFT, by explicitly allocating supervision and the distribution serves only as residual. We compare with TAID and other distillation methods directly (Table 1).
Importantly, our central contribution is the Target framework, rather than the particular interpolation in Target-SFT, which is one instantiation to show the design value of our framework. To clarify this, we further show that Target can systematically improve existing approaches beyond our own example instantiation (Table 2).
We compare with 8 new distillation methods, including the suggested TAID, LFT, ATKD, AdaKD, and 4 additional objectives Veto, HPD, CSD, and AMiD. All methods followed their official implementations. Methods such as EOPD require on-policy student responses, which fall outside our setting and will be discussed in the revised related work.
Table 1: Comparison with distillation methods
Target-SFT outperforms all new baselines, with 4.26% gains over the strongest baseline AMiD and 7.33% over TAID. These results establish our empirical strengths against recent distillation methods.
We hope to clarify our contribution in the supervision-design principle: separating how strongly to imitate the label γt from how to allocate supervision among alternatives π~t, and explicitly coordinating these two choices. This provides concrete guidance for future fine-tuning designs.
To show the value of this framework, we apply Target to existing distillation methods. This considers LFT, Veto, TAID, and AMiD, which are token-level distributions that can be directly used as the residual π~t. It allows us to preserve their distribution-design choices under our supervision structure. Each original and Target-augmented pair uses identical hyperparameters reported in the original paper, to isolate the effect of our Target formulation.
Table 2: Improving existing methods with Target framework
Target framework improves all existing methods with an average gain of 6.14%. Notably, TAID within Target shows substantial additional gains. This also extends to stronger methods such as AMiD, showing that the benefits are not restricted to weaker baselines or our specific residual design.
These results highlight an actionable framework for supervision design beyond algebraic decomposition. While linearity permits the decomposition, Target makes these choices and intuition explicit, providing a reusable structure for objective designs.
We appreciate the reviewer's constructive feedback, and this opportunity to present our contribution in a clearer and more extensive way.