- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CIG-SFT, a token-level weighting method that uses conditional information gain to guide supervised fine-tuning. It estimates the contribution of the task input to each demonstrated token using a frozen reference model and derives a KL-regularized, gradient-equivalent weighted SFT objective. Experiments on mathematical reasoning and code generation show improvements over standard SFT and several adaptive fine-tuning baselines.
Critical strengths
S1. The paper distinguishes task-conditioned information from token predictability and introduces a principled KL-regularized weighting objective.
S2. CIG-SFT consistently achieves competitive performance across mathematical reasoning and code generation, supported by ablation and sensitivity analyses.
Critical weaknesses
W1. The proposed method mainly combines a conditional log-likelihood ratio with an existing KL-regularized policy projection. However, the final token weights still depend on the current model's predictive probability, making the distinction from existing probability-based adaptive SFT methods less substantial.
W2. The mathematical reasoning experiments rely on a single training data source, without dedicated evaluation on high-quality reasoning trajectories distilled from stronger teacher models. Such demonstrations may contain valuable reasoning steps that are difficult for the student model to predict and consequently receive small training weights. It remains unclear whether CIG-SFT can effectively learn from these demonstrations.
W3. Tables 2/4 and Tables 7–9 report inconsistent Pass@16 results (58.6 vs. 56.7) for the default Qwen2.5-Math-1.5B configuration, which should be clarified.
Questions for authors
Q1. Can the authors provide stronger evidence that conditional information gain reflects token learning value rather than task-related predictability or distribution shifts?
Q2. Does CIG-SFT remain effective on high-quality teacher-distilled reasoning datasets? How does it perform against existing baselines in this setting?
4 comments
Weakness 3: Inconsistent Pass@16 Results
We thank the reviewer for identifying this inconsistency. After checking the original evaluation records, we confirmed this was a transcription error in the appendix tables, not a difference in experimental settings.
Specifically, the appendix mistakenly combined Avg@16 from seed 42 (31.5%) with Pass@16 from seed 43 (56.7%). The results in Tables 2 and 4 are correct and correspond to seed 42 (31.5% Avg@16 and 58.6% Pass@16).
For transparency, we provide the results across all five random seeds below:
| Method | seed 42 | seed 43 | seed 44 | seed 45 | seed 46 | Average |
|---|---|---|---|---|---|---|
| SFT | 13.8/42.3 | 13.9/40.9 | 13.9/40.5 | 14.1/43.9 | 14.3/41.4 | 14.01/41.79 |
| DFT | 26.3/49.0 | 26.7/49.8 | 26.1/49.4 | 26.0/49.5 | 26.7/50.6 | 26.37/49.66 |
| IDFT | 25.2/48.9 | 26.4/49.3 | 26.0/45.2 | 25.6/43.9 | 27.0/49.6 | 26.04/47.37 |
| PriFT-prob | 30.3/56.2 | 29.1/56.3 | 29.3/55.4 | 29.3/55.2 | 29.2/55.8 | 29.45/55.75 |
| CIG-SFT | 31.5/58.6 | 30.5/56.7 | 30.6/54.7 | 31.0/56.4 | 31.4/57.4 | 31.00/56.76 |
We apologize for the oversight and will correct the affected entries in Appendix Tables 7–9 to ensure consistency with the original evaluation records.
Question 1: Relationship Between CIG and Learning Value
We thank the reviewer for this important question. We would like to clarify the motivation behind CIG and provide additional evidence linking task-conditioned information to downstream learning benefits.
-
Learning value as our guiding objective.
We introduce "learning value" to highlight a limitation of existing SFT methods: a token's predictive compatibility does not necessarily reflect how beneficial its supervision will be for downstream learning. Ideally, we aim to allocate supervision according to such learning benefits. However, the corresponding optimal weights are intractable to obtain and cannot be directly evaluated against reliable token-level ground truth.
Rather than directly estimating these unknown optimal weights, CIG takes a practical step toward learning-value-aware supervision by incorporating the additional predictive information provided by the task prompt. We will accordingly refine the phrase "quantify learning value" in Lines 81–82 to clarify this intended contribution while preserving learning value as our guiding objective.
-
Why task-conditioned information may indicate useful supervision.
CIG measures how much conditioning on the task prompt reduces a token's surprisal. By the symmetry of pointwise mutual information, this also characterizes how much information the token carries about the task. Such tokens may reflect task-specific reasoning strategies or their concrete execution steps, motivating stronger supervision to learn task-dependent input–output relationships.
Our case study in Appendix A illustrates this intuition. CIG-SFT strengthens supervision on the core reasoning operations "substitute" and "check", as well as the initial execution example "a=1", while reducing emphasis on some subsequent repetitive steps and generic expressions.
-
Additional controlled evidence of downstream learning benefits.
To examine whether CIG-guided adjustments provide learning benefits, we conduct additional controlled experiments on Qwen2.5-Math-1.5B, following the same settings as our main experiments. In addition to DFT and CIG-SFT, we considered four variants:
- Up only: Keep only CIG's weight increases; use DFT weights elsewhere.
- Down only: Keep only CIG's weight decreases; use DFT weights elsewhere.
- Shuffled up: Shuffle the weight increases among similar upweighted tokens in Up only, preserving the upweighted positions and total increase.
- Shuffled down: Shuffle the weight decreases among similar downweighted tokens in Down only, preserving the downweighted positions and total decrease.
Method Avg@16 Pass@16 DFT 26.3 49.0 Up only 28.6 52.8 Shuffled up 28.1 49.8 Down only 31.0 55.9 Shuffled down 30.8 55.4 CIG-SFT 31.5 58.6 The results reveal three findings:
-
Both upweighting and downweighting are beneficial. Compared with DFT, "Up only" improves Avg@16/Pass@16 by 2.3/3.8 points, while "Down only" achieves larger gains of 4.7/6.9 points. This suggests that both strengthening and reducing supervision at CIG-selected positions contribute to downstream learning.
-
Combining both adjustments performs best. Full CIG-SFT achieves improvements of 5.2/9.6 points over DFT, outperforming either individual adjustment. This supports the benefit of jointly allocating stronger and weaker supervision based on task-conditioned information.
-
The assignment of weight magnitudes also matters. Shuffling the adjustment magnitudes among similar tokens reduces performance, particularly for upweighting (−0.5/−3.0 points). This suggests that CIG benefits not only from adjusting overall supervision strength but also from assigning different weights to individual tokens.
These results suggest that CIG's effectiveness goes beyond simply measuring predictive changes: using this information to guide token-level supervision yields measurable downstream learning benefits. The shuffled controls further suggest that the assignment of adjustment magnitudes, not just their overall amount, contributes to these gains.
In summary, learning value defines our guiding objective, CIG provides a tractable step toward it, and downstream learning improvements offer indirect empirical evidence for the effectiveness of this step. We do not claim that CIG exactly captures oracle learning values. Instead, we show that incorporating task-conditioned information yields more effective supervision than the evaluated alternatives.
We provide further clarification, detailed case studies, and controlled experimental analyses in our response to Reviewer nESP. We will incorporate the corresponding clarifications and evidence into the revised manuscript.
Looking Forward to Your Feedback
Dear Reviewer fCbf,
Thank you again for your careful reading and thoughtful feedback. We have provided detailed responses to your comments, together with additional experiments and clarifications. As the discussion period is coming to a close, we would appreciate hearing your thoughts on our responses and whether you have any remaining questions or concerns. We would be happy to provide further clarification to support the discussion.
Thank you for your time and consideration.
Best regards,
The Authors
We thank the reviewer for raising this concern. We agree that KL-regularized projection builds on established optimization techniques. Our key contribution is introducing task-conditioned information into supervision allocation, complementing the model's predictive compatibility rather than replacing it. We clarify the distinction in three aspects:
CIG introduces information beyond predictive compatibility.
Existing methods such as DFT assign supervision weights based on the model's predictive probability given the full context. In contrast, CIG additionally compares token predictions with and without the task prompt, while keeping the demonstrated response prefix fixed. This comparison captures the additional predictive information provided by the task, which cannot be determined from the full-context probability alone.
Probability dependence does not imply probability-only weighting.
Model probabilities remain part of our weighting mechanism to maintain predictive support, while CIG guides how supervision is adjusted. Consequently, tokens with similar predictive probabilities can receive different supervision weights depending on their task-conditioned information. For example, in our case study (Appendix A), token "substitute" receives a weight of 0.436 under CIG-SFT versus 0.098 under DFT, while token "is" receives 0.000 versus 0.983. These examples illustrate how CIG can strengthen supervision on selected task-related operations while suppressing some highly predictable generic expressions.
Ablation results confirm the practical importance of this distinction.
On Qwen2.5-Math-1.5B, CIG-SFT achieves 31.5% Avg@16, compared with 26.3% for DFT. Removing the task-information component reduces performance to 21.9%, while removing KL regularization yields 28.4%. These results support the contributions of both the task-conditioned information signal and the information-proximal optimization mechanism.
In summary, CIG-SFT differs from probability-based adaptive SFT not by eliminating predictive probability, but by incorporating task-conditioned information to guide supervision allocation. We will emphasize this distinction and the respective contributions of the two components in the revised manuscript.
We thank the reviewer for this valuable suggestion. To examine whether CIG-SFT can effectively learn from high-quality reasoning trajectories generated by stronger teachers, we conducted additional experiments using teacher-distilled mathematical reasoning data.
Experimental setup. We use Qwen2.5-Math-1.5B as the student model and selected 30K complete reasoning trajectories from the CoT subset of nvidia/Nemotron-SFT-Math-v4, generated by DeepSeek-V4-Pro (High mode). All trajectories fit within 4096 tokens without truncation. We compared SFT, DFT, iDFT, PriFT-prob, and CIG-SFT under identical training settings, evaluating on the same mathematical benchmarks as our main experiments.
The results show that CIG-SFT remains effective on strong-teacher demonstrations. It achieves the highest average Avg@16 and Pass@16 across the six benchmarks, improving over DFT by 2.5/3.2 points and over PriFT-prob by 1.6/0.2 points, respectively.
These findings provide additional evidence that CIG-SFT can effectively learn from reasoning trajectories distilled from stronger teachers, despite the potential predictive mismatch between teacher demonstrations and the student model. By combining task-conditioned information with model predictive support, CIG-SFT maintains its advantage over probability-based supervision methods in this setting.
We will include these additional experiments in the revised manuscript.