- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The authors study how using the most common loss function, mean cross-entropy, could lead to imbalance when the training set contains examples of variable length. By averaging all token cross-entropies in a batch, the shorter examples get a lower weight when paired with longer examples. This is a known problem, which has already been tackled in RL training. Nevertheless, the current study controls the experimental variables to isolate the effect of the loss function in different scenarios. The experiments are focused on finetuning on reasoning traces, which are generated by a teacher model prompted for different reasoning depths. Different scenarios are tested to see how the mix of data influences the downstream performance, which is measured on several public math and reasoning datasets.
Critical strengths
- The studied potential issue with the loss function is somehow overlooked in the literature. This study brings important empirical validation of the effect of training with classical mean cross-entropy compared to an equal-example loss function.
- The experimental setup is well designed and covers a diverse set of conditions. Confidence intervals and significance tests are included for each experiment.
Critical weaknesses
- The main issue with the paper is the clarity of the presentation. It is quite difficult to follow, mostly due to the high number of named recipes and dataset variations, which appear in multiple configurations. The structure, which mostly enumerates setups and experiments in an LLM-like fashion, lowers the cohesion and readability of the text. I would have liked more discussions and interpretations of the results instead of an enumeration of experiments.
Questions for authors
- The paper would be more valuable if the authors included some recommendations and guidelines for best practices. The series of experiments is meant to isolate where the performance decrease comes from, but I couldn't find a complete explanation in the end.
- As far as I understand, the mixed dataset with equal example loss obtains a lower performance because direct answers get a higher weight, so the model tends to answer with short or without reasoning with confidence some questions that would require longer reasoning. I am thinking that including some instruction at the beginning of the prompt used in finetuning could steer the model towards longer or shorter reasoning. Then, the evaluation could be done by forcing a higher reasoning effort (or on different levels). Is this a correct assessment, or am I missing something?
2 comments
Thank you for adding the new experiments and your answer. However, I will not change my current assessment because the paper still has limited impact. It shows that it is important to know the distribution of the data mix and that changing the loss function has a high influence in the performance, which is somehow expected. As also highlighted in your answer, the difference comes from steering the model towards shorter answers, which is a side effect of the equal-weight loss function, and could be mitigated by including instructions in the finetuning step.
We thank the reviewer for the careful reading. We answer the three points in order.
1. Presentation. Concern: the paper is difficult to follow because of the high number of named recipes and dataset variations.
Two corpora carry the main comparison: Fixed-T2 and Guided-Mix-T2sub, on the same 11,853 questions (Section 4). The interventions of Section 5 run on the full mixture Guided-Mix. Table 1 lists every experiment with what it holds fixed and what it changes. The other named recipes belong to the supporting results of Section 6 and to the registered recipe comparison of Appendix B.
2. Recommendations and explanation. Concern: the paper would be more valuable with recommendations and guidelines for best practices, and a complete explanation is hard to find in the end.
In the mixed-depth corpus the direct answers (T1) are 42.1% of the questions but receive 5.6% of the loss coefficients under the token mean. The equal-example objective gives them their example share. Moving only the T1 targets to their example share costs 7.74 points on Qwen3 and 5.22 on OLMo; equalizing the examples within the T1 and the T2-T5 groups costs less (Table 2). A new experiment agrees (see our reply to Reviewer uj6D): when every T1 target of Guided-Mix-T2sub is replaced by the T2 target of the same question, the cost of the switch falls from 7.91 and 5.46 points to 1.34 and 1.42. The paper recommends stating the loss normalization with every comparison of recipes, and reporting each group's share of the examples and of the loss (Sections 7 and 8).
3. The reviewer's assessment. Concern: with the equal-example loss the model answers directly some questions that would require longer reasoning, and an instruction or a forced higher reasoning effort at evaluation could steer it.
The first part agrees with what we measured (Appendix J). Under equal-example weighting, students of the mixture answer directly on 46% instead of 22% of MATH-500 questions on Qwen3, and their mean response length falls from 125 to 48 tokens. Students of the fixed-depth corpus never answer directly. We also ran the evaluation half of the suggestion: we evaluated the students of Guided-Mix-T2sub again on GSM8K and MATH-500 with the instruction "Give a brief solution showing the necessary steps before the final answer" added to the prompt. The instruction hardly changed their behaviour: on Qwen3 the students trained under the equal-example objective still answer directly on 41.5% of MATH-500 questions, against 44.7% with the unchanged prompt in the same run. Their accuracy changes by -1.24 to +1.42 points across the two students and the two benchmarks, with every interval including zero, and the gap between the objectives does not close. Because the students mostly kept answering directly, this evaluation does not show whether more reasoning would recover the accuracy. An instruction in the fine-tuning prompts would change the training rows, which the comparisons in the paper hold fixed; it was not tested.