- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies whether probability-based token-weighting strength should depend on a pretrained model’s familiarity with each demonstration. It identifies a gap between token-local weighting and sample-level information that can also reshape within-response weights. FamiFT computes each response’s geometric-mean target probability under the frozen initial model and sets a fixed exponent on detached online token probabilities. Experiments fine-tune two math models and one code model on task-specific 10,000-example datasets. The authors report the highest benchmark-averaged accuracy among the compared methods and use fixed, shuffled, reversed, and matched-weight controls to investigate the mechanism.
Critical strengths
- Concrete sample-adaptive extension: The technical delta is replacing DFT’s shared probability exponent with an initial-familiarity-dependent exponent. This is a novel combination, not a new problem or theorem: unlike DFT it changes weighting strength per example; unlike TALR it does not share a batch temperature; unlike CADFT it changes token ratios rather than only sample scale; unlike PriFT it aggregates frozen-model probabilities to modulate online weights instead of directly weighting tokens. My judgment is that this incremental delta could meet ICLR’s bar if the proposed insight is reliably established.
- Controls test the correspondence, not merely the full system: Shuffling preserves the exponent distribution while breaking example-score matching, and reversal tests assignment direction. Reported averages are 37.20 for FamiFT, 35.88 for shuffling, and 35.64 for reversal. These are more diagnostic than comparing only against SFT, although the protocol concern below remains.
- Low-complexity training modification with matched backbones/data: All methods share checkpoints, task data, and training configurations, reducing backbone and extra-data confounds. Scoring is performed once; the training rule uses stored scalars and current probabilities, with no added inference mechanism. Reported gains over the strongest weighting baseline are 0.44 and 0.98 math points and 0.51 code points. Modest gains are not intrinsically a weakness.
Critical weaknesses
- Sampling provenance prevents validating controlled comparisons: Table 1 labels results Avg@16, whereas Table 8 labels identical per-benchmark FamiFT/DFT rows Avg@8; Table 9 likewise repeats every FamiFT/DFT Pass@16 entry from Table 5 as Pass@8. Appendix B.2 explicitly distinguishes these budgets. Equality is not proof of an error, but the text does not establish whether controls and references used matched samples. This affects the mechanism evidence, not merely presentation. Provide generation-file provenance and recompute all four matched controls at one budget; if this is only a labeling error, the concern becomes minor.
- Reliability and tuning fairness remain unestablished: Training uses seed 1, and uncertainty across training runs is not reported in the main text or appendix. The principal incremental margins are 0.44–0.98 points. Baseline thresholds/coefficients are fixed, while FamiFT receives exponent sweeps; matched tuning budgets and a held-out selection protocol are not reported in the main text or appendix. Shared configurations do not establish equal optimization effort. Report at least three paired training seeds, uncertainty, and equal-budget baseline selection on held-out data.
- Attribution supports a joint rule more than an independent shape benefit: Shape-only scores 36.48 and scale-only 35.44, both below DFT’s 36.76; only their combination reaches 37.20. The conditional comparison favors shape, but isolated changes do not improve DFT; the mechanistic interpretation should emphasize interaction rather than an independently beneficial component. The controls also recompute weights from their own trajectories.
- Exact reproduction remains underspecified — source code is missing, and data, baselines, and metrics are partially specified.
Questions for authors
(W1) Were all matched controls evaluated with the same eight or sixteen generations? Matched provenance would raise my score; unequal budgets requiring materially changed results would lower it. (W2) Do three paired seeds and equal-budget held-out baseline tuning preserve the incremental gains? Consistent gains would move me toward acceptance; disappearing gains would reinforce rejection. (W3) Does the shape-by-scale interaction remain positive across seeds, and does the familiarity-split effect survive response-weight normalization? (W4) Can you provide exact subset IDs, optimizer configs, trainer revision, and a minimal gradient-reduction example reproducing Appendix A.1?
4 comments
3. Hyperparameter fairness (W2).
FamiFT uses the fixed rule in all main experiments. We did not tune this rule for individual benchmarks. The default FamiFT rule was fixed across all models and benchmarks. The exponent sweeps were conducted only to analyze sensitivity and were not used to select the main experimental results.
For the tunable baselines, we followed their recommended settings and further evaluated five candidate values for each method. We used a fixed 500-problem development set from the official 7,500-problem MATH training split. Each configuration was trained with three seeds and evaluated with six samples per question. We report the average across seeds. The recommended configurations achieved the highest Avg@6 among the tested values:
| Method | Selected Parameter | Avg@6 | Pass@6 |
|---|---|---|---|
| ASFT | λ = 0.01 | 68.9 | 87.4 |
| ASFT | λ = 0.03 | 72.5 | 87.8 |
| ASFT | λ = 0.05(default) | 74.7 | 89.6 |
| ASFT | λ = 0.07 | 72.2 | 88.8 |
| ASFT | λ = 0.10 | 72.3 | 89.0 |
| TALR | = 0.003 | 68.0 | 79.8 |
| TALR | = 0.007 | 71.4 | 85.2 |
| TALR | = 0.01(default) | 71.8 | 84.6 |
| TALR | = 0.02 | 69.8 | 80.2 |
| TALR | = 0.03 | 69.7 | 85.8 |
| ProFit | = 0.05 | 68.8 | 86.6 |
| ProFit | = 0.10(default) | 73.8 | 87.0 |
| ProFit | = 0.30 | 72.1 | 82.6 |
| ProFit | = 0.20 | 70.1 | 82.0 |
| ProFit | = 0.50 | 69.3 | 82.0 |
These results support our choice of baseline hyperparameters. We will include the full validation results in the appendix.
4. Shape–Scale Interaction and Familiarity-Split Analysis (W3)
We repeated the matched-weight ablation across three training seeds.
| Method | Mean | Ratios | Avg@8 (Mean ± Std) | Pass@8 (Mean ± Std) |
|---|---|---|---|---|
| DFT | D | D | 36.21 ± 0.50 | 54.79 ± 1.02 |
| Scale-only | F | D | 34.82 ± 0.69 | 53.40 ± 1.61 |
| Shape-only | D | F | 36.81 ± 0.31 | 55.72 ± 0.33 |
| FamiFT | F | F | 37.20 ± 0.08 | 57.25 ± 1.53 |
Shape-only now slightly outperforms DFT in the three-seed average, while Scale-only performs worse. Combining both effects in FamiFT gives the best performance.
We also computed the shape–scale interaction:
$ I=\mathrm{FamiFT}-\mathrm{Shape}-\mathrm{Scale}+\mathrm{DFT}. $
The interaction is positive for all three seeds, with an average of +1.78 ± 0.24 points in Avg@8 and +2.92 ± 1.89 points in Pass@8.
This supports the reviewer's observation that the two components interact. We will revise the mechanism discussion to emphasize this interaction.
5. Familiarity-Split Experiment with Response-Weight Normalization (W3)
To further investigate whether the familiarity-split effect comes from response-level scaling or relative token weighting, we extended the motivating experiment in Figure 2 with Shape-only and Scale-only controls across three seeds.
Shape-only preserves the exponent-dependent token ratios while matching each response's mean weight to DFT. Scale-only retains the exponent-dependent response-level mean weights while using DFT's token ratios. Each entry reports Avg@8 ± Std / Pass@8 ± Std (%).
| Setting | Original | Shape-only | Scale-only |
|---|---|---|---|
| DFT | 33.79±1.70 / 55.81±2.09 | 33.80±2.58 / 54.71±1.99 | 32.11±1.34 / 52.33±0.85 |
| Fixed 0.9 | 32.98±0.39 / 55.48±2.68 | 33.15±2.58 / 55.13±2.60 | 35.01±0.44 / 56.64±1.79 |
| Fixed 1.1 | 32.75±1.65 / 53.46±3.05 | 32.51±2.25 / 53.31±2.42 | 34.05±2.20 / 56.60±3.31 |
| Mixed | 35.31±0.62 / 58.15±0.11 | 34.49±0.99 / 57.43±0.13 | 33.45±2.07 / 55.29±2.12 |
| Reversed | 31.95±0.36 / 52.21±1.50 | 33.33±0.10 / 55.08±0.61 | 32.91±1.87 / 55.76±3.30 |
Mixed achieves the highest mean Avg@8 and Pass@8 in both the original and Shape-only settings. In contrast, under Scale-only weighting, Fixed 0.9 achieves the highest average accuracy, while Mixed no longer performs best.
These results suggest that response-level scaling alone does not explain the familiarity-split effect. The advantage of Mixed remains when relative token weights are preserved and response-level mean weights are normalized.
Together with the positive shape–scale interaction observed in our multi-seed ablation, these findings support the importance of familiarity-adaptive token ratios and their interaction with response-level scaling.
6. Reproducibility (W4)
We have added the training code, trainer source snapshot, and reproduction scripts for Tables 1 and 2 to the supplementary material.
Both math models use the same 10,000-example NuminaMath-CoT subset. The code model uses a 10,000-example OpenCodeInstruct subset. The supplement provides the exact training files, ordered sample IDs, and trainer checksums.
We use AdamW with , , weight decay 0.01, gradient clipping at 1.0, cosine learning-rate scheduling, and a global batch size of 256. The math models train for one epoch at with 10% warmup. The code model trains for three epochs at with two warmup steps.
We also added a minimal CPU example for Appendix A.1. It verifies response-token normalization, data-parallel gradient averaging, and microbatch gradient accumulation. The division by the number of microsteps affects only the logged loss, as it occurs after backward().
We thank the reviewer again for the suggestions. These additional experiments and reproduction materials strengthen the evidence for FamiFT and clarify the scope of our conclusions.
Dear Reviewer gT91,
Thank you again for your detailed and constructive feedback. We have now addressed your concerns (W1–W4) with additional experiments, and reproducibility materials.
We would greatly appreciate it if you could take another look at our responses and reconsider your assessment in light of the new evidence. Please let us know if any concerns remain. We would be happy to provide further clarification.
Best regards,
The Authors
We thank the reviewer for the detailed and constructive feedback. We appreciate the recognition of our method and controlled analyses. We conducted additional experiments to address the concerns about evaluation consistency, robustness, hyperparameter fairness, and the shape–scale interaction. We also added reproduction materials to the supplementary material.
1. Evaluation Consistency (W1)
We apologize for an error in reporting the evaluation budget. The Qwen2.5-Math-7B results in Table 1 were mistakenly reported using 8 samples instead of 16. The corresponding Pass@n results in Table 5 also need correction. Tables 8 and 9, which report the matched-weight ablations, correctly use 8 samples for all methods. We re-evaluated the main Qwen2.5-Math-7B results using 16 samples. The corrected Avg@16 scores are:
All matched-weight controls in Tables 8 and 9 were evaluated using the same sampling budget (n=8). Thus, the controlled comparisons themselves used matched evaluation budgets. We have also re-evaluated the main Qwen2.5-Math-7B experiments using n=16. FamiFT still achieves the highest average accuracy (37.28), outperforming DFT (36.68) by 0.60 percentage points. We will correct Tables 1 and 5 and clarify the evaluation budgets.
2. Multi-Seed Robustness (W2)
We ran two additional training seeds for all three original models. The math models were evaluated using Avg@8 and Pass@8, while the code model was evaluated using pass@1. The tables below report the results averaged across three training seeds.
Qwen2.5-Math-7B
Qwen3-8B-Base
Qwen2.5-Coder-3B-Instruct
FamiFT retains the highest benchmark-averaged accuracy on all three models. It also achieves the highest mean accuracy on four of five benchmarks for each math model and all four code benchmarks.