acceptodds
Under review as a conference paper at ICLR 2027

DETECT LLM FINE-TUNING OVERFITTING EARLY

Abstract

Four epochs out of fifty decide whether a fine-tuned model overfits, and four epochs are enough to catch it. Across twenty-two matched-pair comparisons spanning 3.8B to 32B parameters, an early check at epoch 4 calls the final test-set gap for every readable run, with the remaining pair settling by epoch 10. Both thresholds were locked in advance. The results uncover a clear pattern: models distilled from reasoning checkpoints overfit downstream tasks more than their standard instruction-tuned twins. Across four base models they show 1.06 to 4.60x larger drift and 1.05 to 1.84x wider final test gaps. In contrast, models built directly by vendors for reasoning or domain tasks behave much like their standard twins. Pairing models from the same base keeps the comparison clean by holding architecture, vocabulary and size constant. Using only raw training and validation loss curves, our metric splits the gap into two plain numbers: roughness a, how far the training losses sit from a smooth descent, and drift c, how far the validation curve pulls away. Their ratio c/a pinpoints what kind of overfitting is happening. On an unseen 32B pair run across two seeds, the rule flags heavy overfitting at epoch 4, correctly anticipating final drifts over 4.4x. While standard loss gaps top out at sixteen correct calls even with best-case hindsight tuning, our early rule scores a perfect eighteen for eighteen.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Overfitting, supervised fine-tuning, large language models, checkpoint selection, reasoning distillation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Dxe2OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper aims to predict the test-validation gap of a model at the end of post-training with early training dynamics, claiming that the test-validation gap at 50 epochs can be effectively predicted at epoch 4. It utilize roughness, how far the training losses sit from a smooth descent, and drift, how far the validation curve pulls away, as well as the ratio between them to characterize the extent of overfitting. The paper also shows that the checkpoints of distilled models are easier to overfit on downstream task.

Critical strengths
  1. The problem of predicting the future training results with early training dynamics is interesting.
Critical weaknesses
  1. The paper is poorly written and very hard to follow. Taking the abstract alone as an example:
  • At Line 11 the paper says 'for every run', but in the next sentence it begins with 'The remaining pair', which is very confusing.
  • Right after that sentence, at Line 12, the paper says 'Both threshold were locked in advance', but the previous context did not give any clue about even a single threshold.
  • Right after that, at Line 13, the paper claims that 'The results uncover a clear pattern...' and begins to compare distilled model with standard instruction tuned models. However, as there are no additional settings or experiments in the previous context, if 'the results' refer to the previous 'thresholds were locked in advance', it gives no clue about the comparison between distilled and standard models.
  • At Line 17, the paper says 'fine-tune like the twin', but doesn't specify what does 'twin' stand for and what does it mean to fine-tune like it.
  • At Line 25 the paper says 'No single-epoch reading of the same curves is right at both.' but doesn't specify what is 'read' from single epoch, and what is 'right at both'.
  • This is outside of abstract but super confusing. At Line 190 the paper mentions 'audit.py'. Also at Line 191, the paper says 'One number indexes; three numbers diagnose'. What is one number and three numbers here?
  1. The paper contains many basic mistakes. Taking the first few paragraphs of introduction as example:
  • At Line 36, the paper says 'Fine-tune the wrong one and the annotation budget is gone, hardest where labels are most expensive'. It does not make sense because the author supposed we already hold the labeled data at Line 34, and there is no annotation cost after we have already obtained the data.
  • At Line 37, the paper says 'Overfitting is normally legible only once a run is over', which is not true as we can always monitor the training and validation curve and don't have to wait until the end of run.
  • At Line 38, the paper says 'A small fine-tune drives the training loss to zero, ...', which is not likely to be true, depending on the definition of 'small'. A few epochs of SFT usually would not drive the loss to zero and usually people don't 50 epochs of post-training as the paper does.
  1. 50 epochs of training is an uncommon setting of LLM post-training. Typically standard instruction tuning only takes 3-5 epochs[1], reasoning post-training like s1[2] also use 5 epochs. It does not make sense to me to do 50 epochs of training and measure overfitting based on that.

[1] Taori et al. (2023). Stanford Alpaca: An Instruction-following LLaMA model.

[2] Muennighoff et al. (2025). s1: Simple test-time scaling.

Questions for authors

At Line 37, the paper asks 'Which of two released checkpoints will overfit my data sooner?', but the paper seems to try to answer which models will overfit 'more', rather than 'sooner'. Can you clarify on that?

Since we have early stopping approach at select when to stop training based on the lowest validation loss or best validation performance, why do we need to select checkpoints based on the final test-validation gap?

7 comments
AuthorsOpenReview ↗

The cited "simpler metric" uses our own ORI-AUC; epoch-15 ground truth leaves all 22 verdicts unchanged

We thank the reviewer for the comments and answer each with the reviewer's words quoted. Page and line numbers refer to the uploaded revision, where changes are in blue; the review's own line numbers refer to the original.

1. Fifty epochs

Review: "It does not make sense to me to do 50 epochs of training and measure overfitting based on that."

Epoch 50 is only "the settled endpoint the verdict read from the first E = 4 epochs is checked against" (p. 3, lines 129–130); the paper cites s1, LIMA and LIMO at up to 15 epochs (p. 3, lines 127–128). We re-read the ground truth earlier, from the released per-epoch test losses (new Appendix X, Table A22):

Truth read at8 separating runs14 matched runsMargin
Epoch 151.18–1.600.91–1.030.157
Epoch 101.06–1.610.79–1.050.012
Epoch 50 (paper)1.09–1.840.94–1.020.072

Entries are test-gap ratios A/B. At epoch 15 the assignments are identical and the early rule is right on 22 of 22. At epoch 10 they are the same with a thin margin. The revision states the scope: the claim covers horizons from epoch 6, and a run stopped at epoch 15 still carries the difference, the separating members' test gap being 1.18 to 1.60 times their twins' (p. 3, lines 130–133).

2. Early stopping (Q2)

Review: "why do we need to select checkpoints based on the final test-validation gap?"

Early stopping decides when to stop one run. The paper asks which of two checkpoints to start from; the revision says so in Section 3: "Early stopping decides when to stop one run; the rule decides which checkpoint to start from" (p. 2, lines 102–103). The quantity is also different: the paper's truth is test loss minus training loss (Eq. 1, p. 3, lines 111–112). The term "test-validation gap" does not occur in the paper.

3. The three "basic mistakes"

Review: "'Overfitting is normally legible only once a run is over', which is not true as we can always monitor the training and validation curve"

Table A9 gives what that monitoring shows at epoch 4. The raw validation-gap ratio of the separating pair P8 is 1.06–1.08, while the matched pairs P3 and P7 are at 1.07–1.12. The ranges overlap, so the curves do not yet tell the two groups apart, which is the point of the sentence. The revision now carries this evidence in the sentence itself (p. 1, lines 040–041; Figure 6e).

Review: "'A small fine-tune drives the training loss to zero, ...', which is not likely to be true"

The words left out are "numerically zero at the final epoch in every run here" (p. 1, lines 041–042). It is a measured fact about these runs, and "small" refers to the thousand examples of line 036.

Review (line 36, annotation budget): A fair point. The sentence now reads "Fine-tune the wrong one and the labels buy a model that overfits them" (p. 1, lines 036–037).

4. The abstract

ReviewText of the revision
"the paper says 'for every run'""on the twenty runs readable there. The two remaining runs, one pair, are read at epoch 5" (lines 012–013)
"if 'the results' refer to the previous 'thresholds were locked in advance'""The verdicts also show a pattern" (lines 013–014): the verdicts of the twenty-two pair runs of line 010
"doesn't specify what does 'twin' stand for""the instruction-tuned twins released from the same base" (line 015)
"what is 'right at both'""No simple aggregate of the same curves, read at one epoch, is right at epoch 4 and at every later horizon" (lines 026–028)
"What is one number and three numbers here?""ORI-AUC indexes; the three distances a, b, c diagnose" (p. 4, line 194)

The other fixes are made: the abstract names the two thresholds ("The rule uses two thresholds, both locked in advance", line 013); audit.py is introduced as "The released script audit.py" at first use (p. 4, line 193); and "sooner" now reads "more" (p. 1, line 038; Q1).

Request

Every line the review cites lies in lines 11–38 or 190–191 of the original 42-page paper. No experiment, table or theorem is questioned, yet soundness is rated as having concerns that "do not appear easy to address". The one soundness point, the 50-epoch horizon, is answered in Section 1 with the released logs. We would be grateful if the reviewer would reconsider the scores.

P.S. To the Area Chair

This review rates soundness 1 without discussing any result, table or theorem; its line references stop at line 191. Its main objection (50 epochs) is answered above: at epoch 15 the grouping of all 22 runs is unchanged. We would be grateful if this were taken into account.

AuthorsOpenReview ↗

Summary of paper updates: epoch-15 truth 22/22, accuracy 32/32

We thank all reviewers and the Area Chair. The revised PDF is uploaded; every change is in blue and the main text stays within 10 pages.

  1. Ground truth at earlier epochs (WvNW Q2, Dxe2). Read at epoch 15, the test gap gives the same grouping on all 22 pair runs, and the epoch-4 rule agrees on 22 of 22. With a pair's two seeds averaged, the two groups are apart at every epoch from 6 to 50. A run stopped at epoch 15 still carries the difference: 1.18 to 1.60 times the twin's test gap. (Section 3, p. 3, lines 130–133; Appendix X)

  2. Accuracy (WvNW Q1). On the full GSM8K test set, all four separating pairs, eight checkpoints: the flagged member is the less accurate one at 32 of 32 readings; at epoch 50, pooled, 74.0% against 79.4% (p = 2 × 10⁻¹¹). (Section 5.2, p. 6, lines 315–324; Appendix Y)

  3. Simple aggregates (es6e W1, WvNW Q3). The rule is right on all 20 runs at epoch 4 and at each of the five later horizons; no single-epoch reading matches it. Paired with ORI-AUC, which this paper introduces, the RMS gap gives the same numbers, as Theorem 2 predicts. Calibrated out of family, the drift reads 19/20 with one false alarm, the validation-loss ratio 17/20 with three. (Section 5.3, p. 7, lines 352–367)

  4. Other protocols (es6e W2). At rank 64 and on a difficulty-matched split, the thresholds fixed at rank 16 flag all three re-run pairs at epoch 4. The verdict holds across four families, three tasks and 1.5B to 32B parameters; a new paragraph names full-parameter fine-tuning as the next step. (Figure 5e; Section 6, p. 10, lines 524–538)

  5. Writing (WvNW W1, Dxe2). The abstract names the two thresholds, the pair read at epoch 5 and the twins; "index" means ORI(t) only; Section 5 has numbered heads 5.1 to 5.4 tied to the contributions; new Figure 6 puts the epoch-4 reading beside every check made on it.

Reviewer_Dxe2OpenReview ↗

Official Comments by Reviewer Dxe2

Thanks for your rebuttal. However, I still have significant concern over the motivation and presentation quality of the paper.

  • I understand the authors are trying to predict the train-test gap (test loss minus training loss), at 50 or 15 epochs based on the information in first 4 epochs, but why it is practical is still unclear to me. If our goal is to select which checkpoint to start tuning with, then the correct objective of this is to predict the test loss or downstream performance of the model before it starts to overfit. Which means, for every checkpoint, we use early stopping to terminate the run when the model is overfitting. The validation loss or the downstream performance of model we get through this process are valid objectives to predict and compare with, rather than how each model overfits at a same epoch across different models. Whether a fine-tuned model will overfit on a task and how much will the model overfit on the task depend on the training schema and are not properties determined by the base model checkpoint. In another word, if you tune and stop wisely (or stop more aggressively), the worst base model in your experiment may don't overfit at all.
  • The current abstract is still difficult to understand. For one example, in line 13 the revised paper says 'the rule uses two thresholds', but nowhere before or near line 13 does the paper explain what the thresholds are or what are the principles of these thresholds. Only at line 21 the paper mentions two numbers before it explains in highly descriptive ways about a, c, and the ratio between them, which are three numbers and only a/c is one of the thresholds that the paper use. The reader can only speculate what are the thresholds.

Due to the reasons above, I cannot raise my score.

Reviewer_WvNWOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

The authors propose a method for predicting, early in training, which of two model checkpoints will overfit more on a given dataset.

Using two metrics, ORI-AUC and drift, computed from training and validation losses, they predict the relative degree of overfitting at the end of training based on measurements from as early as epoch 4 (or epoch 5), across 22 matched-pair runs.

They also provide a theoretical analysis of the relationship between ORI and drift, and empirically demonstrate that reasoning-distilled checkpoints are more prone to overfitting than instruction-tuned checkpoints, followed by an approximate mathematical analysis of this observation.

Critical strengths

The motivation of detecting overfitting early during fine-tuning is meaningful in practical settings, and the authors provide a theoretical analysis of the proposed approach.

Critical weaknesses

1. The writing has several notable shortcomings.

Details that are not essential to understanding the paper are frequently inserted into sentences, making the overall argument difficult to follow, and several key concepts are introduced without sufficient explanation or clear definition.

There are also inconsistencies in terminology, such as referring to both ORI and ORI-AUC as an "index." Furthermore, some of the paper's contributions do not seem to be sufficiently emphasized outside the introduction.

Overall, these writing issues prevent the paper from effectively communicating its main ideas and key takeaways to readers.

2. There are several concerns regarding the experimental setup.

The authors state that their small-sample, multi-epoch setting is inspired by LIMA, s1, and LIMO. However, all three studies train for only 5–15 epochs, rather than 50 epochs.

Therefore, using the results at epoch 50 as the ground truth for determining overfitting does not seem to reflect a realistic training setting.

Moreover, the validation loss in Figure 2 begins to increase after approximately 5 epochs. Considering that early stopping would likely have been applied around this point in practice, it is questionable whether predicting the outcome after 50 epochs of training represents a realistic scenario.

In addition, although the proposed overfitting detection method outperforms other gap-based methods, achieving 20/20 correct predictions compared to 14–18/20 for several gap-based baselines, the sample size of only 20 makes it difficult to determine whether the proposed method genuinely offers an advantage over the alternatives.

Questions for authors

1. The paper uses test gap as the metric for determining the final degree of overfitting. However, since downstream performance can sometimes improve even when test or validation loss increases, end-to-end performance on benchmarks also seems to be an important evaluation metric.

Currently, the appendix reports inference performance only at epochs 4 and 50 for two pairs. If time and computational resources permit, could you provide benchmark evaluation results for more pairs using checkpoints from epochs 5 to 50, at intervals of 5 or 10 epochs?

2. Given that LIMA, s1, and LIMO train for only 5–15 epochs, could you investigate whether the separates/matches group assignments change when the ground truth is defined using the test gap at epoch 10 or 15, and whether the predictions made at epoch 4 remain consistent with this alternative ground truth?

It would also be helpful to compare the proposed method against the other gap-based baselines under this setting.

3. In the Experiments section, you state that the validation-loss ratio, as well as other gap-based metrics combined with the index, also achieve 20/20 correct predictions. If so, the proposed method no longer appears to have a distinct advantage in making accurate predictions at early epochs.

Furthermore, Table A19 reports that the proposed method is more stable at later horizons, but the RMS gap appears to exhibit similar stability. What, then, are the specific advantages of the proposed method over the alternative approaches?

5 comments
AuthorsOpenReview ↗

Both tests you asked for are done: grouping unchanged at epoch 15 (22 of 22 runs); flagged model less accurate at 32 of 32 readings

We thank the reviewer for a close reading; the summary states our method, theory and main finding accurately. Each point is answered below; new Figure 6 puts the checks in one place. Page and line numbers refer to the revision, where changes are in blue.

1. Ground truth at epoch 50 (W2, Q2)

Review: "could you investigate whether the separates/matches group assignments change when the ground truth is defined using the test gap at epoch 10 or 15"

We did (new Appendix X, Table A22).

Truth read at8 separating runs14 matching runsMargin
Epoch 151.18–1.600.91–1.030.157
Epoch 101.06–1.610.79–1.050.012
Epoch 50 (paper)1.09–1.840.94–1.020.072

Entries are test-gap ratios A/B. At epoch 15 the assignments are identical to the paper's, with a wider margin than at epoch 50. The early rule (epoch 4 on twenty runs, epoch 5 on the PubMedQA pair) agrees with the epoch-15 truth on 22 of 22. With a pair's two seeds averaged, the two groups are apart at every epoch from 6 to 50. Against that truth every count of Section 5.3, baselines included, is unchanged (p. 7, lines 365–366). Appendix X gives the run-by-run reading.

2. Early stopping (W2)

Review: "early stopping would likely have been applied around this point in practice"

The rule answers which of two checkpoints to start from. A run stopped at epoch 15 still carries the difference: the flagged members' test gap is 1.18 to 1.60 times their twins'. The claim covers stopping points from epoch 6, where the two groups are already apart. The revision states this scope (Section 3, pp. 2–3).

3. Sample size (W2)

Review: "the sample size of only 20 makes it difficult to determine whether the proposed method genuinely offers an advantage"

Twenty runs cannot rank 20/20 against 18/20, and the paper does not rest on that margin. It rests on three things. The baselines were each given their best threshold after the fact, while ours were fixed in advance. The fixed thresholds then held on four held-out runs and the 1.5B pair (p. 7, lines 366–367), and in the revision on three pairs re-run at rank 64 and on a difficulty-matched split, all flagged at epoch 4 (Figure 5e). The thresholds sit outside the measured noise: seed-to-seed deviation of the log drift ratio is 0.0189 against a threshold of 0.140 (p. 6, lines 290–291), and under the simulated null the ORI-AUC threshold keeps the false-alarm rate at or below 5% (Table A7).

4. Advantage over alternatives (Q3)

Review: "What, then, are the specific advantages of the proposed method over the alternative approaches?"

The rule is right on all twenty runs at epoch 4 and at each of the five later horizons; no simple aggregate alone matches it at every horizon (Figure 6f). The validation-loss ratio does read 20/20 at epoch 4 (p. 7, lines 362–363), but it is right on all twenty at only two of the five later horizons. The gap-type readings that reach 20/20 do so paired with ORI-AUC, which this paper introduces. Calibrated on other model families (Table A19), the validation-loss ratio reads 17/20 with 3 false alarms, the drift 19/20 with 1. The RMS gap is the drift's upper bound in Theorem 2, so their agreement is a check of the theory; alone it reads 18/20.

5. Accuracy (Q1)

We ran it on all four separating pairs (full test set, eight checkpoints, epochs 4–50; Figure 3e–i, Table A23), and accuracy agrees with the rule. The flagged member is the less accurate one at 32 of 32 readings. At the paper's horizon, epoch 50, it is significantly less accurate in every pair (p = 0.031, 0.001, 10⁻⁷, 0.030); pooled, 74.0% against 79.4% (p = 2 × 10⁻¹¹; Section 5.2, p. 6, lines 315–324). In the 14B and Qwen3-8B pairs the flagged member falls in all five of its runs. The held-out 32B pair, flagged at epoch 4 on both seeds, is lower at 8 of 8 checkpoints. In the held-out Qwen3-8B pair the gap grows from 3.1 to 5.3 points, significant in both runs (p = 0.002, 0.0008).

6. Writing (W1)

Review: "referring to both ORI and ORI-AUC as an 'index.'"

This is a fair point. In the revision "index" means ORI(t) only, and ORI-AUC is written by name wherever the run-level average is meant. Section 5 now has numbered heads tied to the contributions (p. 6, lines 300–301).

Request

The soundness concerns are rated as ones that "do not appear easy to address". Q1 and Q2 are answered above with the tests the review asked for, and Q3 from Table A19. We would be grateful if the reviewer would reconsider the soundness score and the rating.

P.S. To the Area Chair

The central objection of this review is that epoch 50 is an unrealistic ground truth. The reviewer proposed the test (Q2), and we ran it: at epoch 15 the grouping of all 22 runs is unchanged. We also ran the accuracy test of Q1: the flagged member is the less accurate one at 32 of 32 readings. We would be grateful if this were taken into account.

AuthorsOpenReview ↗

Summary of paper updates: epoch-15 truth 22/22, accuracy 32/32

We thank all reviewers and the Area Chair. The revised PDF is uploaded; every change is in blue and the main text stays within 10 pages.

  1. Ground truth at earlier epochs (WvNW Q2, Dxe2). Read at epoch 15, the test gap gives the same grouping on all 22 pair runs, and the epoch-4 rule agrees on 22 of 22. With a pair's two seeds averaged, the two groups are apart at every epoch from 6 to 50. A run stopped at epoch 15 still carries the difference: 1.18 to 1.60 times the twin's test gap. (Section 3, p. 3, lines 130–133; Appendix X)

  2. Accuracy (WvNW Q1). On the full GSM8K test set, all four separating pairs, eight checkpoints: the flagged member is the less accurate one at 32 of 32 readings; at epoch 50, pooled, 74.0% against 79.4% (p = 2 × 10⁻¹¹). (Section 5.2, p. 6, lines 315–324; Appendix Y)

  3. Simple aggregates (es6e W1, WvNW Q3). The rule is right on all 20 runs at epoch 4 and at each of the five later horizons; no single-epoch reading matches it. Paired with ORI-AUC, which this paper introduces, the RMS gap gives the same numbers, as Theorem 2 predicts. Calibrated out of family, the drift reads 19/20 with one false alarm, the validation-loss ratio 17/20 with three. (Section 5.3, p. 7, lines 352–367)

  4. Other protocols (es6e W2). At rank 64 and on a difficulty-matched split, the thresholds fixed at rank 16 flag all three re-run pairs at epoch 4. The verdict holds across four families, three tasks and 1.5B to 32B parameters; a new paragraph names full-parameter fine-tuning as the next step. (Figure 5e; Section 6, p. 10, lines 524–538)

  5. Writing (WvNW W1, Dxe2). The abstract names the two thresholds, the pair read at epoch 5 and the twins; "index" means ORI(t) only; Section 5 has numbered heads 5.1 to 5.4 tied to the contributions; new Figure 6 puts the epoch-4 reading beside every check made on it.

Thank you for the additional experiments on the epoch-15 ground truth and the downstream accuracy, which address part of my concerns about the experimental setup; I will raise my soundness score by one point.

Reviewer_es6eOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a way to select checkpoints such that it does not overfit on the downstream task. The paper proposes combining the existing Overfitting Robustness Index (ORI) with Wasserstein drift to compare the fine-tuning behavior across LLM checkpoints. The results show that they are able to predict the generalization gap relatively well between checkpoints.

Critical strengths
  • The paper addresses a practical problem, and their method is computationally inexpensive
  • The results are shown on different datasets across different model families
Critical weaknesses
  • The proposed method shows limited improvement over a simpler loss-based metric. From Table A19, RMS gap combined with ORI-AUC achieves identical prediction performance as the proposed Wasserstein-based method. It is unclear if the only advantage is a single-metric out-of-family comparison.
  • It is unclear if the selected fine-tuning protocol or hyperparameters generalize to other commonly used fine-tuning protocols.
Questions for authors
  • Please see the weakness section

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.