acceptodds
Under review as a conference paper at ICLR 2027

Same Fit, Different Functions: Evidence Structure and Training Order Shape Held-Out Behavior

Abstract

Training fit alone does not determine behavior away from observed examples. We study this in controlled contradictory-supervision tasks where compared training runs reach matched or saturated training fit but can express different held-out predictive mappings. In Study I, using Qwen2.5-0.5B and a fixed budget of 200 exceptions, increasing the fraction of exceptions aligned with a shared secondary relation raises mean held-out transfer from 0.0808 to 1.0000 across five coherence levels, strictly in all three seeds. A second modular-composition family reproduces the positive fully coherent versus fully idiosyncratic endpoint contrast in three fresh seeds, but not the graded response. In Study II, after first establishing a primary predictor, all 21 Phase-2 arms fit their training examples, yet 200 fully coherent corrections yield complete primary retention and secondary transfer in every seed, whereas 100 coherent corrections produce highly variable transfer. Finally, in an exploratory fixed-source intervention, six schedules with identical correction counts at every optimizer step and matched total exposure yield terminal transfer from 0.1775 to 0.9900; selected extreme schedules replay exactly. Thus, in these controlled tasks, evidence organization and training order can alter which held-out predictive mapping emerges despite matched terminal fit. The results are behavioral and establish neither an internal mechanism nor a universal or cross-model law.

Primary area:
foundation or frontier models, including LLMs
Keywords:
generalization, memorization, inductive bias, contradictory supervision, training dynamics, data ordering, function selection

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_5iwMOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies how models generalize functions in the presence of varying degrees of exceptions / noise, as well as under varying training orders. It finds (consistent with prior work) that both factors can affect generalization.

Critical strengths
  • This work runs reasonable experiments for what it sets out to test, and qualifies its results according to its limitations.
Critical weaknesses
  • These findings simply aren't new. Works the authors cite such as Arpit et al., as well as others they don't like https://aclanthology.org/2020.blackboxnlp-1.21/ or https://arxiv.org/abs/2412.04619 or https://arxiv.org/abs/2103.04514 have clearly shown that 1) fit on (different) training data will obviously affect generalization (and in particular that even if models can memorize training data, they will learn more generalizing structure) and 2) fit on the same training data, including induced by changes in ordering, can substantially change generalization. It is not obvious to me how this paper contributes a publishable new result beyond these, and the authors never make a case for it.

  • The descriptions are extremely hard to follow and the entire work reads like a language model performed all the experiments and wrote it. Of course, having LM experiment and writing assistance is reasonable if the paper makes a useful research contribution, but its unclear that this does.

Questions for authors
  • What new things do we learn from this relative to prior works?
Reviewer_3YBfOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper appears to focus on how models learn exceptions, with a focus on understanding the impact of data ordering and the composition of exceptions. They examine a simplified synthetic setting where the relationship between tokens in the prompt must be used to predict the next token. They then compare a joint-learning setting where “exception” (rule-violating) examples are either presented for joint training (i.e. mixed into the training data) or separately trained on during a “correction phase”. They show that despite all receiving 100% training fit random exceptions are memorized and don’t generalize, that exceptions following a coherent rule do generalize. In a sequential editing setting, they show that mini-batch order can significantly affect generalization.

Critical strengths

None

Critical weaknesses
  • Unclear Motivation for the Setting: The paper presents a rule-learning setting in which the model uses equality relations between tokens to route the model to two different types of rules, as well as a modular relation. This setting is presented with no justification, nor motivation in terms of representing a synthetic task. In general I really struggle to understand why this is a useful/reasonable setting to study. In general, I think that the authors should spend more time either (a) explaining why this setting is likely to isolate a conceptually transferable finding about neural network training (b) further ablate their setting or © explain why this is a valid “simplified/controlled example” used to study a real application. I do not mind studies of simplified setting (such as in-context linear regression [1] or [2]) and they can be highly impactful but they do need to be selected and justified properly

    [1] https://iclr-blogposts.github.io/2024/blog/understanding-icl/

    [2] Nichani, Eshaan, Jason Lee, and Alberto Bietti. "Understanding factual recall in transformers via associative memories." International Conference on Learning Representations. Vol. 2025. 2025.

  • Core Claim is Fairly Well Known: As far as I understand, the core finding of this article is that the same level of training-data fitting can come from different internal functions, with different held-out behavior. I discern that this is the claim on the basis of the title of the article

    Same Fit, Different Functions: Evidence Structure and Training Order Shape Held-Out Behavior

    Training accuracy tells us whether a model has matched the observed labels, but not which predictive rule it will apply elsewhere.

    Unfortunately, this finding is not particularly novel. A significant amount of the classical machine learning literature is focused on this claim, including studies on explicit and implicit regularization. If there is a novel finding in these results that goes beyond this claim, then I feel that the authors should explicitly frame and center that and make clear what they contribute beyond the existing understanding of this phenomena. If not, such a finding is likely not a reasonable contribution to a top-tier conference.

  • Results Appear Unstable: Authors disclose that there are substantial variations in their results depending on seed etc. Given their already limited study of a simplified setting, I believe that this calls into question the generality/significance of their findings.

  • Readability: In general, I found this paper highly challenging to read, to the point that I feel that it may be interfering with my comprehension of the paper. I will list several examples below, but in general the paper needs significant revisions for clarity to be ready for reasonable peer-review, let alone publication at a top-tier conference.

    • micro-order

    This term is first used around line 59, in the introduction, but as far as I can tell is not ever properly defined. I still don’t entirely understand what is the intervention here (though I gather it is relevant to the ordering of how data is presented).

    • or the OOD control, the fit, primary-OOD (≥ 0.80), lineage, repeat, numerical, and resource gates apply to the six P1 c = 0 and c = 1 endpoint runs

    This line appears to be describing some kind of experimental control that is being run, yet I cannot seem to understand what “lineage”, “repeat”, “numerical”, and “resource” gates precisely refer to. This phrase is repeated across multiple parts of the paper and seems to be somewhat important for interpreting the results.

    • A separate preregistered control holds out complete joint cells over c ∈ {0, .25, .5, .75, 1} on one fixed 50/50 cell partition, retaining every atomic feature value and all 25 secondary pairs in training; the c = 0 versus c = 1 contrast is confirmatory and other levels descriptive (Appendix B)

    What is an atomic feature value here? It is never specified in the preceding text I also don’t understand what the feature combination OOD control refers to (the name is sufficiently generic that I cannot infer any information about it.

    • q-profile

    Another term that appears in discussing the experiments, but the authors fail to precisely define

    Other writing concerns are that the exact conclusions of different experiments are not presented in a clear and easy to read manner, excessive numerical results in the main text disrupt reading flow, captions do not contribute to interpretation of figures. Overall I feel that the quality of the writing alone bars this paper from acceptance.

Questions for authors

Please respond to the points that I detailed in the Critical Weaknesses section.

  • Can you describe what exactly is the justification for the setting that you consider. Is it supposed to be indicative of some real-world setting? Does it exemplify some interesting theoretical property? If not why did you select it

  • Can you clarify what your findings contribute besides the "models can implement different solutions at identical fit"? This is well-known in the literature and the basis for implicit/explicit regularization techniques and the study of generalization in general.

  • What novel findings do you contribute on top of this literature? You list some relevant works on models memorizing random-labels etc in your related works. Please add a discussion of how your results add/clarify to this literature

  • What is your proposed mechanism for why mini-batch ordering changes correction training? The mere fact that training order affect model behavior is not novel, so it would be great if you had some kind of further explanation or contribution relative to this.

  • Why do you run these experiments starting from a pre-trained Qwen model -- what is the rationale for this. How sensitive do you suspect your results would be to other pre-trained base model? Why not just train a model from scratch?

Reviewer_ry3ROpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This is a very interesting and longstanding question in ML research, as we all understand the classic intuition that neural networks might interpolate the same data and implement drastically different functions; this is of course worth understanding. This paper sets out to tackle this question on an extremely tightly defined task, based on this "contradictory examples" framework.

Critical strengths

The topic of this paper is highly interesting and I agree with the approach of studying a highly specific, well-defined task. Afterall this is one of the most principled things we can do in ML research. The general conclusion seems to make sense: unseen "secondary" accuracy increases with "exception coherence" ... as well as primary and secondary hold-out performance is related.

Critical weaknesses

I think this paper's largest weakness is a severe lack of exposition and clarity in the experimental setting. Granted, this is not my direct topic of research, so perhaps I am just not as familiar with all the vocabulary as the intended reader. That said, my concerns on readability are explicitly as follows:

  • The abstract reads as an experimental setup or a caption to a main figure rather than an abstract. The main question I agree is motivated well and then immediately all that remains are very specific figures regarding the experimental setup and results. Without first reading the paper I do not know what "exceptions" are, let alone why "200 exceptions" is important or empirically interesting. Reading further, I do not know what "complete-cell holdout" is, nor do I understand the value of "positive fully coherent endpoint contrast." I do not know what "primary retention" and "secondary transfer" are; what does this tell me about machine learning? What does this tell me about different interpolating functions? What do values of G between 0.1775 and 0.9900 mean experimentally; these numbers are meaningless without context. Again, this is the abstract, my very first introduction and first impression of the paper. If these are domain specific words and numbers, then I apologise and I will amend this portion of the review, but as it stands now, the abstract in my opinion should be substantially rewritten.
  • NN is used in paragraph 3 of the introduction without any prior definition. i can infer from context that this is probably the number of exceptions, but this is never stated anywhere. Why is the value of N=200 and 100 chosen in the first place?
  • The related work makes an attempt to connect this study to past literature, but I feel that each of the prior works and results are introduced extremely tersely. Without opening the respective papers, I will not gain an understanding of either the Tolerance Principle nor the Lifted Representation Hypothesis, nor will I understand how either are directly or indirectly related to this work.
  • Equation (3) uses a highly nonstandard (in my opinion) notation for the indicator function (I assume?)
  • Line 151: what does "deterministic random" provenance mean? Is it deterministic or is it random?
  • Again, perhaps this is a lack of domain knowledge on my part, but the relevance for any of the numbers in the experimental design are never discussed or supported. It appears to me that for Family 1, the parameter values are all fixed (100 joint primary, 100 joint secondary, 500A/B, 200 secondary-selector, etc). These numbers are not ablated and I cannot be confident of any effects seen not strongly depending on these choices of numbers. This seems to be true for almost all experiments given.
Questions for authors

I do not have specific questions on this work beyond what I have written in the weaknesses. Many questions come to mind about the definitions, choices of functions, and choices of experiment parameters, which I have detailed above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.