- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes an explanation the generalization versus memorization phenomenon for the linear in-context learning setting as a demonstration of spectral bias—first learning the low-degree components, which allows better generalization, before learning the high-degree parts, leading to close memorization of the training data but worse performance on novel tasks. The work quantifies this behavior by performing a novel eigendecomposition of the closed-form solutions of empirical and population loss for linear ICL, then observing how these components are learned by a kernel machine.
Critical strengths
The perspective is interesting and natural, extending a well-observed framework to understand a relatively new phenomenon and give it a mathematical grounding. The idea is well-motivated.
The subkernel expansion framework and application for an approximate kernel decomposition in this setting is creative and promising, even if its correctness is only empirically observed here.
Critical weaknesses
The evidence, both theoretically and empirically, for an actual spectral bias in this setting is not very convincing. The experiments focus on one case (input and task distribution) and the confirmation is based solely on visual plots.
Some of the conclusions made/suggested seem exaggerated and overblown, especially given there is not an analysis of training dynamics, extensions beyond linear models, or architectures used in practice for ICL (i.e. transformers).
Questions for authors
-
Please clarify the relationship between and . To me, it reads that is the number of training tasks but the model is trained on contexts (tasks) drawn from those , then you take to infinity… so you are reusing samples? What is in Figure 3?
-
You include noise in your model: what is the effect of noise on generalization/memorization? Does the noise act as implicit regularization, like in standard settings? What about noisy observations of the tasks (coefficient vectors)?
-
In line 283 you mention the study is in the regime—is this mentioned earlier? If not, it should be.
-
In Figure 2, the increase in novel task MSE after learning generalizing solution does not look significant, at least visually. Also, this seems different from the usual spectral bias, which appears over optimization steps. Here you train with online SGD (so I guess it technically matches), but in practice training is done on the whole training dateset (or as batches), so I thought you would plot number of steps/epochs—can you elaborate on this difference and why you use online SGD? How is this different from grokking?
-
Any explanation of why the training dynamics would result in spectral bias in this setting? Beyond the empirical results. Is it just like NTK?
-
You take kernel regression with a particular kernel as the learning algorithm, and all analysis is based off of this. Can this algorithm be achieved by actual architectures (i.e. ICL with transformers) and would this same kind of phenomenon necessarily hold? If that isn't obvious, I think it’s a very big jump to suggest this phenomenon generalizes beyond linear tasks and with real-world models.
-
One of the conclusions you make (which is also the title) is that feature learning is not necessary, which is technically true since you use fixed/random features, but it doesn’t seem like that should be the main takeaway. Isn’t this known? Transformers can implement in-context closed form estimators, implicit GD, etc.
-
What does this provide over usual generalization error bounds with bias-variance tradeoffs, which exist for ICL?
-
Typo on line 95: “region.gm”
4 comments
Response to Reviewer Dnh3 (Questions section, part 1)
To the reviewer’s questions:
1. Task diversity versus sample count: The task diversity is fixed from the outset of the problem, we then train models using samples taken from the distribution with tasks. For kernel regression models, this number of samples is fixed, although our theory assumes , as is customary in kernel theory [see e.g. https://arxiv.org/abs/2006.13198, https://arxiv.org/abs/2510.14878]. For Figure 3 (Figure 4 in the updated pdf) which shows the same memorization to generalization transition as is in Raventos et al., we train models on distributions with various task diversity on samples, plotting what their error is at the end of training.
2. Noise: There are various aspects of noise that appear in our model, and one that affects the data itself. There is the noise in the ICL problem itself, which is added to the in-context labels themselves (and sets the scale where if the training tasks have a covariance below the noise threshold, these directions are poorly resolved by model, described in Appendix H.2, line ~1680). Our general finding of the generalization-memorization transition is, however, largely independent of noise effects. We have updated the paper, adding in a sentence in line ~424 to clarify this point for future readers. There is also noise in the eigenfunctions themselves (being a data-dependent observable) along with their fit coefficients, however this noise is averaged over by taking multiple trials. We leave a discussion of this noise to the kernel learning curve papers, such as Simon et al. 2021: https://arxiv.org/abs/2110.03922.
3. Context length regime: We work in a regime of only , although we agree that this was not well articulated in the submitted version of the paper. In our updated version, we make this more transparent, and have added in a section (Section 6.1) that describes the effect finite context length has on the generalizing, memorizing, and kernel solutions.
4. Significance of the rise of the error, SGD: We agree with the reviewer that it is difficult to visually see a dramatic rise in the test error on novel tasks, but note nonetheless that a rise in the error is indeed visible. We are actively working on finding a setting where the rise is more stark. As for the spectral bias, we do indeed use online SGD such that each update uses a fresh sample, with the number of optimization steps and number of seen samples coinciding (up to the batch size of 1024). We chose to use the number of seen samples instead of the number of optimization steps to better align with the kernel results where the solution is found analytically.
4.1 Grokking: Grokking is a separate phenomenon entirely, where the generalizing solution is capable of interpolating the entire training dataset (contrast this against the ICL setting wherein the generalizing solution is always suboptimal and cannot interpolate the training dataset). In grokking, should a model learn the memorizing solution, then it will never generalize without external pressure (i.e. weight decay), whereas in ICL, a model will generalize first then memorize as a byproduct of trying to achieve the lowest possible training error.
Response to Reviewer Dnh3 (Questions section, part 2)
5. Spectral bias: Please see the above point of “evidence for spectral bias.” Yes, for linearized models, the model is well-described by the kernel’s neural tangent kernel (see Jacot et al. 2018: https://arxiv.org/abs/1806.07572).
6. The learning algorithm: We chose kernel regression for the above reason such that we could describe the inductive bias, and thus the error decomposition, of the networks. Practically speaking, any model (including transformers) that is a universal approximator can achieve the results we detail, although we do not have as well developed theories of other model’s inductive biases. Please refer to the general response concerning transformers. On “generalizing beyond linear tasks,” a kernel method is linearized with respect to its weights, not the data, so in theory one could perform a similar kernel analysis on more complicated tasks.
7. Isn’t this known: No. The result that kernel machines are able to perform ICL is not known in the literature. The reviewer is correct in pointing out that feature-learning transformer models can implement e.g. linearized models in-context, however this does not imply that a fixed-feature model will implement such linearized models in-context.
8. Comparison to bounds: The main scientific message we wanted to convey in this paper is the mechanism behind why ICL happens, and the phases of learning, something a bound is unable to express. Additionally, most theoretical accounts of ICL (including e.g. https://arxiv.org/abs/2310.08391 which we believe is the reference the reviewer was referring to) consider only one-layer linear attention models, which are unable to memorize the training tasks and are therefore unable to capture the generalization-memorization transition key to the ICL setup. An error bound will also be unable to capture dip-then-rise behavior in test error throughout time, and would thus not add much to the scientific insight.
9. Typo on line 95: Fixed; we thank the reviewer for pointing this out.
Thanks for the detailed response and updates to the paper; they helped clarify a lot. The transformer experiment is a welcome addition. A more mechanistic study of to see whether those higher degree components emerge during training as predicted would be nice, but even just tracking MSE as is done here gives me more confidence that the theory can offer insight into transformer training.
The spectral perspective is promising but I think the paper as a whole can still be substantially strengthened, including the exposition. Nevertheless, I will increase my score to a 3.
Response to Reviewer Dnh3 (Weaknesses section)
We thank the reviewer for the feedback on our paper. We appreciate that the reviewer found our perspective interesting and natural, in addition to finding our subkernel expansion framework creative. Here, we address the reviewer’s major weaknesses:
Evidence for spectral bias: This seems like a non-issue to us: kernel regression is already well-understood to obey a spectral bias, as has been thoroughly demonstrated theoretically and empirically (see e.g. [https://arxiv.org/abs/cond-mat/0106475, https://arxiv.org/abs/2002.02561, https://arxiv.org/abs/2006.09796, https://arxiv.org/abs/2110.03922]). It is not necessary for the present work to show that kernel regression learns with a spectral bias given by the kernel eigensystem, as this is already known to hold generally.
Evidence for the provided story: Indeed, the strongest evidence that we have correctly computed the kernel eigensystem is the fact that experiment dots fall on theory curves. This is strong evidence (e.g. note the clear match in Figures 1, 8, 9) and is standard. However, we also provide non-visual evidence for our story in Section 3 which discusses decomposing the memorizing and generalizing solutions into the kernels eigenbasis, finding that the two are approximately equivalent when the solutions are truncated at degree-3 polynomials.
No training dynamics: As with spectral bias, the training dynamics of kernel regression follow trivially from its eigensystem. We also note that we do study the training dynamics of RF regression trained by SGD in Figures 2 and 3.
Applicability to non-kernel settings: We refer the reviewer to our general response concerning transformer models.