- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors extend the studies of Naim et al on the limits of transformers and how their performance diverges on regression tasks on inputs from different distribution shifts from what formal algorithms would predict. While Naim et al look at low dimensional linear and polynomial functions, this paper extends the study to large dimensions and defines distribution shifts in terms of spectral diversity (rotation etc). The distribution shifts in particular are dictated around just the choice of input points not on the parameters of the polynomial or linear function and determined by a Gaussian noise. This corresponds to one dimension of the distribution shift in Naim et al.
Critical strengths
This paper advances the study of transformer in context learning for regression problems for multivariate functions. They also offer what to this reviewer are novel bounds on relative risk, which are interesting.
Critical weaknesses
Many of the results echo Naim et al. more than a little bit. For instance at l. 210 the authors find deterioration at the upper end of the pretraining range. This would seem to correspond to Naim et al's boundary values (in KI) but also see arXiv preprint arXiv:2502.03503 for a much more detailed analysis of the causes of failures of generalization. The different training pretraining styles in terms of spectral diversity 238 are similar to the different pretraining regimes countenances in arXiv preprint arXiv:2502.03503. Extending context size in Naim et al's experiments seems very similar to the discussion at the end of 3.4 on extending pretraining counts. And then in section 3.4, unless I misunderstand, the authors find that generalization degrades as the values of the function exceed those in pretraining, also something noted in Naim et al. for both univariate linear functions and polynomials. As the authors acknowledge the layer norm result in 4.2 is also similar to Naim et al.
The bounds given in 4.3 are to new to this reviewer and interesting but it's hard to argue for publishability just on that alone. For this paper to be acceptable, especially in light of the use of gen ai methods on the mathematical formulations, it is important to have an in depth discussion of how the results in this paper go beyond those in Naim et al.
Questions for authors
please motivate the use of noise better. If it's to provide distribution shifts, then the shifts seem very small. What one might call near OOD rather than far OOD. On the other hand, given that many benchmarks are mixtures of quite different problems one would expect that a larger shift would be called for. Can you remark on this?
Please also explain in more detail your experimental setup with covariance
Please describe in depth how you differ from the prior analyses in this area. See papers referenced above.
2 comments
Response [2/2]
Noise and distribution shifts:
Noise is not introduced to generate distribution shifts. Its purpose is to give a nontrivial finite-sample statistical inference problem, where the optimal prediction involves regularisation according to the information available in the observations. The coefficient prior and noise law remain fixed throughout, and the distribution shifts come from changes in input covariance and other aspects of the context. This allows us to vary the statistical problem without changing the optimal prediction rule.
The shifts are not restricted to small deviations from pretraining. We examine context spectra over several orders of magnitude in scale, with widely varying effective dimensions and spectral shapes. We also deliberately investigate behaviour within and near the pretraining range, since understanding the accuracy and limitations of the learned approximation there is as important as identifying failures under extreme changes.
Covariance experimental setup:
Our reference pretraining curriculum samples covariance matrices whose eigenvalues follow either a smoothly varying log-linear spectrum or a two-level spectrum, with independently randomised eigenvectors. Both families vary broadly in scale and effective dimension. When the number of context examples changes, we rescale the covariance so that the expected context Gram matrix remains fixed.
For evaluation, we instead specify the spectrum of the realised Gram matrix (M=X^\top X), allowing us to vary overall scale, effective dimension and the remaining spectral shape separately. We then sample the remaining context variables and responses according to the Gaussian regression model. This distinction between the pretraining covariance distribution and the realised spectra used for evaluation is important to the systematic characterisation in Section 3.
Response [1/2]
Thank you for your detailed review. We agree that Naim et al. are particularly relevant to our work, and that the original introduction did not sufficiently explain the differences between their contributions and ours. We have therefore significantly expanded the introduction, with revisions denoted in red, to clarify these distinctions.
Novelty relative to Naim et al.
We agree that several of our observations have precedents in Naim et al., including failures under changes in input or function magnitude, the effects of widening pretraining distributions, and limitations associated with LayerNorm. These are important contributions which we now discuss explicitly in the introduction.
However, we disagree that our principal contribution is simply to extend their experiments to higher-dimensional regression. Naim et al. principally investigate the limits of recovering underlying functions beyond the pretraining distribution. Their main experiments use noiseless linear and polynomial regression, where sufficiently many observations determine the function exactly. They identify particular extrapolation failures, investigate how these depend on pretraining, and provide explanations for some of the limitations.
Our objective is instead to characterise the approximation to an optimal finite-sample statistical procedure that a trained Transformer has learned. We study noisy regression, where the optimal predictor is ridge, and vary the input covariance while keeping this statistical rule fixed. This allows us to investigate how agreement with ridge depends on the information available in different directions of the regression problem, rather than simply whether a model continues to recover a function outside its pretraining distribution.
Our principal empirical finding is a systematic organisation of generalisation across a rich range of covariance spectra, largely determined by input scale and effective dimension. This organisation remains informative even when we vary the remaining spectral degrees of freedom extensively, and the pronounced dependence on scale persists under substantial changes to pretraining. We investigate both the significant generalisation beyond the two simple spectral families used in pretraining and the structure of the remaining errors. This is a different kind of characterisation from the extrapolation limits studied by Naim et al., and provides a more precise account of the statistical approximation the Transformer has learned.
We also investigate whether the Transformer preserves the mathematical properties of the intended procedure. In particular, Section 3.3 shows that different contexts with identical ridge sufficient statistics can produce substantially different predictions, even after explicitly penalising these discrepancies during pretraining. This also distinguishes our context-count experiment from an ordinary test of generalisation to longer contexts, since we vary the number of examples while keeping the optimal ridge prediction exactly unchanged.
Regarding the large-scale deterioration studied in Section 4.3, it should be noted that we increase the magnitude of the context covariates while keeping the coefficient prior and query distribution fixed. The correct prediction is not required to grow with the context magnitude. Thus, this behaviour is not simply explained by the output-magnitude limitations identified by Naim et al.
More importantly, our systematic characterisation provides a pattern of generalisation against which proposed algorithmic explanations can be tested. A failure to match exact regression does not by itself rule out an approximate iterative algorithm, since such an algorithm may itself fail under distribution shift. In Section 4.3, we show that the finite inverse iteration considered has relative-risk bounds uniform in context scale, whereas a broad class of fixed finite polynomial computations necessarily has diverging relative risk at large scales if it achieves nontrivial accuracy. These results demonstrate how the observed generalisation behaviour can distinguish proposed computational explanations, rather than merely establishing further discrepancies with exact regression.