- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates why large models can learn low-frequency or complex tasks that small models ignore in mixed training distributions, attempting to distinguish sample efficiency gains from persistent task learning bottlenecks. The authors establish a linear regression mixture model for orthogonal tasks, prove that the optimal shared representation selects features according to the product of task frequency and feature spectral strength, and use gradient bounds and a local competition model to explain the preservation of rare signals. Subsequently, the authors pretrain OLMo models of different sizes on Dolma, inject comparison and modular addition tasks with controllable frequencies, and analyze task performance, internal features, and gradient relationships. The results show that scale is associated with rare task learning, feature formation, and retention after updates.
Critical strengths
- The research question is important, and the control approach is clear. Through random token mapping and controllable injection frequency, the authors circumvent some of the difficulties in estimating natural task frequencies, making it easier to study the relationship between scale and long-tail learning.
- The theoretical model is easy to interpret and has testable predictions. The utility provides a clear mathematical basis for ranking optimal representations, and the task boundaries and residual losses can be directly compared with experiments.
- Rich Experimental Analysis. The authors examine test accuracy, DAS sequential features, Fourier representations, and trajectories before and after injection. In particular, modular addition separates training fitting from generalization, providing extensive empirical evidence for mechanistic analysis.
Critical weaknesses
- Existing experiments do not adequately distinguish between forward transfer and subsequent forgetting. Although the authors controlled for the amount of training data and task frequency, large models may still obtain more useful representations from common tasks or learn more from each rare-task observation. The representational differences in Fig. 5 and the absolute loss decrease in Fig. 6a cannot distinguish among these explanations, and identical language model loss does not mean identical transfer ability. We suggest that the authors selectively supplement the analysis by freezing representations after common-task training and testing tail learning, by matching initial learning gains and then comparing forgetting rates, or by changing the task-sharing structure and adding update-protection controls, to further distinguish these mechanisms.
- Uniform hyperparameters mask differences in effective update speed across scales, so less forgetting may not entirely stem from weaker task interference. Appendix C.2 uses the same learning rate and training schedule for different scales, but these settings are not necessarily equally suitable for models of various scales, nor do they guarantee the same effective update magnitude. Large models may make smaller changes to existing representations during non-task updates, thereby exhibiting less forgetting at the same number of steps. We recommend that the authors discuss this alternative explanation, and if time permits, consider adding learning rate sensitivity analyses for representative scales, and compare retention ability under conditions matching initial task ability and non-task learning progress or output changes.
- The gradient cosine results for LLMs do not prove harmful interference. Sec. 4.4 treats nonzero cosine of non-task gradients as interference, and reports for 20M and a near-zero value for 1B. However, under standard gradient descent, the first-order change in task loss is , a positive inner product is usually beneficial to the task, and only a negative inner product indicates harmful conflict. Cosine also ignores gradient magnitude, Adam's actual update, and curvature, and the tendency toward orthogonality across scales may involve high-dimensional geometric effects. Therefore, it cannot show that nonzero cosine of non-task gradients is a form of interference. In addition, Sec. C.5 does not specify the for the top- neurons or the selection process.
Questions for authors
- How do you distinguish better transfer and larger learning gains from reduced forgetting? Could you compare forgetting after matching the initial rare-task learning gain?
- Do the retention differences persist across learning rates and after matching non-task learning progress or output changes?
- Why is the positive cosine similarity between non-task and task gradients interpreted as harmful interference? Could you measure rare-task loss changes caused by actual non-task optimizer updates?
- What value of is used for neuron selection, and how are these neurons selected? Does the cross-scale cosine trend persist with matched dimensions or randomly selected neurons?
5 comments
W2 / Q2: Uniform hyperparameters and effective update speed.
Appendix C.2 uses the same learning rate and training schedule for different scales, but these settings are not necessarily equally suitable for models of various scales
This is a fair point about possible suboptimal convergence, and we agree that the best way to isolate the effect of convergence is to conduct an exhaustive hyperparameter sweep. However, even with the optimal hyperparameters, the retention effect likely will persist: For any finite learning rate, there is a gap beyond which the rare-task signal is overwritten between injections. The critical gap scales roughly as , where the erasure rate shrinks with width (Theorem 4, Corollary 5). The learning rate rescales the critical gap but wider models remain better.
nor do they guarantee the same effective update magnitude. Large models may make smaller changes to existing representations during non-task updates, thereby exhibiting less forgetting at the same number of steps.
Thanks for pressing on alternative mechanisms here. The hypothesized competing mechanism, i.e., the same learning rate + larger model smaller effective update magnitude, can be largely ruled out by observations in Fig 6a. Learning rate schedule and effective update magnitude universally apply to all gradient updates; if a larger model makes smaller changes on non-task updates, such a model would also learn less from each injection, and we do not observe this: larger models do not have smaller immediate gain (Fig. 6a).
If the reviewer had a different mechanism in mind, we would be glad to hear it and see if we can run a targeted analysis.
We recommend that the authors discuss this alternative explanation, and if time permits, consider adding learning rate sensitivity analyses for representative scales, and compare retention ability under conditions matching initial task ability and non-task learning progress or output changes.
Thanks for the suggestions! App. G.1 might have already provided one piece of such evidence by matching non-task learning progress, i.e., general language modeling loss, as the reviewer suggests. At equal general language modeling loss, rare-task loss is similar across sizes at higher frequencies, but larger models achieve lower task loss at 2.4e-7, where smaller models plateau.
W3 / Q3 / Q4: Does a nonzero cosine indicate harmful interference?
Sec. 4.4 treats nonzero cosine of non-task gradients as interference, and reports for 20M and a near-zero value for 1B. However, under standard gradient descent, the first-order change in task loss is, a positive inner product is usually beneficial to the task, and only a negative inner product indicates harmful conflict.
The reviewer is right that, in general, a positive inner product reduces task loss to first order and indicates transfer. However, as we explained above in our design choice, the injected task shares no structure with the pre-training data, so any alignment between non-task gradients and the task direction is not transfer, but rather interference. Moreover, the net effect of these updates is measured directly in Fig. 6a. If the 20M model’s positive alignment were beneficial, its task loss would continue to fall between injections. Instead, the gain from each injection is lost within ~20 steps.
Cosine also ignores gradient magnitude, Adam's actual update, and curvature, and the tendency toward orthogonality across scales may involve high-dimensional geometric effects. Therefore, it cannot show that nonzero cosine of non-task gradients is a form of interference.
We agree that cosine similarity alone does not establish how large the interference is. It also ignores Adam preconditioning, and curvature, and we will soften our wording in Sec. 4.4 accordingly.
However, our primary goal is to determine whether interference is present, and cosine remains a valuable metric for this. Specifically, our mechanism concerns whether task and non-task losses act on shared parameters, which is a directional property. Dimensionality difference is not a concern here as the gradient is computed on top-N neurons. Moreover, raw inner products are not comparable across model scales and are dominated by non-task gradient norms. Adam’s per-coordinate normalization also largely removes raw gradient magnitude from the actual update. Again, as direct evidence to establish inference, the magnitude of the resulting harm is captured instead by Fig. 6a, which integrates the effect of actual optimizer updates over each injection gap.
What value is used for neuron selection, and how are these neurons selected? Does the cross-scale cosine trend persist with matched dimensions or randomly selected neurons?
We use K = 100 neurons in the first-layer MLP, which has the largest causal effect on the task prediction across all models (App. C.5). Within this layer, we select the neurons with the largest gradient magnitude with respect to the task loss. Task features are represented in a distributed way, so the results are not sensitive to the exact value of K.
Regarding randomly selected neurons, they would not have the same causal effects on model outputs, i.e., the model does not even use these neurons in its task computations, hence it is unclear what is the significance of looking at their gradients cosine similarity.
We thank the reviewer for the positive assessment and the detailed feedback. Several of the questions can be explained by one design choice that we should have stated more prominently, so we would like to clarify it first and then address each point.
A key design choice: no feature sharing between rare and common tasks. Both of our settings are built so that rare-task learning cannot benefit from features learned on common tasks:
W1 / Q1: Distinguish between forward transfer and subsequent forgetting
We note that “forward transfer” is a term from the continual learning literature that seems to have little to do with the paper: we study a multi-task learning setting that does not involve curriculum. We hence interpret the reviewer’s concern as whether larger models learn the rare task better because they share more useful features from common tasks, rather than because they forget less.
This would indeed be a concern if features can be shared between tasks. However, as we explained above, the necessary task features cannot be learned from common tasks. This is true both for the linear regression setting and the OLMo setting.
Thanks for the suggestions! In particular, we believe the suggested freezing experiment has an analytic answer in the linear regression setting. A representation trained only on common tasks spans the common block, so the rare-task loss stays at the mean-predictor baseline. Consistent with this, when the common-task residual is large, the rare-task signal is no better than random (App. F.2).
Moreover, varying the task-sharing structure is an interesting extension that we discussed in our response to Reviewer WUwi W1.