- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether context-enhanced learning (CEL) can suppress the transition from in-context learning (ICL) to in-weights learning (IWL) during pretraining. Using a toy setting of interleaved linear dynamical systems, the authors introduce masked backstories, which prepend informative historical states while masking their training losses. The intervention prevents late-stage memorization, preserves ICL generalization, and improves associative recall. Further experiments reveal several surprising behaviors, including near-optimal predictions at positions without direct supervision, ICL degradation when only 0.025% of training systems remain exposed to memorization, and position-dependent retrieval of memorized information.
Critical strengths
-
Effective suppression of ICL transience. The masked-backstory intervention successfully prevents late-stage memorization while preserving held-out ICL performance and improving associative recall (Figs. 2 and 4). The unmasked and uninformative-backstory controls further support the importance of combining informative context with loss masking.
-
Interesting empirical observations about ICL–IWL interactions. The experiments reveal unexpected behaviors, including near-optimal predictions without direct supervision (Fig. 5), ICL degradation from memorizing only ten training systems (Fig. 6), and position-dependent memorization (Fig. 8). These findings raise potentially valuable questions about how competing learning strategies develop in transformers.
Critical weaknesses
-
Insufficient isolation of the benefits of masked backstories. Although the authors examine several backstory variants, they do not compare against simply masking losses on initially unpredictable positions without adding backstories. Such a baseline is important for determining whether the improvement requires informative additional context or primarily results from removing the training incentive to memorize difficult positions.
-
Limited generality and robustness of the findings. The experiments primarily use a single 10.7M-parameter transformer and deterministic five-dimensional orthogonal systems, leaving generalization to other architectures and tasks unclear. Furthermore, the absence of reported variability across independent training runs makes it difficult to assess the robustness of subtler findings, particularly the degradation caused by exposing only ten systems.
Questions for authors
-
How does masked-backstory training compare against a baseline that simply masks losses on the first unpredictable observations of each system, without prepending additional context? Would this baseline similarly suppress IWL and improve associative recall?
-
Are the observed phenomena consistent across independent training seeds and different model sizes or dynamical systems? In particular, how robust are the ICL degradation caused by the ten exposed systems and the position-dependent memorization results?
3 comments
Masking the Loss Without Adding Backstories
We appreciate you pointing out a natural baseline that we missed. We have now run this experiment and the results can be seen in Appendix A. Masking the first seven indices of the first occurrence of a system with a training example does suppress the transition from ICL to IWL just like the masked backstories intervention. Nonetheless, associative recall and performance on later indices that can reach zero error lag behind the performance of the model trained with masked backstories.
Robustness of Findings to Multiple Training Runs, Sources of Randomness and Architectures
In Appendix A, we now have a training run of a 63M-parameter model that shows the same dynamics of transitioning from ICL-IWL, except earlier in training than the 10.7M-parameter model. We also have another run of the ten exposed systems from a different random initialization. We see that the effect still appears. We have also run two more training runs of the 10.7M-parameter model on different random initializations and different random draws of training systems and initial states. The masked backstories intervention works just as it does in our original training run.
Thank you for your time and your thoughtful review. Here are our responses to your constructive criticisms: