- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SCoL, a post-training framework that trains LLMs via meta-RL to generate textual instructions selecting which Transformer layers to update, thereby consolidating long context streams into parameters. A composite reward balancing new-knowledge acquisition and forgetting is used to trade off plasticity and retention in continual data streams.
Critical strengths
- Enabling LLMs to self-regulate where to update via natural-language generation is an interesting and novel form of dynamic plasticity routing.
- Empirical analysis shows SCoL spontaneously produces highly sparse layer selections that overlap strongly with high-Fisher-information layers, demonstrating that preference optimization can internalize parameter importance.
Critical weaknesses
- SEAL baseline reports only 1.30% retention, yet Appendix results show its open-book QA acc. collapses to 0.0%. This looks more like training collapse than memory forgetting. The authors should explain the cause.
- To compute the forgetting penalty over historical streams, SCoL must traverse all past contexts at every step, yielding complexity. The cost is prohibitive.
- Appendix ablations show that a hard-coded Last-10 layers control, with no inner/outer RL, reaches 20.10% retention, versus 20.46% for SCoL's meta-learned routing. A 0.36% gain indicates a severe mismatch between the cost (see W2) and benefit of the Meta-RL architecture. Why not use this stronger, simpler method as a main baseline? Also, does "Last-10" include the decoding layer? Prior work suggests skipping the decoding layer and updating middle-to-late layers may work better. If tuned, could Last-10 exceed SCoL?
- Evaluation data is extremely limited. In LongBench v2, only 60 passages are sampled, with just 20 testing long-context generalization. Why not use the full 503-item benchmark? Such small test sets are highly susceptible to sampling variance, undermining robustness and confidence.
- The paper claims continual knowledge injection, yet meta-training and evaluation are confined to the same distribution and even the same dataset (e.g., SQuAD streams). Real continual learning may involve OOD tasks. Without cross-domain injection tests, the method looks like overfitting to a single dataset distribution rather than a general continual learning paradigm.
- Only SQuAD and LongBench v2 are tested, ignoring degradation of base capabilities after continual updates. Without evaluation on general domains, it is hard to show the model retains its original generalization and reasoning robustness.
Questions for authors
None
General response to all reviewers and the AC (1/2)
We thank all reviewers:
Our research question is whether an LLM can itself determine where in its own weights to write each incoming context at test time. The model is updated at every context, so we frame the LLM at test time as a drifting policy rather than a static model, and train it with meta-reinforcement learning to generate which of its own layers to update, with forgetting penalized in the reward.
The reviewers' main questions are:
During the rebuttal, we added the following experiments to the revised version. The new results further strengthen each claim of the paper. The tables are in our responses to each reviewer and in the revised PDF.
1. SCoL alone forms the Pareto frontier between immediate accuracy and retention (Section 4.2, Tables 1, 5 and 17, Figure 3). For a fair comparison, the submitted version used SEAL's inner adaptation hyperparameters, the LoRA update applied at each context, for every method. We tuned this inner adaptation hyperparameters on a held-out set outside the evaluation stream and found a setting that is twice as fast and better for every method. We also added Random-10, which updates 10 random layers. On the 100-passage SQuAD stream, SCoL outperforms every baseline on both metrics, immediate accuracy right after each update and retention after the whole stream. Against Last-10, SCoL reaches 40.66% vs. 35.07% in immediate accuracy and 31.00% vs. 26.98% in retention, a paired difference of +5.59 ± 1.60 and +4.02 ± 1.54 points (mean ± SE), and its lead over every other baseline is also larger than the standard error. With the tuned hyperparameters, SEAL's retention rises from 1.30% to 21.36% and is still far below SCoL's 31.00%. Random-10 updates as many layers as SCoL and stays below it on both metrics, so the gain comes from learning where to update. Random selection falls further behind when one document is chunked and consolidated over many updates. On LongBench v2, each document is consolidated chunk by chunk and the question is asked only at the end, and Random-10 is 10.75 ± 4.47 (short) and 15.02 ± 5.18 (long) points below SCoL in paired difference.
2. SCoL holds on another model family and on a 27B model (Appendix B.9, Tables 8 to 10). We ran SQuAD on Llama-3.1-8B-Instruct (32 layers) and Qwen3.8-27B (64 layers), against 28 layers for Qwen2.5-7B. SCoL outperforms every baseline on both metrics on both models, so SCoL alone forms the Pareto frontier on all three models. On Qwen3.8-27B, we also meta-trained SCoL at budgets of B=5, 10 and 20 layers, and it outperforms updating the last B layers or B random layers on both metrics at every budget. Llama-3.1-8B-Instruct is meta-trained for 4 rounds without mode collapse, and it still emits 93 distinct selections over the 100 passages at round 4.
3. SCoL generalizes beyond what it is meta-trained on (Section 5.2, Appendices B.9 and C.3, Tables 3, 11, 12 and 17).