acceptodds
Under review as a conference paper at ICLR 2027

Just Enough Experts for Continual Learning of Large Language Models

Abstract

Recent studies on continual learning (CL) have shown that data replay is a simple yet effective strategy for mitigating catastrophic forgetting in large language models (LLMs). However, its effectiveness depends heavily on the quantity and quality of rehearsal data, and substantial forgetting can persist when either is inadequate. Moreover, updating shared model parameters across tasks can introduce interference and further degrade performance. To address these limitations, we propose Adaptive Expansion and Identification (AEI), a framework that combines minimal yet sufficient expert expansion with adaptive expert identification. Specifically, AEI encourages newly introduced experts to capture feature directions complementary to those represented by existing experts. As the represented feature subspace grows across tasks, the remaining complementary space becomes increasingly constrained, encouraging compact and effective expert expansion with reduced redundancy and cross-task interference. Meanwhile, AEI employs a context-aware gating module trained with task-identity supervision to adaptively identify and combine the independent experts relevant to each input. Together, these mechanisms support efficient adaptation to new tasks while preserving previously acquired knowledge. Extensive experiments across multiple LLM backbones demonstrate that AEI outperforms competing continual learning methods using only 1% rehearsal data.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual learning, expert expansion, large language models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Y131OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposed a novel MoE framework, AEI, for continual learning of LLMs. Instead of using a fixed number of experts, AEI utilizes an adaptive expert expansion mechanism encouraging to learn complementary knowledge, which reduces interference and parameter redundancy by constrained complementary spaces. Also, AEI has a context-aware gating module to adaptively identify and combine relevant experts. The experiments across different LLMs and settings show that AEI outperforms existing continual learning methods using only 1% rehearsal data.

Critical strengths

The paper proposes a novel adaptive expandable MoE framework for continual learning. Since previous works mostly focus on using a fixed number of experts, although GoDMoE also proposes an expansion method in MoE for continual learning, the performance of GoDMoE is not better than AEI. The idea and perspective of this MoE for continual learning are novel and interesting to me, since it's different from conventional continual MoE frameworks and it progressively introduces new experts.

Critical weaknesses

It’s not clear for the motivation of the design in AEI. Since AEI already stores 1% rehearsal data for previous tasks and uses them to refine gating vector when learning new tasks, it makes me confused about why not also use this historical data to help update newly introduced expert. We know that historical data can be used as the input representation which can also help new experts reduce interference from old experts in continual learning.

Questions for authors
  1. For newly added experts, how to initialize them? Does the algorithm give high priority to learning new tasks on the newly added experts rather than on old experts?

  2. Can authors explain how to initialize the newly added column in the gating vector in AEI? In line 306, it said that “the parameters learned at stage tt are used to initialize the gating module at stage t+1t+1, and the entire is then updated for the new task”. But if the entire gating vector is updated, the learned parameters would not be the last added column in gating vector, since other columns are also updated. In this case, the dimension of all columns will be a problem for newly added column.

  3. What’s the principle for the asymmetric label-smoothing strategy in Eq(15)?

  4. Can authors compare the performance of AEI and SAME [1]? And compare the cost of GoDMoE in Table 5?

    [1] “SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning”, ICML 2026.

  5. Providing a completed algorithm makes it clear to see how different parts of AEI work together.

4 comments
AuthorsOpenReview ↗

Part1: Responses to Weakness 1, Question 1, Question 2, and Question 3

We sincerely thank the reviewer for the thoughtful and constructive comments. We have carefully addressed the concerns below, and the corresponding revisions are highlighted in blue in the revised manuscript.

Weakness 1: It’s not clear for the motivation of the design in AEI. Since AEI already stores 1% rehearsal data for previous tasks and uses them to refine gating vector when learning new tasks, it makes me confused about why not also use this historical data to help update newly introduced expert. We know that historical data can be used as the input representation which can also help new experts reduce interference from old experts in continual learning.

Response. We apologize for the ambiguity. Indeed, historical replay data are used in both training phases: router training and new-expert training.

At stage tt, both router training and expert training use DtD_t, which contains the current-task data and the retained samples from previous tasks. We first update the router using task-identity supervision. We then freeze the router and optimize the newly introduced expert using the next-token prediction loss on DtD_t, together with the subspace regularizer. In this way, historical samples serve not only as supervision for expert identification and routing, but also as training inputs for the newly introduced expert, allowing it to learn from historical representations.

We have clarified this training procedure in the revised manuscript and provided Algorithm 1 in Appendix F to illustrate how historical replay samples are used in both training phases.


Question 1: For newly added experts, how to initialize them? Does the algorithm give high priority to learning new tasks on the newly added experts rather than on old experts?

Response. The new expert's down-projection matrix is initialized to zero, while its other projection matrices use Kaiming-uniform initialization. The zero down-projection makes the new expert's initial output zero. During expert training, the backbone, historical experts, and trained router are frozen, so parameter updates are confined to the newly introduced expert. Historical experts can still contribute through the router, allowing their knowledge to be reused.


Question 2: Can authors explain how to initialize the newly added column in the gating vector in AEI? In line 306, it said that “the parameters learned at stage t + 1 are used to initialize the gating module at stage , and the entire is then updated for the new task”. But if the entire gating vector is updated, the learned parameters would not be the last added column in gating vector, since other columns are also updated. In this case, the dimension of all columns will be a problem for newly added column.

Response We apologize for the unclear description of the matrix annotations. We expand the router's output projection matrix, not the token-dependent gating vector itself. At stage tt, each attention head's output projection has shape t×dht \times d_h after task tt. At the next stage, we copy these tt rows and append one randomly initialized row, obtaining a matrix of shape (t+1)×dh(t+1)\times d_h and thus outputting a gating vector α∈Rt+1\alpha \in R^{t+1}. The query, key, and value projection dimensions are unchanged, and their learned parameters are also retained as initialization. All parameters of the expanded router are then updated using current and replay data. Updating the values of the existing rows does not change their dimensions, so it causes no dimensional mismatch. After router training, we freeze the router and train the new expert, and the historical experts remain frozen throughout. We have corrected the corresponding descriptions in the revised manuscript.


Question 3: What’s the principle for the asymmetric label-smoothing strategy in Eq(15)?

Response. The principle is to encourage reuse of earlier knowledge while limiting interference from experts introduced later. For a token from task TcT_c with c>1c>1, its corresponding expert receives target weight 1−ε1-\varepsilon, and experts from tasks 1,…,c−11,\ldots,c-1 share the remaining mass ε\varepsilon equally. Experts introduced after task cc receive zero target weight. This preserves the temporal direction of knowledge reuse when training on replay samples (earlier experts share the remaining mass ε\varepsilon). The targets discourage later experts from interfering with earlier tasks (later experts receive zero target weight).

AuthorsOpenReview ↗

Part2: Responses to Question 4 and Question 5

Question 4: Can authors compare the performance of AEI and SAME [1]? And compare the cost of GoDMoE in Table 5?

Table 9. Comparison with SAME on SuperNI-10 using LLaMA-2-7B.

5 tasks

MethodOrder 1 AR ↑Order 1 BWT ↑Order 2 AR ↑Order 2 BWT ↑Order 3 AR ↑Order 3 BWT ↑Average AR ↑Average BWT ↑
SAME52.55-1.0151.97-1.6051.59-2.3152.04-1.64
AEI53.46+0.0353.26-0.0253.26-0.1953.33-0.06

10 tasks

MethodOrder 1 AR ↑Order 1 BWT ↑Order 2 AR ↑Order 2 BWT ↑Order 3 AR ↑Order 3 BWT ↑Average AR ↑Average BWT ↑
SAME62.61-3.1362.89-2.7862.53-3.3662.68-3.09
AEI65.56+0.0465.32+0.0165.64-0.1065.51-0.01

Response. We compare AEI with SAME on SuperNI-10 using LLaMA-2-7B and SAME's official implementation. As shown in Table 9, AEI consistently achieves higher AR and BWT across all evaluated task orders and sequence lengths. The advantage is particularly evident in the 10-task setting, demonstrating better adaptation and knowledge retention than SAME.

Table 10. Computational cost comparison on LLaMA-3.1-8B.

MethodAdditional Params.Training Time (TT)Inference Time (IT)
GoDMoE102M~10.1 h~1.21 h
Gain210M~5.41 h~0.79 h
SERS—~25.6 h~0.40 h
AEI (Ours)235M~5.13 h~0.44 h

We also include GoDMoE in the cost comparison in Table 10. Although AEI introduces more additional parameters than GoDMoE, it requires substantially shorter training and inference times in this setting. These results show that AEI trades a larger number of additional parameters for lower measured runtimes. Both comparisons have been incorporated into the revised manuscript.


Question 5: Providing a completed algorithm makes it clear to see how different parts of AEI work together.

Response. We have provided a complete algorithm (Algorithm 1 in Appendix F of the revised manuscript) to show how expert expansion, router training, expert training, and pruning fit together. In particular, it specifies that both training phases use current and replay data, while the router is frozen during expert training.

Thanks for authors taking time to reply to my concerns. Most of them are addressed, so I would keep my original score. The reason I wanna keep my score is that the current algorithm makes sense to me and has strong performance in experiments. One remaining concern is about the router update rule in the current algorithm. I think it will cause router drift on previously learned tasks due to router weight changes. I think perhaps exploring how to improve the current algorithm to reduce router drift may be future work.

We sincerely thank you for your careful consideration of our responses and for your continued positive assessment. We will consider this constructive suggestion in our future work.

Reviewer_qZMUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Expansion and Identification (AEI) for continual learning of large language models. AEI adds lightweight FFN experts for new tasks while freezing the backbone and previously learned experts. It uses a subspace-based sparsity regularizer and pruning to control expert capacity, together with a context-aware router trained using task-identity supervision. Experiments on SuperNI with different backbones and task sequences show improved performance and limited forgetting using 1% rehearsal data.

Critical strengths
  • The paper is generally well written and easy to follow. The motivation for controlling expert redundancy and improving expert identification is clearly explained.

  • The experiments are comprehensive, covering different model backbones and sequences of up to 50 tasks. The ablation studies, significance tests, and parameter and runtime comparisons provide useful evidence for the effectiveness of the framework.

Critical weaknesses

I have two main concerns:

  1. My first concern is that the benefit of the proposed subspace-based regularization over standard sparsity regularization is not sufficiently established.

The current ablation shows that adding L_reg helps, but does not clarify whether the improvement comes from the parallel–orthogonal decomposition or from sparsity and pruning more generally. A comparison with ordinary neuron-level group-sparsity regularization, using the same router and a similar retained-parameter budget, would help assess the contribution of this design.

  1. I also find the evaluation scope somewhat limited.

Although SuperNI contains diverse tasks, both SuperNI-10 and SuperNI-Long come from the same benchmark family. Moreover, while the 5- and 10-task experiments consider multiple task orders, the longer sequences use a fixed order. Since the proposed expert reuse and expansion mechanism may depend on task similarity and order, the current results leave its robustness across different continual learning settings unclear. Evaluation on an additional benchmark and more long-sequence task orders would strengthen this claim.

Questions for authors
  1. Could you clarify the choice of the routing layer l_0? Section 3.4 motivates using higher-layer representations because they are more task-specific, while Appendix F uses the output of the first block.

  2. What default label-smoothing coefficient is used in Eq. (15)? Appendix F specifies the pruning threshold, but I could not find the default value for label smoothing.

  3. Do the additional parameter counts in Table 5 include the context-aware gating module as well as the retained experts?

4 comments
AuthorsOpenReview ↗

Part1: Responses to weakness 1, question 2, and question 3

We sincerely thank the reviewer for the thoughtful and constructive comments. We have carefully addressed the concerns below, and the corresponding revisions are highlighted in blue in the revised manuscript.

Weakness 1: Subspace-based regularization versus neuron-level sparsity.

Table 4. Comparison of POR and NGR on SuperNI-Long using Qwen3-4B-Base. P-Param. denotes the cumulative number of pruned parameters.

Method20 Tasks AR20 Tasks P-Param.30 Tasks AR30 Tasks P-Param.40 Tasks AR40 Tasks P-Param.50 Tasks AR50 Tasks P-Param.
AEI-NGR66.0110.01M63.2020.10M61.9237.72M60.1456.79M
AEI-POR66.1911.67M63.6023.33M62.3337.14M60.8155.30M

Response. We thank the reviewer for suggesting this comparison. We compare parallel–orthogonal regularization (POR) with neuron-level group-sparsity regularization (NGR) on SuperNI-Long using Qwen3-4B-Base, the same router, and comparable parameter budgets. As shown in Table 4, POR consistently achieves higher AR than NGR across all evaluated sequence lengths, with comparable parameter pruning. These results demonstrate that the improvement does not arise merely from applying sparsity regularization and pruning neurons, but also from explicitly exploiting the parallel–orthogonal structure of historical feature subspaces. These discussions and experiments have been added to Section 4 of the revised manuscript. We further explain their differences as follows.

Difference in regularization terms. The key distinction is that NGR treats each neuron as an indivisible group and penalizes its entire up-projection vector through

LNGR=λdm′∑i=1dm′∥wi∥2.\mathcal{L}_{\mathrm{NGR}} = \frac{\lambda}{d_m'} \sum_{i=1}^{d_m'} \lVert w_i \rVert_2.

Consequently, the parallel and orthogonal components are jointly considered through the overall neuron norm, without explicitly distinguishing their respective contributions to historical knowledge and complementary feature. Its sparsity mechanism therefore primarily determines whether an entire neuron should be retained or removed.

In contrast, AEI decomposes each neuron into its parallel and orthogonal components and penalizes them separately:

LPOR=λdm′∑i=1dm′(∥wi∥∥2+∥wi⊥∥2).\mathcal{L}_{\mathrm{POR}} = \frac{\lambda}{d_m'} \sum_{i=1}^{d_m'} \left( \lVert w_i^{\parallel} \rVert_2 + \lVert w_i^{\perp} \rVert_2 \right).

This component-wise regularization encourages separate shrinkage of the two components, supporting the reuse of previously represented directions, the learning of complementary feature directions, or the removal of redundant neurons.


Question 2: Default label-smoothing coefficient.

Response. The default label-smoothing coefficient is ε=0.4\varepsilon=0.4. We have added this value to the implementation details in Appendix F of the revised manuscript.


Question 3: Additional parameter counts.

Response. Yes. For AEI in the 50-task LLaMA-3.1-8B cost comparison, the additional parameter count includes both the context-aware gating module (25.37M) and all retained experts after pruning (210.02M), totaling 235.39M. The table reports this total rounded to 235M and excludes the pretrained backbone. The revised cost table corrects the original decimal-place errors: 21.0M to 210M for Gain and 23.5M to 235M for AEI.

AuthorsOpenReview ↗

Part2: Responses to weakness 2 & question 1

Weakness 2: Evaluation beyond SuperNI and robustness to task order.

Table 5. Continual fine-tuning results on the mathematical reasoning dataset using Qwen2.5-14B-Instruct.

MethodMMLU-ProIFEvalGeneral Avg.AIME24AIME25Math Avg.
Initial Model65.5186.6976.1016.6713.3315.00
FT w/o Replay57.9970.8664.4330.0026.6728.33
FT w/ Replay59.2280.8270.0220.0026.6723.33
Gain62.5683.7373.1423.3323.3323.33
AEI (Ours)65.0886.2175.6530.0026.6728.33

Response. We address these two concerns with a mathematical reasoning adaptation experiment and two additional task orders for SuperNI-Long.

Evaluation beyond SuperNI. We fine-tune Qwen2.5-14B-Instruct on the long-CoT dataset s1K-1.1 [1], whose average sequence length exceeds 10,000 tokens. As shown in Table 5, AEI achieves mathematical reasoning performance on par with the strongest baseline while maintaining general-domain capabilities close to those of the initial model. This result extends the evidence beyond SuperNI and suggests that AEI can learn from reasoning-intensive data without substantially sacrificing general capabilities. This experiment has been added to Section 4 of the revised manuscript.

[1] Muennighoff et al. s1: Simple test-time scaling. EMNLP 2025.

Table 6. Results on Order 2 of SuperNI-Long using Qwen3-4B-Base.

Method20 Tasks AR ↑20 Tasks BWT ↑30 Tasks AR ↑30 Tasks BWT ↑40 Tasks AR ↑40 Tasks BWT ↑50 Tasks AR ↑50 Tasks BWT ↑
O-LoRA58.53-2.9154.58-3.1857.74-3.6254.03-3.88
Gain62.02-2.4457.11-2.8660.99-3.1157.10-3.53
AEI64.75+0.1560.06-0.0564.04+0.0960.76+0.11

Table 7. Results on Order 3 of SuperNI-Long using Qwen3-4B-Base.

Method20 Tasks AR ↑20 Tasks BWT ↑30 Tasks AR ↑30 Tasks BWT ↑40 Tasks AR ↑40 Tasks BWT ↑50 Tasks AR ↑50 Tasks BWT ↑
O-LoRA61.23-4.6657.52-4.7956.25-4.6153.98-4.30
Gain65.68-1.6260.59-3.5059.99-3.2156.33-4.22
AEI67.70-0.1364.11-0.0863.24-0.1460.75-0.06

Additional long-sequence task orders. We evaluate two additional orders on Qwen3-4B-Base, covering 20, 30, 40, and 50 tasks in each order. Across the eight settings in Tables 6 and 7, AEI achieves higher AR and BWT than both O-LoRA and Gain. Its AR gains over Gain range from 2.02 to 4.42 points, while its BWT remains between -0.14 and +0.15. At 50 tasks, AEI achieves AR values of 60.76 and 60.75 in the two added orders, compared with 57.10 and 56.33 for Gain. Together with the original order, these results support consistent adaptation and knowledge retention across the three evaluated task orders. These experiments have been added to Appendix H of the revised manuscript.


Question 1: Choice of the routing layer.

Response. We thank the reviewer for pointing out this ambiguity. The observation that higher-layer representations can be more task-specific does not imply that a deeper routing layer is always preferable. The router layer involves a stability–plasticity trade-off: because experts are inserted only above l0l_0, a smaller l0l_0 leaves more layers available for task adaptation, whereas a larger l0l_0 provides more task-specific routing features but reduces the number of expandable layers.

Table 8. Sensitivity analysis of the router layer l0l_0.

BenchmarkMetricl0=0l_0=0l0=1l_0=1l0=4l_0=4l0=8l_0=8
SuperNI 5 TasksAR52.9853.2753.1453.00
SuperNI 5 TasksBWT-0.16-0.01-0.01+0.01
Long-CoTAcc13.3320.0028.3323.33
Long-CoTFGT-6.78-3.57-0.45-0.21

Here, l0=0l_0=0 denotes the use of embedding-layer representations. FGT reports the change in general-domain performance, with negative values indicating degradation.

Table 8 compares l0∈{0,1,4,8}l_0\in\{0,1,4,8\}. On SuperNI, l0=1l_0=1 gives the highest AR with near-zero BWT; the AR differences across these choices are small. On the reasoning benchmark, l0=4l_0=4 gives the highest reported reasoning score, whereas increasing the depth to l0=8l_0=8 reduces this score to 23.33. These results support l0=1l_0=1 for our SuperNI setting and show that the preferred depth can depend on the adaptation task. More task-specific representations may help routing during long reasoning sequences, but the layer ablation also changes the number of expert-augmented layers. These experiments have been added to Appendix H of the revised manuscript.

Thank you for the detailed responses and additional experiments. I prefer to maintain my original positive rating.

We would like to thank you again for your time and your positive rating. This is a great affirmation of our work.

Reviewer_1EFAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Expansion and Identification (AEI), a framework for mitigating catastrophic forgetting in the continual learning of large language models. AEI addresses two key challenges: interference between tasks and redundant expert expansion. Specifically, it introduces (1) an adaptive expert expansion mechanism that uses structured regularization and pruning to encourage complementary feature learning while limiting parameter redundancy, and (2) a context-aware gating module trained with task-identity supervision and temporally asymmetric label smoothing to identify and combine relevant experts. Previously learned experts are frozen to preserve historical knowledge, while only a small amount of rehearsal data is required to maintain effective routing. Experimental results show that AEI consistently outperforms competitive baselines in average ROUGE-L and backward transfer, achieving near-zero forgetting with only 1% rehearsal data.

Critical strengths
  1. This paper demonstrated strong empirical performance across different LLM backbones and long task sequences. AEI consistently outperforms competitive continual learning baselines across multiple task orders, model architectures, and sequences of up to 50 tasks, while requiring only 1% rehearsal data.
Critical weaknesses
  1. While the paper evaluates AEI on multiple LLM backbones, including Qwen3-4B, it does not clarify whether Qwen3's thinking mode is enabled. Moreover, the evaluation is restricted to SuperNI instruction-following tasks, without examining continual learning on reasoning-intensive benchmarks. Given the growing importance of reasoning capabilities in modern LLMs as of 2026, it would be valuable to investigate whether AEI can preserve reasoning performance across sequential tasks, particularly when long chain-of-thought generation is involved. If this method doesn't apply to chain-of-thought style RL-finetuned LLM , please document the reason.

  2. Although AEI achieves near-zero and occasionally positive backward transfer, these results primarily demonstrate effective knowledge retention rather than substantial positive transfer across tasks. Moreover, the proposed temporally asymmetric label smoothing explicitly discourages newly introduced experts from contributing to previously learned tasks, potentially limiting beneficial backward knowledge transfer. In an ideal world we will want to knowledge from different tasks benefit from each other. Please provide some forward transfer measurements and task-level transfer analyses of the proposed approach.

Questions for authors
  1. Whether Qwen3's thinking mode is enabled for all experiments?
  2. In table 5's caption, it's mentioned that "Cost comparison using HuggingFace on a single NVIDIA RTX 6000 GPU", however, there's no such GPU as "NVIDIA RTX 6000", which exact GPU are you referring to? NVIDIA Quadro RTX 6000 or NVIDIA RTX A6000 or NVIDIA RTX 6000 Ada or NVIDIA RTX PRO 6000?
5 comments
AuthorsOpenReview ↗

Part1: Responses to weakness 1 & question 1

We sincerely thank the reviewer for the thoughtful and constructive comments. We have tried our best to address your concerns below, and the corresponding revisions are highlighted in blue in the revised manuscript.

Weakness 1 & Question 1: Qwen3 thinking mode and evaluation of Post-training LLM.

Response: We thank the reviewer for this important question. AEI is designed as a downstream adaptation framework that adaptively expands experts according to the requirements of incoming tasks, rather than being tied to a particular training stage of the backbone model. Specifically, AEI keeps the given backbone and previously learned experts frozen, while introducing new expert branches and updating the router to accommodate new tasks. Therefore, its design does not inherently depend on whether the backbone is a pretraining-only checkpoint or has undergone post-training.

Regarding the Qwen3 experiments, we clarify that the original experiments used Qwen3-4B-Base, a pretraining-only checkpoint without SFT or RL post-training, and thus did not involve thinking mode. To further substantiate the applicability of AEI, we conduct two complementary experiments involving long-CoT adaptation and sequential adaptation evaluated with thinking mode enabled.

Table 1. Continual fine-tuning results on the mathematical reasoning dataset using Qwen2.5-14B-Instruct.

MethodMMLU-ProIFEvalGeneral Avg.AIME24AIME25Math Avg.
Initial Model65.5186.6976.1016.6713.3315.00
FT w/o Replay57.9970.8664.4330.0026.6728.33
FT w/ Replay59.2280.8270.0220.0026.6723.33
Gain62.5683.7373.1423.3323.3323.33
AEI (Ours)65.0886.2175.6530.0026.6728.33

Learning from long-CoT reasoning data. We fine-tune Qwen2.5-14B-Instruct on s1K-1.1 [1], whose average sequence length exceeds 10,000 tokens. As shown in Table 1, AEI achieves mathematical reasoning performance comparable to the best-performing baseline, while substantially better preserving general-domain capabilities. In particular, AEI maintains performance close to the initial model, demonstrating its ability to learn from long-CoT reasoning data without substantially sacrificing previously acquired knowledge. This experiment is added in the Section 4 of our revised manuscript.

[1] Muennighoff et al. s1: Simple test-time scaling. EMNLP 2025.

Table 2. Performance of Qwen3-4B with thinking mode enabled after sequential fine-tuning on five SuperNI tasks.

MethodGSM8KOpenBookQABBHIFEvalSuperNI Avg. ROUGE-L
Initial Model89.288.696.883.621.0
O-LoRA87.688.095.278.349.6
InfLoRA87.688.295.278.250.5
Gain88.488.695.281.350.8
AEI (Ours)89.388.696.083.351.5

Preserving reasoning during sequential adaptation. We additionally fine-tune Qwen3-4B sequentially on five SuperNI tasks and evaluate it with thinking mode enabled. As shown in Table 2, AEI largely preserves the initial model's reasoning and instruction-following capabilities, while achieving the best SuperNI performance among the compared methods. These results suggest that AEI can accommodate sequential instruction adaptation without substantially compromising existing reasoning capabilities. This experiment has been added to Appendix H of the revised manuscript.

AuthorsOpenReview ↗

Part2: Responses to weakness 2 & question 2

Weakness 2: Forward transfer and task-level transfer analysis

Response: We agree that near-zero BWT primarily demonstrates knowledge retention, rather than substantial positive backward transfer. AEI aims to preserve learned knowledge while enabling its reuse for subsequent tasks. We provide additional transfer measurements and an ablation of asymmetric label smoothing to examine this trade-off.

How asymmetric smoothing affects transfer. For a sample from task TcT_c, asymmetric label smoothing assigns positive target weights to its corresponding expert and previously learned experts, but zero target weights to experts introduced after task cc. This encourages forward reuse while discouraging later experts from interfering with earlier tasks. These are training targets, not a hard inference-time mask: the router produces a soft mixture over all available experts and is updated at each stage. Consequently, earlier-task performance can still change, and positive BWT remains possible.

Forward transfer and smoothing ablation. We rerun the 20- and 30-task experiments on Qwen3-4B-Base, comparing Gain, AEI with asymmetric label smoothing (AEI-ALS, our default), and AEI with symmetric label smoothing (AEI-SLS). For AEI-SLS, a sample's corresponding expert receives target probability 1−ε1-\varepsilon, and the remaining mass ε\varepsilon is distributed uniformly over the other available experts, including those introduced after the sample's task. Following the definition used in Appendix H of the revised manuscript, we compute

FWT=1T∑t=1T(Ptt−Ptind),\mathrm{FWT}=\frac{1}{T}\sum_{t=1}^{T}\left(P_t^t-P_t^{\mathrm{ind}}\right),

where PttP_t^t is the performance on task tt after learning that task in the continual learning sequence, and PtindP_t^{\mathrm{ind}} is the performance obtained by training independently on task tt. This measures the benefit of learning previous tasks for subsequent task acquisition.

Table 3. Forward transfer (FWT) and backward transfer (BWT) comparison on Qwen3-4B-Base.

Method20 Tasks AR ↑20 Tasks FWT ↑20 Tasks BWT ↑30 Tasks AR ↑30 Tasks FWT ↑30 Tasks BWT ↑
Gain63.350.75-1.8860.320.52-1.60
AEI-SLS65.921.06-0.1063.110.57-0.16
AEI-ALS66.271.210.0863.450.78-0.06

As shown in Table 3, AEI-ALS consistently achieves better overall performance and forward transfer than both Gain and AEI-SLS, while maintaining BWT close to zero. These results reveal that the knowledge required by each task has been sufficiently captured by its current expert together with previously learned experts, and subsequently introduced experts provide limited additional benefits to earlier tasks and may instead interfere with previously learned experts.

Task-level backward transfer. In the 50-task setting, among the 49 earlier tasks, 26 improve, 21 decline, and two remain unchanged, with an average BWT of +0.033. For example, the score on task223 is 40.919 at the reference stage and reaches 42.423 and 41.749 at two subsequent training stages, both above the reference score. These results show that a near-zero average can coexist with both positive and negative task-level changes. We therefore interpret AEI's main advantage as effective retention and forward reuse, with modest, task-dependent backward improvements, rather than uniformly positive or substantial backward transfer. These discussions are added in the Appendix H of our revised manuscript.


Question 2: GPU specification in the cost comparison.

Response. We apologize for the imprecise GPU name. The cost comparison was conducted on a single NVIDIA RTX PRO 6000 GPU. We have corrected the GPU specification in the revised manuscript.

Dear Reviewer 1EFA,

Many thanks for your time and efforts in reviewing our paper.

We have made extensive efforts to address your concerns and answer your questions by providing an additional experiment on a post-trained LLM and further transfer analyses, as well as incorporating the corresponding revisions into the manuscript and highlighting them in blue.

If you have any further concerns or questions, please do not hesitate to let us know. We would be happy to address them promptly.

All the best,

Authors of submission 39422

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.