acceptodds
Under review as a conference paper at ICLR 2027

MemSFT: Mitigating Alignment Tax with an External Parametric Memory

Abstract

Adapting Large Language Models (LLMs) to specialized domains often incurs an alignment tax, as fine-tuning on domain-specific tasks can cause catastrophic forgetting on general tasks. We propose MemSFT, which mitigates the alignment tax by decoupling domain specialization from backbone LLM parameter updates through a plug-and-play parametric memory. Building on Memory Decoder (Cao et al., 2026a), which primarily studies external memory for pretrained LLMs, we develop supervised fine-tuning on external memory to support domain specialization of post-trained LLMs. To selectively invoke domain expertise, we introduce a dynamic memory injector that fuses memory and backbone output distributions at each decoding step. We evaluate MemSFT across three domains, namely biology, geoscience, and law, covering small dense and large mixture-of-experts (MoE) LLMs ranging from 8B to 397B parameters. MemSFT consistently improves domain performance without catastrophic forgetting of general capabilities, whereas full SFT suffers severe forgetting on general tasks. Our results demonstrate a practical path to domain specialization of modern LLMs, enabling memory reuse across multiple backbones with only 0.27× and 0.41× the adaptation FLOPs of full SFT and LoRA for a 397B model.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Parametric Memory, SFT

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UyV2OpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

MemSFT is designed to specialize post-trained language models while preserving their general capabilities. It keeps the backbone frozen and trains a separate domain-memory model using gold answer tokens together with soft targets derived from similar training examples. During decoding, a lightweight injector determines how to combine the backbone’s and memory model’s predictions at each step. The memory can be reused with different backbones that use compatible tokenizers, although each backbone requires its own injector. The authors evaluate MemSFT on biology, geoscience, and law tasks with Qwen3, LLaMA2, and a 397B Intern model. Compared with full SFT, LoRA, and several retrieval and fusion baselines, MemSFT achieves substantial gains on domain-specific tasks while producing little change on the selected general-capability benchmarks. The paper also estimates the reduction in adaptation compute and measures inference latency in the 397B setting.

Critical strengths
  1. Strong specialization with little loss of general ability. On Qwen3-14B, MemSFT improves BioIns from 6.64 to 42.92, while the five-benchmark general average changes only from 83.22 to 83.62. Similar results across model sizes and domains support the paper’s main claim under the evaluated conditions.

  2. Ablations support the proposed mechanism. The learned token-level injector substantially outperforms fixed interpolation. With the same injector, retrieval-trained memory also outperforms the standard SFT-trained external expert on BioIns (42.92 versus 37.34). These controls provide stronger evidence for the proposed design than a comparison with backbone fine-tuning alone.

Critical weaknesses
  1. The main generalization claim depends on an unvalidated retrieval target. The teacher distribution comes from next-token predictions of neighbors selected by distance in a frozen language model’s representation space. The paper does not show that these neighbors provide answers compatible with the current question. Similar-looking expert questions may still require different entities, numerical values, or conclusions. The KL term could therefore encourage incorrect continuations. Most of the evidence comes from BioIns and OpenSWI, whose structured task distributions largely overlap with those used for adaptation. The direct comparison with an SFT-trained external expert is reported only on BioIns. The broader claim would be more convincing with neighbor-answer agreement analyses, teacher probabilities on gold tokens, controlled β=0\beta=0 ablations across domains, and tests on fact-sensitive or genuinely shifted questions. Without these checks, the results support domain gains on the evaluated tasks, but not reliable domain specialization in the broader sense.

  2. The experiments do not show whether the backbone gains useful domain ability or whether MemSFT mainly routes between a generalist and a small specialist. With the same 8B memory, BioIns stays around 42–43 for Qwen3 backbones from 8B to 235B, while OpenSWI RMSE remains 0.47 at every size. Most domain outputs may therefore be determined by the memory, with little dependence on backbone size. Yet the crucial memory-only setting (λ=1\lambda=1) is not reported. General-capability retention may also come from the injector learning to separate clearly domain-formatted inputs from unrelated general benchmarks. Table 18 is consistent with this explanation, because an ordinary SFT expert gives similar retention. Direct alignment and safety evaluations are also missing, although the paper presents MemSFT as a way to reduce the “alignment tax.” The efficiency analysis estimates FLOPs for fine-tuning the large backbone, but does not compare them with a small SFT expert plus router or report the corresponding throughput measurement.ut for the smaller-backbone settings. These omissions leave the claimed synergy, alignment benefit, and practical advantage unresolved.

Questions for authors
  1. What is the performance in the memory-only setting (λ=1\lambda=1) on BioIns and OpenSWI, including the 4B memory paired with the 397B model? Domain scores remain nearly unchanged as the Qwen3 backbone grows. What evidence shows that larger backbones contribute to the domain predictions?

  2. How reliable are the retrieval teacher targets? What probability does the teacher assign to the gold next token, and how often do retrieved neighbors provide answer-compatible targets? Please report these statistics by task type and include matched β=0\beta=0 ablations across domains.

  3. How does the injector behave when domain and general content overlap, or when similarly formatted domain questions require different factual answers? How much general-capability retention remains after removing the signed regularizer (αs=0\alpha_s=0)?

  4. Have the authors evaluated safety or preference-alignment behavior after memory injection? If not, what evidence supports the “alignment tax” claim beyond retention on the reported capability benchmarks?

8 comments
AuthorsOpenReview ↗

Response to Reviewer UyV2 (Part 1/3)

Thank you for the thoughtful feedback. We appreciate the reviewer's recognition of MemSFT’s strong specialization–retention trade-off across model sizes and domains, and the evidence our ablations provide for the proposed mechanism.

Weakness 2 & Question 1. Memory-only performance and backbone contributions

We appreciate the reviewer’s thoughtful question about domain capability. As shown below, MemSFT slightly outperforms memory alone in each of these configurations. With a domain-capable backbone such as the Intern 397B model, MemSFT benefits more from the backbone and achieves better domain performance than either model alone.

TaskBackboneBackbone onlyMemory onlyMemSFT
BioIns ↑Qwen3-14B6.6442.6342.92
OpenSWI RMSE ↓Qwen3-14B4.390.47610.4740
BioIns ↑Intern-S2-Preview-397B56.9259.2860.32

The Qwen3 and Intern configurations use 8B and 4B memories, respectively.

Indeed, domain scores remain nearly unchanged across Qwen3 backbone sizes. These backbones lack the required domain expertise, which increasing model size alone does not provide. This is exactly why domain specialization is necessary even for highly capable general-purpose models. Larger backbones provide stronger general capabilities, while the memory supplies the missing domain expertise. We evaluate different backbones to demonstrate that our method remains effective across model scales.

The challenge, therefore, is to acquire domain expertise without losing the general capabilities of the backbone. Conventional SFT or CPT adapts the model by updating the backbone, but often causes severe forgetting of capabilities established during post-training. MemSFT achieves what previous methods have not: strong domain expertise alongside intact general capabilities, resolving a longstanding challenge in domain specialization.

Weakness 1 & Question 2. Retrieval supervision

Thank you for highlighting this critical question about retrieval supervision. We have expanded the β = 0 ablation to both BioIns and OpenSWI across Qwen3-8B, 14B, and 32B. These results show that the advantage of retrieval supervision holds across domains and backbone sizes.

DomainBackboneDomain: β = 0Domain: MemSFTGeneral avg.: β = 0General avg.: MemSFT
BioIns ↑Qwen3-8B37.0542.8480.0481.08
BioIns ↑Qwen3-14B37.3442.9283.8183.62
BioIns ↑Qwen3-32B37.5142.8285.0785.47
OpenSWI RMSE ↓Qwen3-8B0.49760.4680.9080.80
OpenSWI RMSE ↓Qwen3-14B0.50570.4783.4882.76
OpenSWI RMSE ↓Qwen3-32B0.51660.4684.9584.94

The KL target aggregates a weighted distribution over retrieved neighbors rather than treating one retrieved answer as ground truth. It provides additional supervision from the outputs associated with similar contexts, allowing the memory to learn beyond the single observed target. Individual neighbors can be incompatible while the aggregate distribution still supports the correct answer. Meanwhile, the CE term directly supervises the gold answer, anchoring the memory to the reference target.

The reviewer also suggests a valuable way to examine why this supervision works. Following this suggestion, we analyzed the retrieval-teacher distributions for 1,000 sampled BioIns training QAs. The following results show that the teacher distribution supports the reference targets and that compatible neighbors receive most of the retrieval weight. This provides useful additional supervision alongside the gold CE objective.

Task typeMean gold-token probabilityGold in top-5Mean compatible retrieval weight
Classification83.79%98.95%79.31%
Numerical prediction49.15%81.29%82.37%
Structured output75.84%93.03%81.57%
Overall69.52%91.36%80.84%

The gold token denotes the next token in the reference answer. Compatible retrieval weight is computed by summing the weights of answer-compatible neighbors for each evaluated QA.

Following the reviewer’s suggestion, we also evaluated whether similar inputs requiring different answers remain distinguishable. We paired similar sequences with opposite labels from the BioIns test set, prioritizing pairs with fewer differing positions. The results are shown below:

ModelBoth answers correct
Qwen3-14B backbone3.33%
SFT46.67%
MemSFT60.00%

MemSFT distinguishes these similar pairs more accurately than the SFT model. Together with the cross-domain ablations, these results show that retrieval supervision improves domain learning rather than simply forcing similar inputs toward identical answers. We will include these additional results and diagnostics in the revision.

AuthorsOpenReview ↗

Response to Reviewer UyV2 (Part 2/3)

Weakness 2 & Question 3. Injector performance and regularization

Thank you for raising this point about overlapping domain and general content. In our setting, a request is considered domain-specific if it involves domain content, even when it also requires general capabilities. Domain and general content can therefore coexist within a domain-specific request. Producing a specialized answer may also involve explanation, instruction following, or further processing of the domain prediction.

The injector predicts λ at every decoding step, dynamically adjusting the contributions of the backbone and memory within a single response. In the OpenSWI example in Figure 4, it favors the backbone for ordinary language and the memory for specialized numerical predictions.

In response to the reviewer’s question, we conducted an additional experiment requiring domain expertise together with general skills for processing the resulting predictions, as can arise in practical domain applications. Specifically, on 200 DNA questions, the model must classify a new sequence and use its judgment to update an existing count of positive samples.

MethodBio accuracyCounting completionJoint accuracy
Backbone49.00%100%49.00%
Memory only85.00%0%0%
MemSFT84.50%100%84.50%

The results show that neither the backbone nor the memory alone adequately handles both parts of the task within a single response: the backbone completes counting but struggles with biological judgments, while memory alone makes accurate biological judgments but fails to complete counting. In contrast, MemSFT performs both tasks within the same response.

Regarding the signed regularizer, domain/general labels are typically available for the training data, making this supervision readily applicable in practice. These labels are used only during injector training. At inference, no domain label or prior identification of the request’s domain is required: the injector predicts λ from the models’ hidden representations at each decoding step.

Nevertheless, the reviewer raises a valuable point about the regularizer’s contribution, and we conducted an additional ablation with αs=0\alpha_s = 0 on Qwen3-14B. Even without regularization, MemSFT retains strong domain performance and substantially better general capabilities than SFT. The regularizer further strengthens general-capability preservation while maintaining domain performance.

ConfigurationBioIns ↑MATH-500C-EvalIFEvalMMLU-ReduxINCLUDEGeneral avg. ↑
Qwen3-14B (Backbone)6.6496.483.185.790.060.983.22
SFT41.1631.279.718.972.756.751.83
MemSFT without regularization42.786.1683.0674.3186.9961.0478.31
MemSFT with regularization42.9296.58385.490.562.883.62
AuthorsOpenReview ↗

Response to Reviewer UyV2 (Part 3/3)

Weakness 2 & Question 4. Alignment tax and safety

The reviewer raises an important question about safety and alignment. To clarify, in our paper, alignment tax refers to the loss of a post-trained model’s existing capabilities during domain specialization, including knowledge, reasoning, instruction following, and safety behavior. As discussed in the Introduction, our focus therefore extends beyond safety alone, and our main results already demonstrate mitigation of this tax.

To provide direct evidence on safety, we additionally evaluated Qwen3-14B, its SFT counterpart, and MemSFT on XSTest and HarmBench. XSTest measures refusal of unsafe requests and over-refusal of safe requests, while HarmBench measures attack success in eliciting harmful responses.

BenchmarkMetricBackboneSFTMemSFT
XSTestRefusal rate on unsafe prompts ↑96.50%89.50%96.50%
XSTestRefusal rate on safe prompts ↓5.20%3.20%4.80%
HarmBenchAttack success rate ↓22.08%42.50%22.50%

The results show that SFT substantially degrades safety behavior, reducing refusal of unsafe requests and nearly doubling the HarmBench attack success rate. In contrast, MemSFT maintains essentially the same safety performance as the backbone: unsafe-prompt refusal remains unchanged, while refusal rate on safe prompts is even slightly lower.

Together with our main results, these findings demonstrate that MemSFT mitigates the alignment tax associated with domain specialization, including degradation of safety behavior. We will include these results and evaluation details in the revision.

Weakness 2. Inference efficiency

Thank you for acknowledging our adaptation-FLOPs results and highlighting the efficiency question. We would like to clarify that the 8B+8B configuration serves as a controlled experimental setting. In a practical deployment scenario, the backbone is typically a frontier model with hundreds of billions of parameters. Crucially, memory size does not need to scale with backbone size.

To address the reviewer’s request, we have also measured latency, throughput, and peak GPU memory for the 8B+8B and 14B+8B configurations, and additionally evaluated Qwen3-235B-A22B+8B Memory. We used A100-80GB GPUs, BF16, and vLLM, with two GPUs for the smaller configurations and eight GPUs for the 235B configurations. All configurations used the same input workload and a fixed output length of 512 tokens. Results are averaged over three runs.

ConfigurationE2E latency (s)Throughput (tokens/s)Peak memory (GiB/GPU)
8B backbone5.6754035.7565.69
8B+8B MemSFT7.4403553.8065.92
14B backbone8.0133116.5865.86
14B+8B MemSFT11.6132530.0665.96
235B backbone15.031896.0464.46
235B+8B MemSFT16.341846.5063.65

On the 235B model, MemSFT retains 97.4% of the backbone’s throughput while adding only 1.31 seconds of end-to-end latency. Even in the smaller configurations, where memory is comparable in size to the backbone, latency and throughput overhead remain manageable, although these are not our primary deployment settings.

Once trained, the memory performs inference like a standard language model, without retrieval or KL computation. A same-sized SFT expert + injector therefore has the same inference-time computation as MemSFT, while MemSFT achieves stronger domain performance. We will include these additional measurements in the revision.

Reviewer_hf35OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MemSFT, a domain-specialization framework that freezes a post-trained backbone and stores domain expertise in an external parametric memory trained through supervised learning and retrieval imitation. A lightweight token-level injector dynamically combines the output distributions of the backbone and memory. Experiments across biology, geoscience, and law with models ranging from 8B to 397B show substantial domain gains with minimal degradation of general capabilities, while allowing the same memory to be reused across compatible backbones.

Critical strengths
  • MemSFT effectively improves domain performance while preserving the general capabilities of the frozen backbone.
  • The evaluation is extensive, covering three domains, multiple model families, and backbones ranging from 8B to 397B.
Critical weaknesses
  • The novelty over Memory Decoder is limited, with dynamic token-level injection appearing to be the main methodological extension.
  • Inference efficiency is insufficiently evaluated, particularly for configurations in which the memory is comparable in size to the backbone.
Questions for authors
  • What are the key methodological advances of MemSFT over Memory Decoder beyond applying the external-memory framework to supervised domain adaptation?
  • The comparison with a dynamically routed standard SFT expert is limited to Qwen3-14B on BioIns; does the advantage of retrieval-trained memory hold across other domains and backbone sizes?
  • Since MemSFT runs both models during inference, please report latency, throughput, and peak memory usage for the 8B+8B and 14B+8B configurations under matched serving conditions.
2 comments
AuthorsOpenReview ↗

Response to Reviewer hf35

Thank you for recognizing MemSFT’s effectiveness in domain specialization and general capability preservation, and our broad evaluation across domains, model families, and scales.

Weakness 1 & Question 1. Contribution beyond Memory Decoder

We appreciate the reviewer’s question and would like to clarify MemSFT’s contribution beyond Memory Decoder. MemSFT enables strong domain specialization of frontier post-trained models without sacrificing their existing general capabilities, which previous works have not achieved. We validate this result across domains, model families, and scales. On Qwen3-14B, MemSFT achieves strong domain performance while maintaining general capabilities, whereas SFT and LoRA incur substantial losses:

ModelDomain Performance: BioIns ↑General Performance: Avg. ↑
Qwen3-14B6.6483.22
+ SFT41.1651.83
+ LoRA32.0773.61
+ MemSFT42.9283.62

Even at the scale of 235B parameters, MemSFT delivers a substantial improvement in domain performance while fully retaining the model’s strong existing general capabilities:

ModelDomain Performance: BioIns ↑General Performance: Avg. ↑
Qwen3-235B-A22B1.1487.07
+ MemSFT42.0587.11

Memory Decoder and previous related methods have not established such lossless domain specialization. The challenge is to resolve the dilemma between strong domain specialization and preservation of capabilities established through post-training. As discussed in our Introduction, Ji et al. (2025) demonstrate the elasticity of post-alignment models. Further fine-tuning can rapidly erode the instruction-following and aligned behavior acquired during post-training.

Weakness 2 & Question 3. Inference efficiency

We appreciate your attention to inference efficiency. We would like to clarify that the 8B+8B configuration serves as a controlled experimental setting. In a practical deployment scenario, the backbone is typically a frontier model with hundreds of billions of parameters. Crucially, memory size does not need to scale with backbone size.

To address the reviewer’s request, we have also measured latency, throughput, and peak GPU memory for the 8B+8B and 14B+8B configurations, and additionally evaluated Qwen3-235B-A22B+8B Memory. We used A100-80GB GPUs, BF16, and vLLM, with two GPUs for the smaller configurations and eight GPUs for the 235B configurations. All configurations used the same input workload and a fixed output length of 512 tokens. Results are averaged over three runs.

ConfigurationE2E latency (s)Throughput (tokens/s)Peak memory (GiB/GPU)
8B backbone5.6754035.7565.69
8B+8B MemSFT7.4403553.8065.92
14B backbone8.0133116.5865.86
14B+8B MemSFT11.6132530.0665.96
235B backbone15.031896.0464.46
235B+8B MemSFT16.341846.5063.65

On the 235B model, MemSFT retains 97.4% of the backbone’s throughput while adding only 1.31 seconds of end-to-end latency. Even in the smaller configurations, where memory is comparable in size to the backbone, latency and throughput overhead remain manageable, although these are not our primary deployment settings. We will include these additional measurements in the revision.

Question 2. Memory advantage across domains and scales

Thank you for suggesting this broader comparison. We now compare the two approaches on Qwen3-8B, 14B, and 32B across BioIns and OpenSWI. These results show that the advantage of retrieval-trained memory holds across domains and backbone sizes. We will include these expanded results in the revision.

DomainBackboneDomain avg.: SFT expert + injectorDomain avg.: MemSFTGeneral avg.: SFT expert + injectorGeneral avg.: MemSFT
BioIns ↑Qwen3-8B37.0542.8480.0481.08
BioIns ↑Qwen3-14B37.3442.9283.8183.62
BioIns ↑Qwen3-32B37.5142.8285.0785.47
OpenSWI RMSE ↓Qwen3-8B0.49760.4680.9080.80
OpenSWI RMSE ↓Qwen3-14B0.50570.4783.4882.76
OpenSWI RMSE ↓Qwen3-32B0.51660.4684.9584.94

Thank you for the response and additional experiments. They address some of my concerns, and I will maintain my original score.

Reviewer_DtoHOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces MemSFT, which builds on Memory Decoder to specialize a frozen LLM through an external domain-specific memory LM and a learned, token-dependent mixing weight. The memory is trained using domain SFT data with language-modeling and retrieval-imitation objectives, and a dynamic injector combines its output distribution with that of the backbone. Experiments evaluate adaptation to individual domains and retention of general capabilities across LLMs ranging from 8B to 397B parameters.

Critical strengths
  1. Evaluation across a broad range of model sizes. The experiments cover LLMs ranging from 8B to 397B parameters. This breadth strengthens the empirical evidence that the approach is useful across substantially different model scales, rather than being specific to a single backbone or size.
Critical weaknesses
  1. Limited methodological novelty and insufficient positioning relative to modular continual learning. I view the approach as closely related to the established strategy of adding trainable modules and learning how to route or combine their contributions while preserving existing capabilities. In MemSFT, the memory LM supplies the additional module, and the injector controls its contribution through output-distribution mixing. Relevant work includes DEMix, ModuleFormer, MoCL, and L2R, as well as related multimodal continual-learning methods such as MoELoRA in CoIN and CL-MoE. These methods differ in architecture and setting, but they are relevant to the broader module-and-routing perspective. The related-work discussion and experimental comparisons do not sufficiently establish MemSFT's contribution relative to this literature. Moreover, the external memory design builds on Memory Decoder. In this context, I find the methodological increment limited, and the reported results provide little additional insight beyond what I would expect from combining a domain module with learned mixing.

  2. The adaptation setting limits the significance of the continual-learning contribution. Each experiment adapts the backbone to a single new domain and evaluates retention of its general capabilities. Although several domains are studied, they are evaluated separately, without a sequence of newly learned domains or tasks. Consequently, the experiments do not establish whether the approach can retain previously added domain capabilities as further domains are learned. In my assessment, this narrower setting limits the significance of the work from a continual-learning perspective.

Questions for authors

Clarification request: Which hidden representations are supplied to the injector? For both the backbone and the memory LM, please specify the layer and the point within that layer from which the representation is extracted.

4 comments
AuthorsOpenReview ↗

Response to Reviewer DtoH

We sincerely thank the reviewer for the thoughtful and constructive feedback, and we are pleased that the reviewer recognized the breadth of our evaluation across models ranging from 8B to 397B.

Weakness 1. Novelty and positioning

Thank you for raising the question of MemSFT’s contribution. To clarify, MemSFT achieves what previous methods have not—strong domain specialization of frontier post-trained models without sacrificing their existing general capabilities. Existing modular architectures and routing mechanisms alone do not solve this challenge. We validate MemSFT across multiple domains, model families, and scales. On Qwen3-14B, MemSFT achieves strong domain performance while maintaining general capabilities, whereas SFT and LoRA incur substantial losses:

ModelDomain Performance: BioIns ↑General Performance: Avg. ↑
Qwen3-14B6.6483.22
+ SFT41.1651.83
+ LoRA32.0773.61
+ MemSFT42.9283.62

Even at the scale of 235B parameters, MemSFT delivers a substantial improvement in domain performance while fully retaining the model’s strong existing general capabilities:

ModelDomain Performance: BioIns ↑General Performance: Avg. ↑
Qwen3-235B-A22B1.1487.07
+ MemSFT42.0587.11

We appreciate these connections to modular continual learning and will expand the related-work discussion accordingly. These studies primarily focus on domain language modeling or mitigating forgetting of previously learned tasks, at substantially smaller model scales. They do not demonstrate strong domain specialization of frontier post-trained models while preserving their broad existing general capabilities.

Weakness 2. Scope of continual learning

We appreciate the reviewer’s question and would like to clarify our focus: preserving a post-trained model’s existing capabilities when adapting it to a specialized domain. Even adaptation to a single domain can substantially degrade general capabilities, as our SFT and LoRA comparisons demonstrate. Addressing this problem at large scales is the central objective of our work.

Our study does not target sequential learning of new domains or tasks, and therefore makes no claim about retaining previously added domain capabilities. We will clarify this scope in the revision. Extending MemSFT to sequential domain adaptation is an important and particularly promising direction for future work, and we thank the reviewer for highlighting it.

Question 1. Injector inputs

Thank you for raising this question. The paper describes the injector’s hidden-state inputs but does not explicitly specify their extraction points. For both the backbone and the memory LM, we use the hidden representation after the final decoder block and the model’s final normalization, immediately before the LM head—the same representation used to compute that branch’s next-token logits. We will make this explicit in the revised method section.

Thank you for the response. I appreciate the authors' willingness to expand the related-work discussion. My concerns remain:

  • The response has not convinced me that "domain specialization" is sufficiently distinct from "domain language modeling or mitigating forgetting of previously learned tasks". If it refers to sth not pretrained, like medical, science, ... It should be a very common machine learning task.
  • The response does not provide any comparison with any modular approaches, either in terms of method novelty or empirical performance.

Thank you for clarifying your remaining concerns.

Research objective. We agree that domain specialization overlaps with these research settings. Our distinction lies in retaining the broad capabilities of frontier post-trained models during strong domain adaptation, at scales of hundreds of billions of parameters. We will clarify this positioning.

Modular baseline. Following the reviewer’s suggestion, we additionally evaluated RouteLLM’s MF router. RouteLLM is a well-known framework for model routing, learning to select between frozen language models. It successfully routes domain requests to the expert and nearly all general requests to the backbone.

MethodBioIns ↑General avg. ↑
Qwen3-8B5.0780.52
+ RouteLLM37.3480.45
+ MemSFT42.8481.08

However, selecting one model for an entire request does not enable it to use both models’ capabilities within that response. We therefore conducted an additional DNA + counting test: the model must classify a new sequence and use its judgment to update an existing count of positive samples.

MethodBio accuracyCounting completionJoint accuracy
Backbone49.00%100%49.00%
RouteLLM85.00%0%0%
MemSFT with adapted injector84.50%100%84.50%

MemSFT’s token-level injection enables both capabilities within the same response, while RouteLLM’s request-level selection does not, as neither model alone can adequately handles both parts of the task.

Thank you for the clarification and additional experiments. So we are aligned that the research objective is common and maybe important, but not novel. Regarding the new experiment, I understand that the DNA-plus-counting illustrates the benefit of memsft's token-level routing, while routellm is request-level routing. However, this comparison has limited scope. The result is expected and not new; token-level routing was already investigated even in three years ago, like ModuleFormer (2023). My concern about novelty and significance still remains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.