- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
MemSFT is designed to specialize post-trained language models while preserving their general capabilities. It keeps the backbone frozen and trains a separate domain-memory model using gold answer tokens together with soft targets derived from similar training examples. During decoding, a lightweight injector determines how to combine the backbone’s and memory model’s predictions at each step. The memory can be reused with different backbones that use compatible tokenizers, although each backbone requires its own injector. The authors evaluate MemSFT on biology, geoscience, and law tasks with Qwen3, LLaMA2, and a 397B Intern model. Compared with full SFT, LoRA, and several retrieval and fusion baselines, MemSFT achieves substantial gains on domain-specific tasks while producing little change on the selected general-capability benchmarks. The paper also estimates the reduction in adaptation compute and measures inference latency in the 397B setting.
Critical strengths
-
Strong specialization with little loss of general ability. On Qwen3-14B, MemSFT improves BioIns from 6.64 to 42.92, while the five-benchmark general average changes only from 83.22 to 83.62. Similar results across model sizes and domains support the paper’s main claim under the evaluated conditions.
-
Ablations support the proposed mechanism. The learned token-level injector substantially outperforms fixed interpolation. With the same injector, retrieval-trained memory also outperforms the standard SFT-trained external expert on BioIns (42.92 versus 37.34). These controls provide stronger evidence for the proposed design than a comparison with backbone fine-tuning alone.
Critical weaknesses
-
The main generalization claim depends on an unvalidated retrieval target. The teacher distribution comes from next-token predictions of neighbors selected by distance in a frozen language model’s representation space. The paper does not show that these neighbors provide answers compatible with the current question. Similar-looking expert questions may still require different entities, numerical values, or conclusions. The KL term could therefore encourage incorrect continuations. Most of the evidence comes from BioIns and OpenSWI, whose structured task distributions largely overlap with those used for adaptation. The direct comparison with an SFT-trained external expert is reported only on BioIns. The broader claim would be more convincing with neighbor-answer agreement analyses, teacher probabilities on gold tokens, controlled ablations across domains, and tests on fact-sensitive or genuinely shifted questions. Without these checks, the results support domain gains on the evaluated tasks, but not reliable domain specialization in the broader sense.
-
The experiments do not show whether the backbone gains useful domain ability or whether MemSFT mainly routes between a generalist and a small specialist. With the same 8B memory, BioIns stays around 42–43 for Qwen3 backbones from 8B to 235B, while OpenSWI RMSE remains 0.47 at every size. Most domain outputs may therefore be determined by the memory, with little dependence on backbone size. Yet the crucial memory-only setting () is not reported. General-capability retention may also come from the injector learning to separate clearly domain-formatted inputs from unrelated general benchmarks. Table 18 is consistent with this explanation, because an ordinary SFT expert gives similar retention. Direct alignment and safety evaluations are also missing, although the paper presents MemSFT as a way to reduce the “alignment tax.” The efficiency analysis estimates FLOPs for fine-tuning the large backbone, but does not compare them with a small SFT expert plus router or report the corresponding throughput measurement.ut for the smaller-backbone settings. These omissions leave the claimed synergy, alignment benefit, and practical advantage unresolved.
Questions for authors
-
What is the performance in the memory-only setting () on BioIns and OpenSWI, including the 4B memory paired with the 397B model? Domain scores remain nearly unchanged as the Qwen3 backbone grows. What evidence shows that larger backbones contribute to the domain predictions?
-
How reliable are the retrieval teacher targets? What probability does the teacher assign to the gold next token, and how often do retrieved neighbors provide answer-compatible targets? Please report these statistics by task type and include matched ablations across domains.
-
How does the injector behave when domain and general content overlap, or when similarly formatted domain questions require different factual answers? How much general-capability retention remains after removing the signed regularizer ()?
-
Have the authors evaluated safety or preference-alignment behavior after memory injection? If not, what evidence supports the “alignment tax” claim beyond retention on the reported capability benchmarks?
8 comments
Response to Reviewer UyV2 (Part 2/3)
Weakness 2 & Question 3. Injector performance and regularization
Thank you for raising this point about overlapping domain and general content. In our setting, a request is considered domain-specific if it involves domain content, even when it also requires general capabilities. Domain and general content can therefore coexist within a domain-specific request. Producing a specialized answer may also involve explanation, instruction following, or further processing of the domain prediction.
The injector predicts λ at every decoding step, dynamically adjusting the contributions of the backbone and memory within a single response. In the OpenSWI example in Figure 4, it favors the backbone for ordinary language and the memory for specialized numerical predictions.
In response to the reviewer’s question, we conducted an additional experiment requiring domain expertise together with general skills for processing the resulting predictions, as can arise in practical domain applications. Specifically, on 200 DNA questions, the model must classify a new sequence and use its judgment to update an existing count of positive samples.
| Method | Bio accuracy | Counting completion | Joint accuracy |
|---|---|---|---|
| Backbone | 49.00% | 100% | 49.00% |
| Memory only | 85.00% | 0% | 0% |
| MemSFT | 84.50% | 100% | 84.50% |
The results show that neither the backbone nor the memory alone adequately handles both parts of the task within a single response: the backbone completes counting but struggles with biological judgments, while memory alone makes accurate biological judgments but fails to complete counting. In contrast, MemSFT performs both tasks within the same response.
Regarding the signed regularizer, domain/general labels are typically available for the training data, making this supervision readily applicable in practice. These labels are used only during injector training. At inference, no domain label or prior identification of the request’s domain is required: the injector predicts λ from the models’ hidden representations at each decoding step.
Nevertheless, the reviewer raises a valuable point about the regularizer’s contribution, and we conducted an additional ablation with on Qwen3-14B. Even without regularization, MemSFT retains strong domain performance and substantially better general capabilities than SFT. The regularizer further strengthens general-capability preservation while maintaining domain performance.
| Configuration | BioIns ↑ | MATH-500 | C-Eval | IFEval | MMLU-Redux | INCLUDE | General avg. ↑ |
|---|---|---|---|---|---|---|---|
| Qwen3-14B (Backbone) | 6.64 | 96.4 | 83.1 | 85.7 | 90.0 | 60.9 | 83.22 |
| SFT | 41.16 | 31.2 | 79.7 | 18.9 | 72.7 | 56.7 | 51.83 |
| MemSFT without regularization | 42.7 | 86.16 | 83.06 | 74.31 | 86.99 | 61.04 | 78.31 |
| MemSFT with regularization | 42.92 | 96.5 | 83 | 85.4 | 90.5 | 62.8 | 83.62 |
Response to Reviewer UyV2 (Part 3/3)
Weakness 2 & Question 4. Alignment tax and safety
The reviewer raises an important question about safety and alignment. To clarify, in our paper, alignment tax refers to the loss of a post-trained model’s existing capabilities during domain specialization, including knowledge, reasoning, instruction following, and safety behavior. As discussed in the Introduction, our focus therefore extends beyond safety alone, and our main results already demonstrate mitigation of this tax.
To provide direct evidence on safety, we additionally evaluated Qwen3-14B, its SFT counterpart, and MemSFT on XSTest and HarmBench. XSTest measures refusal of unsafe requests and over-refusal of safe requests, while HarmBench measures attack success in eliciting harmful responses.
| Benchmark | Metric | Backbone | SFT | MemSFT |
|---|---|---|---|---|
| XSTest | Refusal rate on unsafe prompts ↑ | 96.50% | 89.50% | 96.50% |
| XSTest | Refusal rate on safe prompts ↓ | 5.20% | 3.20% | 4.80% |
| HarmBench | Attack success rate ↓ | 22.08% | 42.50% | 22.50% |
The results show that SFT substantially degrades safety behavior, reducing refusal of unsafe requests and nearly doubling the HarmBench attack success rate. In contrast, MemSFT maintains essentially the same safety performance as the backbone: unsafe-prompt refusal remains unchanged, while refusal rate on safe prompts is even slightly lower.
Together with our main results, these findings demonstrate that MemSFT mitigates the alignment tax associated with domain specialization, including degradation of safety behavior. We will include these results and evaluation details in the revision.
Weakness 2. Inference efficiency
Thank you for acknowledging our adaptation-FLOPs results and highlighting the efficiency question. We would like to clarify that the 8B+8B configuration serves as a controlled experimental setting. In a practical deployment scenario, the backbone is typically a frontier model with hundreds of billions of parameters. Crucially, memory size does not need to scale with backbone size.
To address the reviewer’s request, we have also measured latency, throughput, and peak GPU memory for the 8B+8B and 14B+8B configurations, and additionally evaluated Qwen3-235B-A22B+8B Memory. We used A100-80GB GPUs, BF16, and vLLM, with two GPUs for the smaller configurations and eight GPUs for the 235B configurations. All configurations used the same input workload and a fixed output length of 512 tokens. Results are averaged over three runs.
| Configuration | E2E latency (s) | Throughput (tokens/s) | Peak memory (GiB/GPU) |
|---|---|---|---|
| 8B backbone | 5.675 | 4035.75 | 65.69 |
| 8B+8B MemSFT | 7.440 | 3553.80 | 65.92 |
| 14B backbone | 8.013 | 3116.58 | 65.86 |
| 14B+8B MemSFT | 11.613 | 2530.06 | 65.96 |
| 235B backbone | 15.03 | 1896.04 | 64.46 |
| 235B+8B MemSFT | 16.34 | 1846.50 | 63.65 |
On the 235B model, MemSFT retains 97.4% of the backbone’s throughput while adding only 1.31 seconds of end-to-end latency. Even in the smaller configurations, where memory is comparable in size to the backbone, latency and throughput overhead remain manageable, although these are not our primary deployment settings.
Once trained, the memory performs inference like a standard language model, without retrieval or KL computation. A same-sized SFT expert + injector therefore has the same inference-time computation as MemSFT, while MemSFT achieves stronger domain performance. We will include these additional measurements in the revision.
Response to Reviewer UyV2 (Part 1/3)
Thank you for the thoughtful feedback. We appreciate the reviewer's recognition of MemSFT’s strong specialization–retention trade-off across model sizes and domains, and the evidence our ablations provide for the proposed mechanism.
We appreciate the reviewer’s thoughtful question about domain capability. As shown below, MemSFT slightly outperforms memory alone in each of these configurations. With a domain-capable backbone such as the Intern 397B model, MemSFT benefits more from the backbone and achieves better domain performance than either model alone.
The Qwen3 and Intern configurations use 8B and 4B memories, respectively.
Indeed, domain scores remain nearly unchanged across Qwen3 backbone sizes. These backbones lack the required domain expertise, which increasing model size alone does not provide. This is exactly why domain specialization is necessary even for highly capable general-purpose models. Larger backbones provide stronger general capabilities, while the memory supplies the missing domain expertise. We evaluate different backbones to demonstrate that our method remains effective across model scales.
The challenge, therefore, is to acquire domain expertise without losing the general capabilities of the backbone. Conventional SFT or CPT adapts the model by updating the backbone, but often causes severe forgetting of capabilities established during post-training. MemSFT achieves what previous methods have not: strong domain expertise alongside intact general capabilities, resolving a longstanding challenge in domain specialization.
Thank you for highlighting this critical question about retrieval supervision. We have expanded the β = 0 ablation to both BioIns and OpenSWI across Qwen3-8B, 14B, and 32B. These results show that the advantage of retrieval supervision holds across domains and backbone sizes.
The KL target aggregates a weighted distribution over retrieved neighbors rather than treating one retrieved answer as ground truth. It provides additional supervision from the outputs associated with similar contexts, allowing the memory to learn beyond the single observed target. Individual neighbors can be incompatible while the aggregate distribution still supports the correct answer. Meanwhile, the CE term directly supervises the gold answer, anchoring the memory to the reference target.
The reviewer also suggests a valuable way to examine why this supervision works. Following this suggestion, we analyzed the retrieval-teacher distributions for 1,000 sampled BioIns training QAs. The following results show that the teacher distribution supports the reference targets and that compatible neighbors receive most of the retrieval weight. This provides useful additional supervision alongside the gold CE objective.
The gold token denotes the next token in the reference answer. Compatible retrieval weight is computed by summing the weights of answer-compatible neighbors for each evaluated QA.
Following the reviewer’s suggestion, we also evaluated whether similar inputs requiring different answers remain distinguishable. We paired similar sequences with opposite labels from the BioIns test set, prioritizing pairs with fewer differing positions. The results are shown below:
MemSFT distinguishes these similar pairs more accurately than the SFT model. Together with the cross-domain ablations, these results show that retrieval supervision improves domain learning rather than simply forcing similar inputs toward identical answers. We will include these additional results and diagnostics in the revision.