acceptodds
Under review as a conference paper at ICLR 2027

Modular Skill Internalization for Agentic Foundation Models

Abstract

Augmenting large language models with reusable procedural skills via in-context prompting suffers from an inherent bottleneck in capability acquisition. Because a model must infer execution procedures entirely from textual descriptions at runtime, the efficacy of injected skills remains strictly bounded by its pre-existing instruction-following and context-comprehension abilities. Furthermore, in-context skill descriptions can only steer model behavior through conditioning on the current input, whereas the model's learned capabilities are instantiated in its parameters. This creates a fundamental mechanistic mismatch: textual skill injection asks the model to reconstruct a new procedure through transient, context-dependent computation, rather than incorporating that procedure into the parameterized transformations that constitute its learned capabilities. To address this limitation, we introduce Modular Skill Internalization (MSI), a framework that compiles declarative skill specifications into modular, lightweight parameter updates. For each skill, MSI trains an independent parameter module from synthetic skill-specific demonstrations, such that attaching the module at inference time equips the base model with the corresponding procedural capability without requiring the original skill description. These modules are learned independently with behavioral anchors to encourage preservation of the backbone’s existing behavior, enabling skills to be acquired, stored, and invoked as modular parameterized capabilities. Extensive evaluations across four agentic tasks and four foundation backbones demonstrate that MSI consistently outperforms in-context prompting in task performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Agentic Foundation Models, Skill Internalization, Procedural Knowledge, Parameter-Efficient Fine-Tuning, Agent Skills

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_g8HXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents MSI, a framework for turning skill knowledge into parameters for an agentic model. To learn a skill, the framework generates synthetic tasks that require it; then a teacher model generates demonstrations associated with those skills. Finally, these demonstrations, together with an anchor set that is used to preserve the agent's other behaviors, are used to train the model to learn the skill. The paper evaluates this approach on 5 different benchmarks and 4 different models. Results show that the method performs better compared to Direct Skill Injection and random skill adaptor.

Critical strengths
  1. The paper is generally well-written and easy to follow.

  2. The problem that the paper studies, which is to turn skill knowledge into parametric representations in the model, is interesting and important.

Critical weaknesses

The two major weaknesses that I identify are as follows:

  1. Lack of novelty compared to existing methods. The main method the paper proposes is to train LoRA as a parametric representation of the skill. This approach has essentially been discussed in at least two previous papers [1] [2]. Notably, these papers were public at least three months prior to the ICLR deadline. However, the paper entirely does not mention these two papers. The paper needs to clearly position itself against these two prior works and justify its novelty.

  2. Lack of sufficient baseline comparisons. Currently, the baselines that the method chooses are weak and naive baselines, including full skill retrieval, random skill adaptor, oracle skill with demos. However, the paper does not compare itself with other methods, including the training-based skill internalization methods [1] [2], and also test-time skill selection/adaptation methods. Without these, it's unclear whether the improvement that the method gets is solid.

[1] Skill-to-LoRA: From Using Skills to Learning Behaviors for Token-Efficient LLM Agents. https://arxiv.org/abs/2606.16769 [2] LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents https://arxiv.org/abs/2606.06087

Questions for authors
  1. The explanation of the 'complementarity of skill text and MSI' is not fully justified. The author does not present clear insights into why this happens or what exactly the 'orthogonal contribution' of skill text and MSI is. The explanation of the occurrence of sub-optimality of MSI + skill text is also not fully convincing, as the authors did not explain why this happens.

  2. The cost analysis is insufficient. For example, what is the cost of the whole pipeline for internalizing a single skill? What about for each benchmark, considering there are hundreds of skills?

  3. (This can be 'critical weakness 3', but since we are suggested to write '1 or 2' critical weaknesses, I write it here) The comparison strategy of Table 3 is weird. Table 3 compares the performance of MSI and baseline models under the same (imperfect) skill selector. However, the (possibly imperfect) selector design should only affect MSI, not baselines.

3 comments

Thank you for the review and for pointing us to the related work.

W1: Novelty relative to Skill-to-LoRA and LatentSkill.

  • Concurrent work. Both papers are recent arXiv preprints that have not been published at a peer-reviewed venue. The ICLR reviewer guidelines do not require comparison with such work, so we respectfully ask that the missing comparison not be held against the submission.
  • Our response. That said, the papers are clearly relevant. We will cite them and clarify how MSI differs, and we report a comparison below where it was feasible.
  • Skill-to-LoRA (S2L). S2L shares the idea of distilling skills into per-skill LoRAs. MSI differs in four ways:
    1. Data source. S2L self-distills from a model prompted with the skill document. MSI generates tasks and trajectories with a separate, stronger teacher and filters them for quality.
    2. Behavioral anchors. With a wrong adapter loaded and the gold text, anchors reduce the share of correct answers it breaks. The share drops from 17.0% to 13.7% on Qwen, and from 37.0% to 28.2% on Llama (see R-X8ar).
    3. Routing. S2L takes the skill id from the benchmark. MSI selects adapters from a 408-skill library with a learned router, and we evaluate both gold and retrieved assignment.
    4. Scale. S2L evaluates 21 skills (210 tasks) on one backbone. We evaluate 408 skills (4,037 questions) across four domains and four backbones.
  • LatentSkill. LatentSkill makes a different trade-off: it generates adapters without per-skill training. On our tasks, the released compiler did not transfer well (results below).

W2: Baseline comparisons.

We applied the released LatentSkill compiler (Qwen3-8B) to all 408 skill documents, with gold skill assignment. Accuracy (%):

Qwen3-8BTheoremQALogicBenchToolQAMedCalcOverall
Direct42.8476.9729.7233.7342.14
Gold skill text61.5887.6349.0975.1865.77
LatentSkill46.3270.7922.8025.0936.81
  • LatentSkill. The compiler improves over Direct only on TheoremQA, and scores 5.3 pp below the unadapted model overall. The released compiler does not transfer well to our procedural skills.
  • S2L. S2L has not released code, so we could not include it.
  • Shared LoRA. We also added a Shared LoRA trained on our demonstrations (see R-X8ar). With gold skill assignment, MSI outperforms it on all four backbones.

Q1: Why skill text and MSI are complementary.

  • Setup. In Table 1, MSI + Skill Text also changes training. To isolate inference, we fixed Qwen's no-text adapters and retrieved skills, and added only the retrieved document.
  • MedCalc. Accuracy rises from 65.82% to 77.45%. Of the 128 net additional correct answers, 125 come from correctly retrieved skills, where the document supplies exact formulas and coefficients.
  • LogicBench. Accuracy drops from 80.39% to 76.97%. Most of the loss (22 of 26 questions) is on misrouted queries, where the wrong document misleads the model.
  • Takeaway. The adapter carries the procedure. Text adds precise details when it matches the question, and can mislead when it does not. We will replace the "orthogonal" wording with this analysis.

Q2: Cost.

Per skill, we use 200 training demonstrations, 60 anchors, and 80 validation questions. Single-GPU training hours:

BackboneTheoremQA (320 skills)LogicBench (19)ToolQA (14)MedCalc (55)Total hmin / skill
Qwen51.143.0814.7310.5179.4711.7
Llama56.022.9043.529.35111.8016.4
Phi56.693.8785.1512.98158.7023.3
Apertus114.796.8677.8122.76222.2232.7
  • Training. Averaged over all 408 skills for each backbone, training took 12–33 GPU-minutes per skill, and this cost is paid once.
  • Inference. MSI cuts total input tokens per query by 44–76% (see R-VYEb, Q3).
  • Not included. These hours do not include teacher generation and filtering.

Q3: Selector use in Table 3.

  • Why text injection uses the selector. Full-Skill Injection needs the same selector as MSI: with a library of 408 skills, it cannot place every document in context, so it must also pick one.
  • Why LLM Direct does not. It uses no skill, so its scores match Table 1.
  • Why the setups are shared. All skill-based methods use the same top-1 result, so Table 3 compares text vs. adapter under an identical selection. Selector quality alone is reported in Appendix C.
  • Revision. We will clarify this in the caption.

Thank you for your well-structured and concise response. The following concerns remain:

  • While directly comparing the two Skill-to-LORA baselines is not required per ICLR guidelines, the novelty of this paper compared to previous methods is incremental to me.

  • There are very few baseline comparisons with other existing methods that include skill and/or training. The baseline comparison is insufficient and fails to establish that the method achieves better performance compared to previous methods.

  • The cost of training is high (I believe the authors should also compare it to other training-based methods), and it scales linearly with the number of skills.

  • For Q3, the question is not about 'why it needs a selector', but 'why the baseline does not use a golden selector'. A non-perfect selector might be a weakness of your method, but it should not be that of a baseline.

In summary, I think the novelty of this paper compared to existing works is incremental and not well justified, and the experiments are insufficient to establish the methods' advantage compared to existing methods. Considering these, I decide to maintain my score.

Thank you for clarifying Q3. The text baseline receives the gold document in Table 1, while MSI retrieves its own skill in Table 3. We place the results side by side below. All entries are micro accuracy (%) on the same 4,037 questions.

BackboneGold text (Table 1)Gold text + 2 demos (Table 1)Retrieved text (Table 3)Retrieved MSI (Table 3)
Qwen3.5-2B57.259.354.559.3
Llama-3.2-3B34.233.333.748.6
Phi-3.5-mini36.540.734.743.3
Apertus-8B41.840.640.346.4

Retrieved MSI exceeds gold-text prompting on all four backbones. Adding two demonstrations to the gold prompt yields a tie on Qwen at the reported precision, while MSI leads on the other three.

Table 3 addresses deployment when the skill library is available but the question does not identify the correct skill. Text injection must then select a document, just as MSI must select an adapter. Using the same retrieved skill lets us compare how well each method applies it.

Baselines. We agree that comparison with additional prior methods would strengthen the evaluation. Our response to Reviewer X8ar (Q2) reports a Shared LoRA comparison, and our earlier reply to you reports results for the released LatentSkill compiler. We will scope our claims to the comparisons evaluated.

Cost. We agree that MSI’s training cost grows approximately linearly with the number of skills at a fixed training budget per skill. This cost is paid once per skill and backbone, while the resulting adapter can be reused across queries and save inference input tokens by omitting the document. In our Qwen experiment, training a Shared LoRA on the same per-skill positive-example budget used a similar amount of training GPU time. As the library grows, MSI can add a skill by training a new adapter, whereas adding it to Shared LoRA requires updating weights used by existing skills.

Reviewer_VYEbOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The previous language model needs skill documentation to follow procedural actions. The paper tries to internalize the skill into the language model. The proposed method, Modular Skill Internalization (MSI), first constructs skill demonstrations from the teacher model. During training, the LoRA adapter learns from no skill text demonstrations and selects unrelated demonstrations to prevent overfitting. At deployment, the LoRA adapter is retrieved by a learned router. The authors run experiments on four benchmarks; the averaged results show that MSI can outperform the oracle skill and perform even better with skill text and a learned LoRA adapter.

Critical strengths

The paper's main strengths lie in its effectiveness across models and benchmarks.

Critical weaknesses

The paper's main weakness is that the results do not support the original motivation. The original method aims to learn the skill text into a LoRA adapter, but the results show that the skill text is needed to achieve the best performance.

Questions for authors
  1. You use Qwen3.5-122B-A10B as the teacher model. I want to know the teacher model's performance on both direct response and skill text.
  2. I would like to know the performance of the random skill adapter with the correct skill text. The experiment can evaluate the effectiveness of the anchor demonstration design.
  3. What is the token usage of all methods? Does MSI help improve token efficiency?
  4. I believe that the LoRA adapter not only learns how to follow the skill but also distills knowledge from the teacher model. You need further experiments to separate the contributions of distillation and the MSI designs.
  5. In more complex problems, language agents may need to use more than one skill. How will MSI address this?
4 comments

Thank you for the review and the concrete experiment suggestions. We ran new experiments for Q1–Q4.

Main concern: The results do not support the motivation because skill text is still needed for the best performance.

The motivation is that text injection depends on the model's ability to follow the text, and that internalization removes this dependence. It does not claim that text becomes useless. The results support this:

  • MSI alone beats text injection. This holds on all four backbones, under both gold and retrieved routing. With a retrieved skill, MSI even beats text injection with the gold skill.
  • MSI helps most where text works least. Ordered by accuracy with gold text, the backbones are Llama (34.2%), Phi (36.5%), Apertus (41.8%) and Qwen (57.2%). MSI's advantage over text shrinks in the same order: +17.9, +8.5, +7.6 and +4.9 pp.
  • Text is optional on top of the adapter. To isolate the effect of text at inference, we kept each retrieved no-text adapter fixed and added only the retrieved text. Accuracy (%):
Retrieved skillQwenLlamaPhiApertus
Text only54.5533.6934.6840.28
Adapter only59.2548.5843.2746.37
Adapter + text63.3948.1849.0747.83

For Llama and Apertus, adding text changes little. For Qwen, it helps mainly on MedCalc, where skills contain exact formulas and coefficients. We see two practical deployment options: adapter only, with no prompt overhead, or adapter plus text when context budget allows.

Q1: Teacher performance.

Qwen3.5-122B-A10B reaches 65.62% with direct prompting, 75.72% with gold skill text, and 73.74% with retrieved text. Skill documents help even this much stronger model, by 10.1 pp.

Q2: Random adapter with the correct skill text.

Gold text and tools are fixed. The wrong adapter is a non-gold adapter from the same domain, averaged over three assignments. Accuracy (%):

Gold text +QwenLlamaPhiApertus
Wrong adapter, no anchors58.9439.04––
Wrong adapter, with anchors61.5143.5643.8743.70
Correct adapter68.0752.4651.0352.32
  • Correct vs. wrong adapter. The correct adapter beats the wrong one by 6.6–8.9 pp on every backbone, even with the right text in context.
  • Effect of anchors. Anchors reduce the cost of a wrong adapter by 2.6 pp on Qwen and 4.5 pp on Llama. Our response to R-X8ar gives intervals and flip rates.

Q3: Token usage.

Mean tokens per question, text injection → MSI. Input sums all model calls, including tool turns.

StudentRetrieved inputRetrieved outputGold inputGold output
Qwen21,110 → 11,6461,309 → 92921,229 → 10,8591,227 → 862
Llama30,735 → 11,201493 → 74242,314 → 10,266596 → 734
Phi30,080 → 16,714568 → 87033,575 → 15,535637 → 881
Apertus25,798 → 13,770850 → 85037,174 → 13,9631,256 → 872
  • Input. MSI cuts total input by 44–64% with retrieved skills and 49–76% with gold skills. Text injection resends the skill document at every tool turn.
  • Output. Saved output length varies by backbone.
  • Raw generation on ToolQA. Some runs log raw generation for all calls, and there MSI also reduces generation. With retrieved skills, Llama generates 9,120 → 1,246 tokens per question and Qwen 2,336 → 996.

Q4: Distillation vs. MSI design.

We separate the contributions with controls that use no skill text and the gold skill. Accuracy (%):

BaseWrong-skill adapterShared LoRAMSI
Qwen37.337.553.762.1
Llama19.222.347.052.1
  • Per-skill vs. shared. With the same amount of teacher data, per-skill MSI outperforms a shared adapter by +8.5 pp (Qwen) and +5.1 pp (Llama).
  • Wrong-skill control. An adapter trained for another skill gives almost no gain, so the gain is skill-specific rather than a generic effect of teacher-style outputs.

Q5: Multiple skills.

Our benchmarks annotate one skill per question, so we evaluated single-skill tasks. Because adapters are independent and the router works on the query, MSI could switch adapters between steps of a multi-skill task. Standard multi-LoRA serving supports per-request adapter switching. We will discuss this as future work.

Reviewer_VYEbOpenReview ↗

Main concern

Thank you for your replies and clarifications. The ablation and the contribution of MSI and distillation are clear now; both skill internalization and distillation from the teacher model help. The top-line teacher-model performance is now clear. However, I still have some questions about the weaknesses.

Weaknesses

It is clear that MSI+Skill outperformed MSI on macro average. I don’t think you can say the skill text is an optional extra to improve performance. You need to justify when and how we don't have the skill text. Why is making the skill text and learned skill adapter independent important? Or can you provide clear evidence of a trade-off between MSI and MSI+skill? Or, in extreme circumstances, is the skill text missed and cannot be recovered?

Thank you for the follow-up. We agree that MSI+Skill is the better choice when overall accuracy is the priority: it achieves higher overall accuracy on all four backbones. The document remains available in our setting. When the input budget is limited, MSI can load the retrieved adapter without including the document in the prompt.

We re-evaluate Qwen3.5-2B on all 4,037 questions using the same retrieved skills. MSI omits skill text during both training and inference, while MSI+Skill includes it at both stages. Including the document increased the average input used to present a question from 1,906 to 2,874 tokens. The document and its formatting accounted for 968 tokens, or 33.7% of the input before any tool calls.

MethodTotal inputUncached input
MSI11,5551,333
MSI+Skill12,9441,752

Total and uncached input are summed over all model calls and averaged per question. We grouped requests by retrieved skill to allow prefix-cache reuse across questions. Under this workload, MSI+Skill gained 4 percentage points in accuracy while using 12.0% more total input and 31.4% more uncached input. We will revise the motivation to make this trade-off explicit.

Reviewer_VYEbOpenReview ↗

Raise to 3.

I am convinced that there is a trade-off between MSI and MSI+Skill. I will raise my score to 3 with clearer motivation.

Reviewer_X8arOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The key point to me is that the novelty is limited. It is a work that combines some existing modules. The experiments are also not very complete.

The paper proposes modular skill internalization (MSI), which converts text program skills into parameter modules that can be loaded on demand. The approach uses a teacher model to generate and filter execution traces based on skill specifications, and then trains independent LoRA adapters for each skill while keeping the backbone frozen. The method also introduces behavioral anchors to encourage retaining the model's original behavior on non-target tasks. The experiments cover four benchmarks, four backbone models, and 408 skills under oracle and retrieval skill assignments.

Critical strengths

The assessment covers theorem application, logical reasoning, tool usage, and medical calculations, and improves overall accuracy across multiple backbones. Evaluating oracles and retrieved skill assignments helps distinguish the impact of skill execution performance and retrieval errors. Comparisons between matched and random adapters also provide evidence that performance gains are skill-specific.

Critical weaknesses
  1. The technical contribution relative to existing methods has not been fully determined. As discussed in Section 2, MSI is closely related to context distillation, PRAG-style independent LoRA modules, and existing skill internalization methods, while its training uses standard supervised learning objectives. There is practical value in applying these techniques to procedural skills. However, these experiments mainly compare MSI with unadapted models and cue-based skill injection, and do not include fine-tuned baselines trained on the same synthetic data.

  2. Behavioral anchors are an important part of the proposed method, but this paper does not report removing anchors or using only ablation of random or near-miss anchors. It also lacks a systematic evaluation of behavioral retention on non-target tasks.

Questions for authors
  1. Are these results based on a single training seed? Can the authors report the mean and standard deviation of multiple independent trainings using different seeds?

  2. Can the authors include a shared LoRA baseline trained on the same instructor demonstration to evaluate the benefits?

  3. Can the authors quantify the impact of behavioral anchors on target skill performance?

2 comments

Thank you for the review. We address each point below.

W1 & Q2: Fine-tuned baseline on the same data.

We trained one rank-8 Shared LoRA per backbone on 200 positive teacher demonstrations per skill, pooled across all 408 skills. MSI uses the same positive-example quota and adds behavioral anchors. Neither method sees skill text. Accuracy (%) on all 4,037 questions, gold skill assignment:

BackboneBaseShared LoRAMSI
Qwen3.5-2B37.353.762.1
Llama-3.2-3B19.247.052.1
Phi-3.5-mini24.442.845.0
Apertus-8B25.247.249.4
  • MSI outperforms a shared adapter trained on the same amount of data by 2.2–8.5 pp.
  • With the learned retriever, the margin narrows to −0.8 to +5.6 pp, because about a quarter of queries are routed to the wrong skill.
  • On correctly routed queries, MSI leads Shared LoRA by 9.7 pp (Qwen) and 5.7 pp (Llama). The benefit of modularity therefore grows as routing improves.

On contribution, MSI does build on context distillation and modular LoRA. What we add is:

  • a pipeline that turns procedural skill documents into filtered tool-use trajectories;
  • behavioral anchors that limit damage from wrong adapters;
  • a 408-skill evaluation that separates skill execution from skill selection.

W2 & Q3: Anchor ablation and behavior retention.

We trained a second adapter library without anchors for Qwen and Llama. "Wrong" means a non-gold adapter from the same domain, averaged over three fixed assignments. Accuracy (%), without → with anchors:

AdapterSkill textQwen3.5-2BLlama-3.2-3B
Correctnone62.55 → 62.1353.70 → 52.09
Correctgold66.81 → 68.0751.82 → 52.46
Wrongnone33.76 → 37.5320.64 → 22.29
Wronggold58.94 → 61.5139.04 → 43.56
  • Effect on the target skill (Q3). With the correct adapter, accuracy changes by −1.6 to +1.3 pp, so anchors have little effect on target-skill performance.
  • Retention on non-target questions (W2). Loading a wrong adapter is exactly the situation where the adapter sees questions from other skills.
    • Without anchors, a wrong adapter drops Qwen 3.5 pp below the base model (33.76 vs. 37.28). With anchors, it stays at base level (37.53).
    • With gold text, anchors improve accuracy by +2.57 pp on Qwen (paired 95% CI [1.68, 3.44]) and +4.52 pp on Llama ([3.25, 5.82]).
    • On questions the base model answered correctly with gold text, anchors reduce the share that a wrong adapter turns wrong from 17.04% to 13.66% on Qwen, and from 36.97% to 28.18% on Llama.

Q1: Seeds.

Each configuration was trained once; retraining 408 × 4 adapters several times did not fit in the discussion period. Paired bootstrap intervals over test skills (2,000 resamples, clustered by skill) for retrieved MSI vs. retrieved text:

BackboneGain in pp [95% CI]
Qwen+4.71 [1.38, 8.47]
Llama+14.89 [9.99, 20.30]
Phi+8.60 [4.76, 12.57]
Apertus+6.09 [2.60, 10.05]

All intervals exclude zero.

We also note that the reported accuracy is not the result of a single trained model:

  • Each number aggregates 408 adapters, and each adapter is trained independently.
  • The skill-clustered bootstrap resamples over these adapters, so the intervals already reflect variation across separate training runs.
  • The gains are also consistent in direction across four backbones and both routing settings.

The advantage also holds across training budgets. Figure 2 in the submitted paper (Section 5.4) already shows that adapter-only MSI outperforms gold skill text on both Qwen and Llama at every tested per-skill quota of at least 160: 160, 260, 384, and 512. We have since extended the 384/512 evaluation to Phi and Apertus, with the same finding. Micro accuracy (%) over all 4,037 questions, with gold skill assignment and no skill text during MSI training or inference:

BackboneText injectionMSI 160MSI 260MSI 384MSI 512
Qwen3.5-2B57.2059.0362.1362.8963.34
Llama-3.2-3B34.1849.2952.0953.4854.15
Phi-3.5-mini36.54–45.0146.2245.45
Apertus-8B41.81–49.3752.2953.46

The quota includes demonstrations and anchors; validation examples are separate. Thus, the advantage over text injection is observed across multiple training budgets, not only at the default quota of 260.

Thanks for your response. I maintain my rating as the novelty is still the main concern to me.

Comments on the paper

1 comment

We thank the reviewers for their feedback. Several questions came up in more than one review; we answer them briefly here, with details in the individual responses.

1. How does MSI differ from Skill-to-LoRA and LatentSkill? (R-g8HX)

  • Concurrent work. Both papers are recent arXiv preprints that have not been published at a peer-reviewed venue. The ICLR reviewer guidelines do not require comparison with such work, and we respectfully ask the reviewers and the AC to take this into account.
  • What we did. We will cite both papers and clarify how MSI differs. We also ran the comparison that was feasible during the discussion period.
  • Skill-to-LoRA. It uses self-distillation, takes the skill id from the benchmark, and evaluates 21 skills. MSI uses a separate teacher with trajectory filtering, adds behavioral anchors, and selects among a 408-skill library with a learned router.
  • LatentSkill. It generates adapters with a shared compiler and needs no per-skill training. We applied its released compiler to our 408 skills. It scores below the unadapted model overall (36.8% vs. 42.1% on Qwen3-8B).

2. Do the gains come from per-skill modules or just from the teacher data? (R-X8ar, R-VYEb)

  • We trained a Shared LoRA on the same amount of teacher data from all 408 skills. With gold skill assignment, per-skill MSI outperforms it on all four backbones, by 2.2–8.5 pp.
  • Without skill text, an adapter trained for a different skill brings almost no gain over the base model. The improvement is therefore tied to the matching skill.

3. Do behavioral anchors help? (R-X8ar, R-VYEb)

  • We trained adapters with and without anchors, then loaded a wrong adapter on purpose.
  • Anchors improved accuracy in all four wrong-adapter settings (two backbones, with and without text), by +1.7 to +4.5 pp.
  • With the correct adapter, the effect of anchors stayed within 1.6 pp.

4. Does MSI still need skill text? (R-VYEb, R-g8HX)

  • Adapter-only MSI beats text injection on every backbone, with both gold and retrieved skills.
  • MSI with a retrieved skill even beats text injection with the gold skill.
  • Adding text on top of the adapter is an optional extra that helps on some tasks; the model no longer depends on it.

We will add these experiments to the paper, discuss the related work, and clarify the points raised about Table 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.