acceptodds
Under review as a conference paper at ICLR 2027

Skill is Not One-Size-Fits-All: Model-Aware Skill Alignment for LLM Agents

Abstract

LLM agents increasingly retrieve externally curated skills-procedural instructions retrieved at decision time-to improve performance on long-horizon interactive tasks. Existing skill libraries are typically improved in one of two ways: updating model weights-costly, and infeasible for frozen or API-served models-or sharing a single model-agnostic library across backbones with substantially different capacities. However, our controlled experiments across multiple model scales show that skill effectiveness is strongly model-dependent: a skill that benefits one backbone can harm another. Motivated by this, we take a third route-adapting the skills to each model rather than the model to the task-and propose MASA (Model-Aware Skill Alignment), which aligns skills with each target backbone while keeping agent weights frozen. MASA lets skills self-evolve from environmental reward-without human supervision-in two stages: (1) a hierarchical skill evolution pipeline in which a stronger teacher LLM iteratively rewrites general and task-specific skills using hill climbing and UCB-driven tree search, guided by model capability profiles; and (2) a lightweight model-conditioned skill rewriter, trained on these trajectories, that amortizes this search into a single forward pass. Experiments across three interactive environments and four backbones show that MASA consistently achieves the best overall performance, with gains of up to 25.8 points over the strongest baseline. The learned rewriter further generalizes to unseen tasks and environments without additional search, consistently outperforming a much larger teacher LLM at a fraction of the inference cost. Our code is publicly available at https://anonymous.4open.science/r/MASA-BE5F/.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Skill Alignment, Model Adaptation, Agent Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_QBrbOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the model dependence of procedural skills used by frozen LLM agents and proposes MASA, a framework for model-aware skill alignment. MASA combines a teacher-guided hierarchical search over general and task-specific skills with a model-conditioned skill rewriter trained to amortize the search. The evaluation covers four Qwen3 backbones and three types of interactive tasks, together with held-out task and cross-environment rewriter evaluations.

Critical strengths
  1. The paper identifies a meaningful and practically relevant problem: skill effectiveness is not invariant across agent backbones. The controlled granularity experiments provide concrete evidence that model-independent procedural instructions can produce substantially different behaviors across models.

  2. Amortizing expensive skill search into a lightweight rewriter is practically valuable. The model-card ablations and held-out task evaluations support the claim that useful adaptation patterns can be learned and reused without repeating environment-guided search at deployment.

Critical weaknesses

W1. Unclear separation between model-aware skill alignment and basic action-format correction.

A substantial portion of the improvement on WebShop appears to result from correcting verbose reasoning and invalid action-generation patterns in larger Qwen3 models. For instance, the paper reports that Qwen3-14B produces reasoning preambles in approximately 97% of action steps under the baseline setting. This raises the possibility that much of the observed gain could be recovered using explicit action-format constraints, constrained decoding, or a small number of model-specific examples. Without such comparisons, the evidence does not fully establish that hierarchical skill alignment is necessary to achieve these improvements.

W2. Limited evidence of adaptation to genuinely unseen model architectures.

The main MASA experiments and learned rewriter evaluations are conducted on four Qwen3 backbones. Although a supplementary granularity study includes Gemma3 models, the complete skill evolution and rewriter pipelines are not evaluated on a previously unseen model family. Consequently, the claim that MASA-Rewriter serves as general-purpose plug-and-play middleware for new backbones is not adequately supported. A leave-one-backbone-out evaluation, particularly across model families, would substantially strengthen this claim.

W3. Incremental algorithmic novelty over existing prompt and skill optimization techniques remains unclear.

The search pipeline combines teacher-generated revisions, hill climbing for general skills, UCB-based search for task-specific skills, and supervised distillation into a rewriter. These components are individually well established. The main novelty therefore appears to lie in model-conditioned skill optimization rather than in the underlying search algorithms. A direct comparison against strong model-adaptive prompt optimizers using the same feedback and search budgets would better establish the incremental algorithmic contribution.

W4. Model-card construction introduces an additional dependency that is insufficiently characterized.

The capability profiles combine public model metadata with teacher-summarized weaknesses derived from preliminary rollouts. While the ablations demonstrate that these cards contain useful information, the method does not establish how robust optimization is to incomplete, noisy, or inaccurate capability assessments. Moreover, producing these profiles requires additional model-specific evaluation and teacher inference. This profiling process is part of the adaptation pipeline and should be included in its overall cost and robustness analysis.

W5. Some cross-environment generalization improvements are relatively small.

The Cross-env rewriter exceeds the one-shot teacher adapter by approximately 1.5–3.0 percentage points across the four evaluated backbones. These improvements are encouraging but considerably smaller than those achieved in the Cross-task setting, where the rewriter has already observed other task types from the target environment. Without uncertainty estimates, it is difficult to determine how reliably the model transfers rewriting principles across substantially different environments.

W6. The computational advantages depend on amortizing substantial offline optimization costs.

MASA requires teacher-guided environment search, with the reported ALFWorld search-budget analysis reaching 800–1,000 rollouts at larger budgets. The rewriter additionally requires full-parameter supervised fine-tuning of a 4B model. Although deployment requires only a forward pass, the overall cost depends on how frequently the trained rewriter is reused across tasks and target models. The paper does not quantify the break-even point relative to repeated one-shot adaptation or lighter-weight prompt optimization.

Questions for authors
  1. How does MASA compare against a carefully tuned, model-specific formatting and instruction baseline, particularly on WebShop, where action-generation behavior appears to explain much of the performance gap?

  2. Can the trained rewriter generalize to a backbone family excluded entirely from its training data? How do the total teacher inference and environment-search costs compare with competing model-specific optimization strategies?

3 comments

We thank the reviewer for the detailed and constructive feedback. We provide additional controlled comparisons and clarify the contributions, generalization scope, and computational costs of MASA below.

  • A1. Skill Alignment Beyond Action-Format Correction (W1 + Q1)

To determine whether MASA's improvements can be explained by formatting alone, we conduct two additional controlled comparisons.

First, we compare MASA against format-only instructions and constrained decoding on the same WebShop tasks across three sampling seeds.

ModelFormat-onlyMASAImprovement
Qwen3-4B25.7 ± 0.326.5 ± 0.6+0.8
Qwen3-8B7.9 ± 0.328.6 ± 1.0+20.7
Qwen3-14B15.7 ± 1.129.3 ± 0.8+13.6

Second, we explicitly control action formatting through identical guided decoding, enforcing the same action-output structure for all methods. The only difference is the injected skill library.

SettingQwen3-8BQwen3-14B
Guided Decoding Only9.010.8
Guided + Base Skill4.69.0
Guided + MASA Skill27.629.0

With identical formatting constraints, MASA still substantially outperforms the original skill library. Meanwhile, adding the original skill library under guided decoding can even reduce performance relative to using no skills. These findings show that MASA's gains on larger backbones cannot be explained by action-format correction alone. Beyond enforcing valid outputs, evolved skills improve the procedural guidance used for task execution. We also acknowledge that the additional benefit over format-only guidance is small for Qwen3-4B, indicating that the relative importance of skill adaptation varies across backbones.

  • A2. Generalization to Unseen Model Families (W2 + Q2)

Our main experiments intentionally focus on four Qwen3 backbones to isolate model-capability differences while controlling for architectural and training-family variations. The additional Gemma3 study (Appendix E.3) further confirms that model-dependent skill preferences extend across model families. For MASA-Rewriter, Section 4.3 evaluates generalization to unseen tasks and environments. Importantly, the proposed model-conditioned evolution framework is not inherently restricted to the Qwen3 architecture, as it optimizes external skill libraries through environment feedback without modifying target-model parameters.

  • A3. Distinction from Existing Optimization Methods (W3)

We agree that hill climbing, UCB-based search, and supervised distillation are established techniques. Our contribution is not a new search algorithm, but a model-conditioned framework for optimizing dynamically retrieved procedural skill libraries under frozen LLM backbones. Unlike conventional prompt optimization, which typically optimizes monolithic task instructions, MASA jointly addresses backbone-level behavioral guidance and task-type-specific procedures, using environment feedback and capability-aware rewriting.

The original paper already includes comparisons with existing skill-evolution systems (Appendix B, Table 3):

MethodALFWorld 8BWebShop 8B
SkillOps36.4—
SkillX45.0—
SkillOS-Gemini50.713.2
MASA57.928.6

Furthermore, the ablations in Appendix C validate the contributions of both hierarchical optimization and model-aware conditioning. In Table 4, replacing task-specific evolution with one-shot teacher rewriting reduces ALFWorld-8B success from 57.9% to 32.9%, while replacing general-skill evolution reduces WebShop-8B success from 28.6% to 7.2%. Table 5 further shows that, the correct model card improves ALFWorld-4B success from 27.1% to 31.4%.

Together, these results demonstrate the value of model-conditioned, hierarchical skill optimization beyond fixed skill formulations and one-shot adaptation. Our ablations isolate the benefits of model-aware conditioning and the contributions of the two optimization stages, rather than attributing the improvements to novel search algorithms.

  • A4. Model-Card Dependency and Robustness (W4)

The original paper already evaluates model-card sensitivity under a matched search budget (Appendix C, Table 5):

Model-Card SettingALFWorld-4B SR
Random Model Card23.8
Without Model Card27.1
Correct Model Card31.4

All variants use the same search configuration. The correct model card improves success by 4.3 points over the no-card variant, whereas an unrelated card causes a larger degradation. This confirms that capability information contributes beyond additional optimization iterations, while also showing that inaccurate conditioning can be harmful. Appendix C, Table 6 further demonstrates that model-card conditioning remains important for MASA-Rewriter. For example, removing the card reduces Cross-task success on Qwen3-14B from 48.5% to 20.6%.

  • A5. Cross-Environment Generalization and Statistical Reliability (W5)

As reported in Section 4.3, Cross-env MASA-Rewriter improves over DS-Adapter by +1.5, +3.0, +2.9, and +3.0 percentage points on Qwen3-4B/8B/14B/32B, respectively, despite receiving no ALFWorld training examples. Cross-task improvements are larger, reaching +8.8, +13.2, and +7.4 points on 8B/14B/32B. This difference is consistent with the benefit of observing other task types from the target environment, which provides additional supervision about its interaction patterns. We emphasize that Cross-env evaluates the more challenging setting in which the rewriter has no training exposure to the target environment. Its consistent directional improvements across four backbones provide encouraging evidence of transfer.

  • A6. Offline Optimization Cost and Amortization (W6 + Q2)

The original paper already reports a search-budget analysis (Appendix D, Table 7) on ALFWorld with Qwen3-32B. The search gains saturate around 800 rollouts, with no additional improvement at 1,000. The rewriter offers a different performance–cost trade-off: it avoids iterative environment search and teacher inference when adapting skills at deployment. Appendix H also reports the training setup: full-parameter SFT of Qwen3-4B for five epochs, using 499 Cross-env or 769 Cross-task training examples. We recognize that generating these examples, constructing model cards, and training the rewriter incur non-negligible offline costs.

The relevant amortization condition can be expressed as: N(Cteacher−Crewrite)>Csearch+CSFT+CprofileN(C_{\mathrm{teacher}}-C_{\mathrm{rewrite}}) > C_{\mathrm{search}}+C_{\mathrm{SFT}}+C_{\mathrm{profile}}, where NN denotes the number of subsequent skill-adaptation requests, and the costs include the corresponding teacher inference, rewriter inference, offline evolution, training, and profiling expenses. Thus, MASA-Rewriter is most attractive when the trained adaptation mechanism is reused across many skill libraries or adaptation requests. We do not claim that it necessarily achieves a lower total cost for one-off adaptation.

Thank you to the authors for their detailed responses and additional experiments. The rebuttal has partially addressed my concerns. In particular, the format-controlled comparisons provide useful evidence that the improvements on larger backbones extend beyond action-format correction, while the clarifications help better establish the intended contribution and scope of the framework.

However, some concerns remain regarding generalization to unseen model families and comparisons with strong model-adaptive optimization baselines under matched budgets. Taking into account the additional supporting evidence and the authors’ substantial efforts to address the feedback, I have decided to increase my score by one point, from 1 to 2.

Reviewer_34BHOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that one shared skill library does not work for all models. A skill that helps one backbone can hurt another, so the authors propose to adapt the skills to each model instead of changing the model weights. They introduce MASA. A stronger teacher model rewrites general and task-specific skills based on failure cases and a short model card. Then they train a small rewriter to do this adaptation in one pass without extra search. They test it on ALFWorld, WebShop, and search QA, and show it beats shared-skill baselines.

Critical strengths

1.The core point is real and well shown. Keeping the skill content fixed and only changing how detailed it is, then showing different models prefer different versions, is a clean setup.

  1. The split into general skills and task-specific skills makes sense. It matches how agents actually fail, and the paper shows both parts matter.

  2. The small rewriter is a nice practical touch. It removes the need for costly search at deployment and still beats the large teacher.

Critical weaknesses
  1. The novelty is limited. Others have already reported this model-skill mismatch. SkVM reports the same problem directly. MAPO and PromptBridge already do model-adaptive prompts and cross-model transfer. Voyager, ExpeL, and AutoManual already evolve skill libraries.

  2. The model card feels shallow. It is mostly static info like layer count and context size, plus marketing lines from release notes and a teacher-written weakness. The paper does not show what part of the card actually matters. A wrong card hurts more than no card, which suggests the model may just be sensitive to extra text.

  3. The evaluation is narrow. All main results are on Qwen3 in non-thinking mode. There is no full MASA run on another family or on API models. So the claim about different capacities is more about Qwen sizes.

  4. Baselines and reporting are weak. There is no per-model tuning baseline, even though the paper itself shows granularity should be tuned per model. There are no error bars or seeds, and some QA gains are tiny with large swings across datasets. Cost analysis is also missing. The search needs hundreds of rollouts per model, but there is no time or token cost. It is important that the authors provide Such Details.

Questions for authors

See the weaknesses above. Please provide more details, esp wrt the baselines and reporting.

1 comment

We thank the reviewer for the constructive feedback. We clarify the methodological novelty, model-card conditioning, evaluation scope, and additional experimental evidence below.

  • A1. Novelty and Distinction from Prior Work (W1)

MASA adapts dynamically retrieved procedural skills to frozen LLM backbones through environment feedback and distills this adaptation for efficient deployment. Its key distinctions are summarized below.

Prior WorkWhat They DoWhat MASA Does Differently
SkVMProfiles model capabilities and compiles or recompiles skills to address capability gaps.Uses target-agent rewards to search for better general and task-specific skill formulations.
MAPOTrains an optimizer to generate model-adaptive task prompts.Optimizes dynamically retrieved skill libraries** through interactive agent feedback.
PromptBridgeLearns cross-model prompt transformations from calibration tasks.Learns skill rewriting from reward-guided evolution trajectories, rather than source–target prompt mappings.
Voyager / ExpeL / AutoManualAcquire and update reusable skills, insights, or rules from agent experiences.Adapts existing procedural knowledge to different target backbones, rather than primarily acquiring new knowledge.

MASA's key contribution lies in model-conditioned hierarchical skill evolution and its amortization through a learned rewriter. Comparisons with prior skill-evolution systems (Appendix B, Table 3) and hierarchical ablations (Appendix C, Table 4) support the effectiveness of this design.

  • A2. Role and Effectiveness of the Model Card (W2)

The model card is designed to provide three complementary categories of information: architecture metadata, training provenance, and capability descriptions. These serve as contextual information for the teacher to adapt skill formulations to the target backbone. Importantly, the contribution of model-card conditioning has already been evaluated under a matched search budget, as reported in Appendix C (Table 5).

Model-Card SettingALFWorld
Random Model Card23.8
No Model Card27.1
Correct Model Card31.4

All three settings use the same target backbone (Qwen3-4B) and identical two-stage search configurations. The correct model card improves performance over both the no-card and incorrect-card variants, indicating that the conditioning information matters beyond the search budget alone. In addition, Appendix C (Table 6) evaluates model-card conditioning in MASA-Rewriter. Removing the card consistently degrades performance across the evaluated backbones and transfer settings.

  • A3. Evaluation Across Model Families (W3)

Our main experiments deliberately focus on Qwen3 backbones of different scales to study model-dependent skill alignment under a controlled model-family setting. Importantly, the motivating observation of model–skill mismatch is not restricted to Qwen3. Appendix E.3 independently evaluates the Gemma3 family. These results show that skill preferences differ not only across model scales but also across model families. In particular, Qwen3-4B and Gemma3-4B prefer different skill formulations despite having comparable parameter counts.

  • W4. Baseline Fairness, Statistical Reliability, and Computational Cost

We provide additional experiments and clarify the computational efficiency of MASA below.

(1) Per-model Tuning Baseline. We introduce a stronger baseline that independently selects the best fixed skill granularity (No Skill, Concise, Standard, or Detailed) for each backbone.

ModelBest FixedBest FixedMASAGain
Qwen3-4BConcise17.6 ± 1.328.6 ± 0.6+11.0
Qwen3-8BConcise33.8 ± 0.763.3 ± 2.0+29.5
Qwen3-14BDetailed49.5 ± 1.261.2 ± 1.2+11.7

MASA outperforms the best fixed granularity by 11.0–29.5 percentage points, demonstrating that model-specific granularity tuning alone cannot replace reward-guided skill evolution.

(2) Statistical Reliability. We conduct additional three-seed evaluations on ALFWorld (above) and WebShop (below).

ModelMASA
Qwen3-4B26.5 ± 0.6
Qwen3-8B28.6 ± 1.0
Qwen3-14B29.3 ± 0.8

The relatively small standard deviations indicate that MASA's performance remains stable across sampling seeds on both environments.

(3) Computational Cost. Appendix D (Table 7) already analyzes the relationship between search budget and performance on Qwen3-32B ALFWorld.

Rollouts01002004006008001000
SR (%)41.449.354.861.764.365.765.7

MASA achieves substantial improvements with 200 rollouts and saturates at 800 rollouts. Importantly, this search is performed offline. MASA-Rewriter amortizes the optimization into a single rewriting pass, achieving 55.9% SR on held-out ALFWorld tasks without additional search rollouts or teacher calls.

Reviewer_yHizOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MASA (Model-Aware Skill Alignment), a framework that adapts skill libraries to individual LLM backbones rather than applying a shared library across models. MASA uses model-card-conditioned hierarchical skill evolution guided by environmental rewards and distills the resulting adaptations into a lightweight skill rewriter. Experiments across three environments and four Qwen3 backbones demonstrate performance improvements and generalization to unseen tasks and environments.

Critical strengths

1. Practical Problem Formulation Reflecting Real-World Deployment. Real-world LLM agents operate under heterogeneous latency, inference cost, and hardware constraints, making it difficult to assume that a single skill library works equally well across different backbones. The paper addresses this practical challenge and demonstrates that identical skills can even degrade performance on certain models. By evaluating three interactive environments and backbone sizes ranging from 4B to 32B, the work highlights the practical value of model-specific skill adaptation.

2. Efficient Adaptation Without Updating Target Backbone Weights. MASA optimizes external skill libraries without modifying the target agent's weights, making it applicable to frozen or API-served models. Its hierarchical framework separately evolves general and task-specific skills using environmental feedback, then distills the results into a lightweight rewriter for single-pass adaptation. Although offline evolution incurs computational costs, deployment requires no additional teacher calls or environment rollouts. The demonstrated generalization to unseen tasks and environments further supports its practical applicability.

Critical weaknesses

1. Motivation Could Benefit from Stronger Statistical Evidence. The claim that optimal skill granularity cannot be determined by simple heuristics is primarily supported by single-run ALFWorld experiments without repeated trials or confidence intervals. With approximately 140 episodes, a 95% confidence interval can have a margin of 7–8 percentage points, exceeding key differences such as Qwen3-4B's 3.6-point and Gemma3-12B's 0.7-point gaps. Moreover, all models with at least 14B parameters favor Detailed skills, and a simple size-based heuristic selects the best condition for five of seven models. The lower Detailed performance of 32B relative to 14B does not directly contradict this heuristic, as both models favor Detailed skills.

2. Connection Between the Motivation and Methodology Could Be Clarified. The motivation demonstrates model-dependent skill preferences, but it remains unclear which model-card attributes explain or predict these differences. The card ablations (Tables 5–6) evaluate the complete card without distinguishing the contributions of parameter count, architecture, training, and capability information. The different preferences of Qwen3-4B and Gemma3-4B also rely on relatively small within-model gaps of approximately 3.5–3.6 points, while the main experiments focus exclusively on Qwen3 backbones. Furthermore, the qualitative analysis (Table 13) explains skill adaptations through observed rollout failures rather than specific card attributes. A clearer analysis of these relationships would help strengthen the methodological motivation.

Questions for authors
  1. The model card includes various attributes, such as parameter count, architecture, training provenance, and capability profiles. Which of these components contributes most significantly to skill adaptation performance? A component-wise ablation or analysis would help clarify which information is particularly important for effective model-aware conditioning.

  2. How would a skill library evolved for Qwen3-8B perform when directly transferred to Qwen3-14B and Qwen3-32B without further adaptation? Comparing these results against skills independently optimized for each backbone would provide additional insight into the necessity and benefits of model-specific skill adaptation.

2 comments

We thank the reviewer for the constructive feedback and insightful suggestions, which have helped us strengthen the statistical evidence, clarify the role of model-card conditioning, and further investigate cross-model skill transferability.

  • A1. Statistical Evidence and Model-Dependent Skill Granularity (W1)

We thank the reviewer for highlighting the need for stronger statistical evidence. We acknowledge that model size can provide a useful heuristic for selecting skill granularity. Nevertheless, our central observation is that the preferred skill granularity varies across model backbones, and no single fixed granularity consistently achieves the best performance.

To strengthen the statistical evidence, we extend our original ALFWorld granularity study with three additional sampling seeds, yielding four independent runs on the same 140 evaluation tasks. All experimental configurations remain unchanged, and we report the mean and standard deviation across runs.

BackboneNo SkillConciseModerateDetailed
Qwen3-4B17.33 ± 1.1317.30 ± 1.2417.68 ± 1.8711.98 ± 0.54
Qwen3-8B33.23 ± 0.8332.33 ± 2.6323.20 ± 1.9522.88 ± 3.58
Qwen3-14B38.20 ± 1.0538.45 ± 1.4741.05 ± 1.0649.00 ± 1.35

As shown in Table, the best-performing configuration differs across backbones. More importantly, increasing skill detail has different effects across models: compared with Concise skills, Detailed skills reduce success rates by 5.32 and 9.45 percentage points for Qwen3-4B and Qwen3-8B, respectively, while improving performance by 10.55 points for Qwen3-14B.

  • A2. Role and Contribution of Model Card Conditioning (W2 + Q1)

Model cards and rollout feedback provide complementary information for skill evolution. The model card characterizes the target backbone through its architecture, training information, and capability profile, while rollout feedback identifies concrete execution failures. MASA uses both to guide skill rewriting, with environmental rewards determining whether candidate adaptations are retained.

The effectiveness of model-card conditioning is supported by our controlled ablations. As shown in Table 5, under an identical search budget, using the correct target model card achieves 31.4% success on ALFWorld, compared with 27.1% without the card and 23.8% with a randomly assigned card. Table 6 further demonstrates consistent improvements from model-card conditioning in MASA-Rewriter across all four Qwen3 backbones.

  • A3. Cross-Model Skill Transferability (Q2)

To examine the transferability of evolved skills, we conduct additional cross-model evaluations on ALFWorld, directly applying skill libraries evolved for one Qwen3 backbone to other backbones without further adaptation.

Target Backbone4B Library8B Library14B Library
Qwen3-4B31.425.034.3
Qwen3-8B21.457.934.3
Qwen3-14B26.467.964.3

The results show that cross-model skill transfer can be beneficial but is not universally reliable. Transferring the 8B library to 14B improves success from 64.3% to 67.9%, whereas applying the same library to 4B reduces success from 31.4% to 25.0%. Moreover, transferring the 14B library back to 8B substantially degrades performance from 57.9% to 34.3%, revealing asymmetric transfer effects. These findings suggest that skills evolved for one backbone may benefit other models, but their effectiveness cannot be guaranteed across target backbones. While independently evolved libraries are not necessarily optimal, directly reusing skills without adaptation can also result in substantial performance degradation, highlighting the importance of model–skill compatibility.

Thank you for the additional experiments and thoughtful responses. While the new results strengthen the paper, I believe further experiments and analysis would provide more comprehensive support for the proposed approach. The four runs on the same 140 ALFWorld tasks provide stronger evidence that increasing skill detail affects the three tested Qwen3 models differently. However, evaluations across other model families, such as Llama and Gemma, and on models larger than 14B would help establish whether the need to match skills to the target model is a general concern, as the paper’s motivation suggests. Along the same lines, while the 8B library performs well on the 14B model, comparing the 8B library with a library developed for 32B on the same 32B backbone would help clarify what model-specific adaptation offers at larger scales. For smaller models, model-specific skills may help correct recurring issues such as output formatting. The larger-model comparison would help assess whether model-specific skills can also address how a particular model reasons through the task itself. Additionally, although the ablation results support the use of the full model card, evaluating individual fields and selected combinations under the same search budget would help identify which information contributes most to the observed improvements. Given these remaining gaps in the empirical evidence, I will maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.