- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the model dependence of procedural skills used by frozen LLM agents and proposes MASA, a framework for model-aware skill alignment. MASA combines a teacher-guided hierarchical search over general and task-specific skills with a model-conditioned skill rewriter trained to amortize the search. The evaluation covers four Qwen3 backbones and three types of interactive tasks, together with held-out task and cross-environment rewriter evaluations.
Critical strengths
-
The paper identifies a meaningful and practically relevant problem: skill effectiveness is not invariant across agent backbones. The controlled granularity experiments provide concrete evidence that model-independent procedural instructions can produce substantially different behaviors across models.
-
Amortizing expensive skill search into a lightweight rewriter is practically valuable. The model-card ablations and held-out task evaluations support the claim that useful adaptation patterns can be learned and reused without repeating environment-guided search at deployment.
Critical weaknesses
W1. Unclear separation between model-aware skill alignment and basic action-format correction.
A substantial portion of the improvement on WebShop appears to result from correcting verbose reasoning and invalid action-generation patterns in larger Qwen3 models. For instance, the paper reports that Qwen3-14B produces reasoning preambles in approximately 97% of action steps under the baseline setting. This raises the possibility that much of the observed gain could be recovered using explicit action-format constraints, constrained decoding, or a small number of model-specific examples. Without such comparisons, the evidence does not fully establish that hierarchical skill alignment is necessary to achieve these improvements.
W2. Limited evidence of adaptation to genuinely unseen model architectures.
The main MASA experiments and learned rewriter evaluations are conducted on four Qwen3 backbones. Although a supplementary granularity study includes Gemma3 models, the complete skill evolution and rewriter pipelines are not evaluated on a previously unseen model family. Consequently, the claim that MASA-Rewriter serves as general-purpose plug-and-play middleware for new backbones is not adequately supported. A leave-one-backbone-out evaluation, particularly across model families, would substantially strengthen this claim.
W3. Incremental algorithmic novelty over existing prompt and skill optimization techniques remains unclear.
The search pipeline combines teacher-generated revisions, hill climbing for general skills, UCB-based search for task-specific skills, and supervised distillation into a rewriter. These components are individually well established. The main novelty therefore appears to lie in model-conditioned skill optimization rather than in the underlying search algorithms. A direct comparison against strong model-adaptive prompt optimizers using the same feedback and search budgets would better establish the incremental algorithmic contribution.
W4. Model-card construction introduces an additional dependency that is insufficiently characterized.
The capability profiles combine public model metadata with teacher-summarized weaknesses derived from preliminary rollouts. While the ablations demonstrate that these cards contain useful information, the method does not establish how robust optimization is to incomplete, noisy, or inaccurate capability assessments. Moreover, producing these profiles requires additional model-specific evaluation and teacher inference. This profiling process is part of the adaptation pipeline and should be included in its overall cost and robustness analysis.
W5. Some cross-environment generalization improvements are relatively small.
The Cross-env rewriter exceeds the one-shot teacher adapter by approximately 1.5–3.0 percentage points across the four evaluated backbones. These improvements are encouraging but considerably smaller than those achieved in the Cross-task setting, where the rewriter has already observed other task types from the target environment. Without uncertainty estimates, it is difficult to determine how reliably the model transfers rewriting principles across substantially different environments.
W6. The computational advantages depend on amortizing substantial offline optimization costs.
MASA requires teacher-guided environment search, with the reported ALFWorld search-budget analysis reaching 800–1,000 rollouts at larger budgets. The rewriter additionally requires full-parameter supervised fine-tuning of a 4B model. Although deployment requires only a forward pass, the overall cost depends on how frequently the trained rewriter is reused across tasks and target models. The paper does not quantify the break-even point relative to repeated one-shot adaptation or lighter-weight prompt optimization.
Questions for authors
-
How does MASA compare against a carefully tuned, model-specific formatting and instruction baseline, particularly on WebShop, where action-generation behavior appears to explain much of the performance gap?
-
Can the trained rewriter generalize to a backbone family excluded entirely from its training data? How do the total teacher inference and environment-search costs compare with competing model-specific optimization strategies?
3 comments
- A4. Model-Card Dependency and Robustness (W4)
The original paper already evaluates model-card sensitivity under a matched search budget (Appendix C, Table 5):
| Model-Card Setting | ALFWorld-4B SR |
|---|---|
| Random Model Card | 23.8 |
| Without Model Card | 27.1 |
| Correct Model Card | 31.4 |
All variants use the same search configuration. The correct model card improves success by 4.3 points over the no-card variant, whereas an unrelated card causes a larger degradation. This confirms that capability information contributes beyond additional optimization iterations, while also showing that inaccurate conditioning can be harmful. Appendix C, Table 6 further demonstrates that model-card conditioning remains important for MASA-Rewriter. For example, removing the card reduces Cross-task success on Qwen3-14B from 48.5% to 20.6%.
- A5. Cross-Environment Generalization and Statistical Reliability (W5)
As reported in Section 4.3, Cross-env MASA-Rewriter improves over DS-Adapter by +1.5, +3.0, +2.9, and +3.0 percentage points on Qwen3-4B/8B/14B/32B, respectively, despite receiving no ALFWorld training examples. Cross-task improvements are larger, reaching +8.8, +13.2, and +7.4 points on 8B/14B/32B. This difference is consistent with the benefit of observing other task types from the target environment, which provides additional supervision about its interaction patterns. We emphasize that Cross-env evaluates the more challenging setting in which the rewriter has no training exposure to the target environment. Its consistent directional improvements across four backbones provide encouraging evidence of transfer.
- A6. Offline Optimization Cost and Amortization (W6 + Q2)
The original paper already reports a search-budget analysis (Appendix D, Table 7) on ALFWorld with Qwen3-32B. The search gains saturate around 800 rollouts, with no additional improvement at 1,000. The rewriter offers a different performance–cost trade-off: it avoids iterative environment search and teacher inference when adapting skills at deployment. Appendix H also reports the training setup: full-parameter SFT of Qwen3-4B for five epochs, using 499 Cross-env or 769 Cross-task training examples. We recognize that generating these examples, constructing model cards, and training the rewriter incur non-negligible offline costs.
The relevant amortization condition can be expressed as: , where denotes the number of subsequent skill-adaptation requests, and the costs include the corresponding teacher inference, rewriter inference, offline evolution, training, and profiling expenses. Thus, MASA-Rewriter is most attractive when the trained adaptation mechanism is reused across many skill libraries or adaptation requests. We do not claim that it necessarily achieves a lower total cost for one-off adaptation.
Thank you to the authors for their detailed responses and additional experiments. The rebuttal has partially addressed my concerns. In particular, the format-controlled comparisons provide useful evidence that the improvements on larger backbones extend beyond action-format correction, while the clarifications help better establish the intended contribution and scope of the framework.
However, some concerns remain regarding generalization to unseen model families and comparisons with strong model-adaptive optimization baselines under matched budgets. Taking into account the additional supporting evidence and the authors’ substantial efforts to address the feedback, I have decided to increase my score by one point, from 1 to 2.
We thank the reviewer for the detailed and constructive feedback. We provide additional controlled comparisons and clarify the contributions, generalization scope, and computational costs of MASA below.
To determine whether MASA's improvements can be explained by formatting alone, we conduct two additional controlled comparisons.
First, we compare MASA against format-only instructions and constrained decoding on the same WebShop tasks across three sampling seeds.
Second, we explicitly control action formatting through identical guided decoding, enforcing the same action-output structure for all methods. The only difference is the injected skill library.
With identical formatting constraints, MASA still substantially outperforms the original skill library. Meanwhile, adding the original skill library under guided decoding can even reduce performance relative to using no skills. These findings show that MASA's gains on larger backbones cannot be explained by action-format correction alone. Beyond enforcing valid outputs, evolved skills improve the procedural guidance used for task execution. We also acknowledge that the additional benefit over format-only guidance is small for Qwen3-4B, indicating that the relative importance of skill adaptation varies across backbones.
Our main experiments intentionally focus on four Qwen3 backbones to isolate model-capability differences while controlling for architectural and training-family variations. The additional Gemma3 study (Appendix E.3) further confirms that model-dependent skill preferences extend across model families. For MASA-Rewriter, Section 4.3 evaluates generalization to unseen tasks and environments. Importantly, the proposed model-conditioned evolution framework is not inherently restricted to the Qwen3 architecture, as it optimizes external skill libraries through environment feedback without modifying target-model parameters.
We agree that hill climbing, UCB-based search, and supervised distillation are established techniques. Our contribution is not a new search algorithm, but a model-conditioned framework for optimizing dynamically retrieved procedural skill libraries under frozen LLM backbones. Unlike conventional prompt optimization, which typically optimizes monolithic task instructions, MASA jointly addresses backbone-level behavioral guidance and task-type-specific procedures, using environment feedback and capability-aware rewriting.
The original paper already includes comparisons with existing skill-evolution systems (Appendix B, Table 3):
Furthermore, the ablations in Appendix C validate the contributions of both hierarchical optimization and model-aware conditioning. In Table 4, replacing task-specific evolution with one-shot teacher rewriting reduces ALFWorld-8B success from 57.9% to 32.9%, while replacing general-skill evolution reduces WebShop-8B success from 28.6% to 7.2%. Table 5 further shows that, the correct model card improves ALFWorld-4B success from 27.1% to 31.4%.
Together, these results demonstrate the value of model-conditioned, hierarchical skill optimization beyond fixed skill formulations and one-shot adaptation. Our ablations isolate the benefits of model-aware conditioning and the contributions of the two optimization stages, rather than attributing the improvements to novel search algorithms.