acceptodds
Under review as a conference paper at ICLR 2027

From Weights to Words: Meta-Learning Skill Writers with Parameter–Text Adaptation

Abstract

LLM agents increasingly store procedural knowledge in natural-language skills, written by a Writer model and executed by a Reader model. Most methods optimize a skill or its construction prompt for a fixed Reader, without learning a Writer initialization whose construction process can rapidly adapt to new task–Reader pairs. Yet a skill is only as useful as the Reader that executes it. We introduce MetaSkill, which meta-learns a Writer initialization for coupled parameter–text adaptation. Reader feedback temporarily specializes the Writer, and text refinement externalizes that specialization into a persistent skill. Across language, reasoning, and interactive benchmarks, this adaptation is far more effective from the meta-learned initialization than from the original or a multitask-trained Writer, and both stages contribute. MetaSkill also matches the strongest text-space optimizer on non-interactive tasks at a lower token cost and adapts to unseen Reader families. Skills shaped for one Reader serve others less well, tying skill utility to the task–Reader pair. These results show that meta-learning an adaptable Writer enables effective and token-efficient skill acquisition for new tasks and Readers.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
meta-learning, skill generation, test-time adaptation, language model agents, procedural knowledge, execution feedback

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_ncxjOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes a meta-learning approach to learn natural-language skills tailored for a given model (the Reader) and task. The proposed framework is based on two stages and a meta-learning optimization procedure. In the first stage, a skill generating model (the Writer) is trained to generate skills for a particular Reader-task pair with a few steps of GRPO based on accuracy and observed errors from the Reader. In the second stage, the weight-updated Writer iteratively refines a skill based on accuracy and observed errors from the Reader. The meta-learning procedure optimizes the initial Writer parameter θ\theta based on a reward that tracks how well the iterative refinement process of the two stages performs. Experiments on five benchmarks demonstrate that this method rivals text-space optimizer methods like GEPA and TextGrad, and ablations validate the utility of both stages as well as the meta-learning procedure.

I think the proposed method is interesting and the experimental results are positive. My main concerns are about whether the comparison with baselines is fair, and with the presentation of the paper. These concerns make me lean towards rejecting the manuscript in its current form, but I am prepared to increase my score if the authors can address them.

Critical strengths
  1. The approach of learning how to write skills for a specific target model and task is interesting and the proposed method is generally sound.
  2. The ablation experiments show that each component contributes to the performance of the framework, even the small weight-update to the Writer model during T_grad phase.
Critical weaknesses

Major:

  1. Comparison with the baselines is not rigorous. All three baselines and the proposed method use iterative refinement of the skills, but no information is provided for how the baseline models were run, and how they compare with the iteration budget used by MetaSkill. Appendix B.6 just says that they use the same backbone/Reader/support and query set, but does not give any further detail on how methods' search budgets are comparable to MetaSkill's. Figure 2d compares the number of tokens per episode, but it is unclear where these differences come from (eg more iterations, more tokens per iterations, a different prompting setup, etc). Figure 2c is not discussed anywhere in the manuscript and does not seem to take into account the meta-learning phase which is only needed for MetaSkill. I would strongly recommend presenting the results compared to the baselines across a common axis like the number of iterations on the skill, and carefully discussing how the evaluation setup of the baselines makes it a fair comparison.
  2. If my understanding is correct, the meta-training is done over the same tasks used in evaluation. This means that the Writer is only evaluated in-distribution. Again, this might lead to an unfair comparison with the baselines which only perform inference-time search. A stronger evaluation should withhold some environments from the meta-training phase to evaluate performance out-of-distribution.
  3. The presentation could be significantly improved. The experimental setup mentions various terms like "units", "episodes", "queries" without defining them. Some figures are difficult to parse (e.g. Fig1 and Fig2b; Fig2c is not discussed anywhere). It is unclear how the meta-training is applied in the experiments (on a task-by-task basis or once across all the tasks). The experimental details are missing some important information like the batch size.

Minor:

  1. "suggest that weights and words play complementary roles in acquiring procedural knowledge, with weights carrying the capacity to adapt and words carrying what persists." in the conclusion (l.) is misleading. The weight updates operate on the Writer and the procedural knowledge is still only provided to the target model in natural language.

  2. The main Table 1 should report the standard deviation from the 5 repeats for all results rather than just the average.

  3. MetaSkill-4B use a Qwen3 backbone while 7B/14B use a Qwen2.5 backbone. This makes the parameter count comparison misleading and should be clarified in the main text.

Questions for authors
  1. Given how RLVR is generally ran (hundreds of epochs, large batch size), I find it surprising that only a handful of steps of GRPO can significantly improve performance as shown by the α=0\alpha=0 ablation in Table 1B. Can the authors provide more details on how the performance varies with different amount of adaptation of the Writer's weights?
  2. As per my W1 comment, can the authors compare the performance across the number of iterations for both T_grad and T_text steps, alongside comparable iterations from the baseline methods?
  3. Figure 2b is confusing. What is the "single-stage" baseline? Why does it vary depending on "+gradient" or "+text"?
6 comments
AuthorsOpenReview ↗

Response to Reviewer ncxj(Part 1)

Dear Reviewer ncxj:

We greatly appreciate your thoughtful review. We respond to your comments point by point below.

Major W1

How we ran the baselines.

All four methods use the same Qwen2.5-14B Writer backbone, frozen Reader, support and query sets, and skill-output limits. All select skills on support data. The baselines start from the original Writer. MetaSkill starts from its learned initialization and generates up to 22 candidates: 16 for parameter adaptation, then one fresh skill and up to five revisions. The baseline stopping rules and their actual call counts are reported in Table N2 below.

How performance changes at a common budget

We respectfully disagree that the baseline comparison lacks rigor. As described above, we held the Writer backbone, frozen Reader, evaluation data, and skill-output limits fixed across methods. How to match the search budget is a separate design choice. In the original paper, we used the same limit of five text revisions when comparing Writer initializations under MetaSkill’s adaptation procedure. This lets us ask how well each Writer uses the same number of opportunities to revise a skill. For GEPA, TextGrad, and SkillOpt, we used each optimizer’s own stopping rule (Table N2), since a GEPA iteration can end without a revision and a TextGrad step can involve several Writer calls.

To address your suggestion, we also compared the methods under common per-episode token caps (Table N1). At 10k tokens, MetaSkill scored 62.30, 1.76 points above TextGrad, the strongest baseline at that cap. At 5k, TextGrad led by 2.62 points. The caps count Writer and support Reader input and output tokens, including unsuccessful proposals. We evaluate the selected skills on held-out queries and weight the four non-interactive suites equally.

Table N1. Four-suite scores under common construction budgets.

Method5k cap10k cap20k capOriginal scoreOriginal mean tokens
GEPA58.2758.2761.1061.5≈33.2k
TextGrad58.9060.5461.9463.0≈99.8k
SkillOpt54.9256.2256.2058.1≈83.4k
MetaSkill56.2862.30—63.0≈14.0k

Under the original stopping rules, MetaSkill and TextGrad both reached 63.0. MetaSkill used 23.63k tokens on SuperNI, 9.62k on BBH, 10.50k on SCAN, and 12.10k on CLUTRR, for a four-suite mean of 13.96k (about 14.0k). TextGrad used 99.8k tokens per skill on average to reach the same score. We leave MetaSkill’s 20k cell empty because we did not evaluate it under that cap on every episode. Its original mean on SuperNI alone exceeds 20k.

From 5k to 20k, GEPA gained 2.83 points, all after 10k. TextGrad gained 3.04 points across the two intervals. SkillOpt gained 1.28, with little change after 10k (56.22 to 56.20). At 20k, the three methods were 0.40, 1.06, and 1.90 points below their respective original scores. MetaSkill therefore achieved the highest four-suite score at most time and matched TextGrad’s final score under their original stopping rules using about one-seventh as many construction tokens per skill.

(3) Where the token differences in Fig. 2d come from.

In a profile of one episode per non-interactive suite, GEPA and SkillOpt spent most of their tokens executing candidates on the support Reader. TextGrad spent most of its tokens generating feedback and updates with the Writer. Tables N2–N3 report means over these four episodes. The generation limits were 512 tokens for the initial skill, 2,048 for analysis or updates, and 256 for a Reader response.

Table N2. Baseline iteration and call counts in the four-episode profile.

MethodStopping ruleNative loopsNew skill proposalsWriter callsSupport Reader calls
GEPA64 metric calls, checked at iteration boundaries26.753.754.7564.25
TextGradFive TGD steps5.005.0016.0026.50
SkillOptThree epochs, batch size at most two4.504.507.5039.25

GEPA averaged almost 27 loops but fewer than four new proposals. It skips reflection when it already solves the feedback subset, and we disabled merging. TextGrad uses up to two example-level feedback calls and one update call per step. SkillOpt groups successes and failures for reflection, merges or ranks when needed, and makes slow updates across epochs. We disabled its optional meta-skill extension.The same four episodes give the following breakdown in thousands of tokens. We counted unsuccessful proposals and recorded final query execution separately.

AuthorsOpenReview ↗

Response to Reviewer ncxj(Part 2)

Table N3. Token breakdown for the same four episodes (thousands).

MethodWriter inputWriter outputReader inputReader outputProfile totalWith exact-request reuse
GEPA2.912.2245.330.5250.9811.83
TextGrad16.385.5714.480.2436.6836.11
SkillOpt7.861.6423.650.2233.3722.23

Support evaluations accounted for about 90% of GEPA’s tokens and 72% of SkillOpt’s. TextGrad spent about 60% on the Writer, with roughly 1,024 input tokens per Writer call versus 612 for GEPA. In the initial profile we disabled GEPA’s native evaluation cache and retained SkillOpt’s native selection cache. For the last column we applied the same additional rule across methods, reusing the first response to each identical complete request. This made the largest difference for GEPA. Fig. 2d reports the full original evaluation.

What Fig. 2c includes.

It estimates the time to construct one skill at equal token throughput and includes the backward passes and optimizer updates for that skill. Relative to MetaSkill (1.0×), the estimates are 2.5× for GEPA, 6.3× for SkillOpt, and 7.1× for TextGrad. We measured offline training and development separately for one complete checkpoint on eight H100s (Appendix E.4).

Table N4. Offline training and development cost.

SuitesTraining (h)Development evaluation (h)H100 GPU-hours
SuperNI, BBH, SCAN, CLUTRR58.775.28512.36
ALFWorld31.418.55319.70
Total90.1813.83832.06

One checkpoint can serve many task–Reader pairs, and each constructed skill can serve many queries. Whether the deployment savings recover the training cost depends on the number of skills constructed and the GPU-hours saved per skill.

W2 ... A stronger evaluation should withhold some environments from the meta-training phase to evaluate performance out-of-distribution.

We totally agree with this point. We held out tasks or construction units within each of the five main suites and added further evaluation on MBPP as an entirely held-out benchmark.

Our original submission also tested transfer to unseen Reader families (Table 3A): we meta-trained a Writer using only Qwen-family Readers, and adaptation improved the five-suite mean by 9.6 points on Gemma and 9.9 points on Ministral. On OOD tabsk MBPP, we ran the same two-stage adaptation procedure with the original and meta-trained Writers and tested the Reader’s generated code with unit tests. Changing the Writer initialization raised pass@1 from 68.87% to 72.76% (+3.89 percentage points, Appendix E.5).

Table N8. Transfer to held-out MBPP.

MBPP configurationpass@1
GEPA71.98%
TextGrad71.21%
SkillOpt70.82%
Meta-trained Writer, refinement only (error-driven diagnostics)71.21%
Meta-trained Writer, initial skill71.21%
Meta-trained Writer, full two-stage protocol72.76%
Original Writer, full two-stage protocol68.87%

W3 The presentation could be significantly improved ...

Thank you for raising this point. We have revised our paper according to your suggestions. Section 5.1 now defines the evaluation terms with an example. In SuperNI, a unit is a task ID. Each held-out task gives three episodes, each with its own support and query sets. A query is one scored example, or one environment run in ALFWorld. We meta-train one shared Writer across all five suites and training Readers. The original submission reported eight candidate skills per GRPO round (Table A1) and the sampling rule for each outer batch (Appendix B.3). Section 5.1 now states the batch size directly: each outer update combines up to 276 episode visits per Reader. We revised Fig. 1’s caption to distinguish joint meta-training from adaptation to a new task–Reader pair. Fig. 2b’s caption now names the starting point for each comparison: “+gradient” starts from text refinement alone, and “+text” starts from parameter adaptation alone. Section 5.2 now discusses Fig. 2c’s estimate of skill-construction time, including Writer updates. Appendix E.4 reports offline training cost separately.

Minor W1 ... The weight updates operate on the Writer and the procedural knowledge is still only provided to the target model in natural language.

The frozen Reader receives the procedure only through the saved text skill. The Writer retains its shared initialization across tasks. For each new pair, we temporarily update the Writer’s LoRA parameters and discard that update after skill construction.

Minor W2

The main Table 1 should report the standard deviation from the 5 repeats for all results rather than just the average.

Thank you for raising this point. We now have added the sample SD for every score in Table 1D to Appendix E.1, covering all six methods and all five benchmarks.

AuthorsOpenReview ↗

Response to Reviewer ncxj(Part 3)

Minor W3 MetaSkill-4B use a Qwen3 backbone while 7B/14B use a Qwen2.5 backbone.

The 4B comparison varies both family and size. The 7B and 14B Writers share the Qwen2.5 family, so their comparison isolates size more closely. We now identify the families in the main text beside Table 2. With the Reader fixed, the two Qwen2.5 Writers scored 57.0 at 7B and 62.4 at 14B across the five suites. The main optimizer comparison holds the Writer backbone fixed at Qwen2.5-14B.

Q1 Given how RLVR is generally ran (hundreds of epochs, large batch size), I find it surprising that only a handful of steps of GRPO can significantly improve performance as shown by the ablation in Table 1B. Can the authors provide more details on how the performance varies with different amount of adaptation of the Writer's weights?

Our insight is that a few GRPO steps are effective here because we meta-train the Writer specifically to adapt quickly to a new task–Reader pair. During meta-training, we reward the shared initialization according to how well the final skill performs on held-out queries after the Writer adapts to the Reader’s support feedback. Table 1B shows why the starting point matters. Under the same two GRPO rounds and subsequent text refinement, the original Writer scored 41.0 across the five suites, while the meta-trained Writer scored 62.4. To measure how performance changes with the amount of adaptation, we varied the number of GRPO rounds on the same meta-trained checkpoints and kept the text-refinement budget fixed. The first round accounted for 6.9 of the 10.1 points gained over text refinement alone.

Table N5. Effect of GRPO rounds on the same meta-trained checkpoints.

GRPO roundsCandidates evaluatedConfigured optimizer stepsFive-suite mean
00052.3
182–459.2 ± 1.4
2164–862.4 ± 1.9

We also tested whether evaluating more candidates could explain the gain. When we kept all 16 candidate evaluations and the same text-refinement procedure but set the inner learning rate to zero, the mean fell from 62.4 to 54.6 (Table 1B). This isolates a 7.8-point gain from updating the Writer at the same sampling budget. For the five-round setting, we continued meta-training for three epochs using five GRPO rounds per visit and then evaluated with five rounds. This longer schedule scored 61.4 ± 3.2. Because it also changes the checkpoint, Table N5 provides the direct comparison of adaptation depth at a fixed initialization.

Table N6. Five-round continuation experiment.

Additional meta-trainingGRPO rounds at evaluationCandidates evaluatedConfigured optimizer stepsFive-suite mean
3 epochs with 5 rounds per visit54010–2061.4 ± 3.2

Q2 As per my W1 comment, can the authors compare the performance across the number of iterations for both T_grad and T_text steps, alongside comparable iterations from the baseline methods?

We report performance by round for each MetaSkill stage, and by construction tokens for the comparison across optimizers. With the text budget fixed, a second parameter round added 3.2 points (Table N5). With parameter adaptation fixed, allowing three text revisions instead of one improved all four non-interactive suites on the same checkpoint, most on SCAN (49.4 to 65.6). Table N7 uses a single evaluation on the same checkpoint for the one- and three-revision caps.

Table N7. Effect of the text-revision cap.

Suite0 revisionsUp to 1Up to 3Up to 5
SuperNI61.261.265.966.4
BBH51.751.862.563.3
SCAN48.149.465.669.3
CLUTRR40.542.949.652.9
ALFWorld64.260.260.260.4

For the baselines, Table N2 shows why the iteration counts are hard to equate. GEPA averaged 26.75 native loops and 3.75 new proposals, while five TextGrad steps produced five proposals and used 16 Writer calls. Table N1 gives the cross-method score curves at common token caps.

Q3 Figure 2b is confusing. What is the "single-stage" baseline? Why does it vary depending on "+gradient" or "+text"?

“+gradient” starts from text refinement alone, and “+text” starts from parameter adaptation alone. Each bar shows what happens when we add the missing stage. Adding parameter adaptation raises the five-suite mean from 52.3 to 62.4 (+10.1). Adding text refinement raises it from 53.1 to the same 62.4 (+9.3). We revised the Fig. 2b caption to name both starting conditions and report the signed changes on ALFWorld.

We greatly appreciate your thoughtful review and constructive feedback. We revised the setup and figure captions in §§5.1–5.2 and added the standard deviations, adaptation sweeps, cost breakdown, and MBPP results in Appendices E.1 and E.3–E.5. Please let us know if you have any further questions.

Reviewer_WRjmOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes to use a meta-learning approach to learn to produce targetted skills for solving task-reader (-model) pairs. The paper advocates to couple parameter (LoRA) adaptation with text refinement as bi-level optimization in a process that ultimately produces a (skill) writer model that is able to generate and refine skills for a task-reader pair.

Critical strengths
  1. Comprehensive ablations and experiments that justify the method. The different settings in Table 1 were useful for understanding where the method's strength lies
  2. The methodology is a unique application of classical meta-learning methodology in the era of LLMs
  3. The paper is comparatively quite readable overall
Critical weaknesses
  1. Writing clarity can be improved. It was hard to figure out from the notation whether the base model weights were ever updated or if it was only the lora weights that were learned. Also, Equation 8 introduces subscripts and variables that are not defined before hand making it hard to understand on first pass.
  2. The method requires backprop through the writer in order to compute the updates for the loRA params -- this is increases the memory requirements compared to the forward-only skill optimization techniques like GEPA and TextGrad
  3. The datasets and model families explored in the paper are relatively older so it does raise the question of how relevant the results are to the more recent datasets and LLMs
Questions for authors

None at the moment

2 comments
AuthorsOpenReview ↗

Response to Reviewer WRjm

Dear Reviewer WRjm:

We greatly appreciate your thoughtful review. We respond to your comments point by point below.

W1 Writing clarity can be improved. It was hard to figure out from the notation whether the base model weights were ever updated or if it was only the lora weights that were learned. Also, Equation 8 introduces subscripts and variables that are not defined before hand making it hard to understand on first pass.

We greatly thank you for pointing out this. We update only the Writer’s LoRA parameters in both loops. We now define every index in Eq. (8) before using it and identify which tokens contribute to the loss. During meta-training we learn one shared adapter initialization, θ\theta, while keeping the Writer backbone and the Reader frozen. For a new task–Reader pair, we copy that adapter and update the copy, ϕt,m\phi_{t,m}, using the Reader’s scores on the support examples. We then hold the adapter fixed while the Writer refines the skill. Once we have the text, we discard the copy.

We now define each index before Eq. (8). The index ii identifies a trajectory, aa is one Writer output on that trajectory, HaH_a is the prompt for that output, and ℓ\ell indexes its tokens. For example, if the Writer proposes two revisions and we reject the second on the support set, we still include both in the loss because the Writer sampled both. They share the trajectory advantage AiA_i. We use the Reader’s response as feedback in the next prompt. Its tokens do not enter the loss.

W2 The method requires backprop through the writer in order to compute the updates for the loRA params -- this is increases the memory requirements compared to the forward-only skill optimization techniques like GEPA and TextGrad.

Yes. We do need extra memory to backpropagate through the Writer while building a skill. We update a rank-16 adapter and use gradient checkpointing to reduce the activations we store (Appendix B.1). GEPA and TextGrad use only forward passes during construction. We have not measured peak memory for the three methods on the same hardware. To test whether the update helps, we kept the 16 candidate evaluations and the subsequent text refinement the same, and set the inner learning rate to zero. The five-suite mean was 54.6 with updates disabled and 62.4 with updates enabled (Table 1B). We include those updates in the construction-time estimate in Fig. 2c. After construction, the frozen Reader needs one inference request per non-interactive query or per environment step.

The datasets and model families explored in the paper are relatively older so it does raise the question of how relevant the results are to the more recent datasets and LLMs.

Thank you for raising this point. We added an evaluation on IFStruct (2026), and found gains of 6.90–8.75 percentage points across three Reader families. A response passes only if its JSON or YAML output satisfies all requested structural constraints. Benchmark description. We are continuing testing on more models and benchmarks.

Table W1. IFStruct v1.0 pass rates.

ReaderBaselineMetaSkillGain (percentage points)
Gemma-4-12B85.95%92.85%+6.90
Ministral-14B82.45%91.20%+8.75
Qwen2.5-14B78.45%85.50%+7.05

Our original five-suite experiment also tests whether the Writer can adapt to a Reader family excluded from meta-training. We trained the Writer only with Qwen-family Readers. Using feedback from Gemma or Ministral at evaluation improved their five-suite means by 9.6 and 9.9 points, respectively (Table 3A).

Table W2. Five-suite scores before and after adaptation (mean ± SD across training runs).

ReaderSeen in this meta-training run?Before adaptationAfter adaptationGain (points)
Qwen2.5-14BYes43.7 ± 2.761.9 ± 3.7+18.2
Gemma-4-12BNo45.2 ± 6.854.8 ± 4.9+9.6
Ministral-14BNo42.2 ± 5.252.1 ± 7.3+9.9

We added MBPP to test transfer to programming, a domain outside the five training suites. Under the same two-stage adaptation procedure, replacing the original Writer with the meta-trained Writer raised first-attempt unit-test success from 68.87% to 72.76%.

Table W3. MBPP pass@1 (Appendix E.5).

MBPP configurationPass@1
GEPA71.98%
TextGrad71.21%
SkillOpt70.82%
Meta-trained Writer, initial skill71.21%
Meta-trained Writer, text refinement only71.21%
Meta-trained Writer, full two-stage adaptation72.76%
Original Writer, full two-stage adaptation68.87%

We greatly appreciate your thoughtful review and constructive feedback. We clarified the trainable parameters and Eq. (8) in §§3 and 4.3, discussed memory in §6, and added MBPP in §5.4 and Appendix E.5. Please let us know if you have any further questions.

AuthorsOpenReview ↗

Follow-up on Rebuttal

Dear Reviewer WRjm,

We sincerely thank you for your positive assessment. As the discussion period is ending soon, we would appreciate it if you could review our updates or let us know if there is anything for further discussion.

Thank you again for your time and valuable suggestions.

Reviewer_X18uOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MetaSkill, a meta-learning framework that trains an LLM Writer to rapidly generate and refine natural-language skills tailored to specific tasks and frozen Reader models. It improves post-adaptation performance over standard and multitask-trained Writers, while generalizing to unseen Reader models.

Critical strengths

The experiments are detailed.

Critical weaknesses

The proposed meta-learning framework incurs substantial computational overhead due to its nested optimization procedure. For each task–Reader pair, the method requires two rounds of inner-loop GRPO, involving 16 skill generations and 4–8 gradient updates, followed by four independent refinement trajectories, each containing up to five refinement steps (i.e., up to 24 additional skill generations). Even with a relatively small setting of 100 tasks and four Readers, a single pass over all 400 task–Reader pairs would require up to 16,000 Writer generations and 1,600–3,200 inner-loop gradient updates, excluding outer-loop backpropagation and Reader evaluations. Repeating this process for 20 meta-training rounds would result in up to 320,000 Writer generations and 32,000–64,000 inner-loop gradient updates. More importantly, the cost of Reader evaluation can be significantly higher for interactive agents. For example, in ALFWorld, evaluating a single skill may require an entire agent trajectory of up to 40 interaction steps, each involving a separate Reader inference. Assuming two support episodes per skill, the 16 inner-loop skill candidates alone could require up to 1,280 Reader inference calls per task–Reader pair, before accounting for the additional evaluations during refinement and query scoring. This overhead would become even more substantial for realistic long-horizon agents involving hundreds of interaction steps and expensive tool calls. Although the paper reports token efficiency and estimated skill-construction time, it does not fully account for the total meta-training cost, particularly the repeated inner-loop gradient updates and multi-turn Reader executions. This raises concerns about whether the proposed approach can scale efficiently to larger task–Reader distributions and more complex agent environments. A comprehensive analysis of GPU hours, total Reader inference calls, and computational scaling with the number of tasks, Readers, and interaction horizon would strengthen the paper.

Questions for authors

Q1. Despite the substantial computational cost of meta-training, each task–Reader pair contributes only four refinement trajectories per outer-loop visit, with each trajectory potentially containing multiple refinement turns. For example, even with 100 tasks and four Readers, a single pass over all 400 pairs would yield only 1,600 refinement trajectories. Is such a limited number of independent trajectories sufficient to learn a generalizable skill-refinement strategy? How does performance scale with the number of training task–Reader pairs and the number of refinement trajectories sampled per pair?

Q2. The outer-loop objective assigns the same trajectory-level advantage to all Writer-generated tokens across multiple refinement turns. However, a high final reward may be attributable to only a few effective revisions, while ineffective or even harmful revisions receive the same positive advantage. This issue may become more severe as refinement trajectories grow longer, since stochastic generation introduces additional variance and makes credit assignment increasingly difficult. Moreover, the method uses only four trajectories per task–Reader pair to estimate the leave-one-out advantage. How do the authors address the resulting high variance and coarse credit assignment?

Q3. Given the relatively limited number of independent refinement trajectories per task–Reader pair and the potentially noisy trajectory-level training signals, how can the authors ensure that the meta-initialization learns transferable skill-refinement capabilities rather than task-specific or Reader-specific patterns? Although the paper evaluates held-out tasks and unseen Reader families, it remains unclear whether the learned adaptation strategy generalizes to substantially different task distributions, particularly more complex long-horizon agent tasks. Could the authors provide additional OOD evaluations or scaling experiments to demonstrate that increasing the diversity of training tasks leads to improved generalization?

5 comments
AuthorsOpenReview ↗

Response to Reviewer X18u (Part 1)

Dear Reviewer X18u,

We greatly appreciate your thoughtful review. We respond to your comments point by point below.

W1

(1) Offline training cost.

MetaSkill learns one shared Writer initialization that can be reused to construct skills for new tasks and Readers. This offline investment supports repeated deployment, where each new skill requires only short adaptation and can then serve multiple queries. The training cost per constructed skill therefore decreases with use. Below, we report the offline cost, estimated execution workload, and deployment efficiency.

Training and periodically evaluating one checkpoint took 104.01 hours on eight H100 GPUs, totaling 832.06 H100 GPU-hours. This measurement includes Writer generation, Reader execution, backpropagation, and optimizer updates (Appendix E.4). The run trains one Writer jointly across five suites and multiple Readers.

Table X1. Measured cost for one checkpoint.

SuitesTraining (h)Development evaluation (h)H100 GPU-hours
SuperNI, BBH, SCAN, CLUTRR58.775.28512.36
ALFWorld31.418.55319.70
Total90.1813.83832.06

(2) Reader calls and scaling.

We estimate 12.37 million Reader requests across training and development, with 68.9% arising from ALFWorld. The estimates assume three Readers, an average of four Writer submissions per training trajectory and two per development trajectory, and 20 executed steps per ALFWorld rollout.

Table X2. Estimated training and development workload.

SuitesPhaseEstimated Writer generationsEstimated Reader requests
Four non-interactive suitesTraining407,0403,637,920
Four non-interactive suitesDevelopment34,560211,200
ALFWorldTraining122,8808,140,800
ALFWorldDevelopment8,640384,000
TotalTraining and development573,12012,373,920

Each training visit allows up to 40 Writer generations: 16 GRPO candidates followed by four trajectories containing up to six skills each, with 4 to 8 configured inner optimizer steps. At fixed per-visit budgets, doubling the number of scheduled task/Reader visits doubles the generation allowance.

Our schedule samples at most 276 episode visits per Reader per outer round. The per-round workload can therefore remain bounded as the task pool grows, with tasks sampled across successive rounds. Increasing coverage or the number of training Readers increases the total workload.

For interactive tasks, Reader requests also scale with the number of executed rollout steps. Evaluating the initial 16 candidates on two support environments can require 1,280 Reader calls at the 40-step limit, as the reviewer notes. Each additional step requires one Reader call per candidate rollout during construction and one per executed rollout at inference. Longer horizons and expensive tools therefore increase execution cost.

(3) Cost at deployment.

We reuse the trained checkpoint across new tasks and Readers, running one refinement trajectory per skill with at most 22 Writer generations. On the four non-interactive suites, MetaSkill and TextGrad both score 63.0, using 14.0k versus 99.8k construction tokens per skill. Figure 2c includes Writer updates during construction. Once constructed, skills serve subsequent queries through the frozen Reader without further Writer adaptation, spreading the offline training cost across repeated deployment.

Q1. Are the refinement trajectories per task–Reader visit sufficient to learn a generalizable refinement strategy? How does performance scale with the number of training pairs and trajectories per pair?

Four refinement trajectories per training visit were sufficient for transfer in our experiments because each visit contributes to a shared Writer that learns across tasks and Readers over outer rounds (We would first like to clarify a trade-off: As your cost question highlights, evaluating hundreds of candidates for each new task–Reader pair would be impractical at deployment, so we designed the Writer to construct a skill using a few refinement trajectories) And we show that four refinement trajectories per training visit were sufficient for the Writer to learn an adaptation procedure that transferred to held-out tasks and unseen Readers. During training, the Writer can compare four trajectories on similar tasks and Readers, then learn from such comparisons across many other tasks, Readers, and visits. For the performance scaling on refinement trajectories, we show:

Table X3. Refinement trajectories per training visit.

Trajectories per training visit (NN)1248
Four-suite macro-average56.161.263.062.4
AuthorsOpenReview ↗

Response to Reviewer X18u (Part 2)

Four trajectories gave the highest mean in our sweep. At N=4N=4, we compare each sequence of revisions with three alternatives from the same Writer, Reader, and support set. Moving from two trajectories to four improved the four-suite mean by 1.8 points. We expected eight to give a more precise baseline, but additional trajectories on the same small support set may explore similar revisions. This could explain why we saw no further gain at eight.

On the number of distinct training pairs, we are still running the experiment that varies coverage directly. At our current coverage, five independently trained Writers averaged 62.4 ± 1.9 across the five suites on held-out evaluation units. A separate Writer trained only with Qwen-family Readers also improved by 9.6 points on Gemma and 9.9 on Ministral after adaptation (Table X6).

Q2. A high final reward may be attributable to only a few effective revisions, while ineffective or even harmful revisions receive the same positive advantage... How do the authors address the resulting high variance and coarse credit assignment?

We account for progress at individual turns when computing the reward, but all revisions on a trajectory still receive the same advantage. We chose to score the sequence because an early edit changes the feedback available to later turns. The reward therefore includes both final query utility and bonuses for improvement along the way. A poor edit can still share credit for a later correction.

To address variance, we compare each trajectory’s return with the mean return of the other three trajectories on the same task, Reader, and support examples. This accounts for differences in difficulty across episodes. We also clip each task–Reader pair’s gradient to norm one to limit how much any single pair can influence the update. Table X4 shows why we retained the intermediate scores. Starting from the same initialization, we continued training for five epochs under each reward setting. Including both improvement bonuses raised the final four-suite query score from 57.2 to 63.0. This experiment measures the benefit for final skill quality. We did not measure gradient variance directly.

Table X4. Reward ablations after five additional epochs.

Reward during continued trainingFour-suite final query score
Both bonuses (0.5 each), with the prefix rule63.0
Neither bonus57.2
Per-round bonus only57.4
Global bonus only56.9
Both bonuses, without the prefix rule59.2

Q3. ... how can the authors ensure that the meta-initialization learns transferable skill-refinement capabilities rather than task-specific or Reader-specific patterns? ... could the authors provide additional OOD evaluations or scaling experiments to demonstrate that increasing the diversity of training tasks leads to improved generalization?

We have further added experiments on more tasks and found that the learned initialization transfers both to programming tasks outside the training suites and to Reader families excluded from training. On MBPP, changing only the initialization raised pass@1 from 68.87% to 72.76% under the same two-stage procedure (Table X5). On Gemma and Ministral, adapting from a Qwen-only training run improved the five-suite mean by 9.6 and 9.9 points (Table X6):

Table X5. Transfer to held-out MBPP.

MBPP configurationpass@1
GEPA71.98%
TextGrad71.21%
SkillOpt70.82%
Meta-trained Writer, refinement only (error-driven diagnostics)71.21%
Meta-trained Writer, initial skill71.21%
Meta-trained Writer, full two-stage protocol72.76%
Original Writer, full two-stage protocol68.87%

Table X6. Transfer to unseen Reader families.

Unseen ReaderBefore adaptationAfter adaptationGain (points)
Gemma-4-12B45.2 ± 6.854.8 ± 4.9+9.6
Ministral-14B42.2 ± 5.252.1 ± 7.3+9.9

What transfers is the Writer’s ability to use a new Reader’s feedback to write a skill that works for that Reader. In the cross-execution test (Table 3C), Qwen scored 62.1 with skills shaped by its own feedback versus 52.8 with skills shaped by Gemma’s. Gemma likewise scored higher with its own skills (55.0 versus 51.3). Together with the gains on held-out MBPP and both unseen Reader families, these results show that the learned adaptation procedure generalizes across task domains and Reader families even when the resulting skills remain Reader-specific. Our ongoing coverage experiment tests how much additional task diversity improves that generalization. For interactive tasks, our evidence currently covers ALFWorld episodes of up to 40 steps.

We greatly appreciate your thoughtful review and constructive feedback. We added the cost, trajectory-count, and MBPP results to Appendices E.3–E.5 and clarified credit assignment in §4.3 and Appendix A. Please let us know if you have any further questions.

Thank you for the detailed response and additional experiments. A part of my questions are answered. However, my main concerns remain unresolved.

First, the reported computational cost seems surprisingly low given the substantial rollout workload. In particular, the authors estimate 8.14 million Reader requests for ALFWorld training but report only 31.41 hours on eight H100 GPUs. This implies approximately 72 Reader requests per second, despite the sequential nature of interactive agent rollouts and the additional Writer training overhead. The response does not sufficiently explain how this workload is completed within the reported time.

Second, the credit assignment issue remains largely unaddressed. All refinement turns still share the same trajectory-level advantage, regardless of whether individual revisions are helpful or harmful. The reward ablations demonstrate performance improvements but do not directly address the coarse credit assignment or potential high variance from using only four trajectories.

Overall, there are still an important concern not being addressed. I will maintain my original score.

Reviewer_M9pWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MetaSkill, a framework that adapts skill generation to new task–Reader pairs. It first meta-learns a Writer initialization and uses two adaptation stages. The first updates the Writer’s parameters with Reader feedback, while the second fixes the adapted Writer and refines the skill text. The final skill is saved for a frozen Reader. Experiments across five benchmarks show the benefits of the learned initialization and transfer to unseen Reader families.

Critical strengths
  1. MetaSkill offers a practical way to turn parameter adaptation into reusable text skills. The Writer adapts to a task–Reader pair and produces a skill for later use. The temporary parameters are discarded, and the Reader stays frozen throughout.
  2. Ablation studies provide clear evidence for the value of the learned initialization. With the same adaptation procedure, MetaSkill reaches an average score of 62.4. The original and multitask-trained Writers reach 41.0 and 39.9. The zero-learning-rate control also supports the benefit of parameter updates beyond additional sampling.
  3. The evaluation is broad and provides useful evidence of transfer. It covers five benchmarks, multiple model sizes, and five independent training runs. The method can still work on unseen Reader families. In cross-execution tests, each Reader performs better with skills shaped by its own feedback.
Critical weaknesses
  1. The added value of feedback-based text refinement is not fully isolated. The zero-learning-rate control supports parameter updates, but the paper does not compare refinement with independent best-of-N sampling and selection from the same adapted Writer under matched budgets. On ALFWorld, the full method scores below both single-stage variants. A matched comparison and paired stage-level results would help clarify when refinement is useful.
  2. The outer reward design in Equation 7 needs further validation. The paper adds improvement bonuses to final query utility and withholds these bonuses when the first 80 characters remain unchanged. These choices are plausible, but their benefits need empirical support. Separate ablations of the bonuses and the prefix rule would help show whether they improve adaptation.
Questions for authors
  1. Does feedback-based text refinement outperform independent best-of-N sampling and support-based selection from the same adapted Writer under matched candidate and Reader-call budgets?
  2. Why does the full method score below both single-stage variants on ALFWorld? Paired support and query results across stages and runs would help clarify whether this difference is consistent and where it arises.
  3. Do the improvement bonuses in Equation 7 help? What happens if the first-80-character rule is removed?
3 comments
AuthorsOpenReview ↗

Response to Reviewer M9pW (Part 1)

Dear Reviewer M9pW,

We greatly thank you for your thoughtful review. We respond to your comments point by point below.

W1 The added value of feedback-based text refinement is not fully isolated. The zero-learning-rate control supports parameter updates, but the paper does not compare refinement with independent best-of-N sampling and selection from the same adapted Writer under matched budgets. On ALFWorld, the full method scores below both single-stage variants. A matched comparison and paired stage-level results would help clarify when refinement is useful.

We found that revising a skill in response to Reader feedback improved its four-suite mean over independent sampling at the same candidate cap. On ALFWorld, refinement sometimes favored a rule that worked on the support layouts but failed on a new layout.

We chose six independent samples to match the overall budget of up to 22 Writer generations per episode. Both methods spend the first 16 generations on parameter adaptation (two rounds of eight candidates). We discard those candidates after updating the Writer. For the remaining six generations, MetaSkill draws one fresh skill and revises it up to five times. Best-of-6 draws six fresh skills independently from the same adapted Writer. Both use the same support examples to select a skill.

Table R1. Feedback-guided refinement and independent sampling after parameter adaptation.

How candidates are generatedCandidate budgetMean query score across the four non-interactive suites
Independent Best-of-6Six independent draws58.9
Feedback-conditioned refinementOne initial skill and up to five revisions63.0

We also varied the number of revisions while holding parameter adaptation fixed. We evaluated each cap once on the same checkpoint. Increasing the cap from one to three improved all four non-interactive suites. SCAN showed the largest gain, from 49.4 to 65.6. In one SCAN episode, the Reader executed the clauses of “X after Y” in the wrong order even though the skill stated the rule correctly. After seeing the error, the Writer added a worked example, and both support and query scores rose from 25% to 50%.

Table R2. Effect of allowing additional revisions.

Maximum revisionsSuperNIBBHSCANCLUTRRALFWorld
161.251.849.442.960.2
365.962.565.649.660.2

On ALFWorld, the two revision caps gave the same score. In the main evaluation, adding refinement to parameter adaptation lowered mean success from 64.2% to 60.4%. We therefore examined how often refinement changed the outcome on the same query environment. We observed some fewer successful executions after refinement in the paired audit. Table R3 gives the final success rates for the five independently trained Writers. Q2 mainly describes what we found in the execution logs.

Table R3. Final ALFWorld success across training runs.

Full method on ALFWorldRun 1Run 2Run 3Run 4Run 5Mean ± SD
Query success (%)65.263.756.162.854.460.4 ± 4.9

W2 The outer reward design in Equation 7 needs further validation. The paper adds improvement bonuses to final query utility and withholds these bonuses when the first 80 characters remain unchanged. These choices are plausible, but their benefits need empirical support. Separate ablations of the bonuses and the prefix rule would help show whether they improve adaptation.

We found that rewarding improvement during refinement helped the Writer produce better final skills, and that the 80-character rule made those bonuses more effective. We continued meta-training from the same initialization for five epochs under each reward setting. We evaluated all five Writers on the final skill’s held-out query score, so a larger training bonus could improve the reported score only by changing the skill the Writer produced.

Table R4. Reward ablations on the four non-interactive suites.

Reward during continued trainingGlobal weightPer-step weightPrefix ruleMean final query score
Full reward0.50.5On63.0
Neither bonus00Inactive57.2
Per-step bonus only00.5On57.4
Global bonus only0.50On56.9
Both bonuses without the prefix rule0.50.5Off59.2

Removing both bonuses lowered the mean from 63.0 to 57.2. Keeping both bonuses and removing only the prefix rule lowered it to 59.2. Neither bonus alone recovered the performance of the full reward.

AuthorsOpenReview ↗

Response to Reviewer M9pW (Part 2)

Q1 Does feedback-based text refinement outperform independent best-of-N sampling and support-based selection from the same adapted Writer under matched candidate and Reader-call budgets?

Yes under a matched candidate cap. As Table R1 shows, we obtained 63.0 with feedback-guided refinement and 58.90 with Best-of-6. Since both methods start from the same adapted Writer and select by support score, the comparison tests how the Writer spends its remaining proposals. With feedback, it can use one proposal to correct an error in the previous skill. We matched the maximum number of candidates. Actual support Reader calls could differ because refinement can stop early, so the comparison leaves the effect at exactly equal Reader-call counts open.

Q2 Why does the full method score below both single-stage variants on ALFWorld? Paired support and query results across stages and runs would help clarify whether this difference is consistent and where it arises.

We found a recurring ambiguity in the ALFWorld logs. Two rules could succeed on the same support layouts and still prescribe different actions on a query layout. This helps explain why a revision that retained full support credit could lower query success.

We show one cooling task in the logs here to illustrates the issue: the original skill told the Reader to put the cooled object at the destination named in the goal. The revised skill told it to return the object to where it had been found. In the two support environments, the pot and the cup each started at their required destination. Both instructions therefore earned full credit. In the query environment, the potato started on the countertop and the goal specified the dining table. Returning it to the countertop was a legal action, but it left the goal unmet.

Our reading is that the Writer turned a coincidence in the support layouts into a general instruction. The paired audit also shows that revisions helped on some layouts. Four failures became successes, while seven successes became failures. Table R3 summarizes final performance across five runs. The before-and-after records cover the paired audit, so the direction of the change in each individual run remains open.

Q3 Do the improvement bonuses in Equation 7 help? What happens if the first-80-character rule is removed?

Yes. Removing the improvement bonuses cost 5.8 points, and removing the 80-character rule while retaining both bonuses cost 3.8 points (Table R4). We use the global bonus to reward a final skill that improves on its starting point. The per-step bonus also credits a useful intermediate change, including a revision that recovers from an earlier mistake. When we removed the prefix rule, we saw revisions that kept the opening and appended more instructions. The rule withholds the improvement bonus for a comparison if the first 80 characters are unchanged after whitespace normalization. The final query-utility term still credits any edit that helps the Reader, including an appended instruction.

We greatly appreciate your thoughtful review and constructive feedback. We added the sampling comparison and reward ablations in §5.2 and Appendix E.3, and the ALFWorld log example in Appendix F.1. Thank you again for these questions. Please let us know if any part of the revised analysis would benefit from further clarification.

AuthorsOpenReview ↗

Follow-up on Rebuttal

Dear Reviewer M9pW,

We sincerely thank you for your positive assessment. As the discussion period is ending soon, we would appreciate it if you could review our updates and let us know if there is anything for further discussion.

Thank you again for your time and valuable suggestions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Rebuttal

Dear Area Chair and Reviewers,

We sincerely thank you for the time and care you have devoted to reviewing our work. We appreciate your recognition of the practical value of turning temporary Writer updates into reusable text skills (M9pW), the novel application of classical meta-learning to skill writing (WRjm), the ablation evidence that even a few Writer updates improve performance (ncxj), and the detailed experiments (X18u).

Our proposed MetaSkill learns a Writer initialization that adapts to new task–Reader pairs. During meta-training, we reward the initialization according to how well a frozen Reader performs on held-out queries after the Writer adapts. For each new pair, the Writer uses the Reader’s execution scores to update its parameters, then uses observed errors to revise the skill text. We discard the temporary updates and keep the instructions for the Reader to reuse. With the adaptation procedure held fixed, the meta-trained Writer averages 62.4 across five suites, compared with 41.0 for the original Writer and 39.9 for a multitask-trained Writer. The procedure also transfers to unseen Reader families. When Qwen and Gemma exchange skills on the same tasks, each scores higher with skills shaped by its own feedback.

Main revisions:

  • Method and ablations (Sections 5.1–5.2, Figures 1–2, Tables A10–A11). We clarified the training and evaluation setup and added controls for feedback-guided refinement and the meta-training reward. Appendix F.1 traces a refinement failure on ALFWorld.
  • Budget and cost (Table 4, Appendix E.4). We compared optimizers under common token caps and reported offline training costs separately from the cost of constructing each skill.
  • Transfer (Section 5.4, Appendix E.5). We expanded the transfer evaluation to test how well the learned Writer initialization adapts to tasks held out from meta-training.

We kindly refer you to the revised paper for the updated results and discussions. Once again, we greatly appreciate your thoughtful review and constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.