acceptodds
Under review as a conference paper at ICLR 2027

MetaClaw: Just Talk – An Agent That Meta-Learns and Evolves in the Wild

Abstract

Large Language Model (LLM) agents deployed in dynamic environments often remain static, becoming stale as task distributions drift. Existing solutions either store raw trajectories without knowledge distillation, maintain skill libraries disconnected from weight optimization, or require service downtime for retraining. We present MetaClaw, a continual meta-learning framework that jointly evolves a base LLM policy and a skill library of reusable behavioral instructions through two complementary mechanisms. First, skill-driven fast adaptation synthesizes new skills from failure trajectories via an LLM evolver, providing immediate, zero-downtime improvements. Second, opportunistic policy optimization performs gradient-based updates via cloud-based RL during user-inactive windows, managed by a scheduler (OMLS) that monitors activity signals like Google Calendar. To ensure data integrity, a versioning mechanism strictly separates support data (for skill evolution) from query data (for RL updates), preventing stale reward contamination. Built on a proxy-based architecture, MetaClaw scales to production LLMs without local GPUs. Experiments on MetaClaw-Bench and the AutoResearchClaw pipeline demonstrate that skill-driven adaptation alone improves accuracy by up to 32% relative. The full framework elevates Kimi-K2.5 from 21.4% to 40.6% accuracy, nearly matching the GPT-5.2 baseline (41.1%), while achieving an 8.25 gain in end-to-end task completion and an 18.3% boost in robustness.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
meta-learning, agent, evolving

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_73YjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

MetaClaw combines failure-driven skill generation with periodic RL-based weight updates for deployed LLM agents. It uses skill-versioned data buffers and an idle-time scheduler to coordinate the two processes. Experiments on a sequential CLI benchmark report substantial gains for the full system; a separate research-pipeline evaluation reports gains from skill adaptation alone.

Critical strengths

The paper addresses a practical continual-adaptation setting and reports sizable improvements across two benchmark parts. The implementation-oriented discussion may be useful to practitioners.

Critical weaknesses
  1. The “zero-downtime” contribution appears to rely primarily on keeping inference service available while deferring training and swapping weights during idle periods. This resembles established asynchronous update and deployment practices; the paper does not establish what is methodologically new or quantify the availability benefit against an appropriate asynchronous baseline.
  2. The claim that no prior system unifies skill evolution and weight optimization appears too broad. Related work on skill-augmented RL, including work cited by the paper, already combines these ingredients. Without a direct comparison or a precise account of what the proposed coordination adds, it is difficult to determine whether MetaClaw is a distinct methodological contribution rather than an implementation of existing ideas in a more elaborate agent system.
Questions for authors
  1. Relative to prior skill-augmented RL systems, what specific mechanism is unique to MetaClaw, and can its benefit be isolated experimentally?
  2. Against a standard asynchronous training-and-hot-swap baseline, what measurable reduction in service interruption does OMLS achieve?
Reviewer_etE7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents MetaClaw, a framework that lets a deployed LLM agent keep improving by evolving both a skill library and the model weights. An LLM skill evolver turns failed trajectories into short behavioral instructions that take effect in the prompt right away, while cloud LoRA fine-tuning with GRPO updates the policy during idle windows detected from sleep hours, keyboard inactivity, and calendar events. A versioning scheme tags each trajectory with its skill generation and drops older samples from the RL buffer whenever the skills change. On MetaClaw-Bench, a new benchmark of 934 questions over 44 simulated workdays, skills improve both GPT-5.2 and Kimi-K2.5, and the full pipeline raises Kimi-K2.5 from 21.4$%$ to 40.6$%$ accuracy on Part I, close to the GPT-5.2 baseline. Skills alone also improve robustness metrics on the AutoResearchClaw research pipeline.

Critical strengths
  • A timely problem with a coherent system design. Agents that keep learning from use are an important open problem. The paper pairs fast prompt-level skill evolution with slow idle-time RL, and it points out that RL samples go stale once the skills change. The system needs no local GPU and the prompts are documented, so it gives a concrete blueprint others can build on.

  • A sequential benchmark with consistent gains. MetaClaw-Bench runs tasks as a stream of workdays with feedback, so it measures learning from accumulated experience. Skills improve both backbones on both parts, and on Part I the full pipeline brings Kimi-K2.5 close to the GPT-5.2 baseline. If the protocol questions below are resolved, this would be a useful testbed.

Critical weaknesses
  • The gains are not attributed to the proposed mechanisms. The only baseline is the base model without adaptation, and none of the memory, skill, or RL methods the paper positions against is compared. SkillRL, which the paper cites, already evolves a skill library together with the policy during RL, yet the paper groups it with methods whose skills are never coordinated with weight optimization. The novelty then rests on the deployment-time parts, the versioning and the scheduler, and neither is tested. Without an RL-only run or a run without versioning, the claimed synergy between the two loops, the harm of stale rewards, and the claim that the agent becomes better at adapting are unsupported. No result reports variance or repeated runs. The AutoResearchClaw skills come from that pipeline's own failures, so that result shows the mechanism applies there rather than cross-domain transfer. Since these components are the contribution, the evidence so far shows only that adaptation beats no adaptation.

  • The RL protocol is unclear, and Figure 2 is hard to reconcile with it. The paper does not say what data the five-day RL run used, which reward model scored the trajectories, or how GRPO groups are formed when each task is served once. In Algorithm 1, only non-failed trajectories enter the RL buffer, and the buffer appears to be emptied at every skill update, which contradicts the method text where the buffer spans several skill generations. Figure 2 shows the full pipeline far above skills-only Kimi from the first days of Part I, although the paper says RL lags skill adaptation by days. In the late days, it falls below skills-only Kimi. The headline Kimi result rests on the RL component, so these gaps decide whether it reflects the online protocol the paper describes.

Questions for authors
  1. RL protocol What data did the five-day RL run use, and did it run before the first simulated workday or during the stream? Were any MetaClaw-Bench tasks trained on before they were scored? Which reward model was used, how were GRPO groups formed, and how many updates were made? Does the buffer keep only non-failed trajectories from the current skill generation, as Algorithm 1 seems to imply?

  2. Controls Could you add an RL-only run, a run without versioning, and a comparison with SkillRL or a simple memory baseline that keeps all past feedback in the prompt? In the Part II example, the feedback states the rule directly, so the memory baseline is a natural control. Please also report variance over repeated runs and name the LLM used as the skill evolver for each backbone. Since Part II introduces its rules on different days, per-rule learning curves before and after RL would directly test whether the agent becomes better at adapting.

  3. Consistency of results Which model and run produced Figure 3, and why do its GPT completion values differ from Table 1? The case studies and the skill analysis for the 30-day part seem to describe the Part II rules, so do both parts use the same rules? How many topics and runs are behind the AutoResearchClaw table, and which backbone was used?

Reviewer_fL2fOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Metaclaw is a deployment-oriented adaptation framework for LLM-based CLI agents. It maintains a base policy and a retrieved library of natural-language skills. An LLM synthesizes skills from failure, while a slower GRPO/LoRA process updates the policy during user-inactive periods. Skill-generation tags are intended to control which trajectories enter RL training.

The authors introduce MetaClaw-Bench, comprising 934 questions across 44 simulated workdays, and report improvements from skills and from skills plus RL, with the full method evaluated only on Kimi-K2.5. A skills-only evaluation on AutoResearchClaw reports improved pipeline-level metrics. The paper positions these components as continual meta-learning and deployment-time evolution.

Critical strengths
  1. Relevant systems problem. Coordinating immediate behavioral corrections with slower parameter updates addresses a practical tesion in deployed agents. The explicit treatment of skill-context versions and training windows makes the proposal concrete and potentially useful.

  2. Promising within-benchmark results. Table1 reports consisent gains in mean question scores for the tested skill-augmented conditions, and larger gains for the full Kimi system: accuracy rised from 21.4% to 40.6% on Part I and from 21.1% to 39.6% on Part II. The automated file-check criterion and disclosed prompts provide usefull evidence beyond illustrative conversations. Theses results motivate further investigation, although they do not isolate the claimed mechanisms.

Critical weaknesses
  1. The central novelty and causal claims are not established. The cited SkillRL already combines failure-derived skills, retrieval, and skill-policy co-evolution during GRPO (Section 3.3; Algorithm 1). Thus, the Introduction's broad claim that no existing system unifies these adaptation mechanisms is not tenable as written. Deployment scheduling and versioning may be differentiators, but neither is isolated experimentally.

Table 1 omits RL-only, fixed-skill + RL, and a close prior skill/memory-learning baseline. The results therefore do not isolate the proposed coordination or demonstrate an improved ability to adapt to unseen tasks. The submission establishes gains over a non-adaptive baseline, but does not establish that the specific coordination mechanisms are responsible for those gains.

  1. The learning procedure and evaluation protocol are not specified sufficiently to validate the main results. Section 3.3 and Equation (4) describe retaining query data from multiple skill generations, whereas Section 3.4 and Algorithm 1 flush all previous generations. These define different training sets, and the paper does not resolve which procedure produced the reported results.

Algorithm 1 routes failures away from RL, without explaining how GRPO groups and rewards are constructed or how policy/context staleness is handled. The PRM identity, key RL/LoRA settings, and train/evaluate chronology of the five-day run are not adequately specified. In particular, the relationship between this training run and the sequential benchmark evaluation is unclear.

Tables 1-2 do not report independent-run counts or uncertainty estimates. No controlled adaptation-speed or old-task-retention evaluation establishes the stronger continual meta- learning claim. OMLS also lacks interruption and overhead measurements despite Section 3.5 acknowledging a brief inference interruption during weight hot-swapping.

Taken together, these are substantive evidential gaps, not merely presentation issues. They prevent a reliable assessment of the claimed learning mechanism, the attribution of the gains, and the deployment guarantees.

Questions for authors
  1. Novelty and attribution. What specific mechanism distinguishes MetaClaw from SkillRL's skill-policy co-evolution? Do the authors have matched-budget results that isolate its benefit over RL-only or fixed- skill RL? Please distinguish evidence already available from experiments proposed for future work.

  2. Data flow and optimization Does the actual implementation retain old-generation query data, as described in Section 3.3, or flush it, as specified in Section 3.4 and Algorithm 1? Does RL exclude all failure trajectories? Please specify the GRPO grouping/resampling procedure, the reward model, policy/context-version handling, and the key RL/LoRA settings.

  3. Evaluation chronology ls each task scored before its own feedback or trajectory can update the evaluated system? How does the five-day RL run align with the simulated benchmark days? Are prior feedback, tool-use budgets, and context constraints matched across conditions?

  4. Reliability and scope How many independent runs and research topics support Tables 1-2, and what variability was observed? Is there evidence on adaptation to held-out tasks or retention of previously learned behavior? In AutoResearchClaw, which skills transfer unchanged from the source setting, and which are newly synthesized from target-domain failures?

  5. Deployment claims What measured service-interruption durations and pause/resume costs support the no-downtime claim, given the inference interruption acknowledged in Section 3.5?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.