- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies self-evolution of LLM agents when the backbone is frozen and only a textual skill store can change. When a task fails, the same model acts as a teacher and marks at most three verbatim spans that caused the failure. On interactive tasks the target span is picked by a "stall signature", the most frequent recent command whose observation repeats or is empty.
The model then refills only the masked span, up to three times, and each fill is checked by a local probe (exact match, implicated unit tests, or on ALFWorld an admissible unused interact command). A skill is written only if the probe passes (or, on text and code, if there is a local fix), after names and IDs are scrubbed. At test time a router injects a family skeleton plus one or two matched skills. Experiments use a frozen Qwen3.5-4B on HotpotQA, GSM8K, HumanEval, MBPP, ALFWorld and AppWorld against ten baselines with ablation and cost tables, transfer to four other students, a teacher swap, and zero-shot transfer to four held-out QA sets. The main result is ALFWorld 44.0 vs 31.3 for the best baseline.
Critical strengths
-
The span-level repair idea is simple and useful for long-horizon agents. Instead of regenerating the whole episode after a failure, only the stalled command is masked and refilled, and the stall signature (Eq. 4) is read from the environment observations, so no second judge model is needed. I think this matters because cost is the main problem of reflection-style methods on interactive tasks, and the cost tables support it: on AppWorld eval CAVRA uses 16.04 calls / 60.7k tokens vs 33.32 to 41.48 calls for the baselines in Table 8, and the evolve-side token bill is also the lowest on ALFWorld and AppWorld (Table 9).
-
The ALFWorld gain is large and the protocol around it is mostly clean. Evolve and eval splits are disjoint (Table 1), the first-pass verdict is never rewritten, eval is a single student pass, and the ALFWorld margin (44.0±2.1 vs 31.3) is well outside CAVRA's own std. This is the main reason I think there is something real here, and why I am not at clear reject.
Critical weaknesses
The ALFWorld headline is not attributed to the claimed mechanism. Table 7 reports only 2 accesses on ALFWorld, so at most 2 skills were written, yet the score goes from 29.9 (w/o Evolve) to 44.0. At eval the router also injects a family skeleton, which line 316 says is distilled. In the released code these skeletons are eight hand-written recipes (cavra/skills.py, FAMILY_SKELETON, e.g. "heat it with a microwave, then put it in the asked receptacle"), and nothing distills them. The released ALFWorld evolve split has only pick and look tasks, while 92 of the 134 eval games are heat, cool, clean or pick_two, so on most of the eval set the only family-specific guidance is the hand-written recipe.
The released episode loop also replaces the model's command with an unused interact command and adds a hint when it detects a stall (cavra_bench/envs/alfworld.py), which I could not find in the paper. No ablation removes the skeleton alone, and without baseline code I cannot tell whether baselines get the skeleton or the stall override.
To be fair, w/o RM also gets the skeleton and scores 29.9, so the skeleton alone may not explain the gain, but the gain cannot be attributed without this ablation. Finally, "verified" is not measured: the ALFWorld probe accepts any admissible unused interact command, and in Fig. 5 the accepted fix for "place a newspaper on the couch" is "close drawer 18", with an official score of 0. This matters because the paper's contribution is the remask and gated distillation mechanism, and the biggest number could come from hand-designed components.
Questions for authors
-
The released code hard-codes the family skeletons (cavra/skills.py, FAMILY_SKELETON). Do the Table 2 baselines receive them? Please report CAVRA without the skeleton (store kept), and the strongest baselines with the same skeleton added. If CAVRA without the skeleton still beats the best baseline on ALFWorld by clearly more than its std, and the baselines given the skeleton stay below CAVRA, I would raise my rating to 3.
-
Does the stall override and hint in the released ALFWorld episode loop (cavra_bench/envs/alfworld.py) also apply to the baselines and to the w/o Evolve row?
-
Are the Table 3 ALFWorld rows computed on all 134 eval games? cavra/config.py defines ABLATION_LIMITS with 67 ALFWorld games, all five ALFWorld ablation values are multiples of 1/67, and the first 67 eval games contain no heat, cool or pick_two task.