acceptodds
Under review as a conference paper at ICLR 2027

Who Should Learn from Failure? PlayCode-RSI for Verified Co-Evolution of Code and GUI Agents

Abstract

Code agents can repair interactive software, but some defects only appear during execution. GUI agents provide runtime evidence, yet their failures may reflect their own perception, planning, or control mistakes rather than faulty code. This raises a central question: **who should learn from failure?** We study this question in interactive games, where editable code and repeatable interaction allow us to reproduce similar failures with different causes. Automatically verifiable outcomes then help determine whether the game, the player, or both need to change. We introduce **PlayCode-RSI**, a framework for verified co-evolution of code and GUI agents. Its core mechanism, CARE (Causal Attribution and Routed Evolution), combines visual trajectories, execution feedback, program states, and source code. It selects among five decisions: patch the game, update the player, update both, retry, or make no update. Candidate updates are replayed and must pass source-task, transfer, retention, safety, and cross-partner verification. Only verified code experience and GUI skills are inherited by later generations. We also introduce PlayAdaptBench, with Coding, GUI, and Joint RSI subsets, together with multi-turn vision–action–code training data. Evaluation separates partial progress from strict task completion and tracks both independent and joint capability growth. Experiments show sustained improvements in both agents and transfer to unseen tasks, with fewer incorrect updates and less partner-specific co-adaptation. These findings support failure attribution and verified inheritance as a basis for recursive self-improvement. The code, benchmark, and dataset will be publicly released.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Recursive Self-Improvement, Coding agents, GUI agents, Benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pGytOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how two AI agents: one that edits game code and another that plays the game can learn from failures. Its main idea is to determine whether a failure comes from the game, the player, or both before deciding what to change. The system saves useful experience only after checking that the change works and does not damage previous abilities. The paper introduces a benchmark and reports improvements in task completion and fewer updates to the wrong component.

Critical strengths
  1. The paper addresses a meaningful problem. A failed game attempt does not necessarily mean the game is broken. Distinguishing a software problem from a player mistake helps avoid unnecessary or harmful changes.

  2. The system checks more than whether a fix solves the current task. It also checks whether the improvement works on other tasks and preserves previously successful behavior. This is a sensible way to prevent learning an unreliable shortcut.

  3. Testing agents with partners from different generations is a thoughtful experiment. It helps determine whether each agent becomes more capable or simply gets better at working with one particular teammate.

  4. The reported reduction in wrong-target updates supports the main idea. In Table 5, this rate drops from 25.0% without CARE to 8.3% with CARE, while more previously successful behavior is retained.

Critical weaknesses
  1. The paper does not explain the central decision clearly enough. It lists the evidence CARE examines, but gives limited detail about how that evidence leads to a particular decision. A worked example showing how the system rules out competing explanations would make the method much easier to understand and reproduce.

  2. The training comparison does not clearly separate better training content from more training data. The full model receives additional joint-task examples, but dataset sizes and an equal-data comparison are not provided. Some improvement could therefore come from receiving more relevant practice.

  3. Several important performance differences represent only two additional successful tasks. For example, Table 4 reports 23.3% versus 20.0% on 60 joint tasks: 14 versus 12 successes. Table 5’s comparison with and without CARE similarly gives 19 versus 17 successes. Repeated runs are needed to show whether these differences are reliable.

  4. The separation between training, update verification, and final testing is insufficiently explained. The system uses transfer tasks to decide which experiences to keep, then reports performance on held-out tasks. Readers need a clearer description of how these groups are separated to assess whether the improvements extend to genuinely unseen problems.

Questions for authors
  1. Can the authors show one complete example of how CARE distinguishes a game bug from a player mistake?
  2. How many training examples does each variant receive, and does the advantage remain with equal training budgets?
  3. Do the main improvements hold across repeated runs?
  4. Are the tasks used to approve updates separate from the final evaluation tasks, including their underlying game families?
Reviewer_ZAHAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PlayCode-RSI introduces CARE to route interaction failures to code repair, player-skill updates, both, retry, or no update. After supervised fine-tuning, model weights remain fixed, while experience is inherited through source-task, transfer, retention, safety, and cross-partner verification. PlayAdaptBench contains 100 Coding, 100 GUI, and 60 Joint RSI tasks. Across four online generations, completion rates improve from 40.0%, 26.0%, and 23.3% to 50.0%, 36.0%, and 31.7%, respectively. These results are promising, but evaluation independence, methodological detail, and gains over existing reflection approaches require stronger evidence.

Critical strengths
  1. Clear and practically meaningful problem: Distinguishing software defects, player limitations, and execution failures addresses an important issue beyond task success: deciding which component should change.

  2. Verification targets transferable benefits: The five gates and rollback mechanism aim to prevent shortcuts, forgetting, and partner dependence. Compared with joint updates without CARE, retention rises from 83.3% to 95.0%, while wrong-target updates decrease from 25.0% to 8.3%.

  3. Useful decomposition of capability gains: Separate coding, GUI, and joint metrics, together with process scores, strict completion, and cross-generation pairings, help distinguish individual improvement from partner adaptation.

Critical weaknesses
  1. Insufficient evaluation separation and methodological detail: The independence of experience-collection, update-selection, and final test sets is unclear, leaving adaptive validation-set overfitting unresolved. Essential details of CARE’s probing budget, routing criteria, memory retrieval, verification thresholds, and training-data scale are also missing.

  2. Missing related work weakens the novelty positioning: GameVerse [1] already studies visual failure reflection, while OmniGameArena [2] examines iterative skill updates, memory, rollback, and transfer. Towards Generalist Game Players [3] discusses generalist players and a creator–evolution roadmap. The distinctive contribution should therefore be established around code–player update attribution and bilateral verification, rather than game-based reflection broadly and these related works should be discussed in the manuscript.

  3. Limited statistical support and evaluation coverage: On Joint RSI, full SFT achieves 14/60 versus 12/60 for the GPT baseline, and full CARE achieves 19/60 versus 17/60 without CARE. Both advantages amount to two tasks, without repeated runs or confidence intervals. RETRY and NO_UPDATE lack dedicated task categories, and online evolution spans only four generations.

[1] Zhang et al. GameVerse: Can Vision-Language Models Learn from Video-based Reflection? ICML 2026.

[2] Lin et al. OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics. arxiv 2026.

[3] Zhang et al. Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse. arxiv 2026.

Questions for authors
  1. Are verification and final evaluation strictly separated? Please report pool sizes, overlaps in game families or templates, and whether final test feedback influences updates, rollback, or selection.

  2. How are CARE’s attribution and no-update decisions implemented and validated? Please provide the algorithm, probing budget, ground-truth attribution labels, and verification thresholds, alongside RETRY/NO_UPDATE results on correct programs, valid failures, and transient errors.

  3. What do the new mechanisms contribute beyond existing reflection methods? Can the authors compare GameVerse- [1] and OmniGameArena- [2] style reflection with CARE's player side self-improvement? If better, can they be adapted into budget-matched player baselines, with repeated runs, paired tests, and lineage-aware analysis to establish whether the two-task gains are robust?

  4. How do the 90 auxiliary physics tasks map to the 260-task benchmark? Please clarify their training, validation, and testing roles.

  5. What are the raw counts behind the online metrics? Please report numerators, denominators, and total candidate updates.

[1] Zhang et al. GameVerse: Can Vision-Language Models Learn from Video-based Reflection? ICML 2026.

[2] Lin et al. OmniGameArena: A Unified UE5 Benchmark for VLM Game Agents with Improvement Dynamics. arxiv 2026.

Reviewer_VtGaOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies co-evolution between a code agent and a GUI agent in interactive game environments, centered on the question of whether an observed failure should trigger a code patch, a GUI-policy update, both, or neither. The authors propose PlayCode-RSI with a routing module called CARE, introduce the PlayAdaptBench benchmark and multi-turn vision-action-code data, and evaluate both supervised warm-start performance and online experience inheritance across generations.

Critical strengths
  1. CARE unifies failure attribution, update routing, execution verification, and experience inheritance, with safeguards against erroneous updates and capability degradation.

  2. The paper is generally well written and easy to follow.

Critical weaknesses
  1. The evaluation is limited to browser games, and the Joint RSI subset contains only 60 tasks. RETRY and NO UPDATE are not independently evaluated, leaving generalizability to broader software engineering and GUI environments unclear.

  2. CARE's implementation remains underspecified, particularly regarding evidence integration, failure attribution, and probing strategy selection. Its reliability in complex scenarios involving multiple interacting failure causes is also insufficiently validated.

  3. Key implementation details and data partitioning protocols are missing, including probing budgets, memory retrieval rules, validation criteria, and training sample counts. The independence of the inheritance-validation and final test sets is unclear, making reproducibility and generalization claims difficult to verify.

Questions for authors

See Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.