- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces LEAH, a framework for jointly training LLM agents to execute tasks and revise their runtime guidance. At each environment step, the agent first decides whether to add, edit, or delete an entry in its episode-local memory, skills, or additional instructions, and then takes an environment action conditioned on the revised context. The method uses hierarchical credit assignment, with value estimates before and after revision and separate advantage signals for revision and execution decisions.
The paper evaluates LEAH on ALFWorld, AppWorld, and three selected task categories from MedAgentGym. The results show improvements over execution-only training under a fixed harness and flat joint-training baselines. Ablations and behavioral analyses further examine the effects of joint training and changes in revision quality and subsequent execution during training.
Critical strengths
-
The paper addresses an important and practically relevant interdependence between models and harnesses. Agent performance depends jointly on model capabilities and the surrounding runtime mechanisms. Optimizing only the model or only its guidance may overlook the compatibility between these components. Treating this interdependence as a learning problem is valuable and relevant to the optimization of practical agent systems.
-
The training method reflects the revision–execution structure and is supported by empirical evidence. The value of a guidance revision is realized through subsequent actions, making separate credit assignment for revision and execution a reasonable design. The paper reports gains across multiple interactive environments and includes comparisons with joint-training and flat-training alternatives, supporting its effectiveness within the current textual-harness setting.
Critical weaknesses
-
The scope of the editable harness is narrow, limiting the breadth of the contribution to model–harness co-evolution. The method currently edits textual entries in episode-local memory, skills, and additional instructions. In practice, however, a harness also includes context selection and compression, history retention policies, coordination between a main agent and subagents, tool design, and execution control. These components affect information flow and action organization, beyond changing input text.
-
The experiments focus on a relatively simple ReAct setting and do not sufficiently establish the method’s incremental value within a mature harness like codex, opencode, etc. The paper does not discuss whether LEAH consistently adds value on top of these harnesses. This directly affects the assessment of its applicability to practical agent systems.
Questions for authors
-
Are the LLM’s modifications to the harness in LEAH limited to memory and skills? Does LEAH support broader changes to the harness, such as modifying tools, context management mechanisms, or other more complex components?
-
How would LEAH perform if the simple ReAct-based harness were replaced with a more complex open-source harness, such as Codex or OpenCode? What kinds of modifications would LEAH make to such a harness?
-
How is the training cost for LEAH, On-Demand Revision + GRPO, and ReAct + GRPO? I would like to see the relationship between training cost and improvement for the three.
8 comments
W2. The experiments focus on a relatively simple ReAct setting and do not sufficiently establish the method's incremental value within a mature harness like codex, opencode, etc. The paper does not discuss whether LEAH consistently adds value on top of these harnesses. This directly affects the assessment of its applicability to practical agent systems.
Q2. How would LEAH perform if the simple ReAct-based harness were replaced with a more complex open-source harness, such as Codex or OpenCode? What kinds of modifications would LEAH make to such a harness?
Mature harnesses such as Codex or OpenCode. Thank you. We fully agree that ReAct is a relatively simple setting, and LEAH's practicality would be substantially increased with a more complex harness system. However, due to computational constraints, we were unable to evaluate LEAH inside such harnesses. These harnesses are designed for models with long contexts and advanced abilities (larger sizes), and training under them is infeasible with our resources: our runs used 2k–4k-token contexts on 7B–9B models, while these harnesses assume more capable models with much longer contexts (e.g., OpenCode assumes a 200k-token context window by default for models without a known limit [3]). To address your concern, we give a descriptive example on how LEAH would operate under these complex harnesses.
How LEAH would operate in OpenCode.
- Fixed base harness . OpenCode as distributed: its system prompt, built-in tools, agent loop, and context compaction, together with the repository's own AGENTS.md.
- Editable harness . A set of files owned by the agent:
- an instruction file (e.g.,
.opencode/AGENT.md, included through theinstructionsfield of the configuration) for memory and instructions; - skill files (
.opencode/skills/<id>/SKILL.md), which OpenCode lists at every step and loads on demand; - subagent definitions (
.opencode/agents/<name>.md), each with its own prompt and permissions;
- an instruction file (e.g.,
- Deciding to revise. One can add a custom tool,
revise_harness. At each step, the agent either calls an ordinary tool, which acts in the repository, or callsrevise_harness, which is its decision to revise. The tool then calls the same model with LEAH's revision prompt, which writes one ADD, EDIT, DELETE, or NO_CHANGE on the editable files. The edit is applied, and the next model call runs under the revised harness. This is the same act-or-revise structure as in Fig. 2a. - Training. Each task runs headless (
opencode run), with the policy served by vLLM through OpenCode's OpenAI-compatible provider. A proxy between the two records the tokens and log-probabilities of every model call, as in recent systems (LEGO-RL [1], Polar [2]). Calls that decide to revise and calls that write the revision are credited with ; and execution calls are credited with . The reward is whether the task's tests pass.
Q3. How is the training cost for LEAH, On-Demand Revision + GRPO, and ReAct + GRPO? I would like to see the relationship between training cost and improvement for the three.
Training cost. Thank you for raising this point. We acknowledge that LEAH costs more to train than the GRPO baselines, since it involves an additional critic model. However, the comparison with flat PPO in the table below suggests that LEAH's gains do not come from adding a critic. Flat PPO trains the same on-demand revision agent with the same critic and rollout budget, but without hierarchical credit assignment, and reaches only 89.29% on ALFWorld, on par with the GRPO baselines (88.57% and 90.00%). LEAH, at almost the same cost as flat PPO, reaches 96.43%. The gain therefore comes from hierarchical credit assignment, not from the critic itself.
Specifically, on the same hardware (a single node with 4 H100 GPUs) and with the same 128 trajectories per iteration and 150 iterations for all methods, a LEAH run takes 1.33× the training time of ReAct + GRPO and 1.23× that of On-Demand Revision + GRPO. The difference comes mainly from the critic, which has the policy's size in the main LEAH run and could be smaller. A 9B critic also adds about 20 GB of peak memory per GPU.
| Method | Training time per run (h) | ALFWorld success (%) |
|---|---|---|
| ReAct + GRPO | 19.26 | 88.57 |
| On-Demand Revision + GRPO | 20.82 | 90.00 |
| On-Demand Revision + PPO | 24.75 | 89.29 |
| LEAH, Qwen3.5-9B critic | 25.59 | 96.43 |
| LEAH, Qwen3.5-4B critic | 24.23 | 95.00 |
| LEAH, Qwen3.5-2B critic | 23.91 | 93.57 |
Relative to ReAct + GRPO, On-Demand Revision + GRPO gains 1.43 points for 8% more training time and flat PPO 0.72 points for 29% more, while LEAH gains 7.86 points for 33% more. LEAH costs almost the same as flat PPO (25.59 vs. 24.75 hours, 3.4% more): it adds only a second scalar value head, and value evaluations at the post-revision point on the few steps with a revision. With a 2B critic, LEAH trains in 23.91 hours, less than flat PPO, and still reaches 93.57%, above all baselines. Notably, with the 9B critic, LEAH already reaches 92.86% at around iteration 115, when it has used about the same training time as a full ReAct + GRPO run (19.26 hours), surpassing the best baseline performance.
[1] Du, Yiming, et al. "LEGO-RL: Harness-native reinforcement learning for coding agents." arXiv preprint arXiv:2608.17393 (2026).
[2] Xu, Binfeng, et al. "Polar: Agentic RL on Any Harness at Scale." arXiv preprint arXiv:2605.24220 (2026).
[3] OpenCode documentation, "Models." https://opencode.ai/docs/models (accessed November 2026).
Thank you for the careful review and constructive feedback. Below we address your concerns regarding the scope of editable harness, applying LEAH to mature harnesses such as OpenCode and Codex, as well as the training cost.
Scope of the editable harness. Thank you for raising this point. In principle, yes: LEAH can be applied to other harness components such as tools, context management, and subagent coordination and is not tied to text. In our main experiments, we focused on the setting where revisions are limited to three kinds of textual entries as a first study, and also due to the limitation of compute resources. The formulation itself does not depend on the component. Sec. 3 requires only an editable state ct, a deterministic update operator U(ct,zt), and a composition Ht=Hbase⊕ct. Any harness component that fits this form, such as a history-retention setting, a context-compression rule, or the set of enabled tools, uses the same revision–execution process and the same credit assignment. Extending LEAH to such components only changes the space of revisions Z (e.g., discrete settings or code rather than text), and does not change the credit assignment of LEAH.
A preliminary experiment with a non-text harness component. To demonstrate LEAH's feasibility on non-text harness components, we conducted a preliminary experiment, where the agent is allowed to edit the length of its own history window as a revision to its harness. Specifically, we add an editable runtime setting for history length, which allows the agent to change the number of previous action steps retained in its own context (choose from 1 to 10).
LEAH remains effective when its editable harness includes a non-text component. With an editable history length, it reaches 95.71%, on par with LEAH under a fixed window (96.43%) and above ReAct + GRPO with any fixed history length (at most 90.00%). Longer fixed histories help ReAct + GRPO only marginally (88.57% at 2 steps, 90.00% at 5, 89.29% at 10), so LEAH's advantage does not come from seeing more history. In this experiment, we observe that the agent decides to edit the history length in about 6% of all revisions, and these decisions help the agent to either locate previous useful information or reduce the distraction from outdated progress.