- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This article presents EvoHarness-RL, which unifies the agent's memory, state tracking, and progress management into a BPE framework, and trains the agent to autonomously use these auxiliary functions through SFT and GRPO. Experiments demonstrate improvements in task success rates and training efficiency across three benchmarks.
Critical strengths
The research question is reasonable and meaningful, and the method is simple and makes sense..
The experiment is complete and convincing, demonstrating the effectiveness of cross-task generalization.
Critical weaknesses
The novelty of the method is relatively limited. There has been a lot of existing work on external memory, in-context tool using, and RL training.
The authors claim to provide a unified Harness framework, but what is unified is mainly the interface, rather than the actual implementation. This may limit the generality of the framework.
Questions for authors
See Weaknesses
3 comments
Official Reply to Reviewer ujQm (Part 2)
W2: Scope and Generality of Harness Unification
W2: The authors claim to provide a unified Harness framework, but what is unified is mainly the interface, rather than the actual implementation. This may limit the generality of the framework.
Response:
Our objective is not to build a fully plug-and-play universal harness system, but to unify the policy-facing harness coordination interface across heterogeneous environments.
Specifically, BPE abstracts heterogeneous underlying harness support into three shared functional roles (Belief, Progress, and Experience), standardizes the interaction protocol between agents and harnesses, and provides a common learning framework for optimizing their coordination. This unification turns harness coordination into a shared, learnable decision problem, enabling us to systematically investigate the complementary roles and co-evolution dynamics of policies and harnesses across heterogeneous environments, including general harness annealing patterns, environment-dependent coordination strategies, and the cross-domain transferability of learned coordination strategies.
We intentionally preserve environment-specific low-level implementations to maintain flexibility and extensibility. Environment adapters allow existing domain-specific and off-the-shelf support tools to be integrated into the shared BPE interface without modifying the coordination protocol. Meanwhile, core implementations, including BPE workspace management, Experience evolution, and the training pipeline, are reused across environments. As quantified in our response to Reviewer X7qy, only 24.1–34.3% of the core harness code needs to be implemented for each environment. This separation enables BPE to accommodate heterogeneous agentic tasks while minimizing duplicated implementation effort.
To further evaluate generality, we additionally instantiate BPE on SWE-bench Verified in our response to Reviewer jhgw, extending our original three benchmarks to challenging coding tasks. EvoHarness-Base improves a closed-source frontier model (Claude) by 4.4 percentage points without fine-tuning, while EvoHarness-RL achieves a 6.2-point improvement over the reported Qwen3-8B baseline, exceeding the GRPO result by 5.0 points. These gains further support BPE's applicability across heterogeneous agentic tasks.
Happy to have further discussion!
Thank you again for recognizing the strengths of our work and for your constructive feedback. We hope our responses address your concerns, and we are happy to discuss if you have any further questions!
Looking forward to your reply : )
Dear Reviewer ujQm,
Thank you for your thoughtful review and positive assessment of our work! We truly appreciate your constructive feedback.
With approximately one day remaining in the discussion period, we wanted to gently follow up and ask whether our responses have adequately addressed your concerns. We have carefully clarified the scope of our work, provided quantitative evidence, and presented new results on an additional agentic task. If any part would benefit from further clarification, we would be glad to elaborate.
We sincerely appreciate the time and effort you have devoted to evaluating our paper, and we look forward to your reply!
With our best wishes,
#26876 Authors
Official Reply to Reviewer ujQm (Part 1)
We sincerely thank the reviewer for recognizing the significance of our research question and the simplicity and soundness of our method. We also appreciate your thoughtful feedback. We have made every effort to address your concerns through point-by-point clarifications. Our responses to the weaknesses are provided below.
W1: Novelty Compared with Existing Work
Response:
We agree that external memory, tool use, and agent RL have received substantial attention in recent research. Much of the existing work emphasizes either harness-side mechanisms, such as designing, retrieving, and evolving external knowledge and reusable skills (e.g., Reflexion [1], ExpeL [2], ACE [3], SkillOS [4]), or policy-side optimization, including RL training with external skills (e.g., SkillRL [5]). However, our work focuses on the coordination between agent policies and their runtime harnesses, a relatively underexplored research direction at their intersection, to systematically investigate their complementary roles and co-evolution dynamics across heterogeneous environments.
To address this, we introduce a unified BPE abstraction and a shared coordination protocol, making harness use an explicit, learnable policy decision. This enables us to systematically investigate and optimize the evolving division of labor between what agents learn internally and what they delegate to external harness support.
Our experiments demonstrate both practical improvements and new insights into policy–harness co-evolution across heterogeneous environments. We observe harness annealing, where policies progressively reduce their reliance on routine harness operations during training, alongside environment-dependent coordination strategies reflecting different demands for external support. Meanwhile, the harness facilitates policy learning by improving RL rollout efficiency, while accumulated experience progressively enhances its support capabilities. Together, these findings reveal how agent policies and external harnesses can complement and adapt to each other throughout learning.
References
[1] Reflexion: Language Agents with Verbal Reinforcement Learning
[2] ExpeL: LLM Agents Are Experiential Learners
[3] Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models
[4] SkillOS: Learning Skill Curation for Self-Evolving Agents
[5] SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning