- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Harness Continual Learning (HCL), which studies how agents with frozen foundation models can acquire new capabilities by continuously updating their external harnesses while mitigating the degradation of previously learned behaviors. The authors organize the harness into four components: Task Interface, Experience Memory, Capability Map, and Adaptive Router. They further introduce Guarded Harness Evolution, in which a Continual Optimizer proposes updates based on execution feedback, while a Continual Evaluator decides whether to accept them based on current-task performance, historical anchor regressions, and validity constraints. Experiments on ALFWorld, Minecraft, text reasoning, and multimodal perception show that HCL achieves competitive performance in several settings and exhibits a stability–plasticity trade-off under different historical-loss budgets.
Critical strengths
- Practical problem. As agents increasingly rely on external memory, skills, tools, and workflows, preventing regressions caused by harness updates is an important problem.
- Clear framework. The paper separates candidate generation from update acceptance and explicitly considers current-task gains, historical regressions, and validity constraints.
- Clear writing. The paper is generally well-written and easy to follow.
Critical weaknesses
- Insufficient comparison with recent, closely related work on harness evolution. The paper compares HCL with RAG, MemP, and MemRL, but these baselines do not fully cover recent work on self-evolving harnesses. In particular, the paper should better discuss how HCL differs from AHE and Evo-Harness and provide empirical comparisons where possible.
- Inconsistencies in experimental reporting. The main text describes RAG’s 55.56% as the best non-HCL baseline on ALFWorld, whereas Table 1 reports 60.68% for MemRL. In addition, the earlier Stability-HCL results used as the baseline in Appendix E differ from those reported for the same Qwen3.6-27B backbone in Appendix C.3.1. The authors should clarify the sources of these numbers, the experimental settings, and whether the results are directly comparable. The main text’s description of when harness updates are committed also appears inconsistent with the appendix algorithm, which persists each artifact update immediately. Since this could affect update order and subsequent candidate generation, the actual implementation should be clarified.
- Limited evidence of statistical reliability and reproducibility. Although the paper reports results across several experimental settings, it does not systematically provide results from multiple independent runs, standard deviations, or confidence intervals for the main experiments. Such evidence is important for determining whether the reported gains are robust, especially when the improvements over baselines are relatively small.
Questions for authors
See the weaknesses above.
5 comments
W3: Statistical reliability
We repeated the full HCL procedure three times on both the textual and multimodal streams from the same initial harness and task sequence (seeds 11, 22, 33), keeping evaluation sets and decoding settings fixed. Different seeds shuffle the order of adaptation examples within each task, introducing variation into the subsequent harness evolution.
Table R3-3a. Textual results: Mean ± sample SD over three runs.
| Method | MuSiQue ↑ | ProofWriter ↑ | GSM8K ↑ | HotpotQA ↑ | Final Avg. (%) ↑ | Avg.Fgt. (pp) ↓ |
|---|---|---|---|---|---|---|
| MemP | 54.67 ± 3.06 | 91.33 ± 1.15 | 69.33 ± 6.11 | 72.00 ± 0.00 | 71.83 ± 1.04 | 0.44 ± 0.38 |
| MemRL | 58.00 ± 4.00 | 92.00 ± 0.00 | 68.67 ± 5.03 | 72.00 ± 0.00 | 72.67 ± 2.25 | 0.44 ± 0.38 |
| Stability-HCL () | 66.00 ± 0.00 | 94.67 ± 2.31 | ||||
| Plasticity-HCL () | 94.67 ± 2.31 | 78.00 ± 0.00 | 82.50 ± 1.32 | 3.56 ± 1.02 |
Table R3-3b. Multimodal results: Mean ± sample SD over three runs.
| Method | Detection ↑ | Caption ↑ | Grounding ↑ | VQAv2 ↑ | Final Mean. (%) ↑ | Avg.Fgt. (pp) ↓ |
|---|---|---|---|---|---|---|
| MemP | 59.59 ± 1.30 | 29.63 ± 3.20 | 80.84 ± 2.21 | 65.29 ± 1.60 | 0.00 ± 0.00 | |
| MemRL | 60.56 ± 1.08 | 27.89 ± 1.85 | 91.00 ± 0.28 | 64.71 ± 0.81 | 0.00 ± 0.00 | |
| Stability-HCL () | 66.59 ± 1.23 | 35.87 ± 3.48 | 91.26 ± 0.33 | 79.11 ± 0.22 | 68.21 ± 0.71 | |
| Plasticity-HCL () | 90.38 ± 0.22 | 79.37 ± 0.50 | 0.81 ± 0.33 |
The results show limited across-run variation in final performance, with SDs of 0.76 and 1.32 pp on text and 0.71 and 0.10 pp on the multimodal stream, and HCL consistently outperforms MemP and MemRL in FinalAvg across both textual and multimodal streams. These findings provide evidence of reproducibility under both tested settings.
Kind Reminder
Dear Reviewer 92aR,
Thank you again for your constructive feedback. We would appreciate hearing whether our additional experiments and analysis address your concerns. If so, we would be grateful if you could reconsider your assessment. We are happy to clarify any remaining questions.
Best regards, The Authors
To Reviewer 92aR
We sincerely thank you for your constructive comments and for recognizing the importance of harness-level continual learning. We address your concerns below.
W1: Comparison with AHE and Evo-Harness
We agree that the original baselines do not sufficiently cover recent harness-evolution methods. We therefore added six related methods, including AHE and Evo-Harness as you suggested (Table R3-1). Within each stream, all methods use the same frozen model (DeepSeek-V4.1-Flash-1017, Qwen3.6-27B), task order, data splits, and evaluation metrics, while retaining their respective adaptation procedures.
AHE targets coding-agent harnesses, evolving components such as tools, middleware, skills, and sub-agents through observability. Evo-Harness compiles execution experiences into reusable task-specific procedures and cross-task patterns. HCL instead focuses on continual harness adaptation: candidate updates must improve current-task performance, satisfy validity requirements, and limit historical-anchor regression under Bn (Eqs. 9–10).
Table R3-1 shows that Stability-HCL achieves higher final performance and lower forgetting than both AHE and Evo-Harness on both streams. Plasticity-HCL also outperforms both in final performance, although its textual forgetting is higher than AHE’s (2.67 vs. 2.00 pp). Across all six baselines, both HCL variants achieve higher textual FinalAvg, with Stability-HCL matching the lowest forgetting. Stability-HCL also achieves the highest multimodal FinalAvg, while Adaptive Auto-Harness achieves lower forgetting (0.05 vs. 0.22 pp).
Table R3-1. Comparison with related methods on textual and multimodal streams. (In all tables, bold and underlined values indicate the best and second-best distinct results within each column, respectively; ties receive the same formatting. For mean ± SD results, rankings are based on the mean.)
Note: The textual experiments use DeepSeek-V4.1-Flash-1017, an updated version of the original backbone DeepSeek-V4.1-Flash, whose API endpoint is no longer available. The multimodal experiments use Qwen3.6-27B. Due to time constraints, we use training/validation/test splits of 250/50/50 examples per task for the textual experiments and 250/50/250 for the multimodal experiments. The updated backbone and reduced test set make the textual results not directly comparable to those in Table 2 of our paper.
W2: Three reporting inconsistencies
(1) ALFWorld baseline. Thank you for identifying this typo. Table 1 is correct, but Line 318 mistakenly identifies RAG (55.56%) as the best non-HCL baseline. The strongest baseline is MemRL (60.68%). Accordingly, Plasticity-HCL (62.98%) improves over MemRL by 2.30 pp, rather than 7.42 pp; the latter is its improvement over RAG. We will correct Line 318.
(2) Appendix E vs. Appendix C.3.1. We acknowledge that the original manuscript did not clearly distinguish the Qwen3.6-27B configurations. Appendix C.3.1 uses the standard setting, whereas Appendix E explores capability extension with different router prompts, retry limits, and format thresholds (Table R3-2), preventing direct comparison across the two settings. Within Appendix E, the compared runs also differ in memory trajectories, so the gains do not isolate the calculator’s causal contribution. Their results are therefore not directly comparable. We will clarify these configuration differences and the exploratory nature of Appendix E.
Table R3-2. Settings and results of the Qwen3.6-27B runs.
defaultcap_awarecap_aware(3) When updates are committed. The implementation indeed follows Algorithm 1 (Appendix B.2): up to K=3 candidates are evaluated for each artifact through temporary replacement, and the best admissible candidate is committed immediately before optimizing the next artifact. We will correct Sec. 3.3 to match this actual implementation.