acceptodds
Under review as a conference paper at ICLR 2027

Harness Continual Learning: Continual Adaptation beyond Model Parameters

Abstract

Continual learning has largely been model-centric, treating model parameters as the state that changes with sequential experience. Modern agents can also adapt through a harness of prompts, memories, tools, skills, and routing rules. Because these contents jointly shape later execution, a harness update can disrupt previously reliable behavior even when the model is frozen. This raises a new question: how can an agent continually improve its state outside the model while retaining behavior acquired earlier? We formulate Harness Continual Learning (HCL), a new continual learning paradigm in which the harness evolves around a frozen foundation model, and define the resulting loss of earlier behavior as harness-level forgetting. We instantiate HCL with four execution-facing components: the Task Interface, Experience Memory, Capability Map, and Adaptive Router. We further introduce guarded harness evolution to separate update generation from state commitment. A Continual Optimizer proposes candidate harnesses from post-execution feedback, and a Continual Evaluator commits the resulting candidate harness only after checking current improvement, historical retention, and validity. Experiments on open-world capability accumulation, textual reasoning, and multimodal perception demonstrate capability accumulation and failure recovery, with relative gains exceeding 10% over corresponding baselines in multiple settings. Additional experiments further examine knowledge transfer through harness evolution. Component ablations assess the contribution of each harness component, while controlled retention sweeps reveal measurable harness-level forgetting and demonstrate a controllable stability–plasticity trade-off.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
continual learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_92aROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Harness Continual Learning (HCL), which studies how agents with frozen foundation models can acquire new capabilities by continuously updating their external harnesses while mitigating the degradation of previously learned behaviors. The authors organize the harness into four components: Task Interface, Experience Memory, Capability Map, and Adaptive Router. They further introduce Guarded Harness Evolution, in which a Continual Optimizer proposes updates based on execution feedback, while a Continual Evaluator decides whether to accept them based on current-task performance, historical anchor regressions, and validity constraints. Experiments on ALFWorld, Minecraft, text reasoning, and multimodal perception show that HCL achieves competitive performance in several settings and exhibits a stability–plasticity trade-off under different historical-loss budgets.

Critical strengths
  1. Practical problem. As agents increasingly rely on external memory, skills, tools, and workflows, preventing regressions caused by harness updates is an important problem.
  2. Clear framework. The paper separates candidate generation from update acceptance and explicitly considers current-task gains, historical regressions, and validity constraints.
  3. Clear writing. The paper is generally well-written and easy to follow.
Critical weaknesses
  1. Insufficient comparison with recent, closely related work on harness evolution. The paper compares HCL with RAG, MemP, and MemRL, but these baselines do not fully cover recent work on self-evolving harnesses. In particular, the paper should better discuss how HCL differs from AHE and Evo-Harness and provide empirical comparisons where possible.
  2. Inconsistencies in experimental reporting. The main text describes RAG’s 55.56% as the best non-HCL baseline on ALFWorld, whereas Table 1 reports 60.68% for MemRL. In addition, the earlier Stability-HCL results used as the baseline in Appendix E differ from those reported for the same Qwen3.6-27B backbone in Appendix C.3.1. The authors should clarify the sources of these numbers, the experimental settings, and whether the results are directly comparable. The main text’s description of when harness updates are committed also appears inconsistent with the appendix algorithm, which persists each artifact update immediately. Since this could affect update order and subsequent candidate generation, the actual implementation should be clarified.
  3. Limited evidence of statistical reliability and reproducibility. Although the paper reports results across several experimental settings, it does not systematically provide results from multiple independent runs, standard deviations, or confidence intervals for the main experiments. Such evidence is important for determining whether the reported gains are robust, especially when the improvements over baselines are relatively small.
Questions for authors

See the weaknesses above.

5 comments
AuthorsOpenReview ↗

To Reviewer 92aR

We sincerely thank you for your constructive comments and for recognizing the importance of harness-level continual learning. We address your concerns below.

W1: Comparison with AHE and Evo-Harness

We agree that the original baselines do not sufficiently cover recent harness-evolution methods. We therefore added six related methods, including AHE and Evo-Harness as you suggested (Table R3-1). Within each stream, all methods use the same frozen model (DeepSeek-V4.1-Flash-1017, Qwen3.6-27B), task order, data splits, and evaluation metrics, while retaining their respective adaptation procedures.

AHE targets coding-agent harnesses, evolving components such as tools, middleware, skills, and sub-agents through observability. Evo-Harness compiles execution experiences into reusable task-specific procedures and cross-task patterns. HCL instead focuses on continual harness adaptation: candidate updates must improve current-task performance, satisfy validity requirements, and limit historical-anchor regression under BnB_n (Eqs. 9–10).

Table R3-1 shows that Stability-HCL achieves higher final performance and lower forgetting than both AHE and Evo-Harness on both streams. Plasticity-HCL also outperforms both in final performance, although its textual forgetting is higher than AHE’s (2.67 vs. 2.00 pp). Across all six baselines, both HCL variants achieve higher textual FinalAvg, with Stability-HCL matching the lowest forgetting. Stability-HCL also achieves the highest multimodal FinalAvg, while Adaptive Auto-Harness achieves lower forgetting (0.05 vs. 0.22 pp).

Table R3-1. Comparison with related methods on textual and multimodal streams. (In all tables, bold and underlined‾\underline{\text{underlined}} values indicate the best and second-best distinct results within each column, respectively; ties receive the same formatting. For mean ± SD results, rankings are based on the mean.)

MethodText Final Avg. (%) ↑Text Avg.Fgt. (pp) ↓MM Final Mean. (%) ↑MM Avg.Fgt. (pp) ↓
ReasoningBank55.007.3358.270.07‾\underline{0.07}
Evo-Harness51.5010.6762.071.28
Meta-Harness61.0024.0067.507.09
Self-Harness61.0027.3353.579.90
AHE69.502.00‾\underline{2.00}66.016.36
Adaptive Auto-Harness74.501.3368.31‾\underline{68.31}0.05
Stability-HCL (b=0b=0)81.00‾\underline{81.00}1.3368.920.22
Plasticity-HCL (b=∞b=\infty)83.502.6767.960.81

Note: The textual experiments use DeepSeek-V4.1-Flash-1017, an updated version of the original backbone DeepSeek-V4.1-Flash, whose API endpoint is no longer available. The multimodal experiments use Qwen3.6-27B. Due to time constraints, we use training/validation/test splits of 250/50/50 examples per task for the textual experiments and 250/50/250 for the multimodal experiments. The updated backbone and reduced test set make the textual results not directly comparable to those in Table 2 of our paper.

W2: Three reporting inconsistencies

(1) ALFWorld baseline. Thank you for identifying this typo. Table 1 is correct, but Line 318 mistakenly identifies RAG (55.56%) as the best non-HCL baseline. The strongest baseline is MemRL (60.68%). Accordingly, Plasticity-HCL (62.98%) improves over MemRL by 2.30 pp, rather than 7.42 pp; the latter is its improvement over RAG. We will correct Line 318.

(2) Appendix E vs. Appendix C.3.1. We acknowledge that the original manuscript did not clearly distinguish the Qwen3.6-27B configurations. Appendix C.3.1 uses the standard setting, whereas Appendix E explores capability extension with different router prompts, retry limits, and format thresholds (Table R3-2), preventing direct comparison across the two settings. Within Appendix E, the compared runs also differ in memory trajectories, so the gains do not isolate the calculator’s causal contribution. Their results are therefore not directly comparable. We will clarify these configuration differences and the exploratory nature of Appendix E.

Table R3-2. Settings and results of the Qwen3.6-27B runs.

RunRouter promptsRetry limitFormat thresholdCalculatorFinal Avg. (%) ↑
App. C.3.1, Stability-HCLdefault101.00Off62.00
App. E, earlier Stability-HCLcap_aware200.85Off62.50‾\underline{62.50}
App. E, capability-aware HCLcap_aware200.85On74.40

(3) When updates are committed. The implementation indeed follows Algorithm 1 (Appendix B.2): up to K=3K=3 candidates are evaluated for each artifact through temporary replacement, and the best admissible candidate is committed immediately before optimizing the next artifact. We will correct Sec. 3.3 to match this actual implementation.

W3: Statistical reliability

We repeated the full HCL procedure three times on both the textual and multimodal streams from the same initial harness and task sequence (seeds 11, 22, 33), keeping evaluation sets and decoding settings fixed. Different seeds shuffle the order of adaptation examples within each task, introducing variation into the subsequent harness evolution.

Table R3-3a. Textual results: Mean ± sample SD over three runs.

MethodMuSiQue ↑ProofWriter ↑GSM8K ↑HotpotQA ↑Final Avg. (%) ↑Avg.Fgt. (pp) ↓
MemP54.67 ± 3.0691.33 ± 1.1569.33 ± 6.1172.00 ± 0.0071.83 ± 1.040.44 ± 0.38
MemRL58.00 ± 4.0092.00 ± 0.0068.67 ± 5.0372.00 ± 0.0072.67 ± 2.250.44 ± 0.38
Stability-HCL (b=0b=0)66.00 ± 0.0094.67 ± 2.3193.33±1.15‾\underline{93.33 \pm 1.15}72.67±1.15‾\underline{72.67 \pm 1.15}81.67±0.76‾\underline{81.67 \pm 0.76}0.67±0.67‾\underline{0.67 \pm 0.67}
Plasticity-HCL (b=∞b=\infty)64.67±4.16‾\underline{64.67 \pm 4.16}92.67±1.15‾\underline{92.67 \pm 1.15}94.67 ± 2.3178.00 ± 0.0082.50 ± 1.323.56 ± 1.02

Table R3-3b. Multimodal results: Mean ± sample SD over three runs.

MethodDetection ↑Caption ↑Grounding ↑VQAv2 ↑Final Mean. (%) ↑Avg.Fgt. (pp) ↓
MemP59.59 ± 1.3029.63 ± 3.2091.10±0.42‾\underline{91.10 \pm 0.42}80.84 ± 2.2165.29 ± 1.600.00 ± 0.00
MemRL60.56 ± 1.0827.89 ± 1.8591.00 ± 0.2879.38±1.19‾\underline{79.38 \pm 1.19}64.71 ± 0.810.00 ± 0.00
Stability-HCL (b=0b=0)66.59 ± 1.2335.87 ± 3.4891.26 ± 0.3379.11 ± 0.2268.21 ± 0.710.22±0.22‾\underline{0.22 \pm 0.22}
Plasticity-HCL (b=∞b=\infty)66.25±2.07‾\underline{66.25 \pm 2.07}35.53±1.75‾\underline{35.53 \pm 1.75}90.38 ± 0.2279.37 ± 0.5067.88±0.10‾\underline{67.88 \pm 0.10}0.81 ± 0.33

The results show limited across-run variation in final performance, with SDs of 0.76 and 1.32 pp on text and 0.71 and 0.10 pp on the multimodal stream, and HCL consistently outperforms MemP and MemRL in FinalAvg across both textual and multimodal streams. These findings provide evidence of reproducibility under both tested settings.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer 92aR,

Thank you again for your constructive feedback. We would appreciate hearing whether our additional experiments and analysis address your concerns. If so, we would be grateful if you could reconsider your assessment. We are happy to clarify any remaining questions.

Best regards, The Authors

Reviewer_2AmaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Harness Continual Learning (HCL), a framework for continually adapting LLM-based agents by updating their external harness rather than the foundation model parameters. HCL models the harness as a persistent state composed of the task interface, experience memory, capability map, and adaptive router, and introduces the notion of harness-level forgetting to describe regressions in previously reliable behavior after harness updates. To mitigate such regressions, the authors propose guarded harness evolution, which evaluates candidate updates based on current-task improvement, historical retention, and validity before committing them. The method is evaluated on interactive, textual, and multimodal task streams, where harness adaptation generally improves final performance, while varying the historical-loss budget changes the observed balance between capability acquisition and retention.

Critical strengths
  1. The paper offers an interesting system-level view of continual learning by treating the external harness, rather than model parameters, as the evolving state.

  2. It explicitly incorporates historical retention into harness adaptation instead of optimizing only for current-task performance.

  3. The framework is evaluated across diverse textual, multimodal, and interactive task streams, demonstrating that frozen foundation models can acquire additional capabilities through harness evolution.

Critical weaknesses
  1. Although HCL is presented as a new continual-learning paradigm, the experiments do not clearly demonstrate an improved stability–plasticity frontier over existing baselines. In several settings, stronger capability acquisition is accompanied by increased forgetting, suggesting that HCL mainly shifts the operating point of the trade-off rather than fundamentally improving it.

  2. I am not fully convinced that “harness-level forgetting” is sufficiently distinguished from generic behavioral regression caused by changes in prompts, memory, or routing, and its measurement reliability is not thoroughly characterized.

  3. The novelty of guarded harness evolution over a simpler regression-based validation-and-rollback scheme is unclear, since comparisons against simpler retention-gating baselines are limited.

Questions for authors
  1. Distinction of harness-level forgetting. What concrete criterion distinguishes harness-level forgetting from generic behavioral regression? Could the authors also specify which experiment in the paper directly validates this distinction?

  2. Whether HCL improves the stability–plasticity trade-off. Is the main claim that HCL actually improves the stability–plasticity frontier over existing baselines, or that it provides an explicit way to choose an operating point through B_n? If possible, could the authors provide a comparison at matched forgetting levels or matched capability-acquisition levels?

  3. Isolating the effect of guarded harness evolution. Do the authors have results comparing guarded harness evolution against a simple historical validation-and-rollback baseline? If not, how can we determine which part of the observed gain comes specifically from the proposed gating mechanism?

  4. Reproducibility of harness evolution. Have the authors repeated the full HCL procedure multiple times from the same initial harness and task sequence? If so, could they report the run-to-run variance in final performance, forgetting, candidate acceptance, or committed updates? It would also be helpful to clarify the decoding and seed settings for candidate generation and for experiments other than the textual main setting.

  5. Validation of LLM-based retention evaluation in interactive tasks. In ALFWorld and Minecraft, candidate retention is judged by an LLM-based evaluator rather than by actual historical replay. Has this evaluator been validated against real replay outcomes or human judgments? If so, could the authors report its agreement or accuracy?

  6. Sensitivity to anchor selection. Have the authors compared the LLM-selected anchors against random or uniform anchor selection, or evaluated different anchor-set sizes? If not, it is difficult to assess how sensitive the reported retention results are to the anchor-selection strategy and capacity.

  7. Fairness of baseline adaptation capacity. HCL can persistently modify the interface, memory, capabilities, and router, whereas the main baselines appear to have a narrower adaptation space. How do the authors control for differences in adaptation capacity, update budget, and optimization opportunities? Could the authors also explain why more closely related harness-evolution methods such as Self-Harness or Adaptive Auto-Harness were not included?

  8. Numerical inconsistency in the ALFWorld results. The text states that the best non-HCL baseline achieves 55.56%, whereas Table 1 reports 60.68% for MemRL. This would make the gain of Plasticity-HCL 2.30 percentage points rather than 7.42. Could the authors clarify which value is correct?

6 comments
AuthorsOpenReview ↗

To Reviewer 2Ama

We sincerely thank you for your constructive comments. We respond to each point below. Please kindly let us know if any concern remains.

W1 & Q2: Does HCL improve the stability–plasticity frontier, or choose an operating point via BnB_n?

HCL does not claim to universally improve the stability–plasticity frontier over existing baselines. Rather, it makes historical retention an explicit constraint on continual harness adaptation. The budget BnB_n specifies the allowable regressions on historical anchors for each candidate update (Eqs. 9–10), providing an explicit mechanism to select different operating points by adjusting the stability–plasticity balance.

To evaluate whether this mechanism yields more favorable trade-offs, we provide matched comparisons using the additional baselines you suggested (Table R2-1). At the same forgetting level (1.33 pp), Stability-HCL achieves a 6.50 pp higher FinalAvg than Adaptive Auto-Harness. At approximately matched capability acquisition (LearnAvg: 81.50% vs. 81.00%), it reduces forgetting by 26.00 pp compared with Self-Harness. LearnAvg. measures average task performance immediately after learning each task and serves as an indicator of capability acquisition.

Table R2-1. Matched comparisons on the textual stream. (Bold and underlined‾\underline{\text{underlined}} values indicate the best and second-best distinct results within each matched-comparison group, respectively; ties receive the same formatting.)

SettingMethodFinal Avg. (%) ↑Avg.Fgt. (pp) ↓LearnAvg (%) ↑
Matched FgtAdaptive Auto-Harness74.50‾\underline{74.50}1.3374.00‾\underline{74.00}
Stability-HCL (b=0b=0)81.001.3381.50
Matched LearnAvgSelf-Harness61.00‾\underline{61.00}27.33‾\underline{27.33}81.00‾\underline{81.00}
Stability-HCL (b=0b=0)81.001.3381.50

Note: Tables R2-1, R2-2, R2-5, R2-6a, R2-6b, and R2-7 report experiments with seed 11. In tables other than R2-1, bold and underlined values indicate the best and second-best distinct results within each column, respectively; ties receive the same formatting. For mean ± SD results, rankings are based on the mean. The textual results are evaluated via DeepSeek-V4.1-Flash-1017, an updated version of the original backbone whose API endpoint is no longer available. Due to time constraints, we use 250/50/50 examples per task for training/validation/testing, therefore these results are not directly comparable to the test results in Table 2 in our paper.

W2 & Q1: What separates harness-level forgetting from generic regression?

(1) Distinction. Generic behavioral regression refers to performance degradation on evaluated data after system changes, regardless of task history. In contrast, harness-level forgetting specifically refers to performance degradation on previously learned-task data during subsequent harness learning with frozen model parameters. It is the harness counterpart of forgetting in parameter-based CL.

(2) Experimental evidence and measurement reliability. The stage-wise evaluations in Sec. 4.2 and Fig. 5 track earlier-task regression. Table R2-2 provides a concrete validation-set trajectory: during GSM8K adaptation, Plasticity-HCL’s MuSiQue and ProofWriter accuracies fall by 4 and 6 pp, respectively, whereas Stability-HCL’s aggregate scores remain unchanged. With evaluation inputs and rule-based scoring fixed, this illustrates earlier-task regression during subsequent harness adaptation. We will add this trajectory analysis to the revised Appendix D.

Table R2-2. Validation accuracy (%) before and after learning GSM8K.

MethodUpdates committed during GSM8KMuSiQue (before → after)ProofWriter (before → after)GSM8K (after)
Stability-HCL (b=0b=0)1 (Adaptive Router)66 → 6696 → 9692
Plasticity-HCL (b=∞b=\infty)4 (Task Interface, Adaptive Router)68 → 6494 → 8898

W3 & Q3: Novelty and effect of guarded harness evolution

HCL incorporates validation-and-rollback into continual harness learning through a joint acceptance criterion. A candidate must improve current-task performance, keep the loss of previously successful historical anchors within BnB_n, and satisfy validity requirements (Eqs. 9–10).

Table R2-3 compares HCL with simpler rules that check either current-task performance or historical retention. Plasticity-HCL (b=∞b=\infty) achieves higher FinalAvg and lower forgetting than both current-task validation and batch + validation gates. Stability-HCL (b=0b=0) further reduces forgetting from 2.67 to 1.33 pp, with a 2.50 pp decrease in FinalAvg. The history-only gate achieves the lowest forgetting (0.67 pp), but its FinalAvg is clear lower than that of Stability-HCL (i.e., 76.50 vs. 81.00). Stability-HCL (b=0b=0) also reduces forgetting from 5.33 to 1.33 pp compared with the aggregate-history + current-task gate, while maintaining similar FinalAvg. Together, these comparisons support the value of jointly considering acquisition and retention.

Table R2-3. Comparison with current-task and history-only acceptance rules.

"Current-task validation" only requires improvement on current-task validation. "Batch + validation" requires non-decreasing accuracy on both the current batch and validation set, with improvement on at least one. The history-only gate accepts an update whenever overall historical anchor accuracy does not decrease, without requiring current-task improvement or additional validity checks. "Historical aggregate accuracy + current task validation" improvement requires both non-decreasing aggregate historical anchor accuracy and current task validation improvement.

Acceptance ruleFinal Avg. (%) ↑Avg.Fgt. (pp) ↓
Current-task validation only80.504.67
Batch + validation79.506.67
History-only76.500.67
Historical aggregate accuracy + current task validation improvement81.50‾\underline{81.50}5.33
Stability-HCL (b=0b=0)81.001.33‾\underline{1.33}
Plasticity-HCL (b=∞b=\infty)83.502.67

Q4: Reproducibility of harness evolution

As suggested, we repeated the full textual HCL procedure with seeds 11, 22, and 33, starting from the same initial harness. The seeds control the order of adaptation examples within each task, so different seeds capture variation in the resulting harness trajectories under fixed evaluation and decoding settings (for both API-based models and Qwen, we set temperature to 0 and disable thinking). Table R2-4 reports mean ± sample SD across the three runs. Candidate acceptance is the fraction of evaluated candidates passing all applicable gates, while committed updates count the revisions actually applied to the harness.

The low acceptance rates reflect HCL's selective commitment criteria, while accepted candidates across all four learning stages indicate continued adaptation. Across three full HCL runs, performance, forgetting, acceptance rates, and committed-update counts show limited variation (Table R2-4), supporting reproducibility of harness evolution under the tested setting.

Table R2-4. Textual results: mean ± sample SD over three runs with seeds 11, 22, and 33.

MethodFinal Avg. (%) ↑Avg.Fgt. (pp) ↓Candidate acceptance (%)Committed updates (count)
Stability-HCL (b=0b=0)81.67±0.76‾\underline{81.67 \pm 0.76}0.67 ± 0.671.88 ± 0.426.33 ± 1.15
Plasticity-HCL (b=∞b=\infty)82.50 ± 1.323.56±1.02‾\underline{3.56 \pm 1.02}3.19 ± 0.4310.00 ± 1.00

Q5: Validity of the LLM retention judge in ALFWorld and Minecraft

We validate the LLM judge on ALFWorld against strict replay (Table R2-5). Strict replay stores the first four episodes of each task type and re-runs them in the simulator before each update; an update is rejected if the replay success rate drops. Compared with no gate, which commits every proposed update, the LLM judge reduces forgetting from 10.30 to 5.90 pp and improves FinalAvg by 3.00 pp. Strict replay achieves the best retention (1.80 pp) and serves as an upper bound, but it re-executes episodes with the agent and costs about 77× more validation tokens (177.8M vs. 2.3M). The LLM judge is thus a low-cost approximation, and HCL can use strict replay when the verification budget allows.

Table R2-5. Retention gates on ALFWorld.

Retention gateFinal Avg. (%) ↑Avg.Fgt. (pp) ↓Validation tokens per run ↓
No gate60.4010.300
LLM judge (ours)63.40‾\underline{63.40}5.90‾\underline{5.90}2.3M‾\underline{2.3\mathrm{M}}
Strict replay72.401.80177.8M

Q6: Sensitivity to anchor selection and capacity

As suggested, we compare anchor selection and capacity within HCL (Tables R2-6a and R2-6b). For the selection comparison, we varied the anchor-selection strategy while keeping the anchor capacity and other experimental settings fixed. Our default LLM-selected anchors outperform Random (randomly sampled training examples per task) and Uniform (selected at evenly spaced indices) in FinalAvg (82.00% vs. 80.50%) and forgetting (0.67 vs. 4.00 pp) at b=3b=3. The capacity comparison varies the number of anchors allocated per task at b=0b=0. Across the tested capacities, performance and forgetting vary non-monotonically. Five anchors per task yield 75.00% FinalAvg and 8.67 pp forgetting, while 50 yield the lowest forgetting (0.67 pp) with 74.00% FinalAvg. These results suggest that our default capacity of 25 anchors per task offers a reasonable balance between final performance and retention.

Table R2-6a. Anchor selection on the textual stream.

Anchor selectionbbFinal Avg. (%) ↑Avg.Fgt. (pp) ↓
LLM (ours)081.001.33‾\underline{1.33}
LLM (ours)382.00‾\underline{82.00}0.67
Random081.004.67
Random380.504.00
Uniform083.001.33‾\underline{1.33}
Uniform380.504.00

Table R2-6b. Anchor capacity on the textual stream.

Anchors/taskbbFinal Avg. (%) ↑Avg.Fgt. (pp) ↓
5075.008.67
10083.002.67
20080.002.00
25 (ours)081.00‾\underline{81.00}1.33‾\underline{1.33}
40080.502.67
50074.000.67

Q7: Fairness of adaptation capacity; Self-Harness and Adaptive Auto-Harness

We agree that the original baselines have narrower adaptation spaces. We therefore added six related methods, including Self-Harness and Adaptive Auto-Harness, several of which support broader modifications to prompts, tools, skills, and execution policies. Since different methods have different adaptation spaces, we do not enforce uniform controls on adaptation capacity, but retain each method's original adaptation procedure. Within each stream, we use the same frozen model, task order, data splits, and evaluation metrics (Table R2-7).

Table R2-7. Comparison with related methods on the textual and multimodal streams.

MethodText Final Avg. (%) ↑Text Avg.Fgt. (pp) ↓MM Final Mean. (%) ↑MM Avg.Fgt. (pp) ↓
ReasoningBank55.007.3358.270.07‾\underline{0.07}
Evo-Harness51.5010.6762.071.28
Meta-Harness61.0024.0067.507.09
Self-Harness61.0027.3353.579.90
AHE69.502.00‾\underline{2.00}66.016.36
Adaptive Auto-Harness74.501.3368.31‾\underline{68.31}0.05
Stability-HCL (b=0b=0)81.00‾\underline{81.00}1.3368.920.22
Plasticity-HCL (b=∞b=\infty)83.502.6767.960.81

On the textual stream, both HCL variants outperform all six baselines in FinalAvg. Stability-HCL exceeds Adaptive Auto-Harness by 6.50 pp while matching the lowest forgetting (1.33 pp). On the multimodal stream, Stability-HCL achieves the highest FinalAvg (68.92%), although Adaptive Auto-Harness has lower forgetting (0.05 pp vs. 0.22 pp). These comparisons extend the evidence beyond memory-only baselines to methods with broader harness adaptation capabilities.

Q8: ALFWorld numbers (55.56% vs. 60.68%)

Thank you for identifying this typo. Table 1 is correct, but Line 318 mistakenly identifies RAG (55.56%) as the best non-HCL baseline. The strongest baseline is MemRL (60.68%). Accordingly, Plasticity-HCL (62.98%) improves over MemRL by 2.30 pp, rather than 7.42 pp; the latter is its improvement over RAG. We will correct Line 318.

We sincerely thank you again for your helpful comments, which have improved the rigor and clarity of our paper. Please let us know if you have any further questions.

Reviewer_7muHOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies continual learning in the harness evolution around a frozen foundation model, and formulates the resulting loss of earlier behavior as harness forgetting. The authors instantiate the agent harness into four different components, propose a harness optimizer to optimize the harness components from execution level feedback, an evaluator to commit the harness changes by balancing the performance improvement and history retention. The proposed harness continual learning paradigm is evaluated on both long-horizon agentic tasks and knowledge transfer in a streamed setting, and shows improvements compared to baselines including average performance and harness-level forgetting.

Critical strengths
  • This paper formulates the harness-level forgetting during the harness evolution, and defines the harness continual learning to acquire new behaviors while retaining the previously learned knowledge. The harness-level forgetting is overlooked in the existing research in harness evolution. To mitigate the harness-level forgetting, the authors proposed a novel guarded harness evolution algorithm to measure current improvements, historical retention and validity and only commit the harness changes when they are admissible. The authors also conducted stability-plasticity trade-offs in the sequential learning stage.
  • The proposed harness continual learning framework is evaluated on multiple settings including both long-horizon agentic tasks and controlled stream continual learning tasks, and show improvements over the baselines.
Critical weaknesses
  • As there are noises in the harness evolution framework, for example, the evaluator could have noise or variance in multiple runs, it is unclear to me if the proposed harness continual learning framework is robust to those noise. Specifically, in the sequential task streams, would the order of the tasks affect the harness evolution and harness-level forgetting? What if some experiences have conflict knowledge, and how would the method resolve the conflict in order to retain the already learned behaviors?
  • The evaluation results need to include more baselines including those experience learning and harness evolution frameworks to compare against on both average task performance and harness-level forgetting to make the results more complete. For example, the authors could consider including ReasoningBank [1], and Meta-harness [2] in the paper as baselines.

[1] Ouyang, S., Yan, J., Hsu, I., Chen, Y., Jiang, K., Wang, Z., ... & Pfister, T. (2026, April). Reasoningbank: Scaling agent self-evolving with reasoning memory. In International Conference on Learning Representations (Vol. 2026, pp. 94327-94354).

[2] Lee, Y., Nair, R., Zhang, Q., Lee, K., Khattab, O., & Finn, C. (2026). Meta-harness: End-to-end optimization of model harnesses. arXiv preprint arXiv:2603.28052.

Questions for authors
  • In the Continual Optimizer component, the authors designed a sequential component-wise search when several components are selected for evolution and the component-wise search follows a pre-defined order. Could the authors clarify the rationale behind the design, for example, why do the authors only select a component each time and what is the pre-defined order?
  • The detailed prompts used in the paper seem to not included in the paper appendix.
6 comments
AuthorsOpenReview ↗

To Reviewer 7muH

We sincerely thank you for your constructive comments. We respond to each point below. Please kindly let us know if any concern remains.

W1: Robustness to evaluator noise, task order, and conflicting knowledge

(1) Evaluator noise and run-to-run variance. Variability may arise from fluctuations in LLM outputs and from LLM-based selection of historical anchors. In our controlled-stream experiments (Sec. 4.2), rule-based verifiers assign deterministic scores to fixed outputs; thus, evaluation variability can arise from changes in outputs or anchor selection. The seeds control the order of adaptation examples within each task, so different seeds capture variation in the resulting harness trajectories under fixed evaluation and decoding settings. As suggested, we conduct an experiment with three full HCL runs on text with seeds 11, 22, and 33. As shown by the result in Table R1-1a, FinalAvg shows limited variation, with SDs of 0.76 and 1.32 percentage points for Stability-HCL and Plasticity-HCL, respectively. Also, we conduct the experiment of three-run evaluations on the multimodal stream using QWen3.6-27B and the result is reported in Table R1-1b. It also show small FinalAvg SDs of 0.71 and 0.10 percentage points, respectively (Table R1-1b). These results suggest that HCL maintains stable final performance under evaluator noise, with Stability-HCL achieving a favorable balance between final performance and average forgetting.

Table R1-1a. Textual results: Mean ± SD over three runs. (Across all tables, bold and underlined‾\underline{\text{underlined}} values indicate the best and second-best distinct results within each column, respectively; ties receive the same formatting. For mean ± SD results, rankings are based on the mean.)

MethodMuSiQue ↑ProofWriter ↑GSM8K ↑HotpotQA ↑Final Avg. (%) ↑Avg.Fgt. (pp) ↓
MemP54.67 ± 3.0691.33 ± 1.1569.33 ± 6.1172.00 ± 0.0071.83 ± 1.040.44 ± 0.38
MemRL58.00 ± 4.0092.00 ± 0.0068.67 ± 5.0372.00 ± 0.0072.67 ± 2.250.44 ± 0.38
Stability-HCL (b=0b=0)66.00 ± 0.0094.67 ± 2.3193.33±1.15‾\underline{93.33 \pm 1.15}72.67±1.15‾\underline{72.67 \pm 1.15}81.67±0.76‾\underline{81.67 \pm 0.76}0.67±0.67‾\underline{0.67 \pm 0.67}
Plasticity-HCL (b=∞b=\infty)64.67±4.16‾\underline{64.67 \pm 4.16}92.67±1.15‾\underline{92.67 \pm 1.15}94.67 ± 2.3178.00 ± 0.0082.50 ± 1.323.56 ± 1.02

Table R1-1b. Multimodal results: Mean ± SD over three runs.

MethodDetection ↑Caption ↑Grounding ↑VQAv2 ↑Final Mean. (%) ↑Avg.Fgt. (pp) ↓
MemP59.59 ± 1.3029.63 ± 3.2091.10±0.42‾\underline{91.10 \pm 0.42}80.84 ± 2.2165.29 ± 1.600.00 ± 0.00
MemRL60.56 ± 1.0827.89 ± 1.8591.00 ± 0.2879.38±1.19‾\underline{79.38 \pm 1.19}64.71 ± 0.810.00 ± 0.00
Stability-HCL (b=0b=0)66.59 ± 1.2335.87 ± 3.4891.26 ± 0.3379.11 ± 0.2268.21 ± 0.710.22±0.22‾\underline{0.22 \pm 0.22}
Plasticity-HCL (b=∞b=\infty)66.25±2.07‾\underline{66.25 \pm 2.07}35.53±1.75‾\underline{35.53 \pm 1.75}90.38 ± 0.2279.37 ± 0.5067.88±0.10‾\underline{67.88 \pm 0.10}0.81 ± 0.33

Note: The textual experiments use DeepSeek-V4.1-Flash-1017, an updated version of the original backbone DeepSeek-V4.1-Flash, whose API endpoint is no longer available. Due to time constraints, we use training/validation/test splits of 250/50/50 examples per task for the textual experiments and 250/50/250 for the multimodal experiments. The updated backbone and reduced test set make the textual results not directly comparable to those in Table 2 of our paper.

(2) Task order. We evaluated two task orders with retention budgets b=1b=1 and b=3b=3 (seed 11; Tables R1-2a and R1-2b). While task order affects both final performance and forgetting, HCL maintains robust and achieves higher Final Avg. than MemRL and MemP under both orders and both retention budgets, showing consistent performance gains across the tested settings.

Table R1-2a. Task-order sensitivity, O0: MuSiQue → ProofWriter → GSM8K → HotpotQA.

MethodMuSiQue ↑ProofWriter ↑GSM8K ↑HotpotQA ↑Final Avg. (%) ↑Avg.Fgt. (pp) ↓
MemP52.0092.0076.0072.0073.000.67‾\underline{0.67}
MemRL62.00‾\underline{62.00}92.0074.0072.0075.00‾\underline{75.00}0.00
HCL (b=1b=1)62.00‾\underline{62.00}90.00‾\underline{90.00}96.0080.0082.003.33
HCL (b=3b=3)66.0092.0092.00‾\underline{92.00}78.00‾\underline{78.00}82.000.67‾\underline{0.67}

Table R1-2b. Task-order sensitivity, O1: HotpotQA → GSM8K → ProofWriter → MuSiQue.

MethodHotpotQA ↑GSM8K ↑ProofWriter ↑MuSiQue ↑Final Avg. (%) ↑Avg.Fgt. (pp) ↓
MemP76.0066.0094.0056.0073.000.00
MemRL72.00‾\underline{72.00}64.0092.00‾\underline{92.00}54.0070.500.00
HCL (b=1b=1)72.00‾\underline{72.00}96.0094.0072.0083.501.33‾\underline{1.33}
HCL (b=3b=3)72.00‾\underline{72.00}90.00‾\underline{90.00}90.0060.00‾\underline{60.00}78.00‾\underline{78.00}2.00

(3) How does HCL resolve conflicting knowledge. Conflicting knowledge may arise among abstracted experiences accumulated from tasks with different goals. HCL resolves such conflicts through three mechanisms:

  1. The Task Interface identifies the current task's goals and constraints, providing the context for interpreting potentially conflicting experiences.
  2. The Memory component retains raw memories alongside abstracted experiences. The Router uses retrieval and planning to locate relevant examples in raw memory and compose a task-appropriate response, grounding its decisions in concrete examples.
  3. Verification gates evaluate candidate updates against current-task performance, validity requirements, and historical anchors. In particular, BnB_n bounds the allowable regression on historical anchors, constraining changes to previously learned behaviors.

Together, these mechanisms enable HCL to resolve conflicts in the context of the current task, apply task-relevant knowledge, and retain previously learned behaviors within the specified retention budget.

W2: Comparison with ReasoningBank and Meta-Harness

We evaluated ReasoningBank and Meta-Harness using the same frozen backbones (DeepSeek-V4.1-Flash-1017 and Qwen3.6-27B), task orders, data splits, and evaluation metrics as HCL (O0, seed 11). As shown in Tables R1-3 and R1-4, both HCL variants achieve higher FinalAvg than these baselines on both textual and multimodal streams. On the multimodal task stream, although Meta-Harness shows competitive performance, its average forgetting is significantly higher than other methods. The key advantage of HCL lies in its combination of broader harness adaptation and explicit historical-retention constraints. Unlike ReasoningBank, which primarily focuses on experience-memory updates, and Meta-Harness, which lacks explicit constraints on historical-task retention, HCL jointly evolves multiple harness components and evaluates candidate updates against both current-task performance and historical anchors. We will include these baseline comparisons in the revised version.

Table R1-3. Comparison with ReasoningBank and Meta-Harness on the textual task stream.

MethodMuSiQue ↑ProofWriter ↑GSM8K ↑HotpotQA ↑Final Avg. (%) ↑Avg.Fgt. (pp) ↓
ReasoningBank48.0082.0018.0072.00‾\underline{72.00}55.007.33
Meta-Harness14.0088.00‾\underline{88.00}70.0072.00‾\underline{72.00}61.0024.00
Stability-HCL (b=0b=0)66.00‾\underline{66.00}92.0094.00‾\underline{94.00}72.00‾\underline{72.00}81.00‾\underline{81.00}1.33
Plasticity-HCL (b=∞b=\infty)68.0092.0096.0078.0083.502.67‾\underline{2.67}

Table R1-4. Comparison with ReasoningBank and Meta-Harness on the multimodal task stream.

MethodDetection ↑Caption ↑Grounding ↑VQAv2 ↑Final Mean. (%) ↑Avg.Fgt. (pp) ↓
ReasoningBank62.6819.9086.3064.2058.270.07
Meta-Harness66.2132.8686.8084.1367.507.09
Stability-HCL (b=0b=0)65.34‾\underline{65.34}39.4191.6079.3368.920.22‾\underline{0.22}
Plasticity-HCL (b=∞b=\infty)64.1437.31‾\underline{37.31}90.60‾\underline{90.60}79.80‾\underline{79.80}67.96‾\underline{67.96}0.81

Q1: Why search one component at a time, and what is the order?

Design rationale of evolution. Our sequential component-wise search is inspired by block coordinate descent. We treat each harness component as an optimization block, updating one component at a time while keeping the others fixed. Once an update is accepted, it is committed before optimizing the next component. This strategy avoids the combinatorial cost of jointly searching multiple components and allows us to isolate the effect of each update.

We follow the component invocation sequence during HCL inference: Task Interface → Adaptive Router (workflow → memory / capability selection → context organization). We compare this default order with an alternative ordering (Order 2, Table R1-5). Under the same experimental setting (O0, seed 11, b=1b=1), the default order achieves a higher FinalAvg (82.00% vs. 74.50%) and lower Avg.Fgt. (3.33 vs. 4.00 pp). These results indicate that the evolution order affects performance and suggest that our inference-motivated order is a reasonable choice in the tested setting.

Table R1-5. Comparison of different component update orders.

Update orderMuSiQue ↑ProofWriter ↑GSM8K ↑HotpotQA ↑Final Avg. (%) ↑Avg.Fgt. (pp) ↓
Order 1 (Ours)62.0090.0096.0080.0082.003.33
Order 258.00‾\underline{58.00}90.0078.00‾\underline{78.00}72.00‾\underline{72.00}74.50‾\underline{74.50}4.00‾\underline{4.00}

Q2: Detailed prompts

Thank you for pointing this out. We have added Appendix H to the revision. It lists all prompt templates verbatim, with their placeholders and output JSON schemas.

We sincerely thank you again for your helpful comments, which have improved the rigor and clarity of our paper. Please let us know if you have any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.