acceptodds
Under review as a conference paper at ICLR 2027

Recurrent Harness for Agentic Continual Learning

Abstract

Language model agents increasingly operate over long interaction streams, requiring online adaptation to distribution shifts through continual learning. However, existing agent harnesses typically either evolve their states only through forward information flow or rely on additional replay or validation data for harness optimization. To address this, we propose ReHarness, a recurrent harness framework for agentic continual learning that complements forward read and write with backward revision under a strictly online streaming protocol. We ask what information could have been captured differently in the harness state to better inform the current instance in the stream. Specifically, after each instance, we leverage the resulting trajectory as feedback to revise the reflections retrieved for that instance, thereby propagating feedback backward through the textual harness state. Across three continual learning benchmarks and three language model backbones, ReHarness consistently outperforms strong raw-context baselines, memory- and reflection-based methods, and prompt optimization approaches. Further analysis shows that ReHarness produces source-grounded and meaningful revisions, corrects overgeneralized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Harness, Continual Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ePP6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces ReHarness, a framework for agentic continual learning that keeps the underlying LLM fixed while updating external memory. After each task, it uses the observed trajectory and outcome to revise previously retrieved reflections, with revisions constrained by their source trajectories, and stores a new reflection for future use. Experiments across three benchmarks and three LLM backbones demonstrate improved overall performance over memory, reflection, prompt optimization, and raw-context baselines.

Critical strengths
  1. The method is clearly motivated and easy to understand, directly addressing how overgeneralized or outdated reflections can repeatedly misguide an agent.

  2. The empirical evaluation includes diverse baselines, particularly strong raw-context methods, and useful ablations that support the effectiveness of the proposed framework.

Critical weaknesses
  1. The role of downstream feedback is not sufficiently isolated. Current ablation remove the entire reflection operation, it does not establish whether the gains come from feedback-driven correction or simply from giving the model another opportunity to reconsider the original evidence. A control with the same source access and a comparable inference budget, but without downstream feedback, would help distinguish these explanations.

  2. The stability analysis does not directly establish retention. Since ReHarness overwrites earlier reflections, it remains unclear whether these revisions preserve useful knowledge or increasingly tailor memory to the most recent distribution. An A→B→A diagnostic, or evaluation on the earlier distribution after adaptation, would provide more direct evidence for the claimed stability–plasticity balance.

Questions for authors
  1. Revision selectivity. A failure may result from execution errors or environmental randomness rather than an incorrect historical reflection. How does the revision agent avoid damaging valid reflections in such cases? Also, every retrieved reflection is passed to the revision agent, although some may already be correct or may have been retrieved despite having little relevance to the current task. Is there an explicit criterion for deciding whether a reflection needs revision or should remain unchanged?

  2. Retrieval robustness. Retrieval affects both the current decision and the selection of reflections for subsequent revision. The paper does not specify the detail retrieval operation. When retrieval returns many irrelevant or inapplicable reflections, could this both impair the current decision and trigger inappropriate changes to the memory bank?

6 comments
AuthorsOpenReview ↗

Response to Reviewer ePP6 - Part I

We thank the reviewer for the constructive feedback and positive assessment of our work. We address each concern below and will update the manuscript accordingly.


W1: Isolating the contribution of downstream feedback

We conduct three additional ablations on CL-Bench with Claude Opus 4.8 to isolate the contribution of downstream feedback from merely reconsidering the original evidence:

  1. Flagging Ineffective: Flag retrieved reflections as potentially ineffective when the current reward falls below the running average of the task stream, without rewriting them.
  2. Rewriting Without Feedback: Rewrite retrieved reflections using their original source trajectories, without access to the current trajectory as feedback.
  3. Rewriting Differently: Similarly rewrite without downstream feedback, but explicitly instruct the revision agent to produce a different version of each reflection.

We compare these settings against ReHarness and the no-revision baseline. Results are reported as normalized gain:

SettingCohortCodebaseDatabaseSalesSpectrumPokerAverage
No Revision-0.0530.3590.2790.2580.033-0.0120.144
Flagging Ineffective-0.0310.3690.2940.3580.001-0.0270.161
Rewriting Without Feedback-0.0320.3450.2230.3570.1980.0020.182
Rewriting Differently-0.0280.3450.3180.2220.1650.0030.171
ReHarness-0.0190.3780.3610.4080.1620.0090.217

Simply flagging potentially ineffective reflections yields only a modest improvement over no revision (0.161 vs. 0.144). More importantly, both rewriting variants underperform ReHarness (0.182 and 0.171 vs. 0.217), despite revisiting the same source trajectories and performing additional rewriting.

These results suggest that the benefits of backward revision cannot be explained solely by additional reflection or rewriting. Downstream feedback provides a useful signal for correcting previously accumulated experience.


W2: Preserving knowledge under distribution shifts

We agree that an A→B→A evaluation would provide more direct evidence of retention. Our current stability analysis measures transfer at distribution boundaries, rather than explicitly testing whether earlier knowledge remains recoverable after adaptation.

ReHarness prioritizes adapting reflections to the current distribution while retaining source-grounded information from earlier trajectories. Its revision mechanism can also readapt reflections when the environment returns to a previously encountered distribution.

We did not perform retrospective evaluation on previously completed instances because our strictly online protocol processes each instance only once. Replaying previously observed instances after their outcomes have informed memory updates would introduce a different evaluation setting and potential information leakage. However, an A→B→A stream with newly sampled instances from distribution A would be a valid and useful retention diagnostic.

We will distinguish the observed stability gains from direct evidence of long-term knowledge retention in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer ePP6 - Part II

Q1: Revision selectivity and robustness to noisy feedback

We agree that a low reward does not necessarily indicate an incorrect reflection, particularly in stochastic environments such as Exploitable Poker. As discussed in Sec. 4.2, poker provides noisy credit-assignment signals because realized outcomes depend on hidden information and chance. Consequently, ReHarness achieves only marginal gains in this setting.

Rather than applying a fixed reward threshold to determine which reflections should be revised, ReHarness delegates the extent of revision to the revision agent, which independently examines each retrieved reflection using its original source trajectory and the current trajectory as feedback. The revision agent can preserve valid information while refining unsupported hypotheses or overly broad applicability conditions.

To examine whether revisions are selective, we measure character-level edit distance by retrieval rank, averaged across the six CL-Bench tasks:

BackboneRank 1Rank 2Rank 3Rank 4Rank 5
Claude1355.331213.771148.331118.341139.08
GPT1735.281326.081198.281076.951110.99
Gemini1054.801030.45933.38872.63962.04

Revision magnitude generally decreases with retrieval rank, suggesting that the revision agent makes larger changes to more relevant reflections. This is consistent with the intuition that highly relevant reflections offer more opportunities to refine their applicability based on downstream feedback.

This also explains why ReHarness does not require an explicit binary revise-or-retain criterion: the revision agent determines how much each retrieved reflection should change, rather than applying the same modification indiscriminately.


Q2: Retrieval implementation and robustness

As specified in Lines 239–241, ReHarness uses Qwen3-Embedding-8B for retrieval. Given the current query xtx_t, we compute cosine similarity between its embedding and those of previous queries {x}i<t\{x\}_{i<t}, then retrieve the reflections associated with the top-kk most similar queries, where k=5k=5.

We agree that irrelevant or inapplicable retrievals can both misguide the current decision and introduce noise into subsequent revision. ReHarness partially mitigates this through source-grounding constraints and by allowing the revision agent to refine applicability conditions.

However, like other experience-based continual learning methods, ReHarness assumes that the task stream contains reusable structure across instances. When previous experiences are largely unrelated to the current task, retrieval and backward revision become less informative. We will clarify this assumption and the potential sensitivity to retrieval quality.


We thank the reviewer again for the insightful suggestions. We hope our additional ablations and explanations address the main concerns, and we would be grateful if the reviewer could consider raising the score.

AuthorsOpenReview ↗

Gentle Follow-Up on Our Rebuttal

We thank the reviewer again for their time and consideration. As the rebuttal period is approaching its end, we wanted to kindly follow up and ask whether our response has addressed your concerns or if there are any remaining questions we could clarify.

We would greatly appreciate your feedback and consideration of our additional results.

Reviewer_8zg7OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets a problem in agent memory systems, once reflections are written into memory, they are never checked for whether they actually help. To address this, the authors propose ReHarness. After each instance, the k reflections that were retrieved for it are rewritten by a revision agent that uses the instance’s outcome as feedback. The revised reflections then also serve as examples when the new reflection for the current instance is written. Evaluations on three continual-learning benchmarks and three closed frontier models show that ReHarness has the highest average normalized gain and beats the other structured memory methods. Supporting analyses check that revisions stay grounded in their source trajectories, show via counterfactual replay that revision recovers usable information, and decompose the gains into stability and plasticity.

Critical strengths
  1. The idea and method of this paper are very natural. It brings the classic recurrent algorithm idea into the agent harness memory protocol, so that memory is no longer write-once, reflections that turn out to be misleading can be corrected by later feedback. The design is also simple. It adds only one revision step on top of a standard retrieval memory, needs no extra training or validation data.
  2. The experimental setup is solid. Not only is it evaluated across three datasets and three different backbone models, but the authors also run many ablations that show where the gains come from, and the analyses give a complete picture of what ReHarness does.
Critical weaknesses
  1. The revision step costs extra time and tokens on every instance, and I am not sure the gain justifies it. Every reflection gets rewritten each time it is retrieved, so a popular reflection is rewritten many times. Looking at Fig. 6–7, the revised versions are 2–3× longer than the originals, and the added text is mostly hedging (“speculative”, “do not assume”, “may not be”). So I wonder whether reflection length just keeps growing along the stream, which would also inflate the decision agent’s context. The paper does not report how reflection length changes over time, so it is hard to tell if the method stays stable in the long run or slowly turns its memory into a pile of caveats.

  2. Cost and efficiency are a real concern for practicality. In dollar terms ReHarness costs about 3× Top-K Raw (USD 254 vs. USD 85) and is only 19% cheaper than Full Raw (USD 314), for a gain of 0.165 to 0.217 on CL-Bench. Cost is only reported against these two raw baselines, nothing is said about Mem0, RBank, DoT, ACE or GEPA, so the cost-benefit picture across methods is incomplete. More importantly, methods like ACE and GEPA can be optimized in parallel across many trajectories, while ReHarness is strictly sequential: ht+1h_{t+1} depends on $h'_t$, and the authors only consider a single sequential stream. With many agents running against a shared memory, concurrent revisions of the same reflection would conflict, and the paper gives no way to handle this. This limits how far the method can scale beyond the benchmark setting.

Questions for authors
  1. How does the average reflection length change along a stream? Does the decision agent’s context grow with it?
  2. Can ReHarness run with multiple agents updating a shared memory in parallel? If two instances revise the same reflection at the same time, how are the two revisions merged?
  3. Could you report token cost for all baselines, not just the two raw ones?
  4. Why does Full Raw get −0.819 on MemoryArena Search?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 8zg7 - Part I

We thank the reviewer for the constructive feedback and positive assessment of our work. We address each concern below and will update the manuscript accordingly.


W1 & Q1: Reflection length growth and revision stability

We analyze the average reflection length at the beginning and end of each task stream, along with the average length change per revision, across three backbone LLMs on CL-Bench. Length is measured in characters.

BackboneStartEndΔ Length / Revision
Claude2117.75871.5631.5
GPT1833.25866.3557.4
Gemini1276.82495.0153.6

Reflection length indeed increases along the stream, which is expected because backward revision progressively incorporates and clarifies applicability conditions, as specified in our revision prompt in the Appendix.

Meanwhile, Figure 4(a) shows that the magnitude of revision, measured by edit distance, generally decreases as reflections stabilize before a distribution shift, then increases again when outdated reflections require adaptation. Therefore, although reflections become longer, the amount of editing tends to diminish within a stable distribution, suggesting that accumulated experience becomes progressively more stable.


W2 & Q3: Token consumption and efficiency across all baselines

We extend our efficiency analysis to all evaluated baselines, reporting total token consumption (input + output, in millions, excluding hidden reasoning tokens) across the three benchmarks and three backbone LLMs.

CL-Bench (M tokens)

BackboneStatelessMem0RBankDoTACEGEPATopK RawFull RawReHarness
Claude9.0514.5510.2414.8117.5016.0752.66142.1241.57
GPT16.6620.4717.1524.8620.3117.2649.10100.1646.40
Gemini20.6826.9620.9924.0122.4116.2145.50182.9334.29

MemoryBench (M tokens)

BackboneStatelessMem0RBankDoTACEGEPATopK RawFull RawReHarness
Claude0.844.382.075.604.335.985.2416.4125.56
GPT1.104.152.225.663.224.125.6016.4719.00
Gemini0.973.091.785.202.313.425.4516.0116.34

MemoryArena (M tokens)

BackboneStatelessMem0RBankDoTACEGEPATopK RawFull RawReHarness
Claude70.50205.45194.03279.80208.24233.00523.47455.68815.99
GPT330.16388.83498.91761.41530.61490.67846.561,051.461,616.36
Gemini140.27272.00353.12495.49346.88359.48767.27621.53884.65

On CL-Bench, which contains longer, multi-round interactions, ReHarness reduces repeated input context by extracting reusable information into compact reflections. Consequently, it consumes fewer total tokens than Top-K Raw and Full Raw across all three backbones.

In contrast, MemoryBench and MemoryArena involve substantial generation costs, so the additional tokens consumed during reflection and revision outweigh the input-token savings. ReHarness therefore generally consumes more tokens than raw-context baselines in these settings.

We acknowledged this limitation of additional token consumption in our Discussion section. Nevertheless, the consistent performance improvements demonstrate the benefits of investing additional computation in backward revision.

AuthorsOpenReview ↗

Response to Reviewer 8zg7 - Part II

W2 & Q2: Parallel scalability and concurrent memory revisions

We believe ReHarness can be extended to parallel settings, much like ACE and GEPA. In fact, ACE curates a shared playbook after each instance, so concurrent updates may also introduce conflicts. Similarly, prompt optimization methods such as GEPA involve dependencies between successive prompt updates. Thus, concurrent state modification is a broader challenge for stateful agent harnesses rather than a limitation unique to ReHarness.

Importantly, ReHarness revises only the retrieved reflections rather than the entire memory bank, allowing revisions to disjoint reflections to proceed independently.

When two instances retrieve the same reflection, their feedback can be incorporated jointly into the revision agent: Ri′=πR(Ri,τi;τs,τt)R_i' = \pi_R(R_i, \tau_i; \tau_s, \tau_t), where τs\tau_s and τt\tau_t are two subsequent trajectories that both retrieved RiR_i. This extends Eq. (2) by incorporating multiple downstream feedback signals when revising a shared reflection.

Therefore, parallelization is compatible with the ReHarness framework, and we leave it to future work.


Q4: Negative performance of Full Raw on MemoryArena Search

MemoryArena Search imposes a predefined, hard-coded context limit of 130,000 characters. Consequently, the lengthy raw trajectories accumulated by Full Raw and Top-K Raw are frequently truncated, resulting in their unusually poor performance on this task.


We thank the reviewer again for the insightful feedback. We hope these additional results and clarifications address the concerns, and we would be grateful if the reviewer could consider raising the score.

AuthorsOpenReview ↗

Gentle Follow-Up on Our Rebuttal

We thank the reviewer again for their time and consideration. As the rebuttal period is approaching its end, we wanted to kindly follow up and ask whether our response has addressed your concerns or if there are any remaining questions we could clarify.

We would greatly appreciate your feedback and consideration of our additional results.

Reviewer_DE9EOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This is a paper about evaluating how LLM agents can update their own memory under streaming tasks (an online incoming stream of new inputs, where the agent is tasked to finish something). The core contribution is adding a revising agent (or role) which actively uses the current new instance to revise against old “memories” or feedback collected prior. Ie after each instance it goes back to the top-5 reflections that were retrieved for it and rewrites each one using its original source trajectory plus how it did on the new instance, with the rule that any new content has to come from the source trajectory (the new instance is only used as feedback). Then a memory agent writes a new reflection using the revised ones as examples. All three roles use the same backbone and there’s no replay, validation set or gradient update, so it works in a strictly online setting.

Critical strengths

The evaluation is broad: three benchmarks, 16 tasks, three backbones and seven baselines, including raw-context baselines (Top-K Raw and Full Raw) that a lot of memory papers skip. Those raw baselines actually beat most of the structured memory methods here, which the paper says openly. There are also per-task, per-backbone tables in the appendix, and they admit poker as a failure case. This is consequential because even without code, the tables are detailed enough to check where the gains come from, and that’s also how most of my concerns in the weaknesses came up. I think as long as the cost per task is controlled, ReHarness should be solid method improvements.

Critical weaknesses

I’m not sure memory editing is really novel here. Mem0 already has a tool call that can ADD, UPDATE, DELETE or NOOP existing memories, and ACE already does memory attribution, i.e. each time a new instance uses a memory, the agent marks whether that memory helped or hurt. The paper doesn’t mention this, and describes these methods (Section 1) as curation that just keeps the memory bank coherent rather than using feedback from downstream use. So what’s actually new is rewriting the retrieved memory using its source trajectory. But the paper never compares against a simpler version of that (eg just deleting or flagging a memory after it’s used in a failed instance), so I can’t tell if rewriting is what makes the difference. This matters to me because the revising agent is the whole contribution.

Questions for authors

Clarifications:

  1. Why wasn’t supplementary material or code provided? Without it I can’t check how correct the implementation is or whether anything leaks, eg how the revision agent is kept from using the new instance’s answer. Related, what exactly does τ_t contain? The notation says (x_t, y_t, r_t), but the revised reflection in Figure 7 contains “correct 21.99”, so it looks like the trajectories include the gold answer.

  2. Can you report the raw stateless (no harness) scores for each task and backbone, not just normalized gain? The CL-Bench paper reports Sonnet 4.6 with plain ICL at 25.4% normalized gain. Here Full Raw (which is basically ICL) with Opus 4.8 only gets 0.191 on CL-Bench and ReHarness 0.217, so why does Opus 4.8 come out so low in every setting?

  3. A minor one on token cost: the comparison only covers CL-Bench with Claude, one rollout, against Top-K Raw and Full Raw (Appendix A). Can you report it for all methods, benchmarks and backbones, and say whether thinking tokens are counted? In dollars, ReHarness is about 3x Top-K Raw ($254 vs $85), even though the main text calls it token-efficient.

  4. Why do Top-K Raw and Full Raw go so negative on MemoryArena Search, and how is Full Raw handled when the history goes past the context window? Does the Wilcoxon result vs the raw baselines (p_adj 0.039 and 0.043) still hold if you drop Search, and what counts as one observation in the test?

Good to have but minor suggestions:

  1. Did you try a multi-agent setting where some of the components (eg the revision or memory agent) use a weaker or stronger model than the decision agent? Right now all three roles use the same backbone (Section 4.1). This would show whether revision needs a frontier model, and whether a cheaper revision agent could cut the cost.
9 comments
AuthorsOpenReview ↗

Response to Reviewer DE9E - Part I

We thank the reviewer for the constructive feedback. We address each concern below and will update the manuscript accordingly.


W1: Novelty of backward revision compared with Mem0 and ACE

We agree that Mem0 and ACE support memory updates, but their mechanisms differ from the backward revision proposed in ReHarness.

Mem0 supports ADD, UPDATE, DELETE, and NOOP operations when integrating newly extracted information, primarily to consolidate new memories and resolve contradictions with existing entries.

ACE tracks whether individual playbook entries are helpful or harmful and supports incremental updates to maintain and refine its playbook. However, it does not explicitly revisit the original source trajectories of previously retrieved reflections to correct unsupported interpretations or recover missing information based on downstream effectiveness.

In contrast, ReHarness explicitly revises previously retrieved reflections using their original source trajectories and feedback from subsequent instances. Rather than simply consolidating new information or tracking memory effectiveness, our revision agent re-examines what could have been captured differently from earlier experience to better inform future decisions.

We will highlight these distinctions from Mem0 and ACE in the revised manuscript.


W1: Ablations on flagging and rewriting without downstream feedback

We conduct three additional ablations on CL-Bench with Claude Opus 4.8 to isolate the contribution of backward revision:

  1. Flagging Ineffective: Flag retrieved reflections as potentially ineffective when the current reward falls below the running average of the task stream, without rewriting them.
  2. Rewriting Without Feedback: Rewrite retrieved reflections using their original source trajectories, without access to the current trajectory as feedback.
  3. Rewriting Differently: Similarly rewrite without downstream feedback, but explicitly instruct the revision agent to produce a different version of each reflection.

We compare these settings against ReHarness and the no-revision baseline. Results are reported as normalized gain:

SettingCohortCodebaseDatabaseSalesSpectrumPokerAverage
No Revision-0.0530.3590.2790.2580.033-0.0120.144
Flagging Ineffective-0.0310.3690.2940.3580.001-0.0270.161
Rewriting Without Feedback-0.0320.3450.2230.3570.1980.0020.182
Rewriting Differently-0.0280.3450.3180.2220.1650.0030.171
ReHarness-0.0190.3780.3610.4080.1620.0090.217

Simply flagging potentially ineffective reflections yields only a modest improvement over no revision (0.161 vs. 0.144). More importantly, both rewriting variants underperform ReHarness (0.182 and 0.171 vs. 0.217), despite revisiting the same source trajectories and performing additional rewriting.

These results suggest that the benefits of backward revision cannot be explained solely by flagging ineffective memories or reconsidering the original evidence. Downstream feedback provides a useful signal for correcting previously accumulated experience.


Q1: Code availability and implementation reproducibility

Unfortunately, we are unable to release the code due to institutional restrictions on sharing research artifacts. However, we provide the prompts used by all three agents in the Appendix, together with implementation details in Sec. 4.1 to support reproducibility.


Q1: Trajectory contents and potential information leakage

As defined in Sec. 3, τt=(xt,yt,rt)\tau_t=(x_t,y_t,r_t) contains the instance query, agent response, and environment feedback. In CL-Bench, the environment feedback can include the correct answer, such as "21.99" in Figure 7, which comes from the original source trajectory.

Importantly, ReHarness uses the downstream trajectory only as feedback to assess the effectiveness of previously retrieved reflections. Any new content introduced during revision must remain grounded in the original source trajectory, rather than the downstream trajectory. Therefore, including the correct answer from the source trajectory does not constitute information leakage.

Our source-grounding analysis in Table 2 further confirms that over 95% of revised claims remain grounded in their source trajectories, with downstream information leakage limited to 0.8–2.1%.

AuthorsOpenReview ↗

Response to Reviewer DE9E - Part II

Q2: Raw stateless reward for each task and backbone

We report the average stateless reward under each task's native metric, together with the corresponding maximum achievable reward.

CL-Bench

BackboneCohortCodebaseDatabaseSalesSpectrumPoker
Claude0.00410.46450.37170.69900.21960.6633
GPT-0.02330.41050.33670.79490.21962.1667
Gemini-0.01160.04210.09670.83540.21960.5442
max_reward0.16201.00001.00001.00001.00009.4900

MemoryBench

BackboneHB-AcademicHB-CreativeWB-AcademicWB-CreativeWB-Politics
Claude0.65540.64476.28576.73916.5000
GPT0.68090.70697.00007.54357.2273
Gemini0.60540.62896.23816.58706.1818
max_reward1.00001.000010.000010.000010.0000

MemoryArena

BackboneTravelShoppingSearchMathPhysics
Claude0.55020.30000.52490.36780.5343
GPT0.70740.49000.52490.39020.5399
Gemini0.68120.52330.39370.40900.5093
max_reward1.00001.00001.00001.00001.0000

Given these stateless rewards, the raw reward of any method can be recovered from its normalized gain using: rmethod=gnormalized(rmax⁡−rstateless)+rstateless.r_{\mathrm{method}} = g_{\mathrm{normalized}} (r_{\max}-r_{\mathrm{stateless}}) + r_{\mathrm{stateless}}. This allows the raw performance of all methods to be reconstructed from the normalized gains reported in our Appendix.


Q2: Comparison with backbone performance reported in the original CL-Bench paper

We further compare our stateless results against those reported in the original CL-Bench paper:

BackboneCohortCodebaseDatabaseSalesSpectrumPoker
Claude Sonnet 4.60.02890.37110.16330.43150.21952.6392
Claude Opus 4.7-0.00150.46710.15170.36580.21951.3142
Claude Opus 4.8 (Ours)0.00410.46450.37170.69900.21960.6633
GPT-5.40.04970.49740.13830.46640.21961.1133
GPT-5.5 (Ours)-0.02330.41050.33670.79490.21962.1667
Gemini 3 Flash0.00690.40790.08830.43920.21961.6400
Gemini 3.7 Flash (Ours)-0.01160.04210.09670.83540.21960.5442
Gemini 3.1 Pro0.04100.35130.11830.32570.21960.3625
max_reward0.16201.00001.00001.00001.00009.4900

The differences are task-dependent rather than uniformly favoring one model. For example, Claude Opus 4.8 outperforms Opus 4.7 on Database and Sales but underperforms on Poker, where stochastic outcomes introduce substantial variance. Similarly, GPT-5.5 outperforms GPT-5.4 on Database, Sales, and Poker but underperforms on Codebase, while Gemini 3.7 Flash improves over Gemini 3 Flash on Sales but performs worse on Codebase and Poker.

These differences may arise from variations in stream schedules and random seeds across experiments. This is precisely why we report normalized gain relative to the stateless performance of the same backbone under the same seeded schedule, ensuring that each method is evaluated against its corresponding baseline.

Moreover, stronger stateless performance on tasks such as Database and Sales leaves less absolute headroom for further improvement, which can contribute to differences in normalized gain across models and studies.

AuthorsOpenReview ↗

Response to Reviewer DE9E - Part III

Q3: Token consumption across all methods, benchmarks, and backbones

We extend our efficiency analysis to all evaluated baselines, reporting total token consumption (input + output, in millions, excluding hidden reasoning tokens) across the three benchmarks and three backbone LLMs.

CL-Bench (M tokens)

BackboneStatelessMem0RBankDoTACEGEPATopK RawFull RawReHarness
Claude9.0514.5510.2414.8117.5016.0752.66142.1241.57
GPT16.6620.4717.1524.8620.3117.2649.10100.1646.40
Gemini20.6826.9620.9924.0122.4116.2145.50182.9334.29

MemoryBench (M tokens)

BackboneStatelessMem0RBankDoTACEGEPATopK RawFull RawReHarness
Claude0.844.382.075.604.335.985.2416.4125.56
GPT1.104.152.225.663.224.125.6016.4719.00
Gemini0.973.091.785.202.313.425.4516.3416.01

MemoryArena (M tokens)

BackboneStatelessMem0RBankDoTACEGEPATopK RawFull RawReHarness
Claude70.50205.45194.03279.80208.24233.00523.47455.68815.99
GPT330.16388.83498.91761.41530.61490.67846.561,051.461,616.36
Gemini140.27272.00353.12495.49346.88359.48767.27621.53884.65

On CL-Bench, which contains longer, multi-round interactions, ReHarness reduces repeated input context by extracting reusable information into compact reflections. Consequently, it consumes fewer total tokens than Top-K Raw and Full Raw across all three backbones.

In contrast, MemoryBench and MemoryArena involve substantial generation costs, so the additional tokens consumed during reflection and revision outweigh the input-token savings. ReHarness therefore generally consumes more tokens than raw-context baselines in these settings.

We acknowledged this limitation of additional token consumption in our Discussion section. Nevertheless, the consistent performance improvements demonstrate the benefits of investing additional computation in backward revision.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overall Response

We sincerely thank all reviewers for their constructive feedback and thoughtful discussions, and the ACs, SACs, and PCs for their consideration of our submission.


Recap of our contributions:

Our work proposes ReHarness, which introduces an explicit backward feedback path into existing agent harnesses, transforming their predominantly forward information flow into a recurrent learning process. Across three continual learning benchmarks, 16 tasks, and three LLM backbones, ReHarness achieves statistically significant improvements over all seven baselines. Further analyses demonstrate that ReHarness produces source-grounded and meaningful revisions and improves both stability and plasticity under continual distribution shifts.

We are grateful that reviewers recognized the motivation and design of ReHarness (ePP6, 8zg7), the breadth and rigor of our evaluation (ePP6, 8zg7, DE9E), and the value of our analyses and ablations (ePP6, 8zg7).


Addressing concerns:

We have uploaded a revised manuscript with all changes highlighted in blue. The major updates are in Sections 5.4, 5.5, 6, and the Appendix, incorporating additional experiments, analyses, and clarifications from our rebuttal discussions.

  • Reviewer DE9E: We clarified the methodological novelty of backward revision (Sec. 1), added ablations on alternative revision strategies and weaker agents (Sec. 5.5), reported raw stateless rewards and comparisons with the original CL-Bench results (Sec. A.1), and clarified benchmark-dependent token efficiency trade-offs (Secs. 4.2, 6, and A.2).
  • Reviewer 8zg7: We expanded the token consumption analysis across all methods, benchmarks, and backbones (Sec. A.2), compared reflection lengths across methods (Sec. A.4), demonstrated the effectiveness of compact reflections (Secs. 5.5 and A.4), and discussed parallel scalability and concurrent revisions (Sec. 6).
  • Reviewer ePP6: We added ablations isolating the contribution of downstream feedback (Sec. 5.5), evaluated knowledge retention under recurring distribution shifts (Sec. 5.4), and analyzed revision selectivity by retrieval rank (Sec. A.3).

We believe these additions and clarifications address all concerns raised during the review process and further strengthen the methodological and empirical contributions of ReHarness.


We respectfully encourage reviewers to reconsider their ratings in light of the additional experimental results and clarifications, and ask the AC to take these updates and discussions into account in the final decision.

Thank you again for your time and consideration.

Warm regards,
Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.