acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Agentic Continual Learning as Bayesian Discovery

Abstract

Large language models (LLMs) are increasingly used as agents, yet they rarely turn test-time experience into persistent improvements. We connect agentic continual learning (CL) to Bayesian discovery: both accumulate evidence from sequential interactions into persistent knowledge. With the model frozen, what an agent learns can persist only as text, and we represent it as falsifiable hypotheses about the entities of its environment, whose beliefs each task updates by Bayes’ rule. This view clarifies what a task stream must provide: a refutation corrects rather than interferes, forgetting arises when memory revision discards evidence, and a stream becomes silent when later tasks provide no evidence for or against what was learned earlier. We propose DiscoverCL, a training-free harness that approximates these updates with a research loop of proposing hypotheses, acting, and recording evidence. We further introduce DiscoverScore, which measures how much a stream updates an agent’s beliefs per task. With Claude Sonnet 5 and Opus 5 on six CL-Bench streams, DiscoverCL matches or improves over the no-memory baseline in all 12 settings and is the strongest bounded-memory method in 9 of 12. With Opus 5, it recovers 50–97% of full-context ICL’s gain on Database, Spectrum, Sales, and Codebase while using 2.6–9.6× less memory. DiscoverScore ranges from 0.37 to 3.48 bits per task, showing that the streams differ in how much they allow the agent to learn continually.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
language agent, continual learning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_ySbGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a new perspective that interprets agentic continual learning as Bayesian discovery. From this perspective, the authors argue that learning should be represented as belief updates supported by evidence, refutation should correct existing knowledge, and streams without supporting or contradicting evidence are effectively silent. They propose DiscoverCL, a training-free continual-learning framework that maintains memory through a “propose hypotheses–execute tasks–record evidence” loop, and DiscoverScore, which measures how much each task updates the agent’s existing beliefs.

Critical strengths

A meaningful new perspective. The paper reframes agentic continual learning as Bayesian discovery and uses hypotheses and evidence to explain learning, refutation, and forgetting. This perspective is conceptually clear and provides a degree of novelty.

Critical weaknesses

The method relies on somewhat strong assumptions about how environmental knowledge is represented. DiscoverCL assumes that knowledge can be expressed as independent, falsifiable textual hypotheses and that a single contradiction is sufficient to remove a hypothesis. These assumptions may be restrictive for ambiguous, probabilistic, or interdependent knowledge.

Questions for authors
  1. How robust is DiscoverCL when environmental knowledge is probabilistic, interdependent, or only partially falsified? Is deleting a hypothesis after a single contradiction always justified, or would a graded belief-revision mechanism be more appropriate?

  2. Does DiscoverScore correlate with actual continual-learning gains across different LLM backbones, task streams, and memory systems? How sensitive is it to errors or biases introduced by the proposer and summarizer LLMs?

2 comments

We thank the reviewer for the positive assessment and provide new experiments and clarifications below. Baseline values differ from Table 1 because additional runs have completed under the same configuration; the largest change is Full ICL on Poker, where five runs span 0.54 to 2.36. DiscoverCL is the mean ± SD of three runs; new ablation arms are single runs.

[Graded vs. hard deletion] We tested graded revision directly with two alternatives to the original: removal only after two contradictions, and no removal (Table R1; parentheses give the difference from the original).

Table R1: Deletion settings.

StreamOriginalAfter two contradictionsNo removal
Poker0.473 ± 0.2540.102 (−0.371)0.354 (−0.119)
Database0.388 ± 0.1220.427 (+0.038)0.140 (−0.248)
Spectrum (first 40)0.356 ± 0.0170.357 (+0.001)0.356 (+0.000)
Sales0.833 ± 0.0210.625 (−0.209)0.820 (−0.014)

Waiting for two contradictions does not help: it leaves Database and Spectrum unchanged and hurts Sales and Poker, the stochastic stream where it was expected to help. A higher threshold is therefore not the remedy for probabilistic knowledge, which we will state as a limitation; frequency-type hypotheses are a natural next step.

[Deletion audit] To understand why a higher threshold does not help, we sampled 100 removed hypotheses and asked two model judges whether the agent's observations actually contradicted each one (Table R2; κ = 0.73).

Table R2: Audit of removed hypotheses.

Removed hypotheses#Judged unjustified by both (of decidable)
Marked contradicted before removal2523.8%
Never marked contradicted7589.3%

Across all 239 removals, a hypothesis leaves memory in one of three ways: 41% after being marked contradicted, 18% retired by code after ten tasks without being referenced (these can return), and 41% simply no longer written by the proposer. The paper describes only the first. Only 23.8% of removals that follow a contradiction were judged unjustified, against 89.3% of those that do not, so a higher threshold acts on the path that is the smaller source of error. (Table R4 audits reconstructed labels over all hypotheses, while Table R2 audits actual removals under a stricter criterion, both judges, so the two rates are not directly comparable.) We will document all three paths and list the unprompted removals as a limitation.

[Partially falsified knowledge] In practice a hypothesis is revised far more often than discarded: across 3,227 before-and-after comparisons, 17.8% of hypotheses are rewritten and 7.4% disappear, while 74.9% keep their meaning. The implementation is therefore closer to revision than to strict falsification, and we will describe it that way.

[Independence between hypotheses] Independence is encouraged by anchoring each hypothesis to a distinct entity and is not otherwise checked. It is a modeling assumption of DiscoverScore, so interdependent knowledge is not handled explicitly; we will state this as a limitation.

[DiscoverScore and continual-learning gains] We divide the score by the number of hypotheses kept and compare it with CL gain (DiscoverCL − Base) and Full ICL gain (Full ICL − Base), a reference independent of our method (Table R3).

Table R3: Normalized DiscoverScore and learning gains.

StreamNormalized DiscoverScoreCL gainFull ICL gain
Cohort0.057+0.004+0.037
Database0.077+0.209+0.308
Spectrum0.163+0.196+0.165
Poker0.067+0.124+0.870
Sales0.173+0.046−0.000
Codebase0.086+0.092+0.172

The normalized score does not track learning gains (ρ = +0.14 with CL gain; −0.43 with Full ICL gain) and is close to a plain contradiction count (r = 0.92). We will therefore present DiscoverScore as an exploratory statistic, not as a diagnostic of learnability.

[DiscoverScore across backbones and memory systems] Across runs the score varies moderately (SD 0.017–0.064), and it is lower with Opus 5 than with Sonnet 5 on every stream (by 0.042–0.083), so it characterizes a model–stream pair, as the paper states (lines 315–316). It is defined over DiscoverCL's own hypothesis set and does not apply directly to other memory systems; extending it is future work.

[Sensitivity of DiscoverScore to verdict errors] We sampled 423 verdicts, stratified by stream and type, and had Claude Sonnet 5 and Claude Opus 5 re-judge each independently (Table R4). The summarizer's in-run marks are not retained, so the audit covers verdicts reconstructed after the run (see our response to Reviewer U8id).

Table R4: Verdict audit.

Verdict# SampledJudged correct (of decidable)Inter-judge κ
SUPPORTED17347.0%0.35
CONTRADICTED10530.4%0.63
SILENT14554.5%0.44
All42345.6%0.46

Verdicts are noisy, CONTRADICTED most of all, so the score is sensitive to these errors. The audit is model-judged with moderate agreement, so these rates are estimates. App. A.1 already notes that a false refutation counts as learning; we will expand this into an explicit limitation.

Reviewer_upVSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper reframes continual learning for frozen language agents as the accumulation and revision of beliefs about their environments. Instead of storing unrestricted summaries or workflows, it proposes maintaining falsifiable hypotheses paired with supporting evidence. Subsequent tasks may support, contradict, or remain silent about these hypotheses, distinguishing evidence-based correction from accidental forgetting. The authors introduce DiscoverCL, a training-free framework that adds two model calls around task execution: one to propose or remove hypotheses, and another to summarize observational evidence. Uncontradicted hypotheses are preserved, while contradicted ones are retired. They also propose DiscoverScore, which uses Bayesian surprise—the KL divergence between modeled posterior and prior beliefs—to quantify belief updates across a task stream. Across six CL-Bench streams and two model backbones, DiscoverCL matches or exceeds the no-memory baseline in all twelve settings and is the strongest bounded-memory method in nine. Its average memory footprint is approximately one-tenth that of full-context learning. However, its reliance on LLM judgments, hypothesis-independence assumptions, and deletion after a single contradiction limits robustness, particularly in stochastic environments. DiscoverScore measures belief change rather than correctness or downstream usefulness.

Critical strengths
  1. An insightful problem framing. Limited improvement may reflect either poor learning or a lack of new evidence in later tasks. The concept of “silence” helps distinguish these cases and explain why the same memory method performs differently across task streams.
  2. A clear and auditable memory design. Pairing hypotheses with evidence makes it easier to understand what the agent believes, why it believes it, and why a belief is removed. The principle that “not mentioned again” does not mean “refuted” is particularly useful.
  3. A practical connection between theory and implementation. Support, contradiction, and silence lead to distinct update behaviors, while hypothesis management and evidence collection are handled separately. Although the implementation only approximates Bayesian updating, the theory meaningfully guides the design.
  4. Experiments reveal both strengths and limitations. Across six task streams and two models, the method matches or exceeds the no-memory baseline in all settings and leads the compared bounded-memory methods in most. Its weaker performance on poker also clarifies its limits: the approach handles testable environmental regularities better than probabilistic patterns in stochastic behavior.
Critical weaknesses
  1. The metric cannot distinguish useful learning from repeated mistakes. DiscoverScore measures changes in beliefs. An agent that repeatedly makes incorrect judgments, deletes correct knowledge, and later relearns it can also accumulate a high score. The metric therefore tracks belief updates, but does not establish whether tasks help the agent accumulate useful knowledge. The paper’s interpretation of the metric is stronger than the evidence supports.

  2. Deleting a hypothesis after a single contradiction is too strict. Real-world patterns often have exceptions. For example, “this opponent usually plays conservatively” should not be discarded after one aggressive move. This rule is better suited to deterministic regularities than probabilistic patterns or noisy observations. The weaker results on poker are consistent with this limitation.

  3. The experiments do not adequately rule out the benefit of larger memory. The methods use the same tasks and task order, which is appropriate, but their actual memory sizes differ. For example, on Database with Sonnet, DiscoverCL uses approximately 11.9K characters, compared with 2.4K for ReasoningBank. Without comparisons under matched memory budgets, it is difficult to separate the benefits of the proposed design from those of retaining more information.

  4. The ablations do not directly test the core design choices. Showing that revision helps does not establish that hypotheses should be deleted only when contradicted. Key comparisons under matched budgets are also missing: does hypothesis–evidence memory outperform ordinary summaries? Does managing hypotheses and evidence separately outperform updating them together? The overall system is effective, but the contribution of its specific design choices remains unclear.

Questions for authors
  1. How can new hypotheses be merged while existing hypotheses remain unchanged?
    Appendix B.3 instructs the proposer to merge new and existing hypotheses about the same entity, while also requiring uncontradicted hypotheses to be preserved verbatim.

    For example, suppose the agent initially learns that “product prices are in US dollars” and later discovers that “prices exclude tax.” Is this new information added to the hypothesis, stored only as evidence, or left out for now? A worked example would clarify how this update is handled.

  2. Which operations are disabled in the “no revision” ablation?
    Section 10 compares with revision and no revision, but does not clearly specify whether:

    • Only the deletion of contradicted hypotheses is disabled.
    • New hypotheses can still be added.
    • Evidence continues to be updated and contradictions recorded.

    These settings would support different conclusions. The results show that the ablated version performs worse, but it is unclear exactly which mechanism the experiment tests.

  3. How is memory handled when it exceeds the length limit?
    Appendix B.4 specifies a target evidence budget of 3,000 characters and a hard cap of three times that amount, but does not clearly explain how the limit is enforced.

    Does the model compress the evidence again, is the output truncated, or is information removed according to a priority rule? This matters because the method emphasizes preserving evidence and preventing accidental forgetting. Readers need to know which information is sacrificed when memory capacity is insufficient.

3 comments

We thank the reviewer for the careful and constructive feedback and provide new experiments and clarifications below. Baseline values differ from Table 1 because additional runs have completed under the same configuration. DiscoverCL is the mean ± SD of three runs; new ablation arms are single runs.

[DiscoverScore and learning signals] We agree that DiscoverScore measures belief change, not usefulness, as the Ethics Statement notes. The cycle the reviewer describes does occur: 14.2% of removed hypotheses (34 of 239) later reappear in near-identical wording. The normalized score is also close to a plain contradiction count (r = 0.92). We will therefore present DiscoverScore as an exploratory statistic, reported alongside the contradiction rate and the reappearance rate, not as a measure of useful learning.

[Accuracy of verdicts] To check how often an update is warranted, we sampled 423 verdicts, stratified by stream and type, and had Claude Sonnet 5 and Claude Opus 5 re-judge each independently (Table R1). The summarizer's in-run marks are not retained, so the audit covers verdicts reconstructed after the run (see our response to Reviewer U8id).

Table R1: Verdict audit.

Verdict# SampledJudged correct (of decidable)Inter-judge κ
SUPPORTED17347.0%0.35
CONTRADICTED10530.4%0.63
SILENT14554.5%0.44
All42345.6%0.46

Verdicts are noisy, CONTRADICTED most of all. The audit is model-judged with moderate agreement, so these rates are estimates. App. A.1 already notes that a false refutation counts as learning; we will expand this into an explicit limitation.

[Graded vs. hard deletion] We tested the concern directly with two alternatives to the original: removal only after two contradictions, and no removal (Table R2; parentheses give the difference from the original).

Table R2: Deletion settings.

StreamOriginalAfter two contradictionsNo removal
Poker0.473 ± 0.2540.102 (−0.371)0.354 (−0.119)
Database0.388 ± 0.1220.427 (+0.038)0.140 (−0.248)
Spectrum (first 40)0.356 ± 0.0170.357 (+0.001)0.356 (+0.000)
Sales0.833 ± 0.0210.625 (−0.209)0.820 (−0.014)

Waiting for two contradictions does not help: it leaves Database and Spectrum unchanged and hurts Sales and Poker, the stochastic stream where it was expected to help. Never removing hypotheses hurts only on Database. We agree that single-contradiction removal is unsuited to probabilistic knowledge and will state this as a limitation; a higher threshold is not the remedy.

[Deletion audit] To understand why a higher threshold does not help, we sampled 100 removed hypotheses and asked two model judges whether the agent's observations actually contradicted each one (Table R3; κ = 0.73).

Table R3: Audit of removed hypotheses.

Removed hypotheses#Judged unjustified by both (of decidable)
Marked contradicted before removal2523.8%
Never marked contradicted7589.3%

Across all 239 removals, a hypothesis leaves memory in one of three ways: 41% after being marked contradicted, 18% retired by code after ten tasks without being referenced (these can return), and 41% simply no longer written by the proposer. The paper describes only the first. Only 23.8% of removals that follow a contradiction were judged unjustified, against 89.3% of those that do not, so a higher threshold acts on the path that is the smaller source of error. (Table R1 audits reconstructed labels over all hypotheses, while Table R3 audits actual removals under a stricter criterion, both judges, so the two rates are not directly comparable.) We will document all three paths and list the unprompted removals as a limitation.

[Matched memory budget comparison] To isolate the effect of memory size, we capped DiscoverCL at 3 hypotheses and 500 characters of evidence on Database, Sales, and Codebase, so that its memory never exceeds ReasoningBank's (Table R4; average memory in thousands of characters in parentheses). Spectrum requires tracking 13 concurrent channels, which three hypotheses cannot cover (under that cap it scores 0.256, still above ReasoningBank's 0.217), so there we capped evidence at 1,000 characters with no hypothesis cap, which matches ReasoningBank's size.

Table R4: DiscoverCL with memory capped to baseline size. The Spectrum row uses the first 40 tasks in every cell.

StreamNearest-memory baselineDiscoverCL (capped)DiscoverCL (uncapped)
DatabaseReasoningBank 0.157 (2.4k)0.542 (1.8k)0.388 ± 0.122
Spectrum (first 40)ReasoningBank 0.217 (2.3k)0.343 (2.3k)0.356 ± 0.017
SalesReasoningBank 0.756 (3.4k)0.750 (1.7k)0.833 ± 0.021
CodebaseReasoningBank 0.385 (2.8k)0.400 (1.3k)0.389 ± 0.008

With equal or smaller memory, capped DiscoverCL outperforms ReasoningBank on Database and Spectrum and matches it on Sales and Codebase. On Database, the case the reviewer cites, it scores 0.542 at 1.8k characters against 0.157 at 2.4k; its margin over the uncapped runs is within that stream's noise. Its advantage therefore does not come from memory size alone.

[Hypothesis–evidence memory vs. ordinary summaries] We tested a free-form notes memory with the same two extra calls per task (Table R5; 2.00 vs. 2.02 calls measured).

Table R5: Free-form notes with the same two extra calls.

StreamFree-form, 2 callsDiscoverCL (capped)
Database0.372 (3.2k)0.542 (1.8k)
Spectrum (first 40)0.331 (2.1k)0.343 (2.3k)
Sales0.779 (2.7k)0.750 (1.7k)
Codebase0.447 (3.0k)0.400 (1.3k)

The hypothesis format has a clear advantage on Database, matches free-form notes on Spectrum, and trails on Sales and Codebase, while using less memory on three of four streams. We will add free-form notes to the main table and state that the benefit of the hypothesis format is stream-dependent.

[Separate vs. joint updates] We compared the two-call design with a single call that updates hypotheses and evidence together (Table R6).

Table R6: Separate vs. joint updates.

SettingDatabaseSpectrum (first 40)SalesPoker
Separate updates (ours)0.388 ± 0.1220.356 ± 0.0170.833 ± 0.0210.473 ± 0.254
Joint update (1 call)0.4970.3620.7980.062

The two are comparable on Database, Spectrum, and Sales, and separate updates are much better on Poker. Because the joint arm also uses one call instead of two, the comparison does not isolate separation from call count. We will present separation mainly as a benefit for auditability.

[Contribution of hypotheses and evidence] We also showed the agent only one half of the memory (Table R7).

Table R7: Agent sees only hypotheses or only evidence.

SettingDatabaseSpectrum (first 40)SalesPoker
Hypotheses + evidence (ours)0.388 ± 0.1220.356 ± 0.0170.833 ± 0.0210.473 ± 0.254
Hypotheses only0.1620.4030.7490.107
Evidence only0.3600.3780.8070.930

Showing only the evidence matches or exceeds the full method on all four streams, whereas showing only the hypotheses degrades Database, Sales, and Poker; the evidence document therefore carries most of the benefit. In the evidence-only arm the evidence is still organized per hypothesis; only the agent's view is masked. We will revise the claim accordingly: hypotheses act as scaffolding that structures and audits the evidence, not as the content the agent relies on.

[Merging and verbatim preservation] Merging is a prompt instruction, not an operation in code. In the reviewer's example, "prices are in US dollars" and "prices exclude tax" are two properties of one entity. One rule asks the proposer to merge them and move the distinguishing detail into that entity's evidence entry; another asks it not to combine claims that could be contradicted separately. The outcome therefore rests on the proposer's judgment. We will make the rule explicit in App. B.3 and include this worked example.

[No-revision ablation] "No revision" freezes the hypothesis set after the first task: no new proposals, hence no removals, while evidence updates continue. Freezing lowers the score on Database (0.388 → 0.347), Spectrum (0.356 → 0.329), and Sales (0.833 → 0.574) and leaves Poker unchanged (0.463). Contradiction-triggered deletion is isolated by the no-removal setting in Table R2, where proposals and evidence updates stay enabled. We will specify in Section 10 which operations each ablation disables.

[Memory budget enforcement] The two budgets are enforced differently. The hypothesis budget is a hard limit: the list is cut after the B-th hypothesis, and hypothesis text is never truncated. The evidence budget is soft: the summarizer is told the character limit and rewrites the document to fit, so the model chooses what to drop; the 3× cap only guards against runaway generations. The capped runs in Table R4 instead truncate the evidence at the limit. We will state this in App. B.4 and note that evidence loss under a tight budget is not governed by a priority rule.

Comments on the paper

7 comments

We thank the reviewer for the thorough and constructive feedback and provide new experiments and clarifications below. Baseline values differ from Table 1 because additional runs have completed under the same configuration.

[Bayesian framing] We agree the framing should be stated more carefully. A Bayesian perspective motivates the design, but the implementation performs evidence-grounded revision; the Beta–Bernoulli model is used only for scoring in DiscoverScore, where a Beta over a universal claim is an approximation. We will revise Sections 4–5 to make this distinction explicit.

[Computation of DiscoverScore] The paper states that DiscoverScore is computed from the judgments the summarizer already records (lines 258–260 and 264–265). In practice these marks are overwritten when a contradicted hypothesis is rewritten on the next step, so only 21 remain in the saved memory. We therefore reconstruct the verdicts after the run: a model compares each hypothesis before and after every task and labels it supported, contradicted, or silent. This labelling covers one of the three runs and produces Tables R1 and R2; the stability figures for the other runs come from a simpler rule based on how much the hypothesis text changed. We will document this procedure in the paper.

[Normalization of DiscoverScore] Because the raw score grows with the number of hypotheses kept, we divide it by that number (Table R1).

Table R1: Raw and normalized DiscoverScore.

StreamRaw DiscoverScoreNormalized DiscoverScore
Cohort0.3720.057
Database0.4660.077
Spectrum3.4830.163
Poker0.5420.067
Sales0.6490.173
Codebase0.5070.086

Normalization removes the dependence on hypothesis count (Spearman 0.43 → −0.03).

[DiscoverScore and continual-learning gains] Table R2 compares the normalized score with CL gain (DiscoverCL − Base) and Full ICL gain (Full ICL − Base), a reference independent of our method.

Table R2: Normalized DiscoverScore and learning gains.

StreamNormalized DiscoverScoreCL gainFull ICL gain
Cohort0.057+0.004+0.037
Database0.077+0.209+0.308
Spectrum0.163+0.196+0.165
Poker0.067+0.124+0.870
Sales0.173+0.046−0.000
Codebase0.086+0.092+0.172

The normalized score does not track learning gains (ρ = +0.14 with CL gain; −0.43 with Full ICL gain): Sales has the highest score but no Full ICL gain, and Database has a low score but the largest CL gain. It is also close to a plain contradiction count (r = 0.92). We will therefore present DiscoverScore as an exploratory statistic, not as a diagnostic of learnability.

[Stability across runs and backbones for DiscoverScore] Across runs the score varies moderately (SD 0.017–0.064), and it is lower with Opus 5 than with Sonnet 5 on every stream (by 0.042–0.083). This quantifies the paper's statement that the score characterizes a model–stream pair, not the stream alone (lines 315–316; App. A.2, lines 714–716). We will add these numbers to the paper.

[Accuracy of verdicts] We sampled 423 verdicts, stratified by stream and type, and had Claude Sonnet 5 and Claude Opus 5 re-judge each independently (Table R3). The summarizer's in-run marks are not retained, so the audit covers the reconstructed verdicts.

Table R3: Verdict audit.

Verdict# SampledJudged correct (of decidable)Inter-judge κ
SUPPORTED17347.0%0.35
CONTRADICTED10530.4%0.63
SILENT14554.5%0.44
All42345.6%0.46

Verdicts are noisy, CONTRADICTED most of all. The audit is model-judged with moderate agreement, so these rates are estimates. App. A.1 already notes that a false refutation counts as learning; we will expand this into an explicit limitation.

[Matched memory budget comparison] To isolate the effect of memory size, we capped DiscoverCL at 3 hypotheses and 500 characters of evidence on Database, Sales, and Codebase, so that its memory never exceeds ReasoningBank's (Table R4; single run; average memory in thousands of characters in parentheses). Spectrum requires tracking 13 concurrent channels, which three hypotheses cannot cover (under that cap it scores 0.256, still above ReasoningBank's 0.217), so there we capped evidence at 1,000 characters with no hypothesis cap, which matches ReasoningBank's size.

Table R4: DiscoverCL with memory capped to baseline size. The Spectrum row uses the first 40 tasks in every cell.

StreamNearest-memory baselineDiscoverCL (capped)DiscoverCL (uncapped)
DatabaseReasoningBank 0.157 (2.4k)0.542 (1.8k)0.388 ± 0.122
Spectrum (first 40)ReasoningBank 0.217 (2.3k)0.343 (2.3k)0.356 ± 0.017
SalesReasoningBank 0.756 (3.4k)0.750 (1.7k)0.833 ± 0.021
CodebaseReasoningBank 0.385 (2.8k)0.400 (1.3k)0.389 ± 0.008

With equal or smaller memory, capped DiscoverCL outperforms ReasoningBank on Database and Spectrum and matches it on Sales and Codebase, where the margins (0.006 and 0.015) are within run-to-run variation. Its advantage therefore does not come from memory size alone. The capped Database score exceeding the uncapped runs is within that stream's noise.

[Matched compute budget comparison] We tested a free-form notes memory with the same two extra calls per task (Table R5; 2.00 vs. 2.02 calls measured; single run).

Table R5: Free-form notes with the same two extra calls.

StreamFree-form, 2 callsDiscoverCL (capped)
Database0.372 (3.2k)0.542 (1.8k)
Spectrum (first 40)0.331 (2.1k)0.343 (2.3k)
Sales0.779 (2.7k)0.750 (1.7k)
Codebase0.447 (3.0k)0.400 (1.3k)

The hypothesis format has a clear advantage on Database, matches free-form notes on Spectrum, and trails on Sales and Codebase, while using less memory on three of four streams. We will add free-form notes to the main table and state that the benefit of the hypothesis format is stream-dependent.

[Number of runs and significance] DiscoverCL uses three runs per cell; baselines use three on Cohort, Database, Spectrum, and Poker, and two on Sales and Codebase. All runs use a 40-turn cap. The agent is driven through the Claude Code CLI, which calls Anthropic's hosted API (App. B.2) without exposing temperature, so runs use default sampling. Table R6 reports a paired permutation test over episodes and a two-sided Welch test over run means, against the best baseline per stream.

Table R6: DiscoverCL against the best baseline per stream.

StreamBest baselineDiscoverCLPaired p (episodes)Run-level p
CohortMemP −0.011−0.001 ± 0.0050.1140.277
DatabaseMemP 0.2370.388 ± 0.1220.0010.219
SpectrumMemP 0.2840.415 ± 0.022<0.0010.006
PokerMemP 0.7930.473 ± 0.2540.3080.219
SalesReasoningBank 0.7560.833 ± 0.0210.0060.292
CodebaseReasoningBank 0.3850.389 ± 0.0080.8960.755

DiscoverCL matches or outperforms the best baseline on five of six streams, with episode-level significance on Database, Spectrum, and Sales and run-level significance on Spectrum; the run-level test has little power with two or three runs per arm. Poker is the exception (0.320 below MemP), as the paper notes. We agree that Sonnet Codebase and Opus Sales are ties, and we will replace the 46% average with per-stream absolute gains (median +0.044, range −0.320 to +0.152).

[Ablation subset and hypothesis number] We agree that the hypothesis number differences are too small to support a regularization reading and will present that sweep as inconclusive. We also extended the ablations to Database and the full Poker stream (Table R7). Each arm is a single run; parentheses give the difference from full DiscoverCL and that difference as a multiple of its run-to-run half-range, so values above 1× exceed run-to-run noise.

Table R7: Ablations on four streams (number of tasks in parentheses).

SettingDatabase (40)Spectrum (40)Sales (12)Poker (104)
Full DiscoverCL (3 runs)0.388 ± 0.1220.356 ± 0.0170.833 ± 0.0210.473 ± 0.254
Joint update (1 call)0.497 (+0.108, 0.82×)0.362 (+0.006, 0.32×)0.798 (−0.035, 1.37×)0.062 (−0.410, 1.47×)
Frozen after first task0.347 (−0.042, 0.31×)0.329 (−0.027, 1.36×)0.574 (−0.259, 10.14×)0.463 (−0.009, 0.03×)
Agent sees hypotheses only0.162 (−0.227, 1.71×)0.403 (+0.047, 2.40×)0.749 (−0.085, 3.31×)0.107 (−0.366, 1.31×)
Agent sees evidence only0.360 (−0.028, 0.21×)0.378 (+0.023, 1.16×)0.807 (−0.026, 1.02×)0.930 (+0.457, 1.63×)

These results clarify how the memory is used. Showing the agent only the evidence is within or near run-to-run noise of the full method on Database and Sales and above it on Spectrum and Poker, whereas showing only the hypotheses degrades Database, Sales, and Poker. In the evidence-only arm the evidence is still organized per hypothesis; only the agent's view is masked. We will revise the claim accordingly: hypotheses act as scaffolding that structures and audits the evidence, not as the content the agent relies on. The Poker result is consistent with the paper's note that a stochastic opponent requires frequency information that universal hypotheses do not retain. Freezing the hypothesis set (the "no revision" setting of Fig. 5a) hurts most on Sales, and the joint update, which uses one call instead of two, is comparable on three streams and worse on Poker. Deletion settings are in Table R8.

[Verbatim preservation of hypotheses] Comparing every hypothesis before and after each task (3,227 cases) shows that meaning is unchanged in 74.9%, but only 17.2% are copied word for word; 17.8% are rewritten, and 7.4% disappear, more than half of them (4.3%) without ever being marked contradicted. An unrefuted hypothesis can therefore be forgotten, and we will replace that claim in the paper with these rates.

[Graded vs. hard deletion] We tested the suggested comparison with two alternatives to the original: removal only after two contradictions, and no removal (Table R8; parentheses give the difference from the original).

Table R8: Deletion settings.

StreamOriginalAfter two contradictionsNo removal
Poker0.473 ± 0.2540.102 (−0.371)0.354 (−0.119)
Database0.388 ± 0.1220.427 (+0.038)0.140 (−0.248)
Spectrum (first 40)0.356 ± 0.0170.357 (+0.001)0.356 (+0.000)
Sales0.833 ± 0.0210.625 (−0.209)0.820 (−0.014)

Waiting for two contradictions does not help: it leaves Database and Spectrum unchanged and hurts Sales and Poker. Never removing hypotheses hurts only on Database. We leave frequency-type hypotheses to future work.

[Deletion audit] To understand why a higher threshold does not help, we sampled 100 removed hypotheses and asked two model judges whether the agent's observations actually contradicted each one (Table R9; κ = 0.73).

Table R9: Audit of removed hypotheses.

Removed hypotheses#Judged unjustified by both (of decidable)
Marked contradicted before removal2523.8%
Never marked contradicted7589.3%

Across all 239 removals, a hypothesis leaves memory in one of three ways: 41% after being marked contradicted, 18% retired by code after ten tasks without being referenced (these can return), and 41% simply no longer written by the proposer. The paper describes only the first. Only 23.8% of removals that follow a contradiction were judged unjustified, against 89.3% of those that do not, so a higher threshold acts on the path that is the smaller source of error. (Table R3 audits reconstructed labels over all hypotheses, while Table R9 audits actual removals under a stricter criterion, both judges, so the two rates are not directly comparable.) We will document all three paths and list the unprompted removals as a limitation.

[Independence between hypotheses] Independence is encouraged by anchoring each hypothesis to a distinct entity and is not otherwise checked. It is a modeling assumption of DiscoverScore, and we will state it as such.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.