- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Evolving Belief Memory (EBM), a non-parametric continual-learning framework for language agents solving successive tasks in environments with shared latent structure. EBM stores reusable environmental propositions together with their supporting evidence and current status, including hypothesized, supported, and contradicted. Before each task, a proposer suggests additions or revisions, a validator assesses the associated evidence, and a solver uses the updated memory and interaction history to perform the task. The evaluation covers adapted Database and Poker environments across three backbone models, supplemented by persistent-state replay, frozen-checkpoint experiments, and component ablations. EBM achieves the highest mean Database reward among the compared non-oracle methods, while Poker results exhibit greater variability.
Critical strengths
The replay experiments provide meaningful evidence that the accumulated memory captures reusable knowledge.
The evaluation examines the usefulness of the retained state beyond aggregate task performance. In the persistent-state replay experiment, previously learned memory is supplied to a fresh solver without raw cross-task interaction history. The frozen-checkpoint experiment further evaluates later tasks without additional belief updates (Section 4.2, Figures 3–4). These experiments support the claim that information acquired from earlier tasks can be retained and reused to improve subsequent task solving. This is a relevant strength for the paper’s continual-learning setting and provides concrete evidence for the practical value of maintaining persistent environmental knowledge.
Critical weaknesses
The distinctive research contribution of evolving belief states is not sufficiently established beyond the effectiveness of the overall framework.
The paper presents a coherent system design combining proposition generation, evidence validation, and persistent memory. Its most distinctive motivation is that unresolved propositions should be retained across tasks and revisited as further evidence becomes available. However, the experiments do not directly establish the independent value of this mechanism. Removing the entire belief state or validator (Figure 5a) does not isolate the effects of retaining unresolved propositions, exposing status labels, or applying the proposed update rules. Likewise, the replay experiments demonstrate that the retained information is useful, but do not establish which benefits specifically require an evolving belief representation. The manuscript acknowledges that these analyses do not isolate the contribution of status labels and that the history-window results cannot distinguish belief-state tracking from other forms of history compression (Section 4.2; Appendix A.4, Figure 7).
Additional inference computation also remains an unresolved factor, as the comparisons are not compute-matched (Section 4.2; Appendix A.4.1). Consequently, the results remain compatible with explanations based on effective information consolidation and additional reasoning or verification. The framework appears practically useful, but the current evidence provides limited insight into when or why explicit belief-state evolution is needed for continual learning. This gap between the distinctive motivation and its experimental validation limits the demonstrated significance of the contribution and is the primary reason for my recommendation.
Questions for authors
- Request for targeted experimental evidence: the role of unresolved propositions and status labels.
On the existing Database setting, how does performance change when HYPOTHESIZED propositions are not retained across tasks, while the remaining proposal and validation process is preserved? Separately, how does replay performance change when the same saved propositions and evidence are supplied without their status labels? These controls would help distinguish the value of preserving unresolved knowledge from the solver’s use of explicit status information.
- Request for targeted experimental evidence: the contribution under comparable budgets.
Does EBM retain an advantage over a conventional summary or proposition-list memory when both use the same proposal and validation procedure with comparable inference and memory budgets? Such a comparison would help determine whether the proposed belief representation and update mechanism provide benefits beyond additional reasoning and information consolidation. Evidence addressing this question would materially affect my assessment of the contribution.
2 comments
Dear Reviewer,
Thank you for your thoughtful review. As the discussion period is approaching its end, we would appreciate any further feedback if you have time, particularly regarding whether our response has addressed your main concerns.
Thank you again for your time and consideration!
We thank the reviewer for recognizing that “The replay experiments provide meaningful evidence that the accumulated memory captures reusable knowledge.” We conducted additional targeted experiments addressing the reviewer's central concern about the distinctive contribution of evolving belief maintenance.
W1/Q1. Hypothesis Retention and Status-Label Ablations
We conducted both requested controls on Database using Qwen3.8-27B.
Hypothesis retention. Removing HYPOTHESIZED propositions across task boundaries reduces reward from 0.6290 to 0.5170 and accuracy from 73.0% to 60.50% percentage points, while average queries increase from 2.090 to 2.230. These correspond to decreases of 0.1120 in reward and 12.5 percentage points in accuracy. Separately, trajectory analysis of the original EBM runs reported in the paper shows that 29 of 39 initially hypothesized propositions (74.4%) subsequently reached definite statuses through revalidation, revision, or contradiction.
Status-label replay. Removing solver-visible status labels from identical saved checkpoints does not reduce average reward (0.6217 with labels vs. 0.6450 without).
These results highlight two distinct roles of belief status. The condition without solver-visible status labels achieved a higher observed mean replay reward, offering no evidence that exposing these labels improves performance in this replay setting. By contrast, retaining unresolved propositions for future revision in light of new evidence remains an important part of EBM's learning process.
See General Response, Sec. 1, for the complete ablations, four replay configurations, and proposition trajectories.
W2/Q2. Proposition Consolidation Under Comparable Budgets
We introduce a proposition-list baseline sharing EBM's proposer–validator–solver architecture, with the validator adapted to the plain-proposition representation as described in General Response, Sec. 1. Optimized EBM and proposition-list use the same raw-history window.
Optimized EBM achieves 0.5880 reward and 69.0% accuracy using 4.164M tokens, versus 0.4673 and 55.5% using 3.902M tokens for proposition-list. Thus, EBM achieves 25.8% higher relative reward with only 6.7% more total inference tokens.
For completeness, average stored memory sizes are:
The larger memory reflects EBM’s retention of unresolved hypotheses, evidence, and status information.
Our preliminary optimizations further reduce total inference consumption by 50.8% on Database and 70.8% on Poker, while preserving strong observed performance.
Together, the architecture-matched comparison and the hypothesis-retention ablation support EBM's central contribution: a belief-maintenance process that goes beyond conventional proposition consolidation by retaining unresolved hypotheses, while achieving these benefits without requiring the substantial inference overhead of the original implementation.
See General Response, Secs. 1–2, for the full performance tables and component-level token breakdown.