- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
BranchMem stores each task-relevant interpretation of a long-horizon search as a persistent branch, including a candidate, local natural-language judgments, source identifiers, and revision records. The same search model proposes optional updates at stage boundaries. Later decisions read a bounded view of history, sources, and up to three branches, allowing temporarily weaker candidates to remain associated with their supporting evidence when delayed clues arrive. On all 221 MemoryArena Search groups, BranchMem achieves 71.49% final success and 59.23% progress, outperforming the strongest listed baseline while using the fewest tokens. Component ablations further show that removing branches, retaining only one candidate, or dropping revision fields reduces both success and progress.
Critical strengths
-
The representation is well aligned with the failure mode motivating the paper. Keeping alternative candidate-specific evidence chains, rather than only salient facts or a compressed trajectory, is a reasonable design for delayed-evidence search. The ablations consistently support the usefulness of candidate branches, multiple candidates, and revision fields.
-
The evaluation is reasonably broad within the chosen setting. The paper evaluates all 221 MemoryArena Search groups under a shared actor, retrieval environment, and tool budget. The trace-level analysis also provides concrete evidence that retained candidates and source references are used in later search stages.
Critical weaknesses
-
The experiments do not directly compare BranchMem with the closest structural alternatives. Table 1 provides a broad comparison with several memory and retrieval systems, but these baselines do not all directly test persistent, candidate-specific evidence chains. ReTree is particularly relevant because the paper itself describes it as maintaining source-linked claims and repairing affected state after contradictory evidence, yet it is not included in the main comparison. In addition, although the paper states that the actor, retrieval environment, task collection, and search budget are shared across baselines, it provides limited detail on how each published method was adapted to the local search harness. This makes it difficult to assess whether the observed gain is primarily due to the proposed representation or partly due to implementation differences.
-
The evaluation does not directly isolate the failure mode that BranchMem is designed to address. The main motivation of BranchMem is that a temporarily weaker candidate may become correct after delayed evidence arrives, and therefore multiple candidate-specific evidence chains should remain available for later revision. However, the experiments mainly report aggregate success and progress scores over the full MemoryArena Search collection. The paper provides only a small number of qualitative traces, while only 39 lifecycle-state changes are observed across 29 groups.
Questions for authors
-
Could the authors clarify whether ReTree was considered or evaluated under the same actor, retrieval environment, and task collection? Given its source-linked revision mechanism, a comparison or discussion of the practical differences would help clarify the contribution of BranchMem.
-
Could the authors provide more detail on how the published baselines were adapted to the local search harness and how implementation fidelity was checked? It would also be helpful to report per-task paired outcomes for the strongest baselines, and repeated runs where feasible, to better assess the stability of the reported improvements.
We thank the reviewer for their suggestions regarding baseline comparisons, implementation details, and mechanism analysis. We have added experiments with ReTree and clarify below the baseline adaptations and the interpretation of the trace statistics.
W1 / Q1: Direct comparison with ReTree.
Following the reviewer’s suggestion, we additionally evaluated ReTree:
ReTree is the strongest baseline in the expanded comparison. Across all 221 task groups, BranchMem correctly answers 158 final questions, compared with 149 for ReTree. BranchMem achieves a 4.07-percentage-point higher success rate and a 6.74-percentage-point higher progress score, while consuming approximately 55.24% fewer tokens on average. This direct comparison extends the original evaluation to a structurally related method.
The two methods also differ in emphasis. ReTree focuses on backtracking and repairing affected state when new evidence contradicts existing claims, whereas BranchMem emphasizes the persistent maintenance of coexisting candidate interpretations within the same search history.
W2 / Q2: Baseline adaptation and implementation details.
The baseline results reported in our paper come from evaluations within the local search framework, rather than being taken directly from the original papers.
In the revision, we will provide the memory writing, reading, and updating rules actually used for each method, along with configurations, prompts, and modifications required to interface with the local search tools. We will distinguish the original mechanisms from the local adaptations so that readers can examine how each implementation corresponds to its original design. For the newly added ReTree baseline, we will also specify the implementation of source linking, conflict handling, and state repair, as well as the scope of token accounting.
W3: Candidate reassessment and lifecycle statistics.
The 39 lifecycle-state changes reported in the paper count only explicit state-label changes for the same candidate across different reads; they do not capture all candidate-information updates or subsequent uses. A candidate can retain the same state while accumulating new judgments, sources, and task-relation information. The model can also use retained candidates to adjust queries or final answers without changing their lifecycle labels. This count therefore does not directly measure how frequently candidates are reassessed when delayed evidence arrives.