- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
BranchMem stores successful trajectories as latent memories for a frozen LLM reasoner. A shared compressor (Llama-3.2-1B) turns each trajectory into a fixed-length latent value, kept in a bank together with a key built from the task input. For a new query, the top-8 memories are retrieved, and each trajectory is conditioned on one memory at a time. The authors compare the retrieved memories under a fixed prefix, using teacher-forced next-token scoring against an anchor trajectory generated with the top-1 memory. Their aligned analysis reports agreement over shared prefixes and first divergences concentrated at low-margin positions. Within rescue cases, retrieval scores provide little separation between successful and unsuccessful candidates at ranks 2–8. Action Branching builds on this: it groups candidates by their first divergent position and alternative token, expands a representative branch, and re-probes the remaining memories in that group under the new prefix. Experiments use Llama-3.1-8B and Llama-3.2-3B on GSM8K, KodCode, TriviaQA, PopQA and StrategyQA, and report solution coverage, where a query counts as solved if any generated trajectory is correct.
Critical strengths
-
Comparing different latent memories under the same reasoner and the same prefix is a good way to ask what a memory actually does, and the association between first divergence and low-margin positions is reported consistently across two models, two training objectives and two tasks (Table 2, Figure 3).
-
The authors are fairly open about limits. Appendix B.3 shows where Action Branching falls short of independent generation from the same top-8 set, and reports the token savings under B=1 (81–86%). The retrieval analysis (RQ3) and the residual intervention (B.1) add useful context to the main table.
Critical weaknesses
-
The main results do not separate the effect of the method from the effect of extra attempts. In Table 1, BranchMem is scored by whether any trajectory in a tree is correct, but the comparisons with Vanilla, TokMem and MemGen do not establish matched candidate counts or compute budgets. When I look at the one-trajectory comparison (BranchMem Top-1), the gain over existing memory methods is small. On Llama-3.1-8B, SFT Top-1 averages 63.27 against 63.81 for TokMem, and GRPO Top-1 averages 64.49 against 64.26 for MemGen-GRPO. On Llama-3.2-3B, SFT Top-1 is below both TokMem and MemGen-SFT. All four Top-1 variants have higher five-task averages than Vanilla, which supports the interface’s usefulness, but the averages do not establish a clear advantage over existing memory methods. The paper's own Table 7 makes the picture less favorable for branching. Independent generation from the same top-8 memories has higher coverage than the full tree in Table 1 in all 20 settings (by 0.07 to 3.05 points, 0.69 on average). These comparisons suggest a potential cost–coverage tradeoff rather than a coverage advantage over Independent Top-8. But the cost of the full tree is never reported, since Table 7 only covers B=1. Teacher-forced scoring of the remaining candidate memories at each node is not included in the token accounting either, and there is no no-memory baseline given the same number of samples. StrategyQA needs a separate note: answer coverage on a yes/no question becomes 100% whenever both valid answers appear in the tree. The rise from 65.93 to 83.41 therefore does not by itself establish better reasoning without the answer distribution or final-answer selection accuracy. The restricted B=1 results provide evidence of a token–coverage tradeoff, but the main-table gains do not establish more effective use of past experience at a matched total cost.
-
It is not shown that the content of a memory matters, or that the "localized intervention" is specific to memory. The statistics in Table 2 condition on the first position where a candidate’s argmax differs from the anchor token. This selection may favor states where perturbations can change the decision. Although the measured log-probability changes and no-memory margins are not consequences of the definition alone, matched perturbation controls are needed to establish whether the pattern is specific to retrieved experience. Random or averaged memories, a learned constant prefix, or controlled prompt perturbations should be compared on both the branch-point statistics and task performance. The residual experiment in B.1 does show that the specific residual matters for which branch events are reproduced (100% for the own residual, 46.7% when shuffled). But it measures event recovery, not task success, so it does not show that memory content helps solve problems. There is also a training issue. By Eq. 3 and Appendix A.2, the latent is encoded from the same target trajectory that the model is then trained to reproduce (SFT) or to condition on (GRPO), whereas at test time the latent comes from a different problem. The interface may learn same-trajectory reconstruction rather than transferable experience, and the paper does not test this. Since the paper presents its contribution as reuse of past experience, and the motivation for Action Branching rests on this analysis, I think the point needs to be settled.
Overall, the aligned-comparison analysis is interesting, and the Top-1 five-task averages improve over Vanilla. But I cannot separate the main gains from the extra attempts, the cost of the full tree is missing, and the role of memory content has not been tested against matched controls. A cost-matched comparison and a content control could change my view.
Questions for authors
- Clarification: What decoding is used (greedy or sampling, temperature, top-p)? Under matched greedy decoding and stopping rules, what accounts for the difference between full-tree and Independent Top-8 coverage? If the anchor is sampled, how do you make sure the divergences measured against the argmax in Eq. 5 are not just sampling noise?
- Assessment-relevant comparison: What is the total cost of the full tree, counting both generated tokens and teacher-forced scoring passes, compared with Independent Top-8 and with no-memory sampling at the same number of samples?
- Assessment-relevant comparison: How do random, shuffled-across-questions and averaged memories compare with real retrieved ones, both in coverage and in the Table 2 statistics?
- Clarification and assessment-relevant comparison: Is the latent for a training example ever built from a different example's trajectory (cross-sample or leave-one-out)? If not, what happens when the interface is trained that way?
- Clarification and assessment-relevant comparison: On StrategyQA, in what fraction of trees do both answers appear, and what is the accuracy if one answer has to be chosen, for example by majority vote? Also, why does full SFT BranchMem on the 3B model fall well below Vanilla on GSM8K (69.90 vs 77.78)?
3 comments
Coverage under the same attempt limit. We also compare solution coverage with eight attempts per query on the frozen Llama-3.2-3B Reasoner. Vanilla samples output tokens at and top-. MemGen-GRPO applies Gaussian perturbations at scale 0.2 to the Weaver latents and decodes greedily. The TokMem retrieval extension generates from each of the eight highest-ranked memories. The complete tree returns up to eight trajectories. A query is covered if any trajectory is correct.
| Dataset | Vanilla | MemGen-GRPO | TokMem retrieval extension | Corrected BRANCHMEM-SFT complete tree | Corrected BRANCHMEM-GRPO complete tree |
|---|---|---|---|---|---|
| GSM8K | 81.72% | 79.45% | 74.83% | 70.65% | 83.54% |
| KodCode | 55.65% | 66.40% | 67.35% | 69.65% | 62.85% |
On GSM8K, the corrected BRANCHMEM-GRPO complete tree reaches 83.54% coverage, compared with 81.72% for eight Vanilla samples. On KodCode, the BRANCHMEM-SFT complete tree achieves the highest coverage in the table. In the single-trajectory setting, all four BRANCHMEM Top-1 configurations in Table 1 have higher five-task averages than Vanilla. Comparisons with TokMem and MemGen vary with the training setup and Reasoner. The single-trajectory comparison shows the effect of the memory interface. Comparing the complete tree with Top-1 under the same interface quantifies the coverage gained by exploring alternatives within the candidate pool.
Q2 · Reasoner cost of the complete tree
The complete tree expands at most seven branches in addition to the anchor. With up to eight attempts per query, the complete tree decodes fewer new tokens than Independent Top-8 and eight Vanilla samples on both tasks (Q1). The table compares Reasoner cost on the same candidate pool and includes probing and prefix replay.
| Dataset | Method | Time, s/query | Processed token positions per query | Share of time spent probing |
|---|---|---|---|---|
| GSM8K | Complete tree | 5.14 | 2,393.8 | 12.51% |
| GSM8K | Batched Independent Top-8 | 4.83 | 2,259.9 | — |
| KodCode | Complete tree | 10.52 | 5,448.7 | 13.09% |
| KodCode | Batched Independent Top-8 | 11.25 | 4,965.4 | — |
Including probing and replay, the complete tree and batched Independent Top-8 have similar Reasoner times on the same pool.
W2 / Q3 · Memory content and retrieval
Retrieval and solution coverage. Retrieving memories for the current query improves coverage over cross-query shuffling by 3.13 and 4.56 percentage points on GSM8K and KodCode, respectively. Cross-query shuffling uses candidates retrieved for another query while retaining memories encoded from successful experiences. We also compare BGE semantic retrieval, random memories, and the memory-bank mean.
| Memory condition | GSM8K solution coverage | KodCode solution coverage |
|---|---|---|
| BRANCHMEM projected retrieval | 83.54% | 69.65% |
| BGE semantic retrieval | 83.32% | 68.88% |
| Random memories | 81.47% | 67.62% |
| Cross-query shuffled memories | 80.41% | 65.09% |
| Memory-bank mean | 77.18% | 63.41% |
RQ3 and Table 6 in Appendix B.2 examine ranking within a retrieved pool. On TriviaQA and PopQA, among queries where Top-1 fails and Top-8 contains a correct solution, initial retrieval scores distinguish successful from unsuccessful candidates with a mean AUC of 0.498. RQ2 offers an explanation. Different memories can take the same actions along a shared prefix and propose different next steps only when they reach a particular reasoning state.
Branch-point statistics. To test whether the low-margin pattern in Table 2 depends on matching experiences to queries, we fix the anchor generated by the actual Top-1 retrieval for each query on both full test sets. We compare the other seven candidates retrieved for that query with seven candidates retrieved for a different query. The margin ratio below is the median no-memory Top-1/Top-2 margin at first divergence divided by the corresponding median at shared-prefix positions. AUC measures how well a low margin distinguishes these two sets of positions.
| Dataset | Candidate memories | Margin ratio | Low-margin AUC |
|---|---|---|---|
| GSM8K | Actual retrieval | 7.69% | 0.917 |
| GSM8K | Cross-query shuffling | 7.35% | 0.916 |
| KodCode | Actual retrieval | 7.08% | 0.920 |
| KodCode | Cross-query shuffling | 7.47% | 0.914 |
Cross-query shuffling preserves the concentration of first divergences at low-margin positions and lowers solution coverage. The branch-point analysis compares actual retrieval with cross-query shuffling. The coverage table also includes random memories and the memory-bank mean. When we select low-margin positions using the no-memory Reasoner, the replacement effect on both tasks is still about three times that at random positions matched for progress through the trajectory.
Q4 · Cross-example training and generalization
Cross-example SFT achieves held-out solution coverage close to that of same-query pairing on both tasks. We freeze the Llama-3.2-3B Reasoner and train the interface from scratch on the original SFT training data for GSM8K and KodCode, separately. For each training query, BGE retrieves the most similar eligible query other than itself. We feed that other query's reference solution to the compressor, while the current query's reference solution remains the supervision target. KodCode uses the original length filter.
| Dataset | Original interface, Top-1 | Cross-example interface, Top-1 | Original interface, answer-pruned coverage | Cross-example interface, answer-pruned coverage |
|---|---|---|---|---|
| GSM8K | 65.57% | 64.44% | 69.90% | 69.90% |
| KodCode | 65.10% | 65.65% | 67.55% | 68.00% |
Under the same answer-pruning protocol, training the interface with experiences from other queries yields held-out coverage close to the original interface. GSM8K remains at 69.90%, and KodCode changes from 67.55% to 68.00%, with a paired 95% bootstrap interval of [−0.65, +1.55] percentage points. These results show that cross-example pairing can train a memory interface that transfers to unseen queries.
Q5 · StrategyQA and SFT performance on GSM8K
Single-answer accuracy on StrategyQA. With Llama-3.1-8B GRPO, 54.59% of complete trees contain both Yes and No. StrategyQA uses the same agentic search setup as TriviaQA. Each trajectory can retrieve information and use that evidence to reach a yes-or-no judgment. Whether a branch reaches a correct conclusion therefore depends in part on the relevance and sufficiency of the evidence it retrieves. In this setting, coverage measures whether retrieval and reasoning yield at least one correct trajectory.
To assess whether that translates into better answer selection, we use a fixed rule to select one answer per query and then compare it with the reference answer. Each distinct trajectory with a valid answer casts one vote. Ties go to the first valid answer in traversal order, and we return Yes if no valid answer is available. With this rule, majority voting achieves 70.74% single-answer accuracy, exceeding the other single-generation methods with the same backbone in Table 1.
| Reasoner | Contains both answers | Majority-vote accuracy | Anchor accuracy |
|---|---|---|---|
| Llama-3.1-8B | 54.59% | 70.74% | 67.39% |
The 3B SFT gap on GSM8K. With Llama-3.2-3B, MemGen-SFT (66.18%) and TokMem (66.94%) also score below Vanilla (77.78%) and close to BRANCHMEM-SFT Top-1 (65.57%). Action Branching raises BRANCHMEM-SFT coverage to 69.90%. These similar results suggest that the reference solutions in the official GSM8K training split may provide limited direct supervision for reusing strategies across problems. With the same backbone and up to eight attempts per query, the corrected BRANCHMEM-GRPO tree achieves 83.54% coverage, compared with 81.72% for eight Vanilla samples. This gain shows that GSM8K remains informative for evaluating the benefit of BRANCHMEM.
Thank you for suggesting these controls. First divergences remain concentrated at low-margin positions after cross-query shuffling. Retrieving memories for the current query yields higher solution coverage. We also find that cross-example training achieves held-out coverage close to that of the original interface on GSM8K and KodCode.
W1 / Q1 · Attempt counts, decoding, and coverage
Decoding setup. The anchor, branch continuations, and Independent Top-8 under the same candidate pool all use greedy decoding in the submitted experiments. Temperature and top-p do not affect those runs. Equation (5) compares the candidate memory's top token with the token chosen by the anchor under the same prefix. The anchor is generated greedily, so this comparison is free of sampling noise from the anchor.
Complete-tree and Independent Top-8 coverage. Under the same candidate pool, a complete grouped tree recovers the independently generated trajectories when next-token decisions and stopping rules agree. The submitted implementation capped the number of tokens scored during teacher-forced probing. A candidate whose first divergence fell beyond that limit was missed. Numerical differences between batched scoring and decoding can also change the top-ranked token when logits are nearly tied. In the rerun, we scored the full anchor and matched the candidate pool, greedy decoding, EOS handling, and generation limit. After the correction, the complete tree shares prefixes among candidates that choose the same actions and expands alternatives when their actions later diverge. The two methods attain the same solution coverage in all 20 settings.
Each cell below gives the solution coverage attained separately by both the complete tree and Independent Top-8 in that setting.
Generation volume in Table 1. In Table 1, Vanilla, MemGen, TokMem, and BRANCHMEM Top-1 each generate one trajectory per query. The complete tree reuses existing prefixes to explore alternatives on GSM8K and KodCode with a similar number of newly decoded tokens to a single generation by Vanilla or MemGen-GRPO.
Independent Top-8 generates a complete trajectory from scratch for each candidate. Action Branching probes candidate next actions under the existing prefix and generates only a new suffix when an action diverges. Memories that propose the same alternative token at the same position share a branch point and are probed again under the new prefix. This sharing reduces the number of newly decoded tokens. The table counts the tokens newly decoded during complete tree construction, including the Top-1 anchor and the new suffixes of all branches. Each branch reuses the prefix before divergence.
On GSM8K and KodCode, Action Branching builds the complete tree with a newly decoded token budget comparable to a single generation from Vanilla or MemGen-GRPO. This supports the comparison with these baselines in Table 1. At a similar generation budget, Action Branching reuses the prefix before divergence and spends its new decoding on alternatives proposed by the candidate memories.