- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes MEMCOLLAB, a framework for constructing a single, reusable memory bank that can be shared across heterogeneous LLMs by contrastively distilling normative reasoning constraints from paired trajectories on the same task. Experiments on math reasoning, code generation, and one agentic benchmark show consistent gains across different model sizes and families, and ablations suggest that cross-model contrast and task-aware retrieval are both important.
Critical strengths
- The central idea that contrastively distills cross-model trajectory differences into abstract constraints is a compelling way.
- The experiments show that the gains are consistent and significantly outperform the baselines.
- The motivation is clear, and the paper is easy to follow.
Critical weaknesses
- It is unclear how the performance will change when the strongest model is absent. Since the pipeline highly depends on the strongest model as the summarizer.
- The data construction details are not clear. It would be great to explicitly state the size and composition of the training set used to build the memory, as well as the split of data.
- The paper claims that the inference time is reduced, but this claim is not justified by the experiments.
- The retrieval step requires that the target model participates in the contrast pairs. This will limit the plug-and-play transfer to new models that are not present during memory construction.
Questions for authors
See Weaknesses.
3 comments
Thanks for the response from the authors during the rebuttal, which addresses most of my concerns. I have accordingly updated my rating.
Thank you for your thoughtful feedback and for updating your rating. We appreciate your careful review and the constructive discussion, which helped us improve the paper.
Response to reviewer hSk6
Thank you for raising this concern. Appendix C.3, Tables 7 and 8, shows that MemCollab remains effective in comparable-size collaboration between Qwen2.5-7B and Llama-3-8B, as well as Qwen2.5-72B and Llama-3-70B. Furthermore, Appendix D.2, Table 11, shows that smaller models can also serve as effective summarizers and yield substantial gains over the baseline. Together, these findings indicate that MemCollab benefits from complementary reasoning across models without requiring a large capability gap or the strongest model as the summarizer.
Thank you for pointing out the missing details. Our data setup is adapted from Kang et al., Distilling LLM Agent into Small Models with Retrieval and Code Tools (https://arxiv.org/abs/2505.17612), with dataset-specific sizes below. Construction examples are used for offline memory construction, not model parameter training.
Thank you for raising this concern. Appendix Table 14 already reports fewer reasoning turns with MemCollab. We additionally measured wall-clock inference time on MATH500 using Llama-3-8B-Instruct under the same evaluation conditions.
These results support the inference-time efficiency benefit of MemCollab in this setting. We have included these efficiency results in the appendix D.4.
We thank the reviewer for the insightful comment. Based on the suggestion, we can extend the algorithm to consider the case of new model joining: in algorithm 2 line 4, we will remove the condition lm=lj when lj is not a registered model from A.
To test the tranferability of the memory bank, we reused a fixed memory bank constructed from Qwen2.5-7B-Instrut and Qwen2.5-32B-Instrut trajectories and evaluated Llama-3-8B-Instruct on MATH500. We kept the bank fixed throughout the evaluation, so no trajectories from Llama-3-8B-Instruct were used to construct or update its entries. We compared its accuracy with and without access to the bank.
This result shows that memory distilled from two models can transfer to a held-out model from a different family on MATH500. We have added this generalization results to the Appendix D.8.