acceptodds
Under review as a conference paper at ICLR 2027

MemCollab: Cross-Model Memory Collaboration via Contrastive Trajectory Distillation

Abstract

Large language models increasingly rely on memory mechanisms to reuse knowledge from past problem-solving experiences. However, existing methods typically construct memory from the experiences of a single LLM and reuse it with the same underlying model. As a result, stored memories may entangle transferable task knowledge with model-specific solution strategies that are less effective when reused by other LLMs. This coupling becomes problematic when memory is shared across heterogeneous LLMs with different sizes, architectures, or reasoning behaviors, raising a key question: can we construct memory that transfers reliably across different LLMs? We find that naive cross-model memory transfer can degrade performance because model-specific solution strategies encoded in memory may not generalize to other LLMs. To address this challenge, we propose MemCollab, a collaborative memory framework that constructs shared cross-model memory by contrasting reasoning trajectories produced by different LLMs on the same problem. By contrasting how different LLMs solve the same task, MemCollab retains solution principles that are consistently useful across models and removes model-specific steps that do not transfer, while suppressing less transferable, model-specific solution patterns. We further introduce a task-aware retrieval mechanism that conditions memory access on task category, ensuring that only relevant constraints are retrieved at inference time. Experiments on mathematical reasoning and code generation benchmarks show that \ourmethod consistently improves accuracy and inference-time efficiency across LLMs of different sizes and model families. These results demonstrate that collaboratively constructed memory can serve as a shared reasoning resource across different LLMs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_hSk6OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MEMCOLLAB, a framework for constructing a single, reusable memory bank that can be shared across heterogeneous LLMs by contrastively distilling normative reasoning constraints from paired trajectories on the same task. Experiments on math reasoning, code generation, and one agentic benchmark show consistent gains across different model sizes and families, and ablations suggest that cross-model contrast and task-aware retrieval are both important.

Critical strengths
  1. The central idea that contrastively distills cross-model trajectory differences into abstract constraints is a compelling way.
  2. The experiments show that the gains are consistent and significantly outperform the baselines.
  3. The motivation is clear, and the paper is easy to follow.
Critical weaknesses
  1. It is unclear how the performance will change when the strongest model is absent. Since the pipeline highly depends on the strongest model as the summarizer.
  2. The data construction details are not clear. It would be great to explicitly state the size and composition of the training set used to build the memory, as well as the split of data.
  3. The paper claims that the inference time is reduced, but this claim is not justified by the experiments.
  4. The retrieval step requires that the target model participates in the contrast pairs. This will limit the plug-and-play transfer to new models that are not present during memory construction.
Questions for authors

See Weaknesses.

3 comments
AuthorsOpenReview ↗

Response to reviewer hSk6

W1. Comparable-size model results and dependence on the strongest Model

Thank you for raising this concern. Appendix C.3, Tables 7 and 8, shows that MemCollab remains effective in comparable-size collaboration between Qwen2.5-7B and Llama-3-8B, as well as Qwen2.5-72B and Llama-3-70B. Furthermore, Appendix D.2, Table 11, shows that smaller models can also serve as effective summarizers and yield substantial gains over the baseline. Together, these findings indicate that MemCollab benefits from complementary reasoning across models without requiring a large capability gap or the strongest model as the summarizer.

W2. Data construction details

Thank you for pointing out the missing details. Our data setup is adapted from Kang et al., Distilling LLM Agent into Small Models with Retrieval and Code Tools (https://arxiv.org/abs/2505.17612), with dataset-specific sizes below. Construction examples are used for offline memory construction, not model parameter training.

DatasetSourceConstruction examplesTest examples
GSM8KGSM8K train / test1,000500
MATHMATH train / test1,000500
MBPPMBPP train / test120257
HumanEval164 Full dataset8282

W3. Inference efficiency

Thank you for raising this concern. Appendix Table 14 already reports fewer reasoning turns with MemCollab. We additionally measured wall-clock inference time on MATH500 using Llama-3-8B-Instruct under the same evaluation conditions.

MethodTime (seconds/task)
Vanilla (with its own memory)27.3
MemCollab26.1

These results support the inference-time efficiency benefit of MemCollab in this setting. We have included these efficiency results in the appendix D.4.

W4. Cross-model transferability

We thank the reviewer for the insightful comment. Based on the suggestion, we can extend the algorithm to consider the case of new model joining: in algorithm 2 line 4, we will remove the condition lm=ljl_m=l_j when ljl_j is not a registered model from A\mathcal{A}.

To test the tranferability of the memory bank, we reused a fixed memory bank constructed from Qwen2.5-7B-Instrut and Qwen2.5-32B-Instrut trajectories and evaluated Llama-3-8B-Instruct on MATH500. We kept the bank fixed throughout the evaluation, so no trajectories from Llama-3-8B-Instruct were used to construct or update its entries. We compared its accuracy with and without access to the bank.

MethodAccuracy (%)
Vanilla27.4
MemCollab (held-out model)34.0

This result shows that memory distilled from two models can transfer to a held-out model from a different family on MATH500. We have added this generalization results to the Appendix D.8.

Thanks for the response from the authors during the rebuttal, which addresses most of my concerns. I have accordingly updated my rating.

Thank you for your thoughtful feedback and for updating your rating. We appreciate your careful review and the constructive discussion, which helped us improve the paper.

Reviewer_kx95OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses memory sharing across heterogeneous LLMs. The authors observe that memory distilled from one model and reused by another can hurt performance, since stored memory entangles transferable task knowledge with model-specific solution strategies. They propose MemCollab, which pairs reasoning trajectories from different models on the same task into preferred / unpreferred sets, contrasts them to distill normative (enforce; avoid) constraints, tags each entry with model-identity labels, and retrieves at inference time via task-category and model-aware filtering. Experiments on math and code benchmarks across several model families and scales report accuracy and efficiency gains over no-memory and single-source memory baselines.

Critical strengths
  1. Memory reuse across heterogeneous LLMs is a real gap—prior work almost exclusively builds and reuses memory within a single model. The paper poses the question clearly, backs it with a concrete failure case of naive transfer, and offers an intuitive solution: contrasting preferred vs. unpreferred trajectories to separate transferable reasoning principles from model-specific choices, distilled into compact, human-readable (enforce; avoid) constraints.
  2. The evaluation spans multiple model families, scales, and task types, with ablations on preference selection, summarizer choice, and prompt sensitivity that address several natural alternative explanations for the gains.
Critical weaknesses
  1. Model-identity filtering is the only component through which cross-model collaboration is explicitly used during inference, yet its contribution is never evaluated in isolation. Figure 3 examines only the task-aware filtering stage and does not test whether filtering memories by the target model’s identity is actually beneficial. An ablation that removes model-aware filtering while keeping task-aware filtering and all other components unchanged would be necessary to validate this key design choice.

  2. The framing of cross-model transfer does not quite match the actual implementation path. The abstract motivates the work by asking whether we can construct memory that transfers reliably across different LLMs, but retrieval is gated by model-aware filtering: only entries whose contrastive construction involved the target model are retained (Eq. 10, Algorithm 2). Under this design, the memory bank is not something a new LLM can simply read from — before it can retrieve anything, the new model must first run its own trajectories over the training set and have memory re-constructed from its own contrastive pairs. In other words, what transfers across models is the construction recipe rather than the memory itself, which is a weaker statement than the question posed in the abstract.

Questions for authors
  1. What is the accuracy of the task classifier in Eq. 8, and how sensitive is the final performance to its mistakes?
  2. Could you provide end-to-end token counts or wall-clock measurements to support the inference-time efficiency claim in abstract? Table 14 reports only reasoning turns, which may not reflect the longer prompts introduced by retrieved memory.
3 comments
AuthorsOpenReview ↗

Response to reviewer kx95

W1. Model-identity filtering ablation

Thank you for suggesting this ablation. We removed model-identity filtering while keeping task-aware filtering and all other settings unchanged, using memories constructed jointly by Qwen2.5-7B, Llama-3-8B, and Qwen2.5-32B. The results below evaluate Qwen2.5-7B using these memories:

SettingAccuracy (%)
Qwen2.5-7B baseline52.2
MemCollab without model-identity filtering62.8
MemCollab with model-identity filtering66.8

Model-identity filtering improvements supports its benefit in this setting. We will include this ablation in the revised paper.

W2. Cross-model transferability

We thank the reviewer for the constructive comment. Based on the suggestion, we can extend the algorithm to consider the case of new model joining: in algorithm 2 line 4, we will remove the condition lm=ljl_m=l_j when ljl_j is not a registered model from A\mathcal{A}.

To evaluate whether the memory itself can transfer, we reused a fixed bank constructed from Qwen2.5-7B and Qwen2.5-32B trajectories and made it available to a held-out new model Llama-3-8B-Instruct on MATH500. The bank was reused as built, with no trajectories from Llama-3-8B-Instruct added.

Llama-3-8B-Instruct settingAccuracy (%)
Vanilla (held-out model with no memory)27.4
MemCollab (held-out model)34.0

The result shows that an existing memory bank can benefit a model from a different family when retrieval allows it to access the bank. We have added this generalization results to the Appendix D.8.

Q1. Task classifier evaluation

We thank the reviewer for this question. We evaluated Llama-3-8B on MATH500 with the same memory bank, varying only the task classifier in Eq. 8.

Task classifierAccuracy (%)
Original42.4
GPT-5.4-mini43.6

Using GPT-5.4-mini as an LLM judge, we found that the original task labels matched the judge’s labels on 84% of examples.

To assess robustness to task classification errors, we randomly replaced the predicted task labels for 5%, 10%, and 15% of the test data.

Task classifier accuracy (%)Accuracy (%)
Original42.4
inject 5% randomness41.0
inject 10% randomness38.4
inject 15% randomness38.0

The similar results suggest that downstream accuracy was relatively insensitive to the choice between these two classifiers in this setting.

Q2. Inference Efficiency

Thank you for this suggestion. To complement the reasoning-turn results in Appendix Table 14, we additionally measured wall-clock inference time on MATH500 using Llama-3-8B-Instruct under the same evaluation conditions.

MethodTime (seconds/task)
Vanilla (with its own memory)27.3
MemCollab26.1

These results suggest that MemCollab improves inference-time efficiency in this setting by helping the model complete tasks more reliably and make decisions with greater confidence. We have included these efficiency results in the appendix D.4.

Thank you for your rebuttal. Regarding the Task classifier evaluation section, it seems that the accuracy is not high, but replacing the predicted task label with random will have an impact on the final task accuracy. What do you think are the possible reasons for this phenomenon.

AuthorsOpenReview ↗

Follow-up response to Reviewer kx95

Thank you for the follow-up. On MATH500, the main category is given, and the classifier in Eq. 8 selects a subcategory within it. One possible explanation is that different subcategories within the same main category can still retrieve guidance relevant to the problem. Randomly replacing a label can route retrieval to an unrelated set of memories and exclude useful guidance. This can affect accuracy on problems where the retrieved memory would otherwise help the model reach the correct answer. This difference in how the labels are chosen may explain why random replacement affects accuracy while switching between the two classifiers yields similar results.

Reviewer_zCccOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies cross-model generalization of agent experience memory, observing that direct transfer can cause negative transfer. It proposes MemCollab, which contrasts trajectories from different models to distill shared reasoning constraints and reduce dependence on model-specific strategies. Experiments on MATH500, GSM8K, MBPP, and HumanEval show improvements in performance and inference efficiency.

Critical strengths
  1. The paper identifies negative transfer in memory reuse and addresses the practical challenge of sharing experience across models.
  2. Cross-model trajectory comparisons produce explicit reasoning principles and error-avoidance constraints that are easy to inspect.
  3. Experiments cover cross-scale, cross-family, and similar-scale collaboration, supported by retrieval ablations and case studies.
Critical weaknesses
  1. Selecting a single preferred trajectory may underutilize complementary strategies from other correct solutions, biasing the distilled memory toward the strongest model’s reasoning preferences.
  2. Model-identity filtering restricts memory reuse to participating models. The paper does not evaluate whether a fixed collaborative memory bank benefits models excluded from its construction, leaving generalization to new models unverified.
Questions for authors

There are some inconsistencies in the reported results that the authors should check and clarify.

  • Page 6 reports 74.4% MATH500 accuracy for Llama-8B, but Table 2 shows 42.4%; 74.4% corresponds to GSM8K.
  • Table 3 lists Qwen-32B’s MATH500 baseline as 58.0%, inconsistent with 63.8% in Tables 1 and 2. Please clarify and correct these discrepancies.
5 comments
AuthorsOpenReview ↗

Response to reviewer zCcc

W1. Single preferred trajectory

Thank you for raising this concern. During contrastive memory construction, we compare the preferred trajectory with every remaining trajectory, including any other correct solutions.

Appendix C.3 (Tables 7 and 8) reports results for two pairs of models of comparable size: Qwen2.5-7B-Instruct with Llama-3-8B-Instruct and Qwen2.5-72B-Instruct with Llama-3-70B-Instruct. Both models in each pair achieved higher downstream accuracy with MemCollab, showing that memory constructed using a single preferred trajectory can benefit both collaborators.

W2. Generalization to new models

We thank the reviewer for the insightful comment. Based on the suggestion, we can extend the algorithm to consider the case of new model joining: in algorithm 2 line 4, we will remove the condition lm=ljl_m=l_j when ljl_j is not a registered model from A\mathcal{A}.

To test this, we reused a fixed memory bank constructed from Qwen2.5-7B and Qwen2.5-32B trajectories and evaluated Llama-3-8B-Instruct on MATH500. We kept the bank fixed throughout the evaluation, so no trajectories from Llama-3-8B-Instruct were used to construct or update its entries. We compared its accuracy with and without access to the bank.

MethodAccuracy (%)
Vanilla27.4
MemCollab (held-out model)34.0

This result shows that memory distilled from two Qwen models can transfer to a held-out model from a different family on MATH500. We have added this generalization results to the Appendix D.8.

Q1. Incosistent reports.

Thank you for catching these reporting errors. We have corrected the dataset name on page 6 from MATH500 to GSM8K, as 74.4% refers to GSM8K. We have also corrected Qwen-32B’s MATH500 baseline in Table 3 to 63.8%, consistent with Tables 1 and 2.

Thank you for the response and additional experiments. The gain on MATH500 is encouraging. Could you provide a analysis of which previously incorrect cases become correct and whether any previously correct cases become incorrect?

AuthorsOpenReview ↗

Follow-up response to Reviewer zCcc

Thank you for the follow-up question. We compared Llama-3-8B-Instruct’s responses on the same 500 MATH500 problems with and without access to the fixed memory bank. With memory, 74 responses changed from incorrect to correct and 41 changed from correct to incorrect. Overall accuracy increased from 27.4% to 34.0%. The two illustrative cases below show one example of each type of change.

Case 1. Incorrect to correct

Problem. If t(x)=3−g(x)t(x)=3-g(x) and g(x)=xg(x)=\sqrt{x}, what is t(g(16))t(g(16))? The correct answer is 11.

Without memory. The model computes g(16)=4g(16)=4 but then evaluates t(16)=3−16t(16)=3-\sqrt{16} and answers −1-1.

Retrieved memory. When dealing with nested functions, enforce the sequential evaluation starting from the innermost function; avoid direct manipulation of the outer function without resolving the inner function first.

With memory. The model substitutes g(16)=4g(16)=4 into tt and obtains t(4)=3−4=1t(4)=3-\sqrt{4}=1.

Case 2. Correct to incorrect in held-out cross-model transfer

Problem. If 0.1331‾0.\overline{1331} is written in lowest terms as ab\frac{a}{b}, what is a+ba+b? The correct answer is 10301030.

Without memory. The model writes the purely repeating decimal as 13319999\frac{1331}{9999}, reduces it to 121909\frac{121}{909}, and obtains 10301030.

Retrieved memory. When converting repeating decimals to fractions, enforce expressing the repeating decimal as a sum of non-repeating and repeating parts before combining into a single fraction; avoid directly manipulating the entire decimal as a single entity.

With memory. The model attempts to split off a non-repeating prefix even though the decimal is purely repeating. This introduces an erroneous factor of 10001000 into the numerator, yielding 13310009999\frac{1331000}{9999} and the incorrect answer 121909121909.

Why memory fails. The source problem involved 0.42153‾0.42\overline{153}, which has a non-repeating prefix. The test problem belongs to the same task category, but its decimal repeats immediately. The retrieved guidance was suited to the source problem’s structure. In the test problem, applying it led the model to introduce a prefix that was absent.

These results show that a fixed memory bank can improve reasoning accuracy for a model that did not contribute to its construction. The illustrative cases highlight why the applicability of retrieved guidance matters. Guidance can correct a reasoning error when it fits the problem’s structure, while guidance retrieved for the right task category can still introduce an error if its assumptions do not hold for the specific problem.

Reviewer_HMupOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to share experience-based memory across heterogeneous LLMs. MemCollab contrasts trajectories from different models on the same problem and uses a summarizer to extract reusable reasoning constraints and error-avoidance guidance. The reported improvements suggest that contrastive experience extraction can improve both weaker and stronger participating models.

Critical strengths

The paper addresses a useful problem for evolving agent systems. Experience should still be useful when the model changes. The negative transfer from naive large-model memory is particularly interesting: for Qwen2.5-7B, memory from the 32B model reduces HumanEval accuracy from 42.7% to 34.1% (Table 1).

Critical weaknesses

The experiments do not fully isolate cross-model contrast from the other changes in memory construction and retrieval. Self-contrast and summarizer ablations are useful, but I would still like a pooled multi-model memory baseline using the same summarizer, extraction format, retrieval filters, and memory-token budget, without pairwise contrast.

Questions for authors

na

1 comment
AuthorsOpenReview ↗

Response to reviewer HMup

W1. Pooled multi-model memory baseline

We thank the reviewer for suggesting this baseline to isolate the contribution of pairwise contrast. Both methods use Llama-3-8B-Instruct as the summarizer and share the same extraction format, retrieval filters, and memory token budget. The only difference is how the memory bank is constructed. The pooled baseline independently builds one bank from Qwen2.5-7B-Instruct trajectories and one from Llama-3-8B-Instruct trajectories, then directly merges the two banks without pairwise contrast. MemCollab constructs its bank by contrasting paired trajectories from the two models.

MethodAccuracy (%)
Vanilla27.4
Pooled multi-model memory26.0
MemCollab (Appendix C.3, Table 7)30.4

The pooled baseline scored below MemCollab, indicating that pairwise contrast adds value beyond directly pooling memories in this setting.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.