MemCollab: Cross-Model Memory Collaboration via Contrastive Trajectory Distillation

Large language models increasingly rely on memory mechanisms to reuse knowledge from past problem-solving experiences. However, existing methods typically construct memory from the experiences of a single LLM and reuse it with the same underlying model. As a result, stored memories may entangle transferable task knowledge with model-specific solution strategies that are less effective when reused by other LLMs. This coupling becomes problematic when memory is shared across heterogeneous LLMs with different sizes, architectures, or reasoning behaviors, raising a key question: can we construct memory that transfers reliably across different LLMs? We find that naive cross-model memory transfer can degrade performance because model-specific solution strategies encoded in memory may not generalize to other LLMs. To address this challenge, we propose MemCollab, a collaborative memory framework that constructs shared cross-model memory by contrasting reasoning trajectories produced by different LLMs on the same problem. By contrasting how different LLMs solve the same task, MemCollab retains solution principles that are consistently useful across models and removes model-specific steps that do not transfer, while suppressing less transferable, model-specific solution patterns. We further introduce a task-aware retrieval mechanism that conditions memory access on task category, ensuring that only relevant constraints are retrieved at inference time. Experiments on mathematical reasoning and code generation benchmarks show that \ourmethod consistently improves accuracy and inference-time efficiency across LLMs of different sizes and model families. These results demonstrate that collaboratively constructed memory can serve as a shared reasoning resource across different LLMs.