- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MemoryOPD, which distills trained per-task specialist teachers into a single student using reverse KL on-policy distillation within a memory harness. Experiments on MemoryCD and on a transfer task from LongMemEval to LoCoMo show improvements over hand-crafted and RL memory baselines.
Critical strengths
- This paper addresses a relevant problem, learning a shared memory policy across tasks without a downstream reward.
- The experiments are extensive with two student families, three teacher sizes, four personalization tasks, eight held out product domains and two conversational benchmarks, and evaluate both task performance and transfer.
Critical weaknesses
- Limited novelty: on-policy distillation with reverse KL and multi-teacher on-policy distillation are already established in prior work the paper cites. The paper's main contribution is their application to memory-conditioned personalization.
- The experiments are not sufficient to establish the contribution of distillation. The comparisons do not include the specialist teachers’ own scores, warm start model before distillation, and multi-task SFT starting from that same warm start. The only SFT comparison starts from the base model instead and yet performs close to MemoryOPD on MemoryCD tasks.
Questions for authors
- Can you report the scores of the specialist teachers, the warm-start model before distillation, and multi-task SFT starting from the same warm start? These comparisons would be helpful to assess the contribution of distillation to the overall performance.