- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces LIP-Rec, a research task studying personalized recommendation from long-term human–MLLM assistant interactions. To support this task, it presents PersonaRec, a benchmark comprising 200 personas, over 30K candidate items across three domains, and LLM-synthesized multimodal interaction histories. The authors show that frontier MLLMs perform poorly on this benchmark and further propose Yo'Rec, which improves open-source MLLMs through a three-step CoT-based SFT, followed by GRPO-style training with preference and recommendation rewards.
Critical strengths
- The proposed task addresses a clear and practically relevant problem, as personal assistants are gaining increasing attention for shopping and content recommendation.
- The benchmark is substantial in scale and covers explicit, implicit, and evolving preferences in both text-only and multimodal sessions, providing a useful resource for future research.
Critical weaknesses
W1: Insufficient ablation of Yo'Rec's design. (1) In Table 3, the transition from the base model to the PRPL-CoT SFT variant (average R@1 from 4.03 to 11.45) changes both the inference procedure (three-step reasoning) and the model parameters through SFT. It is also unclear whether self-reranking is applied to each variant. Therefore, the contributions of structured reasoning and SFT are not clearly disentangled (see Q2). (2) Furthermore, the effectiveness of the proposed three-step reasoning structure itself remains unclear. Section 3.1 claims that explicit session localization reduces interference from irrelevant history, but this is not directly validated against simpler SFT variants, such as directly rewriting the query or extracting preferences without first localizing sessions (see Q3).
W2: Insufficient analysis of the benchmark's key properties. The paper highlights diverse preference dynamics (explicit, implicit, and evolving) and visually grounded inference as two key characteristics of PersonaRec, but Tables 2 and 3 only report domain-level aggregate results. There is no breakdown by preference type or by text-only versus multimodal interactions, making it unclear which properties contribute most to the benchmark's difficulty or where Yo'Rec provides improvements. In addition, performance on Movies is consistently much lower than on the other domains and relatively close to the Query-Only baseline. For example, under U-Query, GPT-5.6-Sol achieves an R@1 of 2.00 on Movies, compared with 8.20 on Fashion and 8.53 on Food, while Query-Only achieves 0.75 on Movies. However, the paper provides limited analysis of this performance gap.
Questions for authors
Q1: Section 5.2 discusses "Shopping Companion (Tou et al., 2026)", but the cited reference is ShoppingComp (arXiv:2511.22978). The work described appears to be Shopping Companion [1], which also recovers user preferences from multi-session histories for product recommendation and uses SFT followed by GRPO-style training with preference-related rewards. Could the authors correct the citation and clarify how their work differs from Shopping Companion in both benchmark design and training methodology?
[1] Yu, Zijian, et al. "Shopping Companion: Benchmarking and Training LLM Agents for Long-Horizon Preference-Grounded E-Commerce Tasks." arXiv preprint arXiv:2603.14864 (2026).
Q2: For Table 3, could the authors clarify whether self-reranking is consistently applied across all variants? Also, could the authors report results for the base model using the same three-step PRPL-CoT pipeline without SFT, while keeping the retrieval and reranking settings identical?
Q3: To validate the claim in Section 3.1, could the authors compare PRPL-CoT SFT with simpler SFT variants, such as directly rewriting the query, or extracting preferences without first localizing sessions?
5 comments
Response to Reviewer fZAR (2/3)
(Part 2/3)
[W2] (1) Tables 2 and 3 report only domain-level results, with no breakdown by preference type (explicit, implicit, evolving) or by text-only versus multimodal interactions. (2) Movies performance is much lower than the other domains and close to Query-Only, with limited analysis.
(1) Preference-type and modality-level analysis. We sincerely thank you for this valuable suggestion. The reason we do not originally report such fine-grained breakdowns is that preference types (explicit, implicit, and evolving) and modalities (text-only and multimodal) characterize individual interaction sessions, whereas each target item is associated with multiple preference-revealing sessions, potentially involving different preference types and modalities. Thus, a single recommendation instance may combine multiple such characteristics, making a direct final performance breakdown of item-level recommendation metrics less computable.
Nevertheless, following your suggestion, rather than analyzing only the final recommendation performance, we introduce an intermediate metric, Category-wise Preference Recovery Rate (CPRR), to examine preference recovery at the fine-grained level you suggested. Specifically, CPRR measures the proportion of ground-truth preferences successfully recovered within each category. We report CPRR across different preference-expression types (explicit, implicit, and evolving) and evidence modalities, categorized by where the preference value is conveyed: text-only, multimodal with the value in text, and multimodal with the value only in the image.
| CPRR (%) | Explicit | Implicit | Evolving | Text-only | Multimodal (value in text) | Multimodal (value only in image) |
|---|---|---|---|---|---|---|
| GPT-5.6-Sol | 70.20 | 59.80 | 54.40 | 68.10 | 61.30 | 49.80 |
| Qwen3.5-27B | 52.40 | 40.60 | 35.80 | 49.70 | 41.30 | 29.60 |
| Qwen3.5-9B | 31.60 | 20.30 | 15.90 | 28.40 | 21.60 | 13.70 |
| Yo'Rec | 62.70 | 49.80 | 43.60 | 58.20 | 50.40 | 36.90 |
We observe that implicit and evolving preferences are more challenging to recover, while multimodal evidence also increases the difficulty. Nevertheless, Yo'Rec consistently improves the preference recovery performance of the base Qwen3.5-9B model across these challenging scenarios.
(2) Movies. We apologize for the limited discussion due to space constraints. Unlike the concrete attributes of fashion and food, movie preferences often involve abstract aspects (e.g., narrative relationships and emotional intensity; Appendix J), making them harder to infer and map to catalog attributes. This challenge persists even with oracle preferences (Table 4). Nevertheless, human evaluation suggests the task remains solvable, and Yo'Rec substantially improves Movie R@1 from 1.62% to 7.50%. We have revised the paper to add this analysis.
Response to Reviewer fZAR (3/3)
(Part 3/3)
[Q1] Section 5.2 discusses "Shopping Companion (Tou et al., 2026)", but the cited reference is ShoppingComp (arXiv:2511.22978). Please correct the citation and clarify how this work differs from Shopping Companion in benchmark design and training methodology.
We sincerely thank you for pointing out this citation issue. Due to the highly similar titles, we apologize for this mistake and have corrected the citation. Our original discussion in Section 5.2 was indeed intended to refer to Shopping Companion. Below, we further clarify the differences between the two works.
Benchmark Design.
-
Task Definition and Evaluation. Shopping Companion considers a shopping-assistant task. However, its final evaluation uses an LLM-judged Success Rate (Succ.), which assesses whether the recommended product(s) satisfy a set of user needs and recovered preference constraints. Importantly, exact matches to the reference products are not required, as any alternative satisfying these criteria is accepted. Thus, its evaluation focuses on preference- and constraint-satisfying shopping outcomes, rather than retrieving a specific target item. In contrast, LIP-Rec aims to retrieve a specific target item from a fixed candidate pool based on underspecified queries and long-term assistant interactions, enabling controlled target-item ranking evaluation with Recall@K and NDCG@K. This more directly reflects the core objective of recommendation: identifying which candidate item is the best match for the user.
-
Persona Modeling. More fundamentally, PersonaRec explicitly models coherent personas, whereas Shopping Companion does not explicitly construct persistent persona identities with cross-target preference consistency. Instead, it generates target-product-conditioned preference dialogues and plants them into existing LongMemEval histories. In contrast, we construct the entire interaction history for each persona, never expose target items during generation, and reveal only a subset of their attributes through individual sessions. Preferences associated with six targets of the same persona remain mutually consistent across the history.
-
Preference Dynamics and Distractors. PersonaRec systematically models explicit, implicit, and evolving preferences, together with diverse distractors, including domain-related interactions without preference expression and negative preferences. This enables more fine-grained investigation of preference recovery under diverse and potentially misleading signals (you also pointed out this).
-
Modality. Shopping Companion builds on textual LongMemEval histories. In contrast, 57% of PersonaRec sessions are multimodal, including cases where preference values are conveyed only through images (Figure 3). This enables the evaluation of visually grounded preference inference, an increasingly important capability for personal MLLM assistants in this era.
Training Methodology.
Shopping Companion trains an interactive tool-using agent with SFT and RL, employing outcome-level and annotation-free tool-wise rewards to supervise memory retrieval and product search. In contrast, Yo'Rec does not train an interactive tool-using policy; it follows a fixed three-step PRPL-CoT pipeline: session localization, preference extraction, and query rewriting. Leveraging PersonaRec's fine-grained provenance, PRC-PO employs F1-based rewards for session localization and preference recovery, and NDCG@K for recommendation-query effectiveness (Section 3.2 and Appendix F). Finally, we emphasize that Yo'Rec is presented as a simple baseline; our primary contributions are the new LIP-Rec task and its dedicated benchmark PersonaRec.
If our responses, additional experiments, and revisions have addressed your remaining concerns, we would be sincerely grateful if you would kindly consider revisiting your assessment, especially in light of your encouraging recognition of our work. We would also be delighted to further discuss any questions about our work.
Response to Reviewer fZAR (1/3)
Thank you for the careful reading and insightful questions and comments. We sincerely thank you for highly recognizing that (1) our proposed task, LIP-Rec, addresses a clear and practically relevant problem in the era of personalized AI assistants, and (2) PersonaRec is a large-scale benchmark that provides a valuable resource for future research. Below, we carefully address your remaining concerns.
Regarding (1), yes. We consistently apply the same self-reranking procedure across all designed ablation variants, controlling for reranking as a potential confounding factor and allowing the ablation to more directly reflect the contributions of SFT and individual RL rewards. We sincerely apologize for the lack of explanation.
Regarding (2) and (3), we conduct additional experiments comparing Yo'Rec with three variants: PRPL-CoT w/o SFT, Rewriting Only SFT, and w/o Session Localization SFT. The results are summarized below. We observe that none of these variants outperforms the full Yo'Rec.
Specifically, PRPL-CoT w/o SFT directly applies our three-step reasoning pipeline at inference time. However, general-purpose MLLMs are not necessarily equipped with such task-specific reasoning capabilities during pretraining, and thus struggle to execute this pipeline reliably without SFT.
For Rewriting Only SFT and w/o Session Localization SFT, directly processing the long histories makes it difficult to identify and aggregate sparse, query-relevant preferences amid abundant irrelevant or potentially misleading interactions. The former lacks explicit supervision for intermediate preference identification, while the latter skips the localization of relevant sessions. These results validate the necessity of our three-step PRPL-CoT design.