- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3(+1)
- Soundness
- 2
Read review
Paper summary
The paper studies query-conditioned preference discovery from long & short noisy user trajectories and proposes to produce a compact preference memo that a downstream task-facing model can use for personalization. The framework can be viewed as a structured teacher-guided preference-learning pipeline where the student is first asked to imitate frontier model-produced demonstrations, then learns to propose additional preference cues under the same frontier model acting as verifier to guide the student model, and finally the student model is expected to generate task-facing memo independently from the history and the query. The methodology separates preference recognition from preference expression, as the proposal stage is intended to improve recognition of supported and query-relevant preferences, and the memo stage learns to express those preferences in a concise and usable form. The paper evaluates both intermediate preference reasoning through coverage, faithfulness, grounding, and proposer acceptance, and downstream personalization through product/movie selection and personalized response generation.
Critical strengths
1. Structured Decomposition of the Approach
The explicit decomposition of preference learning into teacher-guided initialization, verifier-guided preference discovery, and finally independent memo generation is an interesting way of addressing personalization over long histories. Meaningful separation of between preference recognition and expression is useful and evaluation focusing on understanding the model's ability to identify history-supported and query-relevant preference information before expressing it in memo is thoughtful and appreciable.
2. Interesting Results Pattern
The empirical analyses reveal meaningful behavior. The proposer and memo phases affect recognition, coverage, faithfulness, and grounding differently, and the experiments provide evidence that proposer training teaches something that memo-only training does not capture. The matched proposer-vs-memo training comparisons, multi-round preference discovery analysis, and verification-path ablations make internal behavior interpretable.
Critical weaknesses
1. Lack of Evaluation Coverage and Claim Scope wrt (Primary Blocker)
-
Downstream evaluation coverage is incomplete relative to broader personalization positioning. The main comparisons are against alternative history, retrieval, memory, and memo-trained approaches consumed by generic LLMs, with limited comparisons against established task-specific personalization models such as PENS, NAML, EBNR, LSTUR, MAMBA, SSD4Rec, GRU4Rec, and so on. This makes it difficult to determine whether is primarily a better preference interface for a generic LLM or whether it is a competitive benchmark against established specialized personalization models for downstream tasks.
-
The PersonaMovielens results are not convincing as evidence for query-relevant preference discovery. The target movie is retrospectively selected because the user later rated it 5 *, while the query is synthetically constructed using properties related to the target. In this setting, downstream success can entangle query-movie compatibility with history-conditioned personalization. More importantly, a later rating represents \textbf{\textit{post-consumption utility or satisfaction}}, not the true preference dimensions that caused the user to be interested in the movie. A movie may strongly match a user's prior preferences before watching but still receive a low rating because of the experienced quality of the acting, story, or other aspects that are not represented by the historical preference histories and records. Thus, this experiment is a weaker validation of the claim concerning recovery of the query-relevant preference state that should guide a current decision.
2. Dependence on the Frontier model, which is under-examined (Primary Blocker)
-
is substantially dependent on frontier-model supervision. Frontier models provide the initial memo/proposal demonstrations, contribute to the initial preference inventory and evaluation reference sets, and provide the semantic verification signals that determine proposer rewards and memo supervision.
-
The empirical evidence settings carefully ablate the structure of the verification path and study single-judge instability, so the issue is not that verifier quality is completely unexamined. However, these experiments do not establish whether the learned preference reasoning and downstream gains are robust to changing the actual teacher/verifier model or to a weaker or systematically different teacher. The alternative Gemini-built reference-pool experiment addresses robustness of evaluation ordering, but it does not test training-time teacher/verifier dependence because is not retrained under an alternative supervision model.
Questions for authors
-
Any comparisons against specialized SOTA personalized benchmarks like MIND, Movielens, LAMP, and so on, relative to earlier stated models?
-
Any clear ablation showing that pre-consumption interest is a weaker personalization setting than post-consumption rating? How can rating depend on user's prior history in any settings?
-
How sensitive is training to the identity and quality of the frontier teacher/verifier?
-
A direct correlation or stratified analysis between preference-reasoning metrics and downstream task success would help establish whether the improved intermediate reasoning is actually responsible for the downstream gains
9 comments
Response to Reviewer 6zSi (2/3): PersonaMovieLens and post-consumption ratings
W1b/Q2: Post-consumption ratings in PersonaMovieLens
We agree that post-consumption ratings reflect both initial consumption choice and subsequent satisfaction. We built PersonaMovieLens on MovieLens because it is a standard public dataset with long rating histories and with movie genres and tags. Since MovieLens logs ratings post-watching, PersonaMovieLens operationalizes personalized search as identifying the user's highest-satisfaction item (rating = 5.0) among candidate movies consumed post-cutoff. The negatives are lower-rated movies that also yield lower preference-match scores against the user's history. Modeling future ratings based on historical interactions remains a standard task in collaborative filtering [7]. GPT-5.5 writes the query from the target's genre family, without its title, year, tags, or synopsis (Appendix C.1). Amazon product-search benchmarks build queries from item categories in a similar way [8–10].
To address your concern, we vary how queries and candidates are built. One setting is close to choosing before watching (pre-consumption interest): all candidates match the query, but the user rated only the target. To isolate variables, we changed one component at a time. (1) Genre-only queries name only the target's high-level genre. (2) Unrated negatives are drawn either uniformly at random or from query-relevant movies. GPT-5.6 Luna judges this relevance from the query and movie metadata alone, without the user history.
| PersonaMovieLens (Hit@1) | Query only | Full history | D | PREMO |
|---|---|---|---|---|
| As in the paper | 62.4 | 64.9 | 72.8 | 73.7 |
| Genre-only query | 47.4 | 50.9 | 66.7 | 69.7 |
| Random negatives the user never rated | 82.6 | 83.8 | 85.4 | 86.5 |
| Query-relevant negatives the user never rated | 43.8 | 45.1 | 51.8 | 53.6 |
When we change the query or the candidates, PREMO still leads D and the baselines. Its gain over the query alone is largest with the genre-only query. The gain also holds in the setting close to choosing before watching. The history thus helps predict both which movie the user chooses and how they rate it. Our other two tasks do not use ratings. The APeB target is a purchase, and PersonaMem-v3 scores replies.
We also gave PREMO another user's history. Its Hit@1 drops by 8.1 points (73.7 to 65.6). PREMO's gain therefore comes mainly from the user's own history.
[7] Collaborative filtering with temporal dynamics, 2009.
[8] Learning latent vector spaces for product search, 2016.
[9] Learning a hierarchical embedding model for personalized product search, 2017.
[10] A transformer-based embedding model for personalized product search, 2020.
Response to Reviewer 6zSi (3/3): frontier supervision and the metric–downstream link
W2/Q3: Sensitivity to the teacher and verifier
Our method is generally robust to the choice of teacher and verifier. In our current pipeline, GPT-5.4 generates the SFT warm-up data, the initial inventory, and the references E(x), and GPT-5.6 Luna serves as the RL verifier (Appendix C.10). We chose these models for cost and availability. At deployment, the 4B model writes the memo alone, with no frontier calls.
To test the sensitivity to the SFT warm-up with GPT-5.4, we ran an ablation without SFT warm-up. This run also replaces GPT-5.4's initial inventory with GPT-4.1 rubrics.
As shown below, even trained with RL alone (omitting SFT warm-up), PREMO already outperforms the teacher's memo across all tested downstream policies (e.g., +3.0 over GPT-5.4 memo when evaluated with GPT-5.4). SFT gives a better initialization (+3.8), but PREMO does not depend on SFT to surpass the teacher.
| APeB Hit@1 | GPT-5.4 memo | PREMO, RL only | PREMO with SFT |
|---|---|---|---|
| GPT-4.1 | 67.1 | 68.6 | 70.3 |
| GPT-5.4 | 66.6 | 69.6 | 73.4 |
| GPT-5.5 | 69.4 | 70.4 | 72.4 |
We further ablate the verifier with Qwen3.8-27B.
With an open model as verifier, PREMO performs as well as with GPT-5.6 Luna. We retrained PREMO on PersonaMem-v3 for one round with the open model Qwen3.8-27B as verifier and memo judge. It runs on four H100s. The teacher and the evaluation stay the same.
| PersonaMem-v3, after one round | Cov. | Faith. | Ground. | Task | Training cost |
|---|---|---|---|---|---|
| GPT-5.6 Luna as verifier and judge (paper's run) | 74.7 | 83.7 | 64.3 | 82.1 | 80–88 H100-hours, USD 110–133 |
| Qwen3.8-27B as verifier and judge | 75.0 | 87.1 | 65.0 | 81.5 | about 130 H100-hours, no API |
Open-model verification matches or exceeds frontier verification on all intrinsic memo metrics, with comparable downstream performance and no API cost. The improvements of PREMO thus come mainly from the method's design, which does not rely on a specific frontier model. Each ablation removes one frontier dependence. RL-only training uses neither the SFT data nor the initial inventory of GPT-5.4, and the open verifier replaces GPT-5.6 Luna.
Q4: Do memo metrics explain downstream success?
We would like to note that high-quality memos are important to the success of downstream outcomes. The improved memo quality of PREMO can effectively bring downstream improvements. For each task, we sampled 500 memos from PREMO's checkpoints after each round, evenly across bins of the score being tested. We then computed the Pearson correlation between the score and the instance's downstream outcome (Hit@1, or the task score on PersonaMem-v3). We tested coverage alone and an aggregate score, 0.5 × coverage + 0.25 × faithfulness + 0.25 × grounding:
| Pearson r with downstream success | Coverage | Aggregate score |
|---|---|---|
| APeB | 0.249 | 0.215 |
| PersonaMem-v3 | 0.135 | 0.173 |
| PersonaMovieLens | 0.219 | 0.265 |
All correlations are positive. How much personalization can help differs across queries and tasks [11].
We hope these results resolve the main questions and are glad to provide further details.
[11] To personalize or not to personalize: Modeling queries with variation in user intent, 2008.
Response to Reviewer 6zSi (1/3): specialized personalization models
Thank you for the review. We ran new experiments for both primary blockers. This comment covers specialized models (Q1). Comment 2 covers PersonaMovieLens (Q2). Comment 3 covers the frontier teacher and verifier (Q3) and memo metrics (Q4).
W1a/Q1: Specialized personalization models
PREMO is a semantic preference interface for LLM decision-makers. Like other LLM-based personalization methods [1], it could complement ID-based recommenders. We trained four representative specialized models on our training splits and evaluated them on the same candidate sets as PREMO: GRU4Rec [2] and SASRec [3] (sequential recommenders), SVD++ [4] (collaborative filtering), and UniSAR [5] (joint search and recommendation with query input). GRU4Rec and SASRec also represent the listed sequential models MAMBA and SSD4Rec. PersonaMovieLens is built on MovieLens 32M (Appendix C.1). We therefore also trained these models on all 32M ratings, excluding the test users' later ratings. These ratings come from about 200k users, and our training split has 2,500. We skip SVD++ on APeB because it needs explicit ratings. The other listed models (NAML, LSTUR, EBNR, PENS) target news and take no query. We did not evaluate on MIND and LaMP. Neither has a query that states the user's current intent.
With the same training data, every ID-based model stays well below PREMO. SVD++ overtakes PREMO 4B only with all 32M ratings, and a larger LLM reading the same PREMO memo overtakes SVD++. On APeB, the ID-based models are close to random choice. On PersonaMovieLens, SVD++ trained on our training users only ties the untrained 4B that reads the query alone. PREMO itself is trained on our 2,500 training users only.
This is because our tasks depend on understanding the user's history and intent, and personalization data are often too sparse to learn item-ID embeddings. First, ID-based models learn from the relations between item IDs in many users' histories. Text embeddings help UniSAR, but it is still far below the untrained 4B that reads the query alone. Encoding item text does not provide this understanding. Second, APeB [6] has only about 6k sessions. In our APeB split, 29.1% of the test candidates never appear in the training histories, and 11.9% appear in only one history. MovieLens 32M is dense by comparison. With these ratings, SVD++ can learn good ID embeddings.
PREMO and ID-based models capture complementary signals, even when SVD++ is trained on all 32M ratings. ID-based models learn population-level patterns from many users. PREMO extracts the individual user's intent from their records and query. We compared the two on each PersonaMovieLens test instance: the PREMO memo with Qwen3.5-27B as ranker, and SVD++ trained on all 32M ratings (% of instances):
Each is right on more than 70% of the other's errors. An oracle that picks the right one for each instance would reach 94.6% Hit@1. The difference is largest on new items. On APeB, about 25% of the targets never appear in training. The memo-based policy ranks 67% of these targets first, and each ID-based model ranks 2.8% first. Bringing PREMO's memo into conventional recommenders (e.g., via KAR [1]) is a promising direction for future work.
[1] Towards open-world recommendation with knowledge augmentation from large language models, 2024.
[2] Session-based recommendations with recurrent neural networks, 2016.
[3] Self-attentive sequential recommendation, 2018.
[4] Factorization meets the neighborhood: A multifaceted collaborative filtering model, 2008.
[5] UniSAR: Modeling user transition behaviors between search and recommendation, 2024.
[6] APeB: Benchmarking personalization ability of large language model agents, 2026.