- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
In this paper the authors propose PAT which studies personalized text generation for users with very little past writing. It uses other users’ texts to estimate two things. First, how the target user may write and second, what information is relevant to the current product or topic. It then combines these two reasoning paths to generate the final text and improves them through repeated training based on how well the generated output matches the reference. The method is tested on Amazon reviews, hotel reviews, and product-feedback tasks, for both long- and short-text generation, and is compared with personalization baselines using ROUGE, METEOR, an LLM-based reference score, ablations, and sparse-history analysis.
Critical strengths
None.
Critical weaknesses
-
I am not convinced that this is an important cold-start problem in the first place. A review is mainly about what the user actually experienced with the product. As an example, if Alice finds a lipstick irritating, only Alice knows that. Similar users cannot tell us this. For anything to happen, Alice has to tell the system the important facts herself. In that case a normal writing assistant can already turn those facts into a good review. The proposed short-text task is even harder to motivate as a cold-start problem because the model already sees Alice’s full review and only has to generate a title or summary. So even if PAT works perfectly, I am not convinced that it is solving a problem that really needs this solution.
-
I am also unsure what the reported improvement really means in practice. The main result is that PAT produces text that is more similar to the held-out reference review. The paper itself says that its LLM judge measures similarity to the reference ... this is not the same as whether the output better reflects the user’s real preferences. But what matters to Alice is that the review says what she actually experienced, and what matters to a reader is that the review is truthful and useful -- her style (and hence identity) is of very limited value unless she is an influencer in which case why the cold-start problem even arise (and what stops such a professional Alice from using a voice assistant to do the job?)
Questions for authors
I do not think my concerns can be resolved as of now.
For future work, I think it would be more useful to study a setting where cold-start personalization addresses a clear user need, and where preserving an individual user’s style leads to some meaningful benefit beyond matching a held-out reference.