acceptodds
Withdrawn from ICLR 2027

Sparse Personalized Text Generation with Multi-Trajectory Reasoning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

As Large Language Models (LLMs) advance, personalization has become a key mechanism for tailoring outputs to individual user needs. However, most existing methods rely heavily on dense interaction histories, making them ineffective in cold-start scenarios where such data is sparse or unavailable. While external signals (e.g., content of similar users) can offer a potential remedy, leveraging them effectively remains challenging: raw context is often noisy, and existing methods struggle to reason over heterogeneous data sources. To address these issues, we introduce PAT (Personalization with Aligned Trajectories), a reasoning framework for cold-start LLM personalization. PAT first retrieves information along two complementary trajectories: writing-style cues from stylistically similar users and topic-specific context from preference-aligned users. It then employs a reinforcement learning-based, iterative dual-reasoning mechanism that enables the LLM to jointly refine and integrate these signals. Experimental results across real-world personalization benchmarks show that PAT consistently improves generation quality and alignment under sparse-data conditions, establishing a strong solution to the cold-start personalization problem.

Primary area:
generative models
Keywords:
Personalization, Language Modeling, LLM Applications

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_epnAOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

In this paper the authors propose PAT which studies personalized text generation for users with very little past writing. It uses other users’ texts to estimate two things. First, how the target user may write and second, what information is relevant to the current product or topic. It then combines these two reasoning paths to generate the final text and improves them through repeated training based on how well the generated output matches the reference. The method is tested on Amazon reviews, hotel reviews, and product-feedback tasks, for both long- and short-text generation, and is compared with personalization baselines using ROUGE, METEOR, an LLM-based reference score, ablations, and sparse-history analysis.

Critical strengths

None.

Critical weaknesses
  1. I am not convinced that this is an important cold-start problem in the first place. A review is mainly about what the user actually experienced with the product. As an example, if Alice finds a lipstick irritating, only Alice knows that. Similar users cannot tell us this. For anything to happen, Alice has to tell the system the important facts herself. In that case a normal writing assistant can already turn those facts into a good review. The proposed short-text task is even harder to motivate as a cold-start problem because the model already sees Alice’s full review and only has to generate a title or summary. So even if PAT works perfectly, I am not convinced that it is solving a problem that really needs this solution.

  2. I am also unsure what the reported improvement really means in practice. The main result is that PAT produces text that is more similar to the held-out reference review. The paper itself says that its LLM judge measures similarity to the reference ... this is not the same as whether the output better reflects the user’s real preferences. But what matters to Alice is that the review says what she actually experienced, and what matters to a reader is that the review is truthful and useful -- her style (and hence identity) is of very limited value unless she is an influencer in which case why the cold-start problem even arise (and what stops such a professional Alice from using a voice assistant to do the job?)

Questions for authors

I do not think my concerns can be resolved as of now.

For future work, I think it would be more useful to study a setting where cold-start personalization addresses a clear user need, and where preserving an individual user’s style leads to some meaningful benefit beyond matching a held-out reference.

Reviewer_DKHqOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets personalized product (so-called topic) review generation for cold-start users, who have almost no review history. PAT builds a user–topic bipartite graph, embeds nodes with an off-the-shelf content-independent style encoder, and smooths these embeddings with GraphSAGE. It then retrieves two bags of texts: 1) texts written by stylistically similar users, and 2) other users' texts about the target topic. Two LLM agents summarize these two bags, respectively, into a style summary and a topic summary, and a generator writes the output conditioned on both summaries and the raw retrieved texts. The two summarizers are trained with DPO on candidate summaries ranked by the reward of the generator's final output, and the generator is trained with SFT.

Critical strengths
  1. The problem is meaningful, and the design is reasonable and practical. All modules are shared across users, so no per-user training is needed.

  2. The paper also includes a small human study on the validity of the LLM judge.

Critical weaknesses
  1. The experimental setup is too limited to support the claims.
  • All three datasets come from one benchmark (PGraphRAG), and all have the same two templated tasks.
  • PAT is fine-tuned, but its main baselines are zero-shot prompting.
  • The training reward is ROUGE against the reference, which is also the main evaluation metric.
  1. The method is heavily heuristic, over-engineered, yet under-specified and under-evaluated in the paper, which raises questions about its generalizability.
  • The pipeline chains a style encoder, a GraphSAGE smoothing stage, two retrieval strategies, two DPO-ed summarizer agents, and an SFT-ed generator. Each stage carries multiple hand-chosen hyperparameters or hand-written prompt templates. Given the large hyperparameter space, the results on the small datasets used can be easily inflated by extensive searching or coincidence.
  • The paper doesn't specify the exact reward, how the "silver" summaries are chosen, etc.
  • Ablations are coarse. For example, the authors did not test whether GraphSAGE beat simple baselines, or whether DPO adds anything over SFT.
  1. Formatting problems.
  • In Table 9, PAT's judge score is 3.744 in the PAG block but 3.577 in the PerFine and Sonnet blocks, while the other scores remain identical. This looks like a reporting error.
  • The header reads "ICLR 2026".
  • Several citations render as "?".
  • The captions of Tables 7 and 8 refer to "four" datasets when three are listed.
  • The bibliography entry for Claude still contains "Prompt: [Your Prompt Here]".
Questions for authors

Please see weaknesses above.

Reviewer_FX4XOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PAT, a framework for personalized text generation under sparse user histories. The method retrieves auxiliary writings through two channels: stylistically similar users and topic-related users. Two language-model agents summarize these contexts separately, and a generator uses the resulting summaries to produce personalized text.

Critical strengths

The paper addresses a meaningful problem in personalized text generation. When user histories are limited, retrieving writings from other users provides additional information, but this information may contain conflicting styles and opinions. Separating stylistic information from topic-related content is a reasonable approach to addressing this challenge.

Critical weaknesses
  1. A central motivation of PAT is to separate heterogeneous auxiliary information into complementary signals for writing style and topic-specific preferences. However, the proposed retrieval mechanism does not clearly establish this distinction. In Section 3.1.2, topic-related neighbors are selected using the same GraphSAGE representations employed for style-based retrieval, whose initial features are derived from a style encoder. Moreover, the topic reasoning agent in Eq. (4) does not directly condition on the target user's history. Therefore, it is unclear whether the retrieved topic context captures preferences aligned with the target user or primarily provides general topic information from other users. While this information may still improve generation quality, the current experiments do not establish that the two trajectories capture distinct and complementary personalization signals.

  2. The paper attributes its improvements to the ability of the proposed reasoning trajectories to filter noisy information and jointly refine heterogeneous signals. However, the actual role of these intermediate trajectories remains unclear. According to Eq. (5) and Appendix D, the final generator receives both the trajectory summaries and the original retrieved contexts. Thus, it is difficult to determine whether the summaries effectively filter irrelevant information or simply provide additional guidance while the generator retains access to the complete context. Furthermore, the two reasoning agents independently summarize their respective inputs, without explicit information exchange during inference. This makes the distinction between the proposed multi-trajectory reasoning mechanism and a simpler context summarization framework less clear. The existing ablations in Table 5 show that removing individual trajectories or eliminating training reduces performance, but they do not sufficiently isolate the benefits of the proposed design.

  3. Although PAT achieves promising results on reference-based metrics, these metrics do not directly measure whether the generated text reflects the target user's individual preferences. As explained in Appendix A.5, the model still receives the current task input and information associated with the target topic. For short-text generation, the input already contains the user's review. Consequently, performance gains may partly reflect improved content modeling or reference reconstruction, rather than better recovery of previously unknown user preferences.

  4. The paper would benefit from more careful proofreading. There are many grammatical errors and typos throughout the manuscript, such as "an topic knowledge trajectory", "ablation studies that explores", and "In this section, We ablate". Several unresolved citation placeholders also remain in line 351 , 465 and 467. Furthermore, Tables 7 and 8 refer to four datasets or domains, although only three are presented. The authors should also ensure consistent terminology and accurate descriptions of the experimental results, particularly in Section 5.

Questions for authors
  1. How do the authors verify that the topic-related neighbors selected using GraphSAGE representations are aligned with the target user's preferences, rather than merely sharing similar topics or writing styles? For example, two users may frequently discuss electric vehicles but hold opposing views on their adoption. Although their representations may be similar due to shared topical interests, their underlying preferences could differ substantially. It is unclear whether the proposed neighbor selection mechanism can distinguish topical similarity from actual preference alignment.

  2. Can the authors provide comparisons with a single-summary agent or an SFT-only variant under comparable training budgets? In addition, how much performance is retained when the final generator receives only the trajectory summaries without the original retrieved contexts?

  3. What exact reward function is used to construct the DPO preference pairs in Eqs. (6)–(9)? How is generation variability controlled when comparing candidate trajectories? These details are important for understanding the reliability of trajectory-level credit assignment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.