acceptodds
Withdrawn from ICLR 2027

PREX-Rec: Provenance-Aware Experience Transfer for Sequential Recommendation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reusing a recommendation experience across users requires deciding whether its guidance fits the target user's behavior. Textual similarity alone does not explicitly encode the source interactions associated with that guidance. We propose PREX-Rec, which pairs each natural-language experience with LLM-selected items from its source history and uses these associations for retrieval. A user-item-experience relational index connects target histories to experiences through shared items, weighted by evidence frequency and recency. An LLM screens retrieved experiences for applicability; independent experience-guided rankings are then fused with a base ranking. On the evaluated Amazon Games and Arts candidate-ranking tasks, PREX-Rec improves all reported metrics over five LLM-based baselines. Games NDCG@5 increases from the strongest baseline's 0.2918 to 0.3291. With independent integration fixed, item-based retrieval improves over the evaluated vector retriever; perturbing experience-item assignments within source histories lowers Games NDCG@5 to 0.3086. These results support the utility of experience-specific item associations within the evaluated retrieval-and-ranking framework.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Sequential Recommendation, Large Language Models, Recommender Systems, Experience Management, Graph-Based Retrieval, Cross-User Knowledge Transfer

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_wRdmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PREX-Rec, a provenance-aware experience transfer framework for sequential recommendation. It links recommendation experiences to supporting historical items, retrieves relevant experiences through a user–item–experience graph, and integrates experience-guided rankings with base recommendations. Experiments on Amazon datasets demonstrate improvements over existing LLM-based recommendation methods.

Critical strengths
  1. The paper addresses the applicability of cross-user experience transfer in sequential recommendation, a problem with clear research significance.

  2. The method links recommendation experiences to historical interaction items and enables experience retrieval and reuse through an explicit relational graph.

  3. The method outperforms existing LLM-based recommendation baselines across multiple datasets, with ablation studies and different LLM backbones further supporting its effectiveness.

Critical weaknesses
  1. The proposed method mainly combines existing experience retrieval, item association, and ranking fusion techniques, with limited substantive algorithmic innovation.

2.Graph and vector retrieval use different information, while independent and joint ranking are not evaluated under matched computational budgets, making it difficult to isolate the contribution of the core mechanisms.

3.The paper lacks sufficient evidence for the reliability of experience–item associations, and the cross-user experiments do not fully control for differences in the number of applied experiences.

Questions for authors

Please refer to Weaknesses

Reviewer_rJtPOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PREX-Rec, an LLM-based recommendation framework that leverages experiences derived from LLM reasoning across users with overlapping interaction histories rather than relying on textual similarity. It focuses on selecting the appropriate reasoning results to guide recommendation. PREX-Rec consists of an offline phase and an online phase and does not require training. In the offline phase, it extracts experiences from users and constructs an user-item-experience graph. In the online phase, it retrieves relevant experiences and uses them to improve ranking for next-item prediction.

Critical strengths
  • Unlike conventional recommendation models that represent users and items as embeddings, this framework operates on human-readable textual information. Therefore, the rationale behind each recommendation is directly interpretable.
Critical weaknesses
  • The framework depends heavily on LLM inference, particularly in the online phase. Specifically during online phase, LLM inference needed for a single user includes LLM-based applicability selection step, base ranking step and experience-guided ranking step. This requires at least three LLM calls per user. Moreover, up to five LLM calls are needed when three experiences are selected as useful according to the implementation details. Since recommendation is a latency-sensitive task, this overhead raises concerns about whether the framework is suitable for real-time deployment.

  • For a user to receive guidance from an experience, the exact same item must appear in that user’s interaction history. This requirement limits generalization. For example, two users who are both looking for games with strategic depth cannot benefit from each other’s experiences unless their interaction histories share at least one item. Furthermore, this constraint limits the framework’s effectiveness for long-tail items with rare interaction frequency.

Questions for authors
  • Could the authors report the total inference time across all users? The appendix provides an approximate number of LLM calls, but reporting the time would give a clearer picture of the computational cost.

  • Have the authors conducted a hyperparameter analysis? The implementation details state that three trajectories are sampled per user and that at most three experiences are selected per user during inference. These appear to be hyperparameters that could affect both performance and cost, so an analysis of their impact would help understand the practicality of this paper.

Reviewer_me7TOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how to reuse natural-language recommendation guidance across users in LLM-based candidate ranking. Offline, the method samples reasoning trajectories from training cases, compares the resulting rankings with the known next item, and extracts reusable experiences from successful and failed attempts. Each experience contains an applicability condition, actionable guidance, and a set of supporting items chosen from the source user's history, and redundant or ineffective experiences are filtered using the source case. Experiences are linked to their evidence items, and for a new user the experiences whose evidence overlaps with the observed history are retrieved with a fixed weighted score. An LLM then selects a few applicable experiences, each selected experience produces its own ranking, and these rankings are combined with the base ranking by a deterministic rank fusion rule. The method is evaluated on two product domains against several LLM-based recommenders, with additional checks on another domain and another backbone, and with controls on retrieval, evidence assignment, and cross-user reuse. The paper states that it does not establish superiority over matching with the full source history or over compute-matched ensembles of rankings.

Critical strengths

S1. The method is described clearly and in a way that is easy to inspect. The prompts for each stage, the deterministic retrieval and fusion formulas, and a stage-level breakdown of model calls are documented, so a reader can see how each part of the pipeline works and what it costs in calls. This makes the proposed idea and its evaluation easier to follow and to reproduce.

S2. The paper is candid about the limits of its evidence and includes targeted controls. It states what its comparisons do not establish, such as the absence of a full-history baseline, the lack of compute-matched comparisons, and the shared-pool protocol, and it adds controls that remove cross-user experiences, randomize retrieval, and perturb the evidence assignment. This gives readers a realistic view of what the reported gains can and cannot support.

Critical weaknesses

W1. The contribution is incremental, and its main novel claim is not isolated by the experiments. The experience format, its extraction from successful and failed trajectories, and source-case filtering follow existing experience and reflection-based designs, and fusing several independently generated rankings is an ensemble technique. The new element is linking each experience to selected evidence items and retrieving by weighted item overlap, which amounts to an inverted index over item identifiers. The claim that experience-specific evidence is better than the alternatives is not tested directly, since the paper acknowledges there is no comparison against attaching the full source history or a text retriever given the evidence titles. The benefit over collaborative filtering is also not established. It is therefore unclear what is new and whether the gains come from the evidence links, the experience text, or the ensembling.

W2. The evaluation does not firmly support the reported improvements. Each case uses a candidate set of about fifty items that always contains the target, with randomly sampled negatives and no retriever-generated or full-catalog setting, which is known to be inconsistent with full-ranking evaluation. The closest experience-based baseline is reproduced far below its original report, and the adaptation of the other baselines is not described. The comparison is not compute-matched, since the method issues several times as many model calls per test user, latency and tokens are not measured, and random experiences with the same fusion already give part of the gain. Ablation differences are small, come from a single run, and have no significance tests. Together these points make it difficult to judge how much of the improvement is attributable to evidence-linked retrieval.

Questions for authors

Q1. Each test case uses a candidate set in which the target is always included and the remaining items are sampled at random. I wonder how the negatives are sampled, including whether items from the user's history are excluded, and how realistic the authors consider this setting for a deployed recommender. It would be helpful to see how the results change with the size of the candidate set and the difficulty of the negatives, for example popularity-matched ones, and with candidates produced by an actual retrieval stage without inserting the target.

Q2. The method uses several model calls per test user, and the cost is reported only as call counts. I am curious about the average number of tokens and the median and tail latency per request, and whether the calls are executed in parallel where possible. It would also help to see how the performance compares with the baselines under a matched budget of calls or latency.

Q3. I wonder what motivated generating a separate ranking for each selected experience and combining them with the exponentially decaying rank fusion, in which shorter guided lists receive larger weight. It would be helpful to compare against a simple ensemble that uses the same number of calls, for example fusing several base rankings with the same rule, and against a standard rank fusion method, to clarify how much of the gain comes from the experiences rather than from fusing multiple outputs.

Q4. For the experiment with a second backbone, I wonder whether the experience pool was rebuilt with that backbone or reused from the original one. It would be helpful to see the comparison with the baselines on this backbone as well, and an evaluation in which the users used to build the experiences are disjoint from the evaluated users, since the current pool is built from the training cases of the evaluated users.

Q5. I am curious whether the gain comes from the experience text or from the collaborative signal carried by the item-overlap retrieval. It would be informative to compare against simple variants that use the same overlap to retrieve neighbors and give their next items as hints, that attach the full source history instead of selected evidence, and that give a text retriever the evidence titles. This would help me see what the selected evidence contributes beyond these alternatives.

Reviewer_rS2aOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces PREX-Rec, a provenance-aware experience transfer framework for LLM-based sequential recommendation. The authors argue that existing experience-reuse approaches often rely on textual similarity between user histories and previously generated recommendation experiences, which may fail to capture whether an experience is behaviorally applicable to a target user. To address this limitation, PREX-Rec associates each natural-language experience with a subset of items selected from the source user's observed interaction history. These associations serve as item-level behavioral provenance, enabling experiences to be retrieved based on shared interactions rather than textual similarity alone. The framework consists of several components:

  1. Experience construction: Multiple recommendation trajectories are sampled for historical users. Successful and unsuccessful trajectories are distilled into reusable recommendation guidance, together with applicability conditions and LLM-selected supporting items.
  2. Provenance-aware retrieval: A user–item–experience relational index connects target histories to experiences through shared items. Retrieval scores incorporate item overlap, inverse-frequency weighting, and interaction recency.
  3. Experience applicability selection: An LLM determines which retrieved experiences are relevant to the target user's current behavioral trajectory.
  4. Independent experience-guided ranking: Each selected experience generates a separate candidate ranking, which is combined with an experience-independent base ranking using deterministic rank fusion. The authors evaluate PREX-Rec on the Amazon Games and Arts datasets using GPT-4o-mini, with additional experiments on Musical Instruments and Qwen3.5-Flash. PREX-Rec outperforms five LLM-based baselines across all reported metrics on the two main datasets. For example, Games NDCG@5 improves from 0.2918 for the strongest baseline to 0.3291, while Arts NDCG@5 improves from 0.3080 to 0.3307. Ablation studies further suggest that experience-specific item associations, graph-based retrieval, and cross-user experience reuse contribute to performance. Overall, the paper presents a structured approach to retrieving and reusing recommendation experiences based on their associated behavioral evidence.
Critical strengths

S1. Clear motivation and meaningful problem formulation. The paper identifies an important limitation of text-based experience retrieval: semantically similar guidance may correspond to different behavioral contexts, while behaviorally relevant experiences may be expressed using different language. Associating experiences with specific source-history items provides a natural mechanism for incorporating behavioral evidence into retrieval. The formulation is intuitive and addresses a meaningful challenge in LLM-based recommendation systems. S2. Explicit and interpretable experience representation. PREX-Rec represents each experience as a natural-language instruction paired with selected supporting items. This representation makes the retrieval process relatively transparent. A retrieved experience can be traced to specific shared items, allowing its behavioral relevance to be inspected more easily than with purely embedding-based retrieval. The distinction between experience-specific evidence and complete source-user histories is particularly interesting, since different experiences from the same user may be associated with different subsets of interactions.

Critical weaknesses

W1. The contribution of provenance-aware retrieval is not sufficiently isolated from simpler item-based matching. The central claim is that associating each experience with selected source-history items improves cross-user experience retrieval. However, the paper does not compare PREX-Rec against a simple item-overlap retriever that uses the entire source-user history rather than experience-specific evidence subsets. W2. The graph-versus-vector comparison is not information-matched. In Section 5.3, graph retrieval uses exact item identities associated with each experience, whereas vector retrieval uses semantic similarity between user-history text and abstract experience descriptions. The authors explicitly acknowledge that these methods do not receive identical information. Consequently, the observed improvement cannot be attributed solely to the proposed relational retrieval structure. The vector baseline may perform worse because the experience descriptions intentionally exclude concrete item names, while the graph method retains access to those identities. A stronger comparison would provide the vector retriever with the titles or semantic representations of supporting items, ensuring that both methods have access to comparable behavioral information. Without such a control, the superiority of provenance-aware retrieval over semantic retrieval remains insufficiently established. W3. The evidence-randomization ablation is limited and does not establish the reliability of provenance assignments. The evidence-randomization experiment is useful but insufficient to fully validate the proposed item-level provenance mechanism.

Questions for authors

Q1. Necessity of experience-specific provenance Could the authors compare PREX-Rec against a simpler retrieval baseline that associates each experience with the complete source-user interaction history rather than an LLM-selected subset of supporting items? This would help determine whether experience-specific provenance provides benefits beyond conventional item-overlap matching. Q2. Information-matched semantic retrieval The graph retriever uses explicit supporting item identities, while the vector retriever relies on abstract experience descriptions. Could the authors evaluate a vector retrieval baseline that also incorporates the titles or embeddings of the selected evidence items? Would the advantage of graph retrieval remain when both methods have access to equivalent behavioral information? Q3. Reliability of LLM-selected evidence items How accurately do the LLM-selected evidence items reflect the behavioral conditions described by their associated experiences? Could the authors provide qualitative examples or human evaluations of evidence quality? Additionally, would the results remain stable under multiple evidence-randomization seeds or stronger perturbations that substantially reduce overlap with the original evidence assignments?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.