acceptodds
Under review as a conference paper at ICLR 2027

FIRM: Fine-grained Item Relational Memory for LLM-based Recommender Agents

Abstract

LLM-based recommender agents store user preferences and item characteristics in natural-language memory, and reason over item text and interaction history to rerank candidate items. Recent work improves this reranking with two predefined item relation types, substitute and complement, which reveal user intent implicit in interaction data. However, these types cannot capture the specific comparison criterion: regular vs. decaffeinated beans and small vs. bulk packs are both substitutes, yet differ in caffeine content and package size, respectively. Moreover, the relations are re-inferred for every user rather than stored as knowledge shared across users. Simply generating fine-grained relations produces redundant types with inconsistent definitions. In this paper, we propose Fine-grained Item Relational Memory (FIRM), a shared memory that organizes fine-grained item relations into a consistent type hierarchy. FIRM is built by three agents: the Explorer extracts a fine-grained relation for each item pair; the Manager organizes the types into a relation tree by merging redundant ones and splitting those that mix distinct criteria; and the Verifier checks type definitions for consistency. During inference, the relations among the user's history items are summarized into a user memory of behavioral tendencies. The recommender agent then retrieves the relations between history items and candidates from FIRM and reranks the candidates based on how well these relations match the user's tendencies. On three real-world datasets, FIRM improves NDCG@10 over the strongest baseline by up to 10.03%, and a memory built once remains effective across different LLMs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
recommender systems, LLM-based agents, agent memory, item relations

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7Jj4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FIRM builds a shared fine-grained item-relation memory for LLM-based reranking. An Explorer proposes substitute/complement relations for selected item pairs. A Manager merges and splits those proposals into a relation tree. A Verifier checks assignments and revises definitions. At inference, relations among history items are summarized into a short user memory, and relations between history items and candidates are retrieved for the LLM ranker. Candidates come from SASRec (20 items). If the ground-truth item is missing, it replaces the lowest-ranked candidate — this happens for 83.5–88.7% of test users. On Sports / Home / Grocery, FIRM improves NDCG@10 over the strongest baseline by up to 10.03% on Grocery. A memory built once with Qwen2.5-7B can be reused by other reranking LLMs.

Critical strengths
  1. The motivating example is good. Regular vs decaf beans and small vs bulk packs are both "substitutes," but the comparison criteria differ. Storing a shared typed relation with a definition is a more sensible object than regenerating substitute/complement lists per user.
  2. The experimental package is careful and easy to inspect. Prompts, thresholds, and baseline adaptations are documented. The main table uses paired user-level tests. Ablations remove Manager, Verifier, user memory, and fine-grained labels. Transfer across Qwen sizes and to Llama-3.1-8B / Mistral-7B is reported, including cells where MemRec or AgentCF wins. Offline construction is 3.4–6.5 hours (cheaper than AgentCF); online reranking uses two LLM calls.
  3. Table 10 (coverage split) is the most informative analysis. Gains concentrate on users for whom some relation is actually retrieved. Users with no relation line look close to Zero-shot, and are slightly worse on Home/Grocery. That is exactly the kind of check a memory paper should include.
Critical weaknesses
  1. The evaluation is conditional on the target already sitting in a 20-item list. The target is inserted for 88.6% / 88.7% / 83.5% of test users. Since all methods share candidates, the comparison is fair as a reranking study. It does not tell us whether FIRM helps the pipeline a practitioner would actually run, where most of these targets were never retrieved in the first place. The "up to 10.03%" abstract number is a gain on that inserted list.
  2. The hierarchical part — which is the novel piece — moves the metric only a little. Removing both Manager and Verifier drops NDCG@10 by 3.2%, 2.0%, and 3.8%. Removing Verifier alone changes Home NDCG@10 by about 0.03%. There is no flat-clustering / length-matched control that feeds the Explorer's raw strings to the reranker under the same token budget. Extra relation text in the prompt can help without the tree being necessary.
  3. Relation quality is only measured indirectly via NDCG. There is no human (or independent judge) sample of pair→type assignments, and no real subtree example in the main text. The Verifier also leaves known-bad assignments in place: 676 / 2,651 / 1,136 pairs were judged misfit and kept because no revised proposal was produced (~6–10% of assigned pairs). A fluent but partly wrong taxonomy can still change a prompt.
  4. The large subgroup gains are not cleanly causal. On Home, users with relations on both sides already have Zero-shot NDCG@10 = 0.3205 vs 0.2847 overall, and FIRM's gain on that cell is +0.2015. Those users are easier before any relation text is added. A shuffled-relation control on the same users is missing. The global "remove relations" ablation does not answer this.
  5. Coverage is limited, and the relation inventory is narrower than the introduction suggests. History–history relations exist for only 27.1% / 31.5% / 49.5% of test users. The Explorer only emits Sub / Com / None, and most assigned pairs are substitutes. The paper does not analyze what the discarded None pairs actually contain.
Questions for authors
  1. Evidence: what happens if the target is not inserted? Please split users whose target SASRec already retrieved from users who needed insertion. This is the result that would most change my score.
  2. Evidence: against a control that passes the Explorer's raw (category, name, sentence), truncated to the same token budget, with no Manager/Verifier, how much gain remains? The current ablation still changes what text the ranker sees.
  3. Clarification: on a sample of assigned pairs, what fraction would a second judge say match the stored definition? How should the kept misfit pairs be counted?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 7Jj4 (Part 1/2)

Thank you for carefully examining the evaluation protocol and relation quality, and for proposing specific control experiments.
We will incorporate the additional experiments and analyses into the revised manuscript.

 

W1 Q1 Reranking by target retrieval status

We compared NDCG@10 between users whose targets were retrieved by SASRec (without insertion) and users who required insertion (evaluated with insertion). The groups contain 1,241 / 1,418 / 1,211 and 9,632 / 11,139 / 6,128 users on Sports / Home / Grocery, respectively. Values for the insertion-required group were estimated from the overall results in Table 1, the scores for target-retrieved users, and the two group sizes.

MethodSports (without insertion)Home (without insertion)Grocery (without insertion)Sports (with insertion)Home (with insertion)Grocery (with insertion)
FIRM.6393.7482.7665.2469.2851.2846
AgentCF.6090.7257.7095.2173.2635.2561
MemRec.5493.6688.6355.2470.2754.2644
AgentDR.5099.5482.5227.2362.2524.2435
Zero-shot.4701.4769.4770.2395.2602.2526

For users whose targets were retrieved, FIRM achieved the highest NDCG@10 in all three domains, demonstrating superior reranking on retrieved candidates (up to 8.03% over AgentCF on Grocery). For users who required insertion, FIRM ranked highest on Home and Grocery and performed comparably to MemRec on Sports.

 

W2 Q2 Raw and Flat under a matched relation token cap

We compared FIRM with “Raw” and “Flat” under a matched relation token cap. “Raw” directly uses the Explorer's relation categories, names, and explanations. “Flat” merges similar relations within each category from the same Explorer output and applies the same Verifier as FIRM, without constructing a hierarchy. “Raw” and “Flat” relation blocks were capped at the token count of each user's FIRM relation block, and user memories were generated from each condition's relations.

ConditionSportsHomeGrocery
Raw.2827.3304.3511
Flat.2866.3321.3558
FIRM.2917.3374.3641

FIRM outperformed “Raw” by .0070–.0130 and “Flat” by .0051–.0083 in NDCG@10. This demonstrates that FIRM's relational memory is more effective for reranking than “Raw” and “Flat” even under the same relation token cap.

AuthorsOpenReview ↗

Response to Reviewer 7Jj4 (Part 2/2)

This continues our response in Part 1.

 

W3 Q3 Relation quality and retained misfits

1 Independent evaluation of relation quality

We sampled 200 pairs per domain from all final assignments, including retained misfits. Two external LLMs (GPT-6 Astra and Claude Fable 5.2) judged whether each pair matched its stored relation definition, using three labels: match, mismatch, and insufficient information. Excluding insufficient-information judgments, definition fit rates were 75.6–82.2%, showing that most evaluated pairs matched their stored definitions. Cohen's κ across all 200 pairs and three labels was .832 / .698 / .733 on Sports / Home / Grocery.

2 Counting and handling unresolved misfits

Uncorrected misfits are counted as unresolved, separately from assignments that passed verification. They remain in the final memory and may be retrieved for recommendation, so they were included in the sampling population for the external evaluation above.

Processing statusSportsHomeGrocery
Unique pairs retained without a revision proposal6762,6511,136
Of these, explicit misfits remaining unresolved6332,381997

Table 9 counts unique pairs across all rounds. The first row includes failed responses, indeterminate judgments, and cases later judged valid; we separately report explicit misfits that remained unresolved.

3 Examples from the constructed relation hierarchy

The table shows selected branches of the relation hierarchy constructed on Home. Pair counts include assignments to each type and its descendants.

DepthRelation typeStored definition (verbatim)Pairs
1size_differenceItems differ in physical size, allowing for substitution based on specific requirements and space needs, relevant to various sub-types including bedding, storage, and display dimensions.1,437
2dimensionsItems differ in size, allowing for substitution based on spatial needs, covering sub-types like use_occasion, capacity, and form.45
3capacityItems vary in capacity, such as 6-quart vs 4.5-quart.1
2room_dimensionsItems differ in size to fit specific room dimensions or traffic flow.11
1functionalityItems offer similar core functions but differ in specific features or design, suitable for the same tasks.68
2specific_cooking_functionsItems offer a range of cooking functions, allowing buyers to choose based on specific cooking needs.10
2food_preparation_tasksItems are designed for specific food preparation tasks, allowing buyers to choose based on the specific task.11

The path size_difference → dimensions → capacity refines a shared size-based criterion into dimensions and capacity. The 6-foot and 5-foot folding table pair is assigned to dimensions, making dimensional differences available as a comparison criterion.

 

W4 Shuffle control on the same user subgroup

We applied the “Shuffle” control to the same user subgroup with retrieved history–history and history–candidate relations (Table 10). “Shuffle” preserves candidates and item-pair edges, jointly shuffles relation names and definitions, and regenerates user memories.

DomainUsersFIRMShuffleDifference
Sports2,151.2895.2813+.0082
Home2,087.5220.4911+.0309
Grocery2,417.4759.4564+.0195

NDCG@10 gains over “Shuffle” were .0082–.0309. This demonstrates that preserving relation semantics enables FIRM to rerank more effectively even with the same users and edges.

 

W5 Relation coverage and the meaning of None

FIRM targets substitute and complement relations, and the current memory is dominated by substitutes. Our main experiments evaluate all test users, including those with no retrieved relations. We sampled 200 pairs per domain labeled None by the Explorer. The same two external judges classified each pair as no relation identified, outside Sub/Com, valid Sub/Com, or insufficient information. The table reports judgment percentages as Astra / Fable and Cohen's κ across these four categories.

Judgment category or metricSportsHomeGrocery
No relation identified55.5 / 48.083.5 / 84.079.0 / 80.0
Outside Sub/Com35.0 / 44.55.0 / 9.55.5 / 7.0
Valid Sub/Com9.5 / 7.510.0 / 5.514.5 / 12.0
Insufficient information0.0 / 0.01.5 / 1.01.0 / 1.0
Cohen's κ (4 categories).807.757.726

The None samples included relations outside Sub/Com, while 5.5–14.5% were judged valid Sub/Com relations. Thus, None does not imply that a pair has no semantic relation, and some substitute or complement relations may have been missed.

AuthorsOpenReview ↗

Follow-up on Our Responses

Dear Reviewer 7Jj4,

Thank you again for your careful assessment of our evaluation protocol and relation quality. We would appreciate your thoughts on the additional experiments and analyses provided in our responses. Please let us know if any concerns remain or if further clarification would be helpful.

Kind regards,
The Authors

Reviewer_yXyXOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents FIRM, a shared item-relational memory for LLM-based recommender agents. An Explorer proposes fine-grained substitute or complement relations for item pairs selected from textual and collaborative signals. A Manager merges, splits, and organizes these relations into a hierarchy, and a Verifier checks pair assignments and refines type definitions. At inference time, the system summarizes a user's history and the relations among purchased items into a two-sentence user memory. It then retrieves relations between history items and candidates for LLM reranking. Experiments compare FIRM with eleven baselines on the Sports, Home, and Grocery subsets of XMarket. FIRM obtains the best result on all twelve main metrics, with significant gains over the strongest baseline on eleven of them; its relative NDCG@10 gain is 3.62%–10.03%. A memory built with one model also remains useful for rerankers of different sizes and families.

Critical strengths
  1. The paper turns fine-grained item relations into a memory shared across users and separates public item knowledge from user-specific preferences. This is a sensible step beyond user-level, coarse relation methods such as AgentDR.
  2. The evaluation covers three domains, eleven baselines, component ablations, cost analysis, and several reranking LLMs. The results are consistent enough to show that the memory is not tied to one model or dataset.
Critical weaknesses
  1. The paper does not yet establish that the main gain comes from the semantics of the fine-grained relations. Stored edges are selected using textual proximity, collaborative proximity, and co-consumption, so edge presence is already informative. Most of the gain also remains with coarse relations or the Explorer's uncanonicalized relations.
  2. The construction procedure has unclear scaling behavior. Selecting the top 0.1% of neighbors per item makes the number of item pairs grow roughly quadratically with catalog size. The experiments contain only 4,151–7,064 items, and the target is manually inserted into the 20-item candidate set for 83.5%–88.7% of test users. The results therefore measure conditional reranking quality rather than end-to-end recommendation performance.
Questions for authors
  1. If the same item-pair edges are retained but all relations are replaced by a single "related" label, or if fine-grained names and definitions are shuffled within the substitute and complement categories, how much of the gain remains? This would separate edge-presence information from relation semantics.
  2. Why use a fixed fraction of neighbors rather than a fixed number per item or a fixed total construction budget? As the catalog grows, how do relation coverage, construction cost, and reranking quality change?
  3. Each unordered item pair receives only one relation type. How does the method handle pairs with several valid comparison criteria or directional complement relations?

Suggestion:

  1. Please report end-to-end results on the natural candidate sets, or separate candidate recall from reranking quality conditioned on target retrieval. A second data source or a larger catalog would also make the scope clearer.
2 comments
AuthorsOpenReview ↗

Response to Reviewer yXyX

Thank you for suggesting targeted controls to distinguish relation semantics from edge-presence information, and for your feedback on scalability.
We will incorporate the additional experiments and analyses into the revised manuscript.

 

W1 Q1 Contribution of relation semantics given the same edges

Constructing a shared memory of item relations is FIRM's core contribution, and fine-grained semantics provided additional gains even with the same edges. “Related” replaces all relations with a single label, while “Shuffle” jointly shuffles relation names and definitions either within Sub/Com categories or irrespective of category. User memories were regenerated using each condition's relations.

Relation representationSportsHomeGrocery
FIRM.2917.3374.3641
Related.2786.3227.3462
Shuffle within Sub/Com.2811.3248.3507
Shuffle without category constraints.2796.3236.3486

FIRM outperformed “Related” and both “Shuffle” conditions in all three domains, with NDCG@10 gains of .0131 / .0147 / .0179 over “Related”. This demonstrates that fine-grained relation semantics contribute to recommendation performance even when the edges are unchanged.

 

W2 S1 Reranking without target insertion

Following your suggestion, we evaluated reranking without inserting the target into the candidate set. The table reports NDCG@10 for the 1,241 / 1,418 / 1,211 users whose targets were retrieved by SASRec on Sports / Home / Grocery.

MethodSportsHomeGrocery
FIRM.6393.7482.7665
AgentCF.6090.7257.7095
MemRec.5493.6688.6355
AgentDR.5099.5482.5227
Zero-shot.4701.4769.4770

Among users whose targets were retrieved, FIRM achieved the highest NDCG@10 in all three domains without target insertion.

 

W2 Q2 Catalog size and construction cost

To address your question about catalog size and construction cost, we are conducting additional scalability experiments. We will share the results in a follow-up comment as soon as they are available.

 

Q3 Multiple comparison criteria and directional complements

FIRM currently stores one relation type per unordered item pair. When evaluating a candidate, it uses the relations stored for multiple candidate–history item pairs, so several relations can inform that candidate's evaluation. The prompt distinguishes history and candidate roles, but the representation does not encode multiple criteria for the same pair or the direction of complement relations. We will clarify this scope in the manuscript.

AuthorsOpenReview ↗

Follow-up on W2 Q2: Catalog Size and Construction Cost

Dear Reviewer yXyX,

Following up on W2 Q2, we now report the additional scalability experiments promised in our earlier response.

W2 Q2 Catalog size and construction cost

We used the top 0.1% rule to scale neighborhood sizes with each catalog, yielding 4–7 neighbors per item in each embedding space. With a fixed fraction, the number of embedding-based candidate pairs grows approximately quadratically with catalog size. We therefore rebuilt the relational memory with a fixed number of neighbors per item (k = 1 / 2 / 1 for Sports / Home / Grocery, about 0.025% of each catalog) and with a pair budget. The budgeted variant selects pairs globally within each embedding space using the same per-space quota as fixed k. The table reports stored item pairs / NDCG@10.

DomainTop 0.1%Fixed kPair budget
Sports11,490 / .29173,879 / .29013,955 / .2913
Home25,297 / .33749,279 / .33528,988 / .3353
Grocery12,063 / .36414,004 / .36334,042 / .3637

Both alternatives stored roughly one third as many pairs as the top 0.1% setting, with NDCG@10 differences within .0022 in all three domains. These results demonstrate that FIRM can retain comparable reranking performance with fixed k or a pair budget.

We also evaluated construction cost, coverage, and reranking performance across neighbor fractions on the larger Electronics catalog (14,634 items). Construction calls count one Explorer call per selected pair plus completed Manager and Verifier calls. Coverage is the percentage of users with at least one history–candidate relation in the reranking prompt.

Neighbor selectionStored pairsConstruction callsCoverageNDCG@10
0.01% (k=1)13,83594,01045.4%.2881
0.05% (k=7)56,155619,01060.5%.2969
0.1% (k=15)97,2681,805,11365.6%.2970

On Electronics, reducing the neighbor fraction from 0.1% to 0.05% cut construction calls by 65.7% while maintaining nearly identical NDCG@10 (a difference of .0001).

We would be happy to clarify any remaining questions.

Kind regards,
The Authors

Reviewer_HfbkOpenReview ↗
Rating
2
Confidence
3
Informativeness
3(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the use of fine-grained item relations shared across users to improve LLM-based recommendation. Unlike existing approaches that primarily model item relationships using coarse substitution and complementarity categories, FIRM constructs a hierarchical relational memory that captures more specific comparison criteria and usage relationships between items. Three agents are employed to identify, organize, and verify fine-grained relations. During recommendation, the constructed relational memory is used to infer user preferences and provide relation-aware information for LLM-based candidate reranking. Extensive experiments demonstrate the effectiveness of the proposed method.

Critical strengths
  1. This paper is well-written, which is easy to read and clear to grab the key idea of the model.
  2. This paper presents a simple yet effective solution for constructing and reusing fine-grained item relationships across users.
  3. The idea of organizing fine-grained relations into a shared memory is interesting and is demonstrated to be effective for improving the consistency and interpretability of relational knowledge used for recommendation.
Critical weaknesses
  1. The technical novelty is limited and needs to be further verified. The proposed framework mainly combines existing techniques, including LLM-based relation extraction, hierarchical taxonomy construction, semantic merging and splitting, LLM-based verification, and memory-augmented recommendation. Furthermore, relation-aware reranking is implemented by incorporating retrieved relational descriptions and user memory into an LLM prompt, without introducing a new recommendation learning objective or reasoning mechanism.
  2. The hierarchical relation construction process relies heavily on LLM judgments, which may introduce inaccurate or inconsistent relations. Human evaluation or comparisons against annotated item relationships are needed to verify whether the proposed framework actually improves relation quality.
  3. All three evaluation datasets come from the same benchmark, and the evaluation focuses on reranking a fixed set of 20 candidates. Additional evaluations across different recommendation domains and more realistic retrieval settings would strengthen the paper's contribution.
  4. The experiment lacks sensitivity analyses of the heuristic choices, such as split threshold, co-consumption threshold and hierarchy depth constraints.
Questions for authors
  1. Could the authors provide a qualitative analysis of how fine-grained relations influence the final ranking decisions? In particular, examples where FIRM correctly ranks a candidate that coarse-grained relation modeling fails to identify would help illustrate the practical value of the proposed approach.
  2. How does the quality of user memory change with the length and diversity of interaction histories? Are fine-grained relational preferences equally beneficial for users with relatively homogeneous interaction histories and those with more diverse behavioral patterns?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Hfbk (Part 1/2)

Thank you for recognizing the reusability of our shared relational memory and the clarity of the paper.
We will incorporate the additional experiments and analyses into the revised manuscript.

 

W1 Technical contribution of relational memory

FIRM’s core contribution is the shift from individual item memory to shared relational memory. Individual memory characterizes items separately, whereas FIRM organizes comparison criteria and usage roles between item pairs into shared relation types and definitions. To directly evaluate this contribution, we compared FIRM’s relational memory with AgentCF’s individual user/item memories using the same user histories, candidates, reranking setup, and memory token cap.

Memory conditionSportsHomeGrocery
Individual memory (AgentCF user/item memories).2713.2948.2967
Relational memory (FIRM).2917.3374.3641

FIRM outperformed individual memory by .0204 / .0426 / .0674 in NDCG@10 on Sports / Home / Grocery. This demonstrates that relational memory is more effective for reranking than individual memory under the same user histories and memory token cap.

 

W2 Quality of the constructed relations

Two external LLMs not used in relation construction (GPT-6 Astra and Claude Fable 5.2) evaluated the same 200 item pairs sampled from the final assignments in each domain. They judged whether each pair matched its stored relation definition, using three labels: match, mismatch, and insufficient information. Definition fit rate is the proportion of matches after excluding insufficient-information judgments; Cohen's κ measures inter-judge agreement across all 200 pairs using these three labels. Results are reported as Astra / Fable.

Evaluation metricSportsHomeGrocery
Definition fit rate (%)75.6 / 76.980.8 / 80.482.1 / 82.2
Inter-judge agreement (κ).832.698.733

Both judges found that most assigned item pairs matched their stored relation definitions, with definition fit rates of 75.6–82.2% across the three domains.

 

W3 Additional data and retrieved candidates

We evaluated FIRM on H&M [2] to assess its applicability to clothing beyond the three Amazon XMarket [1] domains. We compared NDCG@10 without target insertion for the same 679 users whose targets were retrieved, out of 4,754 users (Candidate Recall@20: 14.3%).

MethodNDCG@10
FIRM.4456
AgentCF.4104
MemRec.3693
Zero-shot.3746

On H&M's retrieved candidates, FIRM outperformed AgentCF by .0352 in NDCG@10, achieving the best reranking performance among the compared methods in the clothing domain.

 

W4 Sensitivity to key hyperparameters

On Grocery, we varied the split threshold, maximum split depth, and co-consumption threshold one at a time, holding the other conditions fixed. Settings are listed in this order, with defaults of 20 / 2 / 2.

Parameter variedSettingsRelation typesNDCG@10Change from default
Default20 / 2 / 2329.3641—
Split threshold10 / 2 / 2340.3579−.0062
Split threshold40 / 2 / 2321.3610−.0031
Maximum split depth20 / 1 / 2319.3601−.0040
Maximum split depth20 / 3 / 2375.3597−.0044
Co-consumption threshold20 / 2 / 3275.3621−.0020
Co-consumption threshold20 / 2 / 5253.3629−.0012

With 253–375 relation types across settings, the maximum absolute change in NDCG@10 was .0062. FIRM outperformed AgentCF, the strongest baseline on Grocery, under every tested hyperparameter setting.

 

References

[1] Bonab et al. Cross-Market Product Recommendation. CIKM, 2021. https://xmrec.github.io/

[2] H&M Group. H&M Personalized Fashion Recommendations. Kaggle, 2022. https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/data

AuthorsOpenReview ↗

Response to Reviewer Hfbk (Part 2/2)

This continues our response in Part 1.

 

Q1 Examples of fine-grained relations and final rankings

We compared the “Coarse relations” variant with FIRM on two Home cases. Both methods rank the same candidate list; arrows between relation types indicate parent–child paths.

FieldCase 1 Folding tablesCase 2 Openers
Recent purchase historyAlarm clock → under-shelf basket → 6-foot folding tableCoffee maker → robot vacuum → insulated shopping bag → 3-piece bottle-cap gripper set
Target candidate5-foot folding table from the same product family as the 6-foot folding table in the history6-in-1 multifunction opener (bottles, cans, lids), from the same brand as the bottle-cap gripper in the history
Related item pair6-foot folding table (history) ↔ 5-foot folding table (candidate)3-piece bottle-cap gripper set (history) ↔ 6-in-1 multifunction opener (candidate)
Coarse relationsubstitutionsubstitution
FIRM fine-grained relationsize_difference → dimensions; substitution based on size and space requirementsfunctionality → specific_cooking_functions; substitution based on functional scope
Target rank Coarse relations → FIRM18 → 220 → 3

FIRM provides the size difference between the 6-foot and 5-foot tables and the functional scope of the bottle-cap gripper and multifunction opener as comparison criteria. The target ranks improved from 18 to 2 and from 20 to 3 relative to “Coarse relations”, illustrating the effectiveness of fine-grained relations for candidate comparison in these two cases.

 

Q2 User memory with varying history length and diversity

Relational memory gains varied with history length and diversity. We kept the history–candidate inputs fixed and varied whether user memory included relational information (Table 5). Within each domain, we split users into tertiles by full training-history length and by diversity, measured as the variance of bge-large embeddings of history items. The length groups are Short, Mid, and Long. The table reports NDCG@10 differences (FIRM − Items-only memory).

User groupSportsHomeGrocery
Short+.0008+.0046+.0032
Mid+.0027+.0120+.0041
Long+.0042+.0176+.0107
Diversity lower 1/3+.0081+.0150+.0072
Diversity middle 1/3+.0050+.0009+.0095
Diversity upper 1/3−.0066+.0130−.0004

Gains were larger for groups with longer histories in all three domains. Because FIRM infers preferences from recurring relations, longer histories may provide more evidence for identifying comparison criteria. In the highest-diversity group, gains were positive on Home and negative on Sports and Grocery.

The same two judges also evaluated two memory sentences per user for consistency with that user's purchase history, using four labels: match, partial match, mismatch, and unable to judge. For the length analysis, judgments for 120 users per domain were aggregated by group. For diversity, we evaluated 20 users per group (40 sentences and 80 judgments). The table reports the percentage of all judgments labeled match or partial match, with Cohen's κ in parentheses.

User groupSportsHomeGrocery
Short95.9 (.759)91.3 (.762)91.9 (.686)
Mid97.7 (.728)96.3 (.905)96.1 (.360)
Long98.1 (.737)99.6 (.765)91.8 (.580)
Diversity lower 1/392.5 (.73)93.8 (.61)96.3 (.72)
Diversity middle 1/3100.0 (.93)91.3 (.88)87.5 (.51)
Diversity upper 1/398.8 (.82)96.3 (.80)88.8 (.64)

Memory–history consistency rates were 87.5–100.0% across all evaluated groups, demonstrating that FIRM consistently generated user memories aligned with purchase histories across different history lengths and diversity levels.

AuthorsOpenReview ↗

Follow-up on Our Responses

Dear Reviewer Hfbk,

Thank you again for your constructive feedback. We are following up on the additional experiments and clarifications provided in our responses. We would appreciate your feedback on whether these address your concerns and would be happy to clarify any remaining questions.

Kind regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.