acceptodds
Under review as a conference paper at ICLR 2027

LDCO: Evidence-Grounded Preference Transfer for User-Side Cold-Start Recommendation

Abstract

Recommender systems connect users with products at scale, yet they remain largely platform-centered and dependent on target-domain histories. When a user enters an unfamiliar domain with an ambiguous query, three difficulties arise together: target-domain cold start leaves little evidence for personalization, the query omits relevant preferences, and the platform's ranking may not match the user's preferred ordering. We address these coupled problems with Layered Dimension-Conditioned Query Optimization (LDCO), a user-side large language model agent framework that operates through an existing retrieval interface. LDCO constructs domain-level and global profiles from authorized source-domain histories, selects relevant evidence, and maps transferable preferences to target-product attributes. It then uses Monte Carlo tree search on a fuzzy-query dataset to optimize the query-generation prompt through retrieval feedback, and reranks the returned candidates by query–profile relevance. On a user-disjoint Amazon benchmark, LDCO raises Hybrid Hit@100 from 0.3169 to 0.4155, MRR@100 from 0.0365 to 0.0758, and NDCG@100 from 0.0872 to 0.1388 over retrieval with the original fuzzy queries. Under live Amazon retrieval, the full framework raises Precision@15 from 0.2628 to 0.4162. These results show that evidence-grounded preference transfer, retrieval-aware query optimization, and user-side reranking jointly improve cold-start recommendation without modifying the platform's retrieval model.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Large language model, Cross-domain recommendation systems, User profile management, Query optimization

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_4Hk8OpenReview ↗
Rating
4(+2)
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Layered Dimension-Conditioned Query Optimization (LDCO), a user-side LLM framework for recommendation under target-domain cold start and underspecified queries. LDCO constructs domain-level and global profiles from source-domain histories, predicts the target domain, selects relevant source profiles, maps supported preferences to target-product attributes, and generates a personalized retrieval query. The query-generation prompt is optimized offline using an MCTS-style search with Hybrid Hit@100 as feedback. An LLM subsequently reranks the retrieved candidates using the generated query and user profile. The method is evaluated on a user-disjoint benchmark derived from Amazon Reviews 2023 and through live Amazon retrieval evaluated by an LLM judge.

Critical strengths

The paper studies a practically relevant interface-constrained setting in which a user-side agent can personalize retrieval without modifying a platform’s index or ranking model. The separation between query rewriting, which changes candidate membership, and reranking, which changes candidate order, is clearly motivated and reflected in the evaluation. The proposed evidence-selection and dimension-mapping stages also provide a sensible safeguard against unrestricted cross-domain preference transfer. The manuscript clearly documents the main pipeline, separates prompt-optimization and test users, freezes the selected prompt before testing, reports the seed and optimized prompts, and provides useful latency and API-cost measurements. The reported gains are sizable, particularly for the Electronics target domain.

Critical weaknesses
  1. The evaluation does not establish that LDCO improves recommendations for actual cold-start users. Every offline fuzzy query is generated from the title, description, and review of one five-star target item, after which that same ASIN is treated as the sole relevant result. Therefore, plausible substitutes are counted as errors, and the task primarily measures recovery of an item that was used to synthesize the request rather than relevance for naturally issued queries. Moreover, the split is only stated to be user-disjoint: no target-ASIN or product-intent overlap analysis is reported, even though the optimized prompt explicitly contains product-specific rules for cases such as an iPad Air 2 case, a 1-inch curling iron, and a surge suppressor learned during optimization. The live experiment does not resolve this limitation because its labels come entirely from GPT-4.1, while the intervention being evaluated is itself an LLM reranker ordering products according to a closely aligned query/profile relevance criterion. Without human preference labels or validation of the judge, the reported gains cannot distinguish improved user relevance from agreement between two LLM-based scoring procedures.

  2. The experiments do not show that the proposed architecture is preferable to existing or simpler alternatives. The comparison consists of the original fuzzy query and internal variants; no competing personalized query-rewriting, cross-domain recommendation, or user-side recommendation method is evaluated. The ablations also remove Layers 1–3 jointly rather than testing target classification, source selection, and dimension mapping separately, and they do not compare the hierarchical profile against domain-only or global-only profiles. Consequently, the results support the complete pipeline as one engineered system, but they do not substantiate the claimed contribution of its individual layers or show that MCTS-optimized LDCO is responsible for the gains relative to reasonable alternative pipelines.

Questions for authors
  1. Are target ASINs, product types, and fuzzy-query intents disjoint between the 568 prompt-optimization users and the 142 test users? What are the corresponding overlap rates, and how does the method perform when target items and intents observed during prompt optimization are excluded from testing?
  2. How is the “fixed reference profile” used by GPT-4.1 constructed, and what exact judging prompt and decision rules are used? Has this judge been compared with blinded human assessments of whether the returned products fit the stated request and user preferences?
  3. How does LDCO compare under the same retriever and execution model with a direct profile-conditioned rewrite, a non-MCTS prompt-optimization method, and representative personalized rewriting or user-side recommendation approaches? Can the three LDCO layers and the two profile levels be ablated individually?
3 comments
AuthorsOpenReview ↗

A Response to Reviewer 4Hk8

Thank you for recognizing the user-side setting's practical value. Your questions prompted a split and Judge audit, additional comparisons, and dimension-context analysis. We hope the following reponses could address your concerns. The main changes and improvements are smmarized as follows.

Main additions and corrections

  • An explicit overlap audit covering users, target ASINs, and exact query strings.
  • A correction of the Judge reference, with its scoring rubric and human-validation status.
  • A profile-conditioned rewriting comparison and dimension-context ablations, distinguished from pipeline-layer ablations.

1. Evaluation scope and independence of the split

Offline single-ASIN evaluation measures recovery of a known positive with a five-star rating. Acceptable substitutes receive no credit. The improvement therefore supports target-item recovery, without establishing real-user satisfaction.

The split audit gives:

\text{Quantity} & \text{Audited value} \\\\ \hline \text{Optimization: cases / users} & \text{568 / 525} \\\\ \text{Test: cases / users} & \text{142 / 140} \\\\ \text{Total distinct users} & 651 \\\\ \text{Shared users} & 14 \\\\ \text{Test cases with shared ASIN} & 5 \\\\ \text{Shared exact query strings} & 16 \end{array}$$ The shared users have different target domains: eight Electronics-to-Beauty and six Beauty-to-Electronics cases. No user-ID/target-domain pair repeats. The 16 shared query strings occur in 18 test cases. Product-type and semantic-intent disjointness have not been established. We apologize for the inaccurate user-disjoint description and will correct the manuscript. Performance after excluding these overlaps remains unmeasured. Different target domains do not eliminate dependence through shared users, so the current results do not establish strict split independence. **2. What reference and rubric does the Judge use?** The released Judge receives **target-product and candidate-product information**. The manuscript's description of a fixed reference profile and original-query input is inaccurate and needs correction. The configured model is GPT-4.1 (`gpt-4.1-2025-04-14`, temperature 0.1). Its rubric assigns: - **3:** a near match in primary function and critical purchase attributes; - **2:** a practical substitute with the same primary function and no critical incompatibility; - **1:** related, but not a reliable substitute; - **0:** irrelevant or incompatible. Grades of at least 2 count as relevant for the loose binary metric. Category or brand agreement alone is insufficient. The Judge uses only supplied product information; the full rubric is in `llm-product-judge/prompt_template.txt`. This accepts alternatives beyond the original ASIN. **3. Direct rewriting, MCTS, and profile contribution** The added LLM4CS-REW-style adaptation generates one query from the request and profile. With GPT-4o-mini, Hybrid retrieval, and no reranking, it obtains Hit/MRR/NDCG@100 of **0.2746/0.0346/0.0784**, versus **0.4155/0.0577/0.1236** for LDCO. This is an adapted rewriting reference. Table 2's profile-free configuration obtains **0.3453/0.0458/0.0993** without reranking. Profile conditioning improves all three point estimates. MCTS optimizes prompts offline;. **4. Dimension-context ablations and harmful transfer** We added the following ablations without reranking: $$\begin{array}{lrrr} \text{Dimension context} & \text{Hit@100} & \text{MRR@100} & \text{NDCG@100} \\\\ \hline \text{All groups} & 0.4155 & 0.0577 & 0.1236 \\\\ \text{No target-type dimensions} & 0.3803 & 0.0613 & 0.1197 \\\\ \text{No cross-type bridge} & 0.3944 & 0.0585 & 0.1204 \\\\ \text{No domain-pair mapping} & 0.4085 & 0.0611 & 0.1238 \end{array}$$ The complete context has the highest Hit@100, but some reduced contexts improve MRR. Routing and source selection are retained. Your questions helped us correct and clarify the evaluation. We welcome further discussion of your main concerns. If this analysis and corrected interpretation sufficiently address them, would you consider reassessing the paper and increasing the score accordingly? Thank you very much!

I thank you for addressing my concerns adequately and I decide to raise my rating. Thank you for your effort.

Thanks a lot for your valuable time in reviewing this paper and for raising the evaluation score. Your comments have significantly helped us to improve the quality of this work.

Reviewer_LPWGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LDCO, a user-side LLM agent framework for cold-start recommendation. It aims to address target-domain cold start, query ambiguity, and user–platform objective mismatch. The method constructs cross-domain user profiles, maps user preferences to target-product attributes, optimizes query-generation prompts using MCTS, and reranks retrieved products with an LLM. Experiments on an Amazon benchmark and live Amazon retrieval show improvements in retrieval and ranking performance.

Critical strengths
  1. The paper presents a user-side recommendation framework that combines cross-domain preference transfer, query optimization, and reranking without modifying the platform's retrieval model.
  2. The paper evaluates LDCO on an Amazon benchmark and through live Amazon retrieval, showing improvements in both target-item recovery and ranking quality.
Critical weaknesses
  1. Limited novelty of the problem setting and insufficient evaluation. Incomplete natural-language queries are common in recommendation systems. Existing studies on conversational recommendation and user agents have already explored similar problems, where users express incomplete preferences and the system needs to infer their intentions. The paper does not clearly explain how its fuzzy query setting differs from these existing studies. Moreover, the fuzzy queries are synthetically constructed from known target products by removing some product attributes, rather than collected from real user requests. The evaluation then measures whether the original products can be retrieved. This may not reflect real user needs and preferences. Although the paper conducts live Amazon retrieval experiments, it does not include real user evaluations. Instead, product relevance is judged by an LLM. Therefore, the experiments provide limited evidence that LDCO better satisfies actual user preferences.
  2. Limited methodological novelty and insufficient comparisons. The proposed framework mainly combines existing techniques. First, the cross-domain module extracts user preferences from source-domain histories and transfers them to target-domain attributes, which is already common in existing cross-domain recommendation methods. Second, the MCTS-based prompt optimization follows the existing AFlow framework, while LLM-based reranking is also a widely used technique. Therefore, the technical novelty of these components appears limited. Moreover, the experiments mainly compare LDCO with the original fuzzy queries and its own variants, without comparisons to existing cross-domain recommendation methods. This makes it difficult to assess the advantages of the proposed framework over prior work.
Questions for authors
  1. How does the proposed fuzzy-query setting differ from existing conversational and agent-based recommendation, and can the authors provide evaluations using real user queries or human judgments?
  2. What is the main novelty of the proposed cross-domain preference transfer, and how does the proposed cross-domain preference transfer differ from existing training-free LLM-based approaches such as AgentCF++?
3 comments
AuthorsOpenReview ↗

A Response to the Reviewer LPWG

Thank you very much for letting us to clarify the methodological contribution and evaluation validity. Your comments led us to sharpen the contribution, extend the comparisons, and audit the evaluation assumptions.

Main additions and corrections

  • A focused account of LDCO's query-time evidence-transfer design and its relationship to conversational and agent-based recommendation.
  • Additional single-query and multi-query reference comparisons using the local Hybrid retriever.
  • Explicit boundaries for synthetic-query evaluation, product-substitutability judgments, and data-split independence.

1. What does LDCO contribute beyond fuzzy queries and existing recommendation pipelines?

We agree that fuzzy queries and incomplete preferences are established problems. Our contribution concerns how a user-side assistant can turn source-domain evidence into useful search constraints when target-domain history is unavailable and the platform's retrieval model cannot be modified.

LDCO identifies the target domain, selects relevant source profiles, and maps supported preferences to target attributes before rewriting the query. The design addresses a specific difficulty: a preference expressed in one product domain is not automatically a valid constraint in another. For example, a durability concern may inform a protective-case query, whereas an unrelated book-genre preference should not.

The contribution is query-conditioned evidence selection and attribute transfer, integrated with retrieval. MCTS, reranking, and profile summarization are supporting components. LDCO focuses on using existing histories to inform a single-turn request, complementing conversational approaches that acquire information through further interaction.

2. Relationship to AgentCF++

AgentCF++ uses a dual-layer memory architecture, two-step fusion, and group-shared memory to support cross-domain user-agent behavior modeling and account for popularity effects. LDCO instead uses selected review-profile evidence to construct an attribute-conditioned query for an existing product retriever.

Both organize cross-domain preferences, but use them differently: agent memory and behavior modeling in AgentCF++, versus query construction and catalog retrieval in LDCO. This distinction does not replace an empirical comparison. We have not evaluated AgentCF++ under our protocol and do not claim superiority over it.

3. Additional comparisons

We added two adapted query-optimization references on the same 142 test cases, using Hybrid retrieval without reranking:

\text{Method} & \text{Hit@100} & \text{MRR@100} & \text{NDCG@100} \\\\ \hline \text{Fuzzy query} & 0.3169 & 0.0365 & 0.0872 \\\\ \text{Search Swarm-style} & 0.2042 & 0.0157 & 0.0492 \\\\ \text{LLM4CS-REW-style} & 0.2746 & 0.0346 & 0.0784 \\\\ \text{LDCO without reranking} & 0.4155 & 0.0577 & 0.1236 \end{array}$$ The single-query adaptation uses the request and profile; the multi-query replay uses query branches and local fusion. LDCO obtains higher values under this protocol. These adaptations extend the evidence beyond internal ablations. **4. Does the evaluation establish real recommendation effectiveness?** The five-star target supplies an observed positive. Recovering that item does not establish that other items are unsuitable or that a current need has been satisfied. This is controlled target-item recovery evidence. The online Judge accepts substitutes but remains target-referenced and lacks blinded human validation. We therefore distinguish product substitutability from personalized relevance. A real-query user study and human relevance evaluation remain unavailable. Our audit finds 710 cases from 651 users, with 14 users shared across splits under different target domains and five test cases sharing optimization target ASINs. We correct the user-disjoint claim here and will update the manuscript; these overlaps limit generalization. Your review has helped us clarify both the contribution and its evidence. We welcome further comments on the novelty argument and the most informative remaining comparison. If these additions address your principal concerns, would you consider a more favorable recommendation and an increased score? Thanks a lot!
AuthorsOpenReview ↗

Follow-up on Our Response and Invitation for Further Feedback

Dear Reviewer,

Thank you again for your careful review and constructive suggestions. We are following up on our response posted on November 16, which presents additional analyses and clarifications motivated by your comments.

As the discussion period approaches its end, we would appreciate your feedback on whether our response addresses your main concerns and which issues, if any, remain most important to your assessment. Your guidance would help us focus any further clarification and improve the paper.

We understand that you are reviewing several submissions and appreciate your time. We remain available to answer follow-up questions and discuss any points that need further explanation.

Thank you for your consideration.

Thank you for your response.

After rereading the introduction, I still find it difficult to identify the paper's main contribution. The paper emphasizes fuzzy intent understanding, cold start, and user objectives in a user-side assistant, but similar problems have already been studied in existing agent-based recommendation methods, including those discussed in the related work. It remains unclear what new technical challenges arise from addressing these problems together.

Moreover, I acknowledge that the authors have added several experiments. However, these experiments mainly focus on individual components within the proposed setting and still lack sufficient comparisons with existing recommendation methods. Since the paper presents LDCO as a user-side recommendation agent, I believe it should also be compared with existing agent-based recommendation approaches, under a comparable evaluation protocol. Therefore, I will maintain my original score and hope the AC will take these concerns into consideration.

Therefore, I will maintain my original score.

Reviewer_uLHiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a user-side agent framework that helps shoppers discover products in unfamiliar categories without changing the underlying e-commerce platform's search or recommendation algorithms. When a user with an empty history in a target category enters an underspecified search query, the proposed method inspects their authorized interaction histories across other domains to build both domain-specific and cross-domain preference profiles. It then maps transferable preferences into concrete target-product attributes, generates a refined query using an offline MCTS-optimized prompt, sends the query to the platform's retrieval interface, and reranks the returned candidates according to the user's personal profile.

Critical strengths
  • Using the agent as a user-side layer that works through an existing retrieval interface with no changes to the platform is realistic and easy to deploy. The paper also clearly says that reranking cannot recover items that were never retrieved, which explains why query rewriting and reranking are both needed.

  • The ablations in Table 2 are informative that show that query optimization drives Hit@100 while reranking drives MRR and NDCG, and Appendix B explains why this must be the case. The Oracle comparison gives a useful upper bound, and the paper reports latency and cost per query.

  • The Guardrails designed against spurious cross-domain transfer in the proposed architecture is interesting and important. The architecture incorporates deliberate structural constraints such as requiring support across at least two source domains for global preferences, that restricts source selection to at most two relevant domains, and discards unsupported mappings which effectively mitigates negative transfer (e.g., preventing book genres from corrupting electronics queries).

Critical weaknesses
  • The test set used in the evaluations is small and narrow. Only 142 test users which is split across just two target domains is the whole set used for all claims in the paper. At this size, differences of a few points in Hit@100 correspond to a small number of of users. No confidence intervals, significance tests, or multiple random splits are reported so it is unclear which gains are robust and repeatable.

  • The used benchmark is synthetic and may not reflect real vagueness. The fuzzy queries are generated by an LLM from the held-out item and the source profiles come from reviews of the same user. The pipeline constructs a setting where preferences hidden from the query are expected to be recoverable from history. It is not shown that real users’ queries or real cross-domain preferences behave this way. The assumption that a user’s comfort or durability preferences transfer from clothing to earbuds is plausible but not tested or supported by proper justifications.

  • The comparisons are only against the raw fuzzy query and the paper’s own ablations. There is no comparison with other query rewriting methods, cross-domain recommenders, or simple strategies like prompting an LLM with the profile and asking for a better query. Without these, it is impossible to tell how much of the gain comes from LDCO’s specific design versus any profile-aware LLM rewrite.

  • The online path takes about 30 seconds per query, which is slow for shopping search, and the paper gives no analysis of how performance changes with longer histories, noisy or sparse source data.

Questions for authors

Please view my concerns in weaknesses.

4 comments
AuthorsOpenReview ↗

A Response to the Reviewer uLHi

Thank you for recognizing the practical setting and the complementary roles of rewriting and reranking. Your concerns about realistic evaluation and simpler alternatives directly shaped our rebuttal work. We hope the following reponse could address your concerns, and hope you can increase the accessment score of this work accordingly. The main changes and improvements are smmarized as following.

Main additions and corrections

  • A profile-conditioned single-query rewriting comparison under the same local retrieval setting.
  • An audit of dataset counts and overlap, correcting the claim of a user-disjoint split.
  • A clearer account of synthetic queries, transferable preferences, and the limits of the latency and robustness evidence.

Next, we explain our revision in details. 1. Test-set size and reliability

The benchmark contains 710 user–target-domain cases from 651 distinct users. The optimization set has 568 cases from 525 users; the test set has 142 cases from 140 users, covering 69 Beauty and 73 Electronics cases. Fourteen users occur in both splits under different target domains, and five test cases share target ASINs with optimization cases. We apologize for the inaccurate user-disjoint description and will correct it in the manuscript.

These overlaps and the two-domain scope limit generalization. Verified confidence intervals for the exact Table 2 comparison are not yet available, so we report its differences descriptively.

2. Synthetic queries and the recoverability of preferences

The benchmark uses real histories and five-star ratings, while an LLM constructs incomplete queries from target products. LDCO receives the query and source profile, without the target product as an additional query-time input. This tests whether source information helps recover an observed positive item.

A synthetic request is not a verified expression of current user needs, and missing preferences need not be recoverable across domains. Evidence selection and attribute mapping are intended to restrict inappropriate transfer; their effectiveness for every user is not established.

The online Judge accepts practical substitutes for the target, but lacks blinded human validation. Our supported conclusion is therefore improved target-item retrieval and assessed product substitutability, rather than demonstrated user satisfaction.

3. Comparison with a simple profile-conditioned rewrite

Following your suggestion, we added an LLM4CS-REW-style adaptation that supplies the current query and profile to an LLM to generate one rewritten query. It uses GPT-4o-mini and the local Hybrid retriever without reranking, as does the LDCO comparison below.

\text{Condition} & \text{Hit@100} & \text{MRR@100} & \text{NDCG@100} \\\\ \hline \text{Fuzzy query} & 0.3169 & 0.0365 & 0.0872 \\\\ \text{Profile-conditioned rewrite} & 0.2746 & 0.0346 & 0.0784 \\\\ \text{LDCO: no profile} & 0.3453 & 0.0458 & 0.0993 \\\\ \text{LDCO: with profile} & 0.4155 & 0.0577 & 0.1236 \end{array}$$ LDCO improves over both the implemented direct rewrite and its profile-free configuration. These are local comparisons, not an equal-budget study or a comprehensive cross-domain recommendation comparison. One adaptation's lower score does not imply that direct rewriting is generally ineffective. A Search Swarm-style multi-query replay obtains 0.2042/0.0157/0.0492. It is another local adaptation, rather than a full reproduction of the online-search system. **4. Efficiency and sparse or noisy histories** The current end-to-end estimate is 32–37 seconds, comprising approximately 2–3 seconds for rewriting, 1–2 seconds for local retrieval, and 28–32 seconds for product fetching and Top-100 reranking. Prompt optimization is performed offline. We agree that the online delay limits interactive use; the present results demonstrate feasibility rather than deployment-ready responsiveness. The benchmark selects users with substantial source histories. Robustness to sparse histories, longer queries, and noisy evidence remains untested. Summarization and source selection offer mechanisms for handling such inputs, but their effectiveness requires dedicated evaluation. Your review prompted a more direct baseline and a more precise interpretation. We welcome further suggestions on the remaining concerns. If these additions sufficiently address your main reservations, we would appreciate your reconsideration of the assessment and, where warranted, a higher score. Thank you very much!
AuthorsOpenReview ↗

Follow-up on Our Response and Invitation for Further Feedback

Dear Reviewer,

Thank you again for your careful review and constructive suggestions. We are following up on our response posted on November 16, which presents additional analyses and clarifications motivated by your comments.

As the discussion period approaches its end, we would appreciate your feedback on whether our response addresses your main concerns and which issues, if any, remain most important to your assessment. Your guidance would help us focus any further clarification and improve the paper.

We understand that you are reviewing several submissions and appreciate your time. We remain available to answer follow-up questions and discuss any points that need further explanation.

Thank you for your consideration.

Thank you for the additional results and clarifications in the rebuttal.

  • I commend the clarity of the authors in finding and disclosing the data leaks. I agree that the user-disjoint claim is inaccurate and it’s nice to have it scoped down. However, the lack of statistical paired test leaves my concern unaddressed. Also, while I appreciate disclosing the data leakage, I don’t think it’s sufficiently addressed. About 10% of the test set overlaps with the optimization users, and especially since prompt optimization was run on the optimization set, this is concerning.
  • Regarding synthetic queries, I appreciate the acknowledgment, but I think the lack of human evaluation, even on a small test set, keeps my concern unaddressed.
  • Thanks for adding the baselines, but the simple GPT-4o-mini profile-conditioned rewrite dropping below the raw query is a bit strange to me. Is the prompt optimized at all for such baseline? LDCO benefits from an offline prompt-optimization phase so comparing it against an unoptimized single-shot prompt is a bit unfair in my opinion.

Overall, while I appreciate the authors’ rebuttal and I find merit in the work, my main concerns about the paper have not been completely addressed.

AuthorsOpenReview ↗

Response to Your Concerns

Thank you again for your valuable comments. Your follow-up led us to think whether the evaluation supports our claims. In the revised manuscript, we have added human judgments, reported a paired test, and clarifed how the rewrite baseline was chosen. We hope these modifications can reduce your concerns.

Human relevance. Two raters each assessed the same five retrieved products for 20 targets using product descriptions alone. On the four-point scale, they agreed on 43 of 100 grades. At the practical-substitute threshold (grade 2 or 3), they accepted 79 and 100 candidates, respectively. Appendix F gives the full grade cross-tabulation and two scored examples, including a tablet paired with a protective case that received grades 0 and 3. This evaluates product substitutability on a small sample. Whether the results satisfy the user's original need remains open (Section 4 and Appendix F).

Overlap and uncertainty. Fourteen of 140 test users also appear in prompt optimization under another target domain. Appendix D.2 reports the user, item, and query overlaps. In a separate paired Hybrid retrieval run without reranking, adding profile evidence increased Hit@100 by 0.0775 (95% user-clustered bootstrap interval [0.0143, 0.1399], Holm-adjusted p = 0.028). This test includes the overlapping users, so it leaves the effect on entirely unseen users unresolved (Appendix F).

Prompt fairness. The one-shot GPT-4o-mini rewrite was not optimized offline. Its Hit@100 of 0.2746, below the raw query's 0.3169, describes that prompt only. Table 3 reports the external rewrite controls and dimension ablations. Table 4 separately reports the manual LDCO seed and selected prompt (Hit@100 of 0.3239 and 0.4155). An equal-budget comparison against a tuned external rewriter remains open.

We appreciate how your questions helped us distinguish the evidence for target recovery from the evidence for practical substitutes. We would welcome your view on whether the revised claims reflect that distinction. Thanks.

Reviewer_hVS1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • This paper proposes LDCO, a user-side LLM agent for cross-domain cold-start recommendation leveraging an existing retrieval interface.
  • It constructs hierarchical domain-level and global user profiles from source-domain interaction histories.
  • To generate personalized queries, the three stage pipeline does —target-domain classification, source-domain selection, and preference-to-attribute mapping.
  • Query-generation prompts are optimized with MCTS using retrieval feedback followed by LLM-based candidate reranking.
  • Experiments are conducted on Amazon Reviews 2023 with 142 held-out test users and also on live Amazon retrieval.
  • Reports improvements in Hybrid Hit@100 (0.3169 →\rightarrow 0.4155), MRR@100 (0.0365 →\rightarrow 0.0758), NDCG@100 (0.0872 →\rightarrow 0.1388), and live Precision@15 (0.2628 →\rightarrow 0.4162).
Critical strengths
  • This paper presents an interesting problem and a practical framework that jointly addresses query ambiguity, cross-domain cold start, and potential user–platform ranking mismatch.

  • Hierarchical profiling and evidence-grounded attribute mapping are intuitive and well structured.

  • Formulation is coherent, and the overall methodology is clearly presented and practically also offers lot of flexibility as this method can work with existing retrieval systems without modification.

  • Results show substantial improvements in retrieval and ranking metrics, particularly MRR@100.

  • Experiments use different retrievers (BM25, vector, and hybrid) and includes ablation studies, prompt-optimization comparisons, and live Amazon retrieval evaluation.

  • Implementation details, prompt examples, and an Oracle query analysis are given in Appendix.

Critical weaknesses
  • Real-world personalization and generalization evaluation is limited.

    • Synthetic fuzzy queries are generated from held-out target products.
    • A single five-star product per user serves as ground truth, limiting relevance evaluation.
    • Most Hit@100 improvements come from Electronics; Beauty shows marginal recovery gains.
  • Confidence intervals and statistical significance tests are missing.

  • Comparisons with existing LLM-based cross-domain recommendation and query-rewriting approaches are missing.

  • 3 LDCO stages are ablated together rather than independently.

  • MCTS lacks comparisons against simpler, iterative, budget-matched prompt optimization methods.

  • Product-specific normalization rules in the optimized prompt may explain some gains independently of preference transfer.

  • Evidence-grounding accuracy and negative-transfer rates are not explicitly evaluated.

  • Online path from target classification through reranking has latency of approximately 30 seconds per query which could limit practical applicability.

Questions for authors
  1. How much improvement in cross-domain personalization comes from user-specific preferences versus generic query rewriting? Could shuffled-profile and profile-free baselines isolate this contribution?

  2. What are the individual contributions of target classification, source selection, and dimension mapping? An ablation might provide useful insights.

  3. How does MCTS compare against random search or iterative prompt refinement under equal computational budgets?

  4. Please provide human relevance judgments, statistical significance tests, and confidence intervals for the reported improvements?

  5. Please report Hit@5/10/50 and NDCG@5/10/50 for ranking metrics. These cutoffs better represent practical recommendation quality than K=100/300/500.

  6. How frequently do transferred attributes introduce unsupported or irrelevant preferences? Can this be quantified?

  7. Could you include successful and failed examples from the actual test set, showing source evidence, generated queries, and ranking changes?

  8. How do latency and costs vary across queries?

2 comments
AuthorsOpenReview ↗

A Response to These Critical Comments

Thank you for the careful review and for recognizing LDCO's practical motivation. Your questions guided our next-stage work: additional comparisons, dimension-context ablations, case analysis, and an evaluation. We respond to your comments below.

Main additions and corrections

  • Additional rewriting comparisons, dimension-context ablations, and ranking results at K = 5, 10, and 50.
  • An initial attribute-support audit, concrete success and failure cases, and a latency breakdown.
  • Corrections to the dataset description and a clearer distinction between target-item recovery and user satisfaction.

1. User profiles versus generic rewriting

The retrieval-only conditions in Table 2 give:

\text{Condition} & \text{Hit@100} & \text{MRR@100} & \text{NDCG@100} \\\\ \hline \text{Fuzzy query} & 0.3169 & 0.0365 & 0.0872 \\\\ \text{LDCO: no profile} & 0.3453 & 0.0458 & 0.0993 \\\\ \text{LDCO: with profile} & 0.4155 & 0.0577 & 0.1236 \end{array}$$ Profile-free rewriting adds 0.0284 in Hit@100; profile conditioning adds another 0.0702. This supports the value of profile information. **2. Component contributions** We added ablations of three dimension-context groups, using the Round 12 prompt and Hybrid retrieval without reranking: $$\begin{array}{lrrr} \text{Dimension context} & \text{Hit@100} & \text{MRR@100} & \text{NDCG@100} \\\\ \hline \text{All groups} & 0.4155 & 0.0577 & 0.1236 \\\\ \text{No target-type dimensions} & 0.3803 & 0.0613 & 0.1197 \\\\ \text{No cross-type bridge} & 0.3944 & 0.0585 & 0.1204 \\\\ \text{No domain-pair mapping} & 0.4085 & 0.0611 & 0.1238 \end{array}$$ The complete context has the highest Hit@100, but some reduced contexts improve MRR. These address your request. **3. External comparisons and MCTS** We added an LLM4CS-REW-style profile-conditioned rewrite and a Search Swarm-style local multi-query replay. With Hybrid retrieval and no reranking, their Hit/MRR/NDCG@100 values are 0.2746/0.0346/0.0784 and 0.2042/0.0157/0.0492, respectively. LDCO performs better under this protocol. MCTS optimizes the prompt offline. Equal-budget comparisons with random search or iterative refinement remain unavailable. The controls also do not independently rule out product-specific prompt information, motivating the overlap audit below. **4. Evaluation validity and statistical reliability** The five-star target is an observed positive, but recovering it does not establish current user satisfaction. Offline metrics miss acceptable substitutes. The online Judge accepts substitutes, but remains target-referenced and lacks blinded human validation. Our audit finds 710 cases from 651 users, including 142 test cases from 140 users. Fourteen users cross splits under different target domains; five test cases share optimization target ASINs. We apologize for the inaccurate user-disjoint description and will correct the manuscript. Verified confidence intervals for this exact Table 2 comparison remain unavailable, so we report descriptive gains. These overlaps and the two-domain scope limit generalization. **5. Results at smaller cutoffs** The full model obtains the following results over the 142 test cases: $$\begin{array}{lrrr} \text{K} & \text{Hit@K} & \text{MRR@K} & \text{NDCG@K} \\\\ \hline 5 & 0.1056 & 0.0613 & 0.0721 \\\\ 10 & 0.1479 & 0.0663 & 0.0852 \\\\ 50 & 0.3380 & 0.0748 & 0.1264 \end{array}$$ These characterize early ranking. **6. Unsupported attributes and concrete examples** An initial audit flagged six unsupported preference-like additions in five of 142 cases (3.5%). This measures textual support. * **Success:** “a case for iPhone 6/6s” becomes “iPhone 6s rugged case with screen protector.” Durability concerns in the history support “rugged.” The target moves from rank 3 to rank 1 through reranking; this is an end-to-end example, not isolated causal evidence. * **Failure:** A beard-conditioner rewrite adds “softening and moisturizing” without explicit profile support. The target remains at rank 289, outside the Top-100 reranking pool, illustrating grounding and candidate-recovery limitations. **7. Latency and cost** Estimated end-to-end latency is 32–37 seconds: rewriting takes approximately 2–3 seconds, local retrieval 1–2 seconds, and product fetching plus reranking 28–32 seconds. Your suggestions have made our analysis more precise. We welcome further comments on the remaining concerns. If these additions sufficiently address the main reasons for your recommendation, would you consider reassessing the paper and increasing the score accordingly? Thank you!
AuthorsOpenReview ↗

Follow-up on Our Response and Invitation for Further Feedback

Dear Reviewer,

Thank you again for your careful review and constructive suggestions. We are following up on our response posted on November 16, which presents additional analyses and clarifications motivated by your comments.

As the discussion period approaches its end, we would appreciate your feedback on whether our response addresses your main concerns and which issues, if any, remain most important to your assessment. Your guidance would help us focus any further clarification and improve the paper.

We understand that you are reviewing several submissions and appreciate your time. We remain available to answer follow-up questions and discuss any points that need further explanation.

Thank you for your consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.