- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RPTune, a framework for LLM-based product search in small merchant catalogs. The method combines a learned catalog curator, which selects and reorganizes products based on downstream LLM feedback, with LLM post-training using a context-relative reward. The framework relies on synthetic training data and does not require merchant-provided relevance labels or user interaction logs. Experiments across seven merchants and nine LLM backbones demonstrate substantial improvements in search accuracy and inference efficiency.
Critical strengths
Clear motivation and practical framework. The paper addresses a meaningful problem for small merchant businesses and proposes a relatively simple framework that integrates catalog curation with LLM post-training.
Comprehensive experimental evaluation. The authors evaluate the approach across multiple merchants and LLM backbones. The results consistently demonstrate improvements in search accuracy and latency. The ablation studies further validate the contributions of context curation, context ordering, and post-training.
Useful empirical observations. The experiments reveal that appropriately pruning and ordering catalog items can improve LLM performance, even when the entire catalog fits within the context window. The generalization experiments on newly added products and unseen merchants further support the practical value of the approach.
Critical weaknesses
Limited algorithmic novelty and insufficient comparisons with closely related methods. The proposed method mainly combines existing embedding models, a lightweight MLP-based reorganizer, and GRPO post-training. Related approaches have already explored aligning context ranking with downstream LLM performance, such as RankRAG and RRPO. The paper should better distinguish its technical contributions from these methods and provide additional comparisons where applicable.
Insufficient justification of the reorganizer optimization objective. The reorganizer is trained using reward-weighted selections generated by a frozen LLM rather than directly optimizing the downstream model's expected reward through a conventional policy-gradient objective. Although the experiments show improvements, the paper does not sufficiently explain why this surrogate objective is effective or how it differs fundamentally from simpler reward-weighted ranking approaches.
Concerns about evaluation reliability. The evaluation queries and reference labels are primarily generated using an LLM. More importantly, the independent evaluation reported in the appendix shows that only 64.9% of the reference labels are accepted, while many queries admit multiple reasonable products. Therefore, single-reference exact-match accuracy may not adequately reflect recommendation quality. Additional evaluation using multiple acceptable products or independently judged relevance scores would strengthen the conclusions.
Section 4.4 “All methods consistently improves the gemma baseline” -> “All methods consistently improve upon the Gemma baseline.”
Questions for authors
-
In light of recent research, the authors should clearly articulate the novel contributions of this work to the recommender systems community.
-
The authors should provide a deeper explanation of why the surrogate loss constructed from the outputs of a frozen LLM can effectively optimize downstream product selection performance.
9 comments
Response to Reviewer q9ie (2/4): W2, Q2
W2, Q2: Thank you for raising this question. We clarify why our surrogate objective improves downstream product selection and how it differs from simpler reward-weighted ranking approaches.
Why our surrogate objective works. Intuitively, we let the frozen LLM pick from the current curated list several times. Picks that beat its average pick move up the list, while worse picks move down or drop out. A conventional policy-gradient approach would instead sample different contexts and reward the context-selection actions based on the LLM's answer quality, without explicitly targeting the products the LLM actually picks. The sampling process introduces two additional drawbacks: first, the space of possible product subsets and orderings grows combinatorially with catalog size, hindering effective exploration; second, each sampled context requires its own costly LLM rollouts, resulting in noisy reward estimates under a fixed rollout budget. Our surrogate instead keeps the context deterministic and directly updates each selected product's priority using its group-relative advantage. Since these priorities control which products appear in the context and where, the updates help the LLM make better selections from subsequent contexts without sampling alternative contexts.
Why simpler reward-weighted ranking falls short. Our objective differs from simpler ranking approaches in two ways. First, closed-loop LLM feedback learns from the products the LLM actually selects, rather than fitting fixed relevance labels. Second, comparing to raw-reward weighting, our group-relative advantages promote better-than-average selections and demote worse ones, whereas the former reinforces inferior products that still receive positive relevance rewards.
We isolate these two components through controlled ablations using the same frozen encoder, training data, and inference setup:
| Reorganizer | Closed-loop LLM feedback | Group-relative advantages | EM (%) | FR (%) |
|---|---|---|---|---|
| None (encoder only) | – | – | 16.4 | 60.6 |
| Supervised | ✗ | ✗ | 16.4 | 60.2 |
| Raw-reward weighting | ✓ | ✗ | 17.1 | 60.3 |
| RPTune (ours) | ✓ | ✓ | 20.7 | 65.1 |
Supervised ranking performs no better than the encoder, while raw-reward weighting improves EM by only 0.7 points. In contrast, our objective reaches 20.7% EM and 65.1% FR. Consistent with this, the average relevance of the LLM's selections increases steadily during our training (Figure 10), whereas under raw-reward weighting it stagnates or declines on five of seven merchants (decreased from 0.586 to 0.577 on average). We summarize these results in Section 4.4 and provide the gradient analysis and training dynamics in Appendix A.12.
Response to Reviewer q9ie (3/4): W3-1 (Reference label reliability)
W3: We clarify our evaluation setup and provide additional empirical evidence, which we believe substantially strengthens our conclusions. Specifically, we address your concerns below by (1) clarifying our audit numbers, and then (2) testing whether RPTune's gains depend on the reference labels through evaluations that do not rely on them.
Human verification and the original audit. We use synthetic queries to avoid relying on sensitive user search logs, following prior LLM-generated evaluation benchmarks with human quality checks [1][2][3], as discussed in Section 4.1 (lines 256–261) of the original submission. Human experts reviewed pilot queries for realism and, blind to the proposed reference, verified labels for the pilot queries, all tied cases, and a 10% sample of the remaining queries (Appendix A.3).
The subsequent audit examined whether judges outside the Gemini family supported our reference labels. Across the 154 held-out queries, our references met the four non-Gemini judges' threshold on 64.9% of queries, versus 55.8% for claude-opus-5's own full-catalog selections, a 9.1-point gap. Notably, claude-opus-5 itself served on the panel, yet the panel favored Gemini's references over Claude's own selections, suggesting that the labels do not merely reflect Gemini's preferences and providing evidence against model-family bias.
Why the reference met the panel’s threshold on 64.9% of queries. We clarify that the panel's threshold measures absolute suitability: whether a product, on its own, adequately satisfies the query's key requirements, rather than whether it is the best available option. Our queries reflect a common real-world situation in which each of a shopper's requirements is reasonable, but for many queries, no product in the merchant's catalog satisfies all of them, such as wedding-grade wear on a strict budget (Q1, Section 2). A useful search system must then recommend the best available trade-off, which may still fall short of the panel's absolute threshold.
Of the 54 queries where the reference missed this threshold, it still receives the panel's highest grade in 52 (96.3%), meaning no other evaluated product was graded higher. Overall, the reference receives the highest grade on 150 of 154 queries (97.4%). Thus, 64.9% measures absolute suitability, whereas 97.4% measures relative best-match quality among evaluated candidates. We have clarified this distinction and revised the terminology in Appendix A.3.
Why several variants can be broadly acceptable. On 44.8% of queries, at least two variants satisfied the shopper’s main priorities well enough to count as reasonable recommendations. A variant could qualify even when the panel preferred another product for the same query. We nevertheless use a single reference label per query because our task aims to identify the most suitable option for the customer rather than any broadly acceptable alternative. Notably, in 100% (69/69) of these queries, the judges’ grades favored a single variant over every other candidate.
[1] Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A benchmark for tool-agent-user interaction in real-world domains, 2024.
[2] Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. Benchmarking large language models in retrieval-augmented generation, 2023.
[3] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. ToolLLM: Facilitating large language models to master 16000+ real-world APIs, 2023.
Response to Reviewer q9ie (1/4): W1, Q1
Dear Reviewer q9ie,
Thank you for your detailed summary and constructive feedback. We are glad that you found RPTune’s motivation clear and its design practical for small merchants. We also appreciate that you highlighted one of our main findings: pruning and ordering products can improve search accuracy even when the entire catalog fits in the context window. Below, we address the mentioned weaknesses and have accordingly revised the paper with changes highlighted in blue:
W1, Q1: Thank you for pointing us to these closely related approaches. We now discuss RankRAG and RRPO in Related Work (Section 5) and compare against both in Table 2. While all three align context ranking with downstream performance, RPTune differs in its learning signal:
To compare with these methods directly, we implemented both in our setting under the same synthetic training data, catalogs, and 25% retention budget, and found that: (1) Our reorganizer objective provides a much stronger learning signal than stochastic context-policy RL: we replaced our reorganizer objective with RRPO's policy-gradient objective, and on
gemini-3.7-flashRRPO reaches 30.5% EM while RPTune reaches 51.4% EM. (2) Decoupling curation and GRPO post-training in the full RPTune pipeline outperforms unified rerank-and-select tuning: Ongemma-4-E4B-it, RankRAG reaches 12.6% EM while the full RPTune pipeline reaches 31.0% EM.We document the detailed numbers in Table 2 (with full breakdowns in Appendix Tables 7 and 15–18). Table 2 also includes TourRank, which ranks candidates with the LLM itself and achieves only 13.6% EM on the same
gemini-3.7-flashbackbone.From these results, our main contributions to the recommender-systems community are clear: (1) closed-loop item-level choice feedback for curation, which substantially outperforms alternative approaches such as RRPO, as shown above, with its effectiveness further validated by our ablations (Section 4.4; response to W2); (2) curation–post-training co-design: a context-relative GRPO reward that credits the best product the curator retains, providing stable learning signals on curated contexts (Section 4.4); and (3) a practical regime for small merchants: in-context catalog search, where learned curation improves every backbone on every merchant without merchant-provided relevance labels or behavior logs.