acceptodds
Under review as a conference paper at ICLR 2027

RPTune: Learned Context Curation for LLM Catalog Search

Abstract

For small merchant businesses (SMBs) whose catalogs fit within a long-context LLM, full-catalog prompting offers a compelling alternative to multi-stage retrieval designed primarily for large marketplaces with millions of items. However, fitting the full catalog into the context window does not ensure that the model can use it effectively, since LLMs do not exploit long contexts uniformly. We therefore study in-context catalog search through two complementary questions: (1) how to curate and present catalogs to the LLM, and (2) how to adapt the LLM for product selection on curated contexts. We propose RPTune, an end-to-end framework that couples learned catalog curation with LLM post-training using automatically generated, catalog-grounded supervision. An encoder-reorganizer curator orders and prunes products guided by downstream LLM feedback, while the resulting curated catalogs in turn improve the effectiveness of LLM post-training with a context-relative reward. We evaluate RPTune on 7 real merchants spanning distinct retail verticals, using 100 complex conversational queries per merchant. RPTune consistently improves search accuracy across both proprietary and open-weight LLMs, with context curation yielding gains of up to 31.4 percentage points and post-training adding a further 10.3 points on average.

Primary area:
foundation or frontier models, including LLMs
Keywords:
catalog search, LLMs

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_q9ieOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RPTune, a framework for LLM-based product search in small merchant catalogs. The method combines a learned catalog curator, which selects and reorganizes products based on downstream LLM feedback, with LLM post-training using a context-relative reward. The framework relies on synthetic training data and does not require merchant-provided relevance labels or user interaction logs. Experiments across seven merchants and nine LLM backbones demonstrate substantial improvements in search accuracy and inference efficiency.

Critical strengths

Clear motivation and practical framework. The paper addresses a meaningful problem for small merchant businesses and proposes a relatively simple framework that integrates catalog curation with LLM post-training.

Comprehensive experimental evaluation. The authors evaluate the approach across multiple merchants and LLM backbones. The results consistently demonstrate improvements in search accuracy and latency. The ablation studies further validate the contributions of context curation, context ordering, and post-training.

Useful empirical observations. The experiments reveal that appropriately pruning and ordering catalog items can improve LLM performance, even when the entire catalog fits within the context window. The generalization experiments on newly added products and unseen merchants further support the practical value of the approach.

Critical weaknesses

Limited algorithmic novelty and insufficient comparisons with closely related methods. The proposed method mainly combines existing embedding models, a lightweight MLP-based reorganizer, and GRPO post-training. Related approaches have already explored aligning context ranking with downstream LLM performance, such as RankRAG and RRPO. The paper should better distinguish its technical contributions from these methods and provide additional comparisons where applicable.

Insufficient justification of the reorganizer optimization objective. The reorganizer is trained using reward-weighted selections generated by a frozen LLM rather than directly optimizing the downstream model's expected reward through a conventional policy-gradient objective. Although the experiments show improvements, the paper does not sufficiently explain why this surrogate objective is effective or how it differs fundamentally from simpler reward-weighted ranking approaches.

Concerns about evaluation reliability. The evaluation queries and reference labels are primarily generated using an LLM. More importantly, the independent evaluation reported in the appendix shows that only 64.9% of the reference labels are accepted, while many queries admit multiple reasonable products. Therefore, single-reference exact-match accuracy may not adequately reflect recommendation quality. Additional evaluation using multiple acceptable products or independently judged relevance scores would strengthen the conclusions.

Section 4.4 “All methods consistently improves the gemma baseline” -> “All methods consistently improve upon the Gemma baseline.”

Questions for authors
  1. In light of recent research, the authors should clearly articulate the novel contributions of this work to the recommender systems community.

  2. The authors should provide a deeper explanation of why the surrogate loss constructed from the outputs of a frozen LLM can effectively optimize downstream product selection performance.

9 comments
AuthorsOpenReview ↗

Response to Reviewer q9ie (1/4): W1, Q1

Dear Reviewer q9ie,

Thank you for your detailed summary and constructive feedback. We are glad that you found RPTune’s motivation clear and its design practical for small merchants. We also appreciate that you highlighted one of our main findings: pruning and ordering products can improve search accuracy even when the entire catalog fits in the context window. Below, we address the mentioned weaknesses and have accordingly revised the paper with changes highlighted in blue:

W1, Q1: Thank you for pointing us to these closely related approaches. We now discuss RankRAG and RRPO in Related Work (Section 5) and compare against both in Table 2. While all three align context ranking with downstream performance, RPTune differs in its learning signal:

  • Because the LLM's answer in catalog search identifies a specific product, RPTune learns directly from the item-level choices, promoting or demoting each product the LLM selects on a deterministically constructed context.
  • RRPO, designed for RAG where the reader's answer is free text rather than one of the retrieved passages, instead uses the reader’s answer quality to reward the reranker’s context-selection steps and thus, when applied to catalog search, does not explicitly target the reader-selected products themselves to link their features to downstream LLM behavior.
  • RankRAG jointly learns ranking and generation from supervised targets, but its ranking supervision does not adapt to which near-substitutes the current reader selects incorrectly.

To compare with these methods directly, we implemented both in our setting under the same synthetic training data, catalogs, and 25% retention budget, and found that: (1) Our reorganizer objective provides a much stronger learning signal than stochastic context-policy RL: we replaced our reorganizer objective with RRPO's policy-gradient objective, and on gemini-3.7-flash RRPO reaches 30.5% EM while RPTune reaches 51.4% EM. (2) Decoupling curation and GRPO post-training in the full RPTune pipeline outperforms unified rerank-and-select tuning: On gemma-4-E4B-it, RankRAG reaches 12.6% EM while the full RPTune pipeline reaches 31.0% EM.

We document the detailed numbers in Table 2 (with full breakdowns in Appendix Tables 7 and 15–18). Table 2 also includes TourRank, which ranks candidates with the LLM itself and achieves only 13.6% EM on the same gemini-3.7-flash backbone.

From these results, our main contributions to the recommender-systems community are clear: (1) closed-loop item-level choice feedback for curation, which substantially outperforms alternative approaches such as RRPO, as shown above, with its effectiveness further validated by our ablations (Section 4.4; response to W2); (2) curation–post-training co-design: a context-relative GRPO reward that credits the best product the curator retains, providing stable learning signals on curated contexts (Section 4.4); and (3) a practical regime for small merchants: in-context catalog search, where learned curation improves every backbone on every merchant without merchant-provided relevance labels or behavior logs.

AuthorsOpenReview ↗

Response to Reviewer q9ie (2/4): W2, Q2

W2, Q2: Thank you for raising this question. We clarify why our surrogate objective improves downstream product selection and how it differs from simpler reward-weighted ranking approaches.

Why our surrogate objective works. Intuitively, we let the frozen LLM pick from the current curated list several times. Picks that beat its average pick move up the list, while worse picks move down or drop out. A conventional policy-gradient approach would instead sample different contexts and reward the context-selection actions based on the LLM's answer quality, without explicitly targeting the products the LLM actually picks. The sampling process introduces two additional drawbacks: first, the space of possible product subsets and orderings grows combinatorially with catalog size, hindering effective exploration; second, each sampled context requires its own costly LLM rollouts, resulting in noisy reward estimates under a fixed rollout budget. Our surrogate instead keeps the context deterministic and directly updates each selected product's priority using its group-relative advantage. Since these priorities control which products appear in the context and where, the updates help the LLM make better selections from subsequent contexts without sampling alternative contexts.

Why simpler reward-weighted ranking falls short. Our objective differs from simpler ranking approaches in two ways. First, closed-loop LLM feedback learns from the products the LLM actually selects, rather than fitting fixed relevance labels. Second, comparing to raw-reward weighting, our group-relative advantages promote better-than-average selections and demote worse ones, whereas the former reinforces inferior products that still receive positive relevance rewards.

We isolate these two components through controlled ablations using the same frozen encoder, training data, and inference setup:

ReorganizerClosed-loop LLM feedbackGroup-relative advantagesEM (%)FR (%)
None (encoder only)––16.460.6
Supervised✗✗16.460.2
Raw-reward weighting✓✗17.160.3
RPTune (ours)✓✓20.765.1

Supervised ranking performs no better than the encoder, while raw-reward weighting improves EM by only 0.7 points. In contrast, our objective reaches 20.7% EM and 65.1% FR. Consistent with this, the average relevance of the LLM's selections increases steadily during our training (Figure 10), whereas under raw-reward weighting it stagnates or declines on five of seven merchants (decreased from 0.586 to 0.577 on average). We summarize these results in Section 4.4 and provide the gradient analysis and training dynamics in Appendix A.12.

AuthorsOpenReview ↗

Response to Reviewer q9ie (3/4): W3-1 (Reference label reliability)

W3: We clarify our evaluation setup and provide additional empirical evidence, which we believe substantially strengthens our conclusions. Specifically, we address your concerns below by (1) clarifying our audit numbers, and then (2) testing whether RPTune's gains depend on the reference labels through evaluations that do not rely on them.

Human verification and the original audit. We use synthetic queries to avoid relying on sensitive user search logs, following prior LLM-generated evaluation benchmarks with human quality checks [1][2][3], as discussed in Section 4.1 (lines 256–261) of the original submission. Human experts reviewed pilot queries for realism and, blind to the proposed reference, verified labels for the pilot queries, all tied cases, and a 10% sample of the remaining queries (Appendix A.3).

The subsequent audit examined whether judges outside the Gemini family supported our reference labels. Across the 154 held-out queries, our references met the four non-Gemini judges' threshold on 64.9% of queries, versus 55.8% for claude-opus-5's own full-catalog selections, a 9.1-point gap. Notably, claude-opus-5 itself served on the panel, yet the panel favored Gemini's references over Claude's own selections, suggesting that the labels do not merely reflect Gemini's preferences and providing evidence against model-family bias.

Why the reference met the panel’s threshold on 64.9% of queries. We clarify that the panel's threshold measures absolute suitability: whether a product, on its own, adequately satisfies the query's key requirements, rather than whether it is the best available option. Our queries reflect a common real-world situation in which each of a shopper's requirements is reasonable, but for many queries, no product in the merchant's catalog satisfies all of them, such as wedding-grade wear on a strict budget (Q1, Section 2). A useful search system must then recommend the best available trade-off, which may still fall short of the panel's absolute threshold.

Of the 54 queries where the reference missed this threshold, it still receives the panel's highest grade in 52 (96.3%), meaning no other evaluated product was graded higher. Overall, the reference receives the highest grade on 150 of 154 queries (97.4%). Thus, 64.9% measures absolute suitability, whereas 97.4% measures relative best-match quality among evaluated candidates. We have clarified this distinction and revised the terminology in Appendix A.3.

Why several variants can be broadly acceptable. On 44.8% of queries, at least two variants satisfied the shopper’s main priorities well enough to count as reasonable recommendations. A variant could qualify even when the panel preferred another product for the same query. We nevertheless use a single reference label per query because our task aims to identify the most suitable option for the customer rather than any broadly acceptable alternative. Notably, in 100% (69/69) of these queries, the judges’ grades favored a single variant over every other candidate.

[1] Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A benchmark for tool-agent-user interaction in real-world domains, 2024.

[2] Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. Benchmarking large language models in retrieval-augmented generation, 2023.

[3] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. ToolLLM: Facilitating large language models to master 16000+ real-world APIs, 2023.

Reviewer_7jUgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • RPTune is a framework for LLM-based product search over small catalogs that fit within long-context windows of modern LLMs.
  • This paper combines a contrastively trained encoder, a learned catalog reorganizer for ranking and pruning, and GRPO-based LLM post-training.
  • Automatically generated labels for catalog-grounded supervision are used as reward signal bypassing the need for merchant-provided relevance labels.
  • Experimental results across seven merchant catalogs and nine LLMs demonstrating consistent improvements in accuracy and latency, with curation gains up to 31.4% and an additional 10.3% from post-training on average for Gemma.
Critical strengths
  • This paper presents an interesting framework and practical approach that combines learned context curation with LLM post-training which is promising for small-catalog search.
  • Strong accuracy gains across proprietary and open-weight LLMs and multiple merchant verticals demonstrate empirical improvements.
  • Several useful ablation studies are reported that evaluate encoder-only, reorganizer, GRPO post-training, context ordering, pruning budgets, and generalization.
  • The methodology is clear and mathematical formulations are straightforward. LLM-feedback training objective is particularly interesting.
Critical weaknesses

Incomplete baselines and evaluation

  • Missing comparisons with hybrid BM25, SPLADE, ColBERT, and structured attribute filtering.
  • Semantic IDs, DSI, and Search-R1 and other relevant literature deserve discussion as related approaches.
  • RAG uses top-10 candidates, whereas RPTune retains 25% of the catalog. Candidate budgets should be matched.
  • Apart from EM and feature rewards, other metrics like Recall@k, MRR, NDCG etc., could provide additional insights and give a holistic view.
  • Evaluation uses synthetic queries and predominantly single-reference labels, despite multiple products often being acceptable.

Unclear learning contributions and scalability

  • Supervised reorganizer achieves only 16.4% EM, compared with 20.7% for the LLM-feedback reorganizer. The underlying reason is not clear and might need further analysis.
  • Reorganizer uses a surrogate objective rather than directly optimizing the downstream LLM's expected reward.
  • Merchant-specific versus shared encoder training is insufficiently explored.
  • Synthetic relevance generation may be expensive. Training costs and scalability are not quantified.
  • Generalization experiments are encouraging but limited. Statistical confidence intervals are also missing.
Questions for authors
  • Are the encoder and reorganizer trained independently for each merchant? How would a shared cross-merchant model perform?
  • Why does supervised reorganizer training perform no better than the encoder alone? Is the improvement from better candidate recall, context ordering, or downstream LLM selection? Additional metrics might be useful.
  • Please compare against hybrid retrieval, SPLADE, or ColBERT under matched candidate budgets? How does this approach relate to semantic-ID retrieval and Search-R1?
  • Report curator Recall@k and ranking metrics such as MRR@k or NDCG@k where ranked outputs are available? Can multiple acceptable products be considered?
  • Do the improvements hold under independently human-graded relevance labels?
  • What are the actual labeling costs, training time, and GPU requirements per merchant? How frequently would models require retraining?
9 comments
AuthorsOpenReview ↗

Response to Reviewer 7jUg (1/6): W1, W2, Q3

Dear Reviewer 7jUg,

Thank you for your detailed summary and constructive feedback. We are glad that you found our framework design promising for small-catalog search and the reorganizer’s LLM-feedback training objective particularly interesting. We also appreciate your recognition of the consistent gains across merchants and model families. Below, we address the mentioned weaknesses and have accordingly revised the paper with changes highlighted in blue:

W1, W2, Q3: Thank you for pointing us to these additional retrieval baselines and related approaches. We added four retrieval and filtering baselines: hybrid retrieval (BM25 and Gemini Embedding 2 fused with reciprocal rank fusion), SPLADE++, ColBERTv2, and structured attribute filtering (an LLM extracts hard constraints like price, size, and shade, and filters on catalog metadata). Each is evaluated both as a context selector at RPTune’s 25% budget and in the top-10 RAG setting, with gemini-3.7-flash as the backbone. As we report in Table 2 and Section 4.2 (lines 304–308), all four new baselines yield lower average EM and FR than raw full-catalog prompting in both the top-10 RAG and 25% context-selection settings. The table below directly compares these baselines with RPTune at the matched 25% retention budget.

Method (gemini-3.7-flash backbone)Avg. EM (%) ↑Avg. FR (%) ↑Latency (s) ↓
gemini-3.7-flash (raw full catalog)36.869.08.0
+ ColBERTv2 (25% retention)23.158.412.8
+ SPLADE++ (25% retention)29.864.414.1
+ Hybrid BM25 (25% retention)34.867.89.3
+ Attribute filtering (25% retention)35.066.922.8
+ RPTune (25% retention)51.477.35.1

These four new baselines degrade EM relative to raw full-catalog prompting in 21 of the 28 selector–merchant combinations, while RPTune improves on all seven merchants, outperforming every new baseline on every merchant at the lowest average latency. Structured attribute filtering, the strongest of the four, falls short because rigid metadata matching over-filters on multi-constraint trade-off queries and misses soft or implicit attributes not captured in structured fields.

We now cite and position the following approaches in Related Work (Section 5). They were designed for large corpora or data-rich platforms, and each faces limitations in the SMB setting, where the full catalog fits in context but behavioral logs and ML infrastructure are scarce.

  • Generative retrieval requires training on the catalog and, for semantic IDs, on user logs: DSI [1] indexes documents by training on them, so catalog updates require further training, and semantic-ID recommenders such as TIGER [2] are trained on user interaction histories, which SMBs rarely have at scale. RPTune instead keeps the catalog in context, so it handles newly added products without retraining (Section 4.3) and requires no behavioral logs.

  • Search-R1 [3] trains an LLM to run multi-round search over a large open-domain corpus, which is unnecessary overhead when the entire catalog fits in context. Search-R1 also updates the LLM's weights and thus cannot be applied to frozen proprietary backbones. RPTune instead learns how to curate and present the catalog with a lightweight curator, which is able to improve 8 frozen backbones, and LLM post-training is an optional further stage for open-weight models.

  • We directly compare with approaches that align ranking with downstream LLM performance: as we show in the updated Table 2, on gemma-4-E4B-it, RPTune's full pipeline outperforms RankRAG [4] by 18.4 EM points, and on gemini-3.7-flash, RPTune’s curator outperforms RRPO [5] by 20.9 EM points.

[1] Yi Tay, Vinh Q. Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, Tal Schuster, William W. Cohen, and Donald Metzler. Transformer memory as a differentiable search index. NeurIPS 2022.

[2] Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Tran, Jonah Samost, Maciej Kula, Ed Chi, and Maheswaran Sathiamoorthy. Recommender systems with generative retrieval. NeurIPS 2023.

[3] Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-R1: Training LLMs to reason and leverage search engines with reinforcement learning, 2025.

[4] Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, and Bryan Catanzaro. RankRAG: Unifying context ranking with retrieval-augmented generation in LLMs. NeurIPS 2024.

[5] Yuhang Wu, Xiangqing Shen, Fanfan Wang, Cangqi Zhou, Zhen Wu, Xinyu Dai, and Rui Xia. Optimizing RAG rerankers with LLM feedback via reinforcement learning. ACL 2026.

AuthorsOpenReview ↗

Response to Reviewer 7jUg (2/6): W3, W4, Q4-1

W3: We would like to clarify that Table 2 in the original submission already includes budget-matched comparisons. In the “Context selectors (25% catalog retention)” block, the selectors select exactly the same 25% budget as RPTune. The top-10 RAG rows in the upper block are an additional comparison that follows the standard RAG setup, in which the LLM selects from a fixed number of top-ranked candidates.

W4, Q4-1: Thank you for suggesting a broader set of ranking metrics to complement our downstream LLM evaluation. We now discuss standalone ranking performance in Section 4.2 (lines 307–308) and provide detailed results in Appendix A.7, including the full selector comparison in Table 8.

Although RPTune's curator is designed to optimize context for downstream LLM selection, it achieves competitive standalone ranking performance. Our original submission reports context recall across retention budgets from 5% to 100% (Figure 6, right; Figure 7 in the revised paper). We now complement this with Recall, MRR, and NDCG at k=10k = 10 and 25% retention, ranking candidates by each selector's relevance score and computing NDCG with feature reward as graded relevance. RPTune's curator achieves Recall of 50.8% (top-10) / 77.1% (25% retention), MRR of 0.241 (top-10) / 0.254 (25% retention), and NDCG of 0.636 (top-10) / 0.686 (25% retention), ranking second overall by average rank across all six metric–budget combinations, behind only Jina Reranker 3.5, which it trails by small margins.

Higher standalone ranking metrics do not necessarily translate to better LLM selection. Jina Reranker 3.5 scores higher than RPTune on every reported ranking metric, yet yields substantially lower downstream EM with gemini-3.7-flash (Table 2). These metrics assess reference retrieval or feature-based relevance, but do not directly capture how the retained alternatives affect the LLM’s final choice: feature-similar products can distract it from selecting the reference. To isolate the effect of candidate-set composition while controlling reference availability and position, we conducted an experiment on the 489 queries where both RPTune and Jina retain the reference at the 25% budget. We place the reference in the same final product slot adjacent to the query and randomly shuffle the remaining candidates across five trials per query. Under these controls, gemini-3.7-flash achieves 45.3% EM with RPTune’s candidate sets versus 38.6% with Jina’s, a 6.7-point gain (95% CI [+3.7, +9.9]).

AuthorsOpenReview ↗

Response to Reviewer 7jUg (3/6): W5, Q4-2, Q5

W5, Q4-2, Q5: We clarify our evaluation setup and provide additional empirical evidence, which we believe substantially strengthens our conclusions.

We use synthetic queries to avoid relying on sensitive user search logs, following prior LLM-generated evaluation benchmarks with human quality checks [6][7][8], as discussed in Section 4.1 (lines 256–261) of the original submission. Human experts reviewed pilot queries for realism and, blind to the proposed reference, verified labels for the pilot queries, all tied cases, and a 10% sample of the remaining queries (Appendix A.3).

We use a single reference label because our task aims to identify the most suitable option for the customer rather than any reasonable one (Section 2). Specifically, all (69/69) audited queries with more than one acceptable variant have a single variant graded higher than every other evaluated candidate. To test directly whether RPTune's gains depend on this single reference, we conducted two new evaluations on the 154 held-out audited queries without using the original reference labels at all: (1) blinded comparisons of product selections by independent LLM judges and human annotators, and (2) accuracy evaluation allowing multiple suitable products. We report these results in Section 4.2 (lines 308–310) and Appendix A.3.

First, when selections differ, RPTune selects better than the baseline in over 85% of cases. Whenever RPTune and a baseline picked different products, we presented both selections to blinded LLM judges without revealing which method made which choice or the original reference label. As we show in Table 4, RPTune's pick was preferred 87% (LLM judges) and 86% (humans) of the time. Specifically:

Across the 154 audited queries and all 23 comparisons in Table 2, spanning all 9 LLM backbones, all 7 context selectors at the matched 25% budget, and all 7 top-10 RAG baselines on gemini-3.7-flash (1,853 per-comparison disagreement cases, covering 1,026 unique product pairs), we prompted claude-opus-5.5, grok-4.7, and gemini-4-argon to compare the two selected product records in randomized order with method identities and reference labels hidden. Counting each unique pair once, the panel rated RPTune's selection better in 87.1% of pairs by majority vote.

For independent human validation, we collected blind human annotations for all disagreements on frozen claude-sonnet-5 (49 cases) and post-trained gemma-4-E4B-it (127 cases), using the annotation interface shown in Figure 11 (Appendix A.3). These 176 comparison cases correspond to 171 unique annotation pairs, because five query–product pairs arise in both backbone settings. Using the same aggregation rule, human annotators rated RPTune's selection better in 86.0% of pairs overall:

Comparison groupOpus 5.5Grok 4.7Gemini 4 ArgonMajority Vote (≥2/3\geq 2/3)Human Annotations
Full (1,026 unique pairs): All 23 Comparisons83.7%82.3%89.3%87.1%-
Subset (171 unique pairs): Overall75.4%81.3%81.3%81.3%86.0%
Subset (49): claude-sonnet-581.6%85.7%81.6%83.7%89.8%
Subset (127): gemma-4-E4B-it73.2%79.5%81.1%81.1%83.5%

Second, when loosening the relevance criterion, RPTune's gains hold. On the same 154 queries, we evaluated accuracy against the set of products marked suitable by our non-Gemini judge panel, counting any product in this set as correct. As we show in Table 5 of our revised paper, across the same 23 comparisons, RPTune's mean gain is +20.0 pp vs. +19.5 pp under strict best-match EM, with statistically significant gains in 22 and 23 of the 23 comparisons, respectively, and method rankings remain consistent across both metrics. Single-reference EM therefore neither penalizes baselines for selecting alternative suitable variants nor inflates RPTune's improvements:

Comparison groupBest-Match EMAcceptable-Match
6 Non-Gemini LLMs+12.6+12.6
2 Gemini LLMs+9.4+8.5
gemma-4-E4B-it+23.6+23.4
7 Context Selectors+19.3+18.7
7 Top-10 RAG+27.9+30.5
All 23 Comparisons+19.5+20.0

[6] Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. τ-bench: A benchmark for tool-agent-user interaction in real-world domains, 2024.

[7] Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. Benchmarking large language models in retrieval-augmented generation, 2023.

[8] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. ToolLLM: Facilitating large language models to master 16000+ real-world APIs, 2023.

Reviewer_YZV5OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies in-context catalog search for small merchant businesses which fit within a long-context LLM. It argues that full-catalog prompting is an alternative to multi-stage retrieval in this regime, and asks two questions: how to curate/present the catalog, and how to adapt the LLM for product selection on curated contexts. The proposed framework, RPTUNE, couples an encoder–reorganizer curator with LLM post-training: the encoder is trained with MNRL; the reorganizer orders and prunes the catalog with scores combining base similarity and an MLP adjustment trained by LLM-in-the-loop RL; the downstream LLM is post-trained with GRPO on a context-relative reward derived from a relevance distribution Drel.

Critical strengths
  1. The core mechanism is isolated by a decisive ablation. The supervised reorganizer essentially matches the encoder-only curator, while the LLM-in-the-loop reorganizer improves on all seven merchants. This directly evidences that optimizing against downstream LLM selections — not relevance supervision or encoder similarity — drives the curation gains; the gradient analysis in App. A.10 explains why the two objectives differ. This is the strongest support for the paper's central claim.

  2. An honest, internally consistent empirical package. Consistent gains across 6 model families, credible data-quality engineering, a quantitative failure analysis, and every headline number recomputes correctly against the tables.

Critical weaknesses
  1. Training and evaluation supervision are generated by the same model, and the training-side supervision is entirely unvalidated. Drel — the only reward source for both reorganizer RL (Algorithm 1) and post-training — is produced by gemini-3.1-pro with no human verification; the evaluation queries and reference labels are also generated by the same model, mitigated only by partial human audit and a judge panel that accepts 64.9% of labels. The cross-family gain argument (Grok +19.3 vs. Gemini +12.5 EM) excludes model-family bias but not a systematic bias toward the labeler's notion of relevance. Part of the headline gains could reflect fitting the labeler's preferences rather than genuine search quality, and this risk is not quantified anywhere.

  2. No variance or significance is reported, and the most important design conclusions rest on single-merchant ablations. Tables 2–3 report 10-run means without STD/CI; several headline gains are only +5–8 pp (e.g., claude-opus +8.6, claude-sonnet +10.7). The reward-design ablation ("context-relative reward is essential: EM drops 10 pp", Sec. 4.4), the curated-vs-raw post-training study, and the ordering/pruning sweeps are all on Beauty Bakerie (115 variants) only. Both issues are fixable but materially weaken the empirical case.

Questions for authors

See weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer YZV5 (1/4): W1-1 (Training label reliability)

Dear Reviewer YZV5,

Thank you for your detailed summary and constructive feedback. We are glad you found the reorganizer’s reward design well motivated and supported by results across all seven merchants. We also appreciate your careful verification of our results and your recognition of the consistent gains across six model families. Below, we address the mentioned weaknesses and have accordingly revised the paper with changes highlighted in blue:

W1: Thank you for this feedback! We have strengthened our findings in two ways: (1) directly validating the training labels through blind judgments from three LLMs other than our labeler and from human annotators, and (2) evaluating RPTune's product selections through blinded comparisons without using the original reference labels at all.

Training labels. In Appendix A.1 lines 670–674 of the original submission, we explain why the labeling procedure described in Section 3.2 (lines 176–186) leaves relatively little room for ambiguity. The training queries are short and specify concrete attributes. For example, “fiery red liquid matte lipstick for date night” specifies the product type, color, and finish. Each call scores only 10 variants against a short query with explicit product attributes, leaving little ambiguity about what to assess. We therefore expected a single frontier model to handle this task reliably and used gemini-3.1-pro to generate the training labels. In our revised paper, we have also surfaced this rationale in the main text in Section 3.2, lines 187–190.

To check the labels empirically, we randomly selected one of the 10 training queries generated for each product variant. This covered every variant across all seven merchants and yielded 2,468 queries, or 10% of the training set. For each sampled query, we drew 3 pairs of randomly sampled product variants across the catalog and 3 hard pairs randomly sampled from the top 25% of variants (14,808 pairs total). With gemini-3.1-pro's scores hidden and the presentation order randomized, we prompted claude-opus-5.5, grok-4.7, and gemini-4-argon to select the better match. Across the 14,808 pairs, the three judges agreed with our training labels on 93.3%–94.0% (with 93.9% majority vote).

We then sampled 22 training queries per merchant (154 queries, 1 random pair and 1 top-25% pair each = 308 pairs) for blind human annotation through the interface we show in Figure 9 (Appendix A.1). As we show below, human annotations agreed with the higher-scored variant on 95.5% of pairs (96.8% on random pairs and 94.2% on top-25% hard pairs), closely matching the independent model panel (95.8% majority vote):

Audit groupOpus 5.5Grok 4.7Gemini 4 ArgonMajority Vote (≥2/3\geq 2/3)Human Annotator
Full (14,808): Overall94.0%93.3%93.6%93.9%—
Full: Random96.1%95.9%95.9%96.0%—
Full: Hard91.9%90.8%91.4%91.9%—
Subset (308): Overall96.1%94.8%94.8%95.8%95.5%
Subset: Random96.8%96.8%96.8%96.8%96.8%
Subset: Hard95.5%92.9%92.9%94.8%94.2%

We have included this full empirical audit in Appendix A.1 (lines 783–797 and Table 3) and surfaced its takeaway in the main text in Section 3.2 (lines 189–190).

Finally, we note that synthetic training-data generation is a modular component of RPTune, and natural extensions include replacing the labeler with another model, an ensemble of models from different families, or merchant-provided relevance signals when available, without changing the curator or post-training stages.

AuthorsOpenReview ↗

Response to Reviewer YZV5 (2/4): W1-2 (Evaluation label reliability)

Evaluation labels. Thank you for noting that RPTune’s gains extend across model families, including when Grok and Claude serve as the search backbones. Beyond the human verification during dataset construction, which covered all pilot queries, all tied cases, and a random 10% sample of the remaining queries (Section 4.1; Appendix A.3), we address your concerns below by (1) explaining what the 64.9% means, and then (2) testing whether RPTune's gains depend on the reference labels through evaluations that do not rely on them.

The 64.9% does not indicate unreliable evaluation labels. In our original audit of 154 held-out queries, our references met the four non-Gemini judges' threshold on 64.9% of queries, versus 55.8% for claude-opus-5's own full-catalog selections, a 9.1-point gap. Notably, claude-opus-5 itself served on the panel, yet the panel favored Gemini's references over Claude's own selections, suggesting that the labels do not merely reflect Gemini's preferences and providing evidence against model-family bias.

We clarify that the panel's threshold measures absolute suitability: whether a product, on its own, adequately satisfies the query's key requirements, rather than whether it is the best available option. Our queries reflect a common real-world situation in which each of a shopper's requirements is reasonable, but for many queries, no product in the merchant's catalog satisfies all of them, such as wedding-grade wear on a strict budget (Q1, Section 2). A useful search system must then recommend the best available trade-off, which may still fall short of the panel's absolute threshold.

Of the 54 queries where the reference misses this threshold, it still receives the panel's highest grade in 52 (96.3%), meaning no other evaluated product was graded higher. Overall, the reference receives the highest grade on 150 of 154 queries (97.4%). Thus, 64.9% measures absolute suitability, whereas 97.4% measures relative best-match quality among evaluated candidates. We have clarified this distinction and revised the terminology in Appendix A.3.

AuthorsOpenReview ↗

Response to Reviewer YZV5 (3/4): W1-3 (Independent recommendation quality checks)

Independent evaluation of recommendation quality. To directly test whether RPTune's gains reflect better recommendations rather than closer agreement with Gemini-generated references, we conducted two additional evaluations on the same 154 held-out queries: (1) blinded comparisons of product selections by independent LLM judges and human annotators, and (2) accuracy evaluation allowing multiple suitable products. We report these results in Section 4.2 (lines 308–310) and Appendix A.3. We believe the following results greatly strengthen our empirical conclusions:

First, when selections differ, RPTune selects better than the baseline in over 85% of cases. Whenever RPTune and a baseline picked different products, we presented both selections to blinded LLM judges without revealing which method made which choice or the original reference label. As we show in Table 4, RPTune's pick was preferred 87% (LLM judges) and 86% (humans) of the time. Specifically:

Across the 154 audited queries and all 23 comparisons in Table 2, spanning all 9 LLM backbones, all 7 context selectors at the matched 25% budget, and all 7 top-10 RAG baselines on gemini-3.7-flash (1,853 per-comparison disagreement cases, covering 1,026 unique product pairs), we prompted claude-opus-5.5, grok-4.7, and gemini-4-argon to compare the two selected product records in randomized order with method identities and reference labels hidden. Counting each unique pair once, the panel rated RPTune's selection better in 87.1% of pairs by majority vote.

For independent human validation, we collected blind human annotations for all disagreements on frozen claude-sonnet-5 (49 cases) and post-trained gemma-4-E4B-it (127 cases), using the annotation interface shown in Figure 11 (Appendix A.3). These 176 comparison cases correspond to 171 unique annotation pairs, because five query–product pairs arise in both backbone settings. Using the same aggregation rule, human annotators rated RPTune's selection better in 86.0% of pairs overall:

Comparison groupOpus 5.5Grok 4.7Gemini 4 ArgonMajority Vote (≥2/3\geq 2/3)Human Annotations
Full (1,026 unique pairs): All 23 Comparisons83.7%82.3%89.3%87.1%-
Subset (171 unique pairs): Overall75.4%81.3%81.3%81.3%86.0%
Subset (49): claude-sonnet-581.6%85.7%81.6%83.7%89.8%
Subset (127): gemma-4-E4B-it73.2%79.5%81.1%81.1%83.5%

Second, when loosening the relevance criterion, RPTune's gains hold. On the same 154 queries, we evaluated accuracy against the set of products marked suitable by our non-Gemini judge panel, counting any product in this set as correct. As we show in Table 5 of our revised paper, across the same 23 comparisons, RPTune's mean gain is +20.0 pp vs. +19.5 pp under strict best-match EM, with statistically significant gains in 22 and 23 of the 23 comparisons, respectively, and method rankings remain consistent across both metrics. Single-reference EM therefore neither penalizes baselines for selecting alternative suitable variants nor inflates RPTune's improvements:

Comparison groupBest-Match EMAcceptable-Match
6 Non-Gemini LLMs+12.6+12.6
2 Gemini LLMs+9.4+8.5
gemma-4-E4B-it+23.6+23.4
7 Context Selectors+19.3+18.7
7 Top-10 RAG+27.9+30.5
All 23 Comparisons+19.5+20.0

Comments on the paper

7 comments
AuthorsOpenReview ↗

Response to Reviewer RM3c (1/3): W1-1 (Reference label reliability)

Dear Reviewer RM3c,

Thank you for your detailed summary and constructive feedback. We are glad that you highlighted the consistent gains across all 63 backbone–merchant combinations and found our component analysis convincing, particularly the evidence for learning from downstream LLM selections. Below, we address the mentioned weaknesses and have accordingly revised the paper with changes highlighted in blue:

W1, Q1: Thanks for pointing this out! We clarify our evaluation setup and provide additional empirical evidence, which we believe substantially strengthens our conclusions. Specifically, we address your concerns below by (1) explaining what the 64.9% means, and then (2) testing whether RPTune's gains depend on the reference labels through evaluations that do not rely on them.

Why the reference met the panel’s threshold on 64.9% of queries. We clarify that the panel's threshold measures absolute suitability: whether a product, on its own, adequately satisfies the query's key requirements, rather than whether it is the best available option. Our queries reflect a common real-world situation in which each of a shopper's requirements is reasonable, but for many queries, no product in the merchant's catalog satisfies all of them, such as wedding-grade wear on a strict budget (Q1, Section 2). A useful search system must then recommend the best available trade-off, which may still fall short of the panel's absolute threshold. For context, claude-opus-5's own full-catalog selections met the same threshold on only 55.8% of queries, 9.1 points below our references. Notably, claude-opus-5 itself served on the panel, yet the panel favored Gemini's references over Claude's own selections, suggesting that the labels do not merely reflect Gemini's preferences and providing evidence against model-family bias.

Of the 54 queries where the reference missed this threshold, it still receives the panel's highest grade in 52 (96.3%), meaning no other evaluated product was graded higher. Overall, the reference receives the highest grade on 150 of 154 queries (97.4%). Thus, 64.9% measures absolute suitability, whereas 97.4% measures relative best-match quality among evaluated candidates. We have clarified this distinction and revised the terminology in Appendix A.3.

Why several variants can be broadly acceptable. On 44.8% of queries, at least two variants satisfied the shopper’s main priorities well enough to count as reasonable recommendations. A variant could qualify even when the panel preferred another product for the same query. We nevertheless use a single reference label per query because our task aims to identify the most suitable option for the customer rather than any broadly acceptable alternative. Notably, in 100% (69/69) of these queries, the judges’ grades favored a single variant over every other candidate.

AuthorsOpenReview ↗

Response to Reviewer RM3c (2/3): W1-2, Q1 (Independent recommendation quality checks)

Evaluation beyond single-reference accuracy. To directly assess whether RPTune improves recommendation quality independently of the original references, we conducted two additional evaluations on the same 154 held-out queries without using the original reference labels at all: (1) blinded comparisons of product selections by independent LLM judges and human annotators, and (2) accuracy evaluation allowing multiple suitable products. We report these results in Section 4.2 (lines 308–310) and Appendix A.3. We believe the following results greatly strengthen our empirical conclusions:

First, when selections differ, RPTune selects better than the baseline in over 85% of cases. Whenever RPTune and a baseline picked different products, we presented both selections to blinded LLM judges without revealing which method made which choice or the original reference label. As we show in Table 4, RPTune's pick was preferred 87% (LLM judges) and 86% (humans) of the time. Specifically:

Across the 154 audited queries and all 23 comparisons in Table 2, spanning all 9 LLM backbones, all 7 context selectors at the matched 25% budget, and all 7 top-10 RAG baselines on gemini-3.7-flash (1,853 per-comparison disagreement cases, covering 1,026 unique product pairs), we prompted claude-opus-5.5, grok-4.7, and gemini-4-argon to compare the two selected product records in randomized order with method identities and reference labels hidden. Counting each unique pair once, the panel rated RPTune's selection better in 87.1% of pairs by majority vote.

For independent human validation, we collected blind human annotations for all disagreements on frozen claude-sonnet-5 (49 cases) and post-trained gemma-4-E4B-it (127 cases), using the annotation interface shown in Figure 11 (Appendix A.3). These 176 comparison cases correspond to 171 unique annotation pairs, because five query–product pairs arise in both backbone settings. Using the same aggregation rule, human annotators rated RPTune's selection better in 86.0% of pairs overall:

Comparison groupOpus 5.5Grok 4.7Gemini 4 ArgonMajority Vote (≥2/3\geq 2/3)Human Annotations
Full (1,026 unique pairs): All 23 Comparisons83.7%82.3%89.3%87.1%-
Subset (171 unique pairs): Overall75.4%81.3%81.3%81.3%86.0%
Subset (49): claude-sonnet-581.6%85.7%81.6%83.7%89.8%
Subset (127): gemma-4-E4B-it73.2%79.5%81.1%81.1%83.5%

Second, when loosening the relevance criterion, RPTune's gains hold. On the same 154 queries, we evaluated accuracy against the set of products marked suitable by our non-Gemini judge panel, counting any product in this set as correct. As we show in Table 5 of our revised paper, across the same 23 comparisons, RPTune's mean gain is +20.0 pp vs. +19.5 pp under strict best-match EM, with statistically significant gains in 22 and 23 of the 23 comparisons, respectively, and method rankings remain consistent across both metrics. Single-reference EM therefore neither penalizes baselines for selecting alternative suitable variants nor inflates RPTune's improvements:

Comparison groupBest-Match EMAcceptable-Match
6 Non-Gemini LLMs+12.6+12.6
2 Gemini LLMs+9.4+8.5
gemma-4-E4B-it+23.6+23.4
7 Context Selectors+19.3+18.7
7 Top-10 RAG+27.9+30.5
All 23 Comparisons+19.5+20.0
AuthorsOpenReview ↗

Response to Reviewer RM3c (3/3): W2, Q2

W2, Q2: Thank you for these questions on preparation costs and retraining, which are critical to real-world deployment and inspired us to consider a transfer-based deployment mode. In short, we recommend a two-tier path for SMBs: (1) zero-cost onboarding, where a merchant directly reuses an RPTune model trained on other merchants, achieving accuracy comparable to merchant-specific preparations without additional labeling or training; and (2) optional one-time merchant-specific preparation, which costs \567onaverage(567 on average (\\73–\$1,677) per merchant and pays back after about 4.5K queries through prompt-token savings.

Tier 1: Zero-preparation deployment through cross-merchant transfer. We consider two practical scenarios in which a new merchant can reuse an existing RPTune model without additional labeling or training:

(1) Transfer from a single merchant. Our original cross-merchant experiments (Figure 3b) show that applying each merchant-specific RPTune model directly to every other merchant improves EM by 18.1 points and FR by 13.6 points on average over the raw backbone with positive gains in every pair, close to the in-domain gains of +20.3 EM and +17.7 FR.

(2) Cold-start onboarding on a multi-merchant platform. A platform such as Shopify can train one curator on its existing merchants and apply it to each new merchant. We simulate this by further evaluating a leave-one-merchant-out (LOMO) curator trained on six merchants and directly deployed on a seventh, unseen merchant. As we show in the new Figure 4a, with frozen gemma-4-E4B-it, LOMO achieves 21.4% EM and 64.9% FR, comparable to merchant-specific curation (20.7% EM and 65.1% FR), without any target-merchant labeling or training.

Tier 2: Optional one-time merchant-specific preparation. For merchants who want a dedicated RPTune model, we derive the following cost model on 80GB H100 GPUs (details in Appendix A.4): for nn variants and CC catalog tokens, teacher usage is ≈10nC+400n2\approx 10nC + 400n^2 input and ≈150n2\approx 150n^2 output tokens, and curator training takes ≈0.026n+0.65nC106\approx 0.026n + 0.65\frac{nC}{10^6} GPU-hours. For merchants who self-host an open-weight LLM and want further accuracy gains, optional post-training adds ≈2.25nC106\approx 2.25\frac{nC}{10^6} GPU-hours.

Specifically, for the 7 merchants we evaluated, the cost averages \567,withthesmallestcatalog(BB)costing567, with the smallest catalog (BB) costing \\73 and 6 of the 7 merchants costing under \$650. We report the detailed teacher API calls and token usage for each merchant in Appendix A.4, alongside the following cost and training breakdown.

MerchantData labeling costTraining (GPU-hours)Curator total costOptional post-training (GPU-hours)
BB\$615.8\$739.7
OON\$24416.1\$27627.2
HK\$15111.7\$17520.2
WCP\$55333.5\$62075.0
WB\$56633.4\$63378.1
FF\$1,53173.0\$1,677184.1
PF\$45530.7\$51679.4
Average\$50929.2\$56767.7

At 25% retention and public Gemini input-token price, RPTune saves an estimated \56–56–\\190 per 1,000 queries, allowing the one-time preparation cost to be recovered after approximately 4.5K queries on average through input-token savings alone. Larger catalogs generally cost more to prepare but also yield greater savings per query.

Regards,

Paper 21868 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.