acceptodds
Under review as a conference paper at ICLR 2027

RecoAtlas: A Benchmark and RL Environment for LLM Recommendation Agents Optimizing Set-Level Utility

Abstract

LLM recommender agents increasingly produce structured reports: sets of items accompanied by natural-language justifications. Yet existing benchmarks often focus on reranking small candidate sets or judging reports primarily by semantic plausibility, providing limited support for evaluating and training agents based on set-level utility. We introduce Recommender Atlas (Agentic Tasks for Learning and Assessment in Shopping), or RecoAtlas, a benchmark and reinforcement learning environment grounded in user behavior. RecoAtlas provides a shopping toolkit for product retrieval, complementarity, and diversity. Given a shopping query, an agent uses these tools to generate a structured recommendation report and receives behavior-grounded feedback, forming an end-to-end interaction that supports both evaluation and reinforcement learning. Its controlled tool environment exposes agents to semantic, behavior-aligned, or faulty tools, enabling systematic analysis of how LLM reasoning, tool signals, and tool-use policies contribute to performance. Across controlled experiments, RecoAtlas exhibits key properties of a meaningful benchmark for agentic systems: performance scales with model capacity and test-time compute, and improves with stronger tools. These experiments also reveal that semantic plausibility does not necessarily translate into behavior-grounded utility. RecoAtlas provides a foundation for training and evaluating shopping assistants to move beyond semantic plausibility toward coherent, behaviorally grounded recommendation sets.

Primary area:
datasets and benchmarks
Keywords:
Agentic Recommender System, LLM Agent, Too Use, LLM Post-Training

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EuybOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces RecoAtlas, a benchmark and RL environment for LLM shopping agents. Each task gives the agent a shopping request and asks it to search a big catalog and return a set of K items, each with a short exaplaination. Ground-truth sets come from real user behavior: items a user interacted with positively within 7 days on Amazon Reviews or H&M, filtered by GPT-5.6 for coherence. GPT-5.6 generates the query from those items. The agent gets three recommendation tools (relevance search, complementarity, de-duplication), each in a “semantic” version and a “behavior-trained” version.

Critical strengths
  1. The pipeline is complete and reusable: data split, query generation, tools, evaluator and RL interface all fit together, and code and artifacts are released. Others could apply it to new datasets.
  2. The cross-domain experiment is the convincing evidence that skills learned in RecoAtlas transfer beyond the training domain.
Critical weaknesses
  1. For the cross-domain experiment, it rests on a single data point. The authors train Qwen3-4B with GRPO for 512 updates on Amazon Musical Instruments and evaluate zero-shot on 406 hidden H&M Clothing queries. SetHit@K rises from 0.82% (base model) to 1.43% with the direct SetHit@K reward and 1.21% with the learned set reward. This is one source–target pair, one run per condition, and no seeds or confidence intervals. At such low absolute values, a 0.4–0.6 pp gap is hard to distinguish from noise.
  2. RL is front and center in the title, abstract, and contributions, but the experiments only use Qwen3-4B, each setting is run once with no seeds, and the two datasets point in opposite directions. That’s not enough to back a main contribution.
  3. For a benchmark paper, the key questions are: are the ground-truth sets reliable, does the query leak the answer, and does the metric reflect real utility? The paper doesn’t spend enough space on these.
Questions for authors

See weakness

5 comments

We thank the reviewer for the careful reading. We address each weakness with new evidence: (i) three-seed RL results on two backbones (Qwen3-4B and Gemma-4-E4B-it), including cross-domain transfer in both directions, and (ii) two human annotation studies on reference-set reliability, query leakage, and metric validity (seven annotators: four co-authors and three external annotators with no AI/RecSys background), and (iii) a benchmark rebuilt with a different LLM family (Gemini-Flash-3.8). Section and appendix numbers refer to the revised PDF.

W1. Cross-domain transfer rests on a single data point

We agree that one run per condition on 406 queries cannot separate small differences from noise. We added three-seed experiments in both directions, evaluated on all agent_hidden queries of the target domain. We ran this study on Gemma-4-E4B-it because all of its in-domain checkpoints were available; due to storage constraints, not all Qwen3-4B checkpoints were retained after their in-domain evaluation. The original Qwen3-4B run shows the same direction (0.82% → 1.21–1.43% on H&M).

Table R1. Cross-domain transfer: SetHit@K (%) on the target domain, base → transferred model, mean ± std over three seeds.

Train → target domainDirect rewardLearned rewardIn-domain training on target (direct / learned)
Musical Instruments → H&M0.29 → 1.85 ± 0.130.28 → 1.89 ± 0.071.81 ± 0.11 / 2.27 ± 0.10
H&M → Musical Instruments1.18 → 7.27 ± 0.211.23 → 8.05 ± 0.038.92 ± 0.75 / 8.08 ± 0.39
  • Transfer is robust, not noise. Every transferred run improves substantially over its base model in both directions, and the gains far exceed the variation across seeds.
  • Transfer comes close to training on the target domain. Without seeing a single target-domain query, transferred policies recover at least 78% of the gain of in-domain training.
  • What transfers is agent behavior. The policy learns to produce complete, well-formed reports and to search the catalog more effectively, and both carry over to the new domain (full validity and discovery metrics in Appendix L).
  • Neither reward transfers consistently better. As you note, the original small gap between rewards was within noise. With three seeds, the learned reward is comparable in one direction and better in the other, so we no longer claim that either reward transfers better and rather say it is domain/data dependent.

Revision. We replace the single-run transfer result with these three-seed results in Section 4.5 and add the full tables (validity, discovery, and retention before and after transfer) in Appendix L.

W2. More RL evidence

We agree that single runs on one backbone could not support the original claim. We now report three seeds on two backbones, evaluated on the full held-out set, and we clarify what the RL contribution is.

Table R2. Held-out SetHit@K (%) after RL, mean ± std over three seeds. Base = untrained model, averaged over the two reward arms.

BackboneDomainBase policyDirect (SetHit@K) rewardLearned reward
Qwen3-4BMusical Instruments6.2010.77 ± 0.468.76 ± 0.38
Qwen3-4BH&M0.931.08 ± 0.171.87 ± 0.17
Gemma-4-E4B-itMusical Instruments1.218.92 ± 0.758.08 ± 0.39
Gemma-4-E4B-itH&M0.291.81 ± 0.112.27 ± 0.10

RL improves mean SetHit@K over the base policy in all 8 settings (the smallest gain, Qwen3-4B on H&M with the direct reward, is within seed variation), and the trained policies transfer across domains (W1). The gains come from better behavior at each stage of the task (all metrics are in the revised RL table, Section 4.5):

  • Search. Discovery recall (the share of reference items surfaced by any tool call) rises in 7 of 8 settings, e.g., Qwen3-4B on Musical Instruments: 15.48% → 20.98%.
  • Selection. The share of discovered reference items kept in the final report rises in all 8 settings, e.g., Qwen3-4B on Musical Instruments: 40.11% → 51.12%.
  • Report completeness. Gemma-4-E4B-it initially produces valid reports only 17–19% of the time; RL raises this to 97–100% under both rewards (Qwen3-4B is already ≥99.88% valid).

This is the RL contribution we will emphasize: RecoAtlas turns logged behavior into an environment where training reliably improves how recommendation agents search, select, and report.

The reward comparison is domain-dependent. On H&M, every learned-reward run beats every direct-reward run on both backbones. On Musical Instruments, direct optimization is better for both backbones. This shows a consistent, domain-dependent pattern. Precisely, GRPO learns from reward differences within a rollout group. When every report in a group is valid but recovers no reference item, the direct reward is constant and gives no learning signal, whereas the dense learned reward can still rank the reports. This matters more when recovery is sparse:

  • H&M is a sparser starting point. Base SetHit@K is 4–7× lower than on Musical Instruments.
  • The learned reward improves discovery where the direct reward stalls. On H&M, it raises discovery recall on both backbones, while direct training barely moves it.

Revision. We revise the contributions and Sections 1, 4.5, and 5 to present the RL contribution as the environment and its consistent training gains, and to state the reward comparison as domain-dependent.

AuthorsOpenReview ↗

Part III (Final)

W3. Reference-set reliability, query leakage, and metric utility

We agree these are the key questions for a benchmark, and we answer each with new evidence that the revised paper moves into a dedicated validity section.

Are the reference (ground-truth) sets reliable? A mined 7-day set is kept only if at least 2 of 3 LLM coherence votes find one common goal, a useful role for every item, and no redundancy (Section 3.1). Human annotators confirm the result: on 50 random held-out H&M query–set pairs, a majority judged 49/50 sets to be coherent bundles and 48/50 to satisfy their request (Table R3). The filter is also not specific to GPT-5.6: rebuilding Musical Instruments with Gemini-Flash-3.8 in every construction role gives 77.9% agreement on keep/reject decisions over 3,189 mined sets, and the leaderboard ranking of the four open-weight agents is unchanged (Section 4.1, Appendix K).

Do queries leak the answer? Each query passes deterministic leakage checks and an LLM query–set consistency check. No query received a majority leakage flag (0/50; 3.7% of individual judgments).

Table R3. Human verification of queries and reference sets.

QuestionYesPartlyNoQueries with majority "Yes"
Does the set satisfy the request?86.2%13.0%0.8%48/50
Do the products make sense together as one bundle?85.4%12.2%2.4%49/50
Does the request read like a real shopper's request?70.7%24.8%4.5%46/50
Does the request leak the exact items? (lower is better)3.7%0.0%96.3%0/50

Does SetHit@K reflect real utility? Reference sets come from logged positive interactions within a seven-day window and are kept only if their products serve distinct, useful roles toward one goal (Section 3.1). Because the report size is fixed at K=∣G∣K = |G|, recovering more reference items means covering more of these roles, and every redundant product occupies a slot that could have recovered a missing role. To test this with humans, annotators compared two equal-size bundles for each of 50 held-out H&M queries, one with SetHit@K > 0 and one with SetHit@K = 0. They were blind to model identity, SetHit@K, and the reference set, and presentation order was balanced. They chose the bundle that better fits the request, checking coverage of every requested piece, the stated colour, style, and occasion, whether the items work as one outfit, and redundancy. Annotators preferred the bundle with positive SetHit@K in 81.0% of judgments (84.3% for external annotators), and SetHit@K agrees with individual annotators (κ = 0.62) more than annotators agree with each other (mean pairwise κ = 0.50) (Table R4). SetHit@K gives no credit to acceptable alternatives outside the reference set, a limitation shared with behavior-based metrics such as purchase-based Recall@K, which we state explicitly.

Table R4. Human agreement with SetHit@K.

Agreement with SetHit@K [95% CI]Cohen's κ with SetHit@K
All annotators81.0% [74.0, 87.6]0.62
Co-authors78.1% [69.4, 86.2]0.56
External annotators84.3% [77.1, 91.1]0.69
Majority vote (ties excluded)88.6%—
Human–human (mean pairwise κ)—0.50

Revision. We add a dedicated benchmark-validity section to the main paper (Section 4.1) covering reference-set reliability, query leakage, and metric validity, with the full annotation protocols in a new appendix.

We are very grateful for the reviewer's time. We hope these additions address the reviewer's concerns, and we would be grateful if the reviewer would consider updating their score.

Reviewer_sMswOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RecoAtlas introduces a benchmark and RL environment for LLM recommendation agents that search large catalogs and construct coherent shopping bundles. It uses behavior-derived reference sets, recommendation-specific tools, and a learned set-level reward to support evaluation and policy optimization. Experiments cover four environments from Amazon and H&M.

Critical strengths
  1. RecoAtlas extends traditional item-ranking evaluation to large-catalog, set-level recommendation, better reflecting the tool-using workflows of LLM recommendation agents.
  2. The benchmark integrates behavioral reference sets, recommendation-specific tools, and an RL interface, providing a useful environment for both evaluating and improving recommendation agents.
  3. The experiments systematically investigate tool alignment, LLM reasoning, model scaling, thinking modes, and RL optimization, revealing useful patterns in recommendation-agent performance.
Critical weaknesses
  1. SetHit@K measures exact reference-item recovery rather than complementarity, compatibility, or overall bundle quality, making its relationship to the paper's central motivation unclear.
  2. Shopping queries are generated from the ground-truth item sets, and temporally proximate interactions do not necessarily represent coherent shopping bundles, potentially limiting the benchmark's validity for real-world user requests.
  3. The LLM agent and non-LLM baseline use different retrieval strategies and candidate discovery procedures, making it difficult to isolate the contribution of LLM reasoning from improved retrieval.
  4. The reported learned-reward improvement on H&M is based on a single training run without uncertainty estimates, while direct reward performs better on Musical Instruments, limiting the generality of the conclusions.
Questions for authors
  1. Can the authors provide independent evidence that higher SetHit@K corresponds to better bundle quality, including complementarity, functional coverage, and acceptable alternative recommendations?
  2. How sensitive are the benchmark results to the query-generation procedure? Would the conclusions hold for independently generated or real-world shopping queries?
  3. How much of the LLM agent's improvement remains when non-LLM baselines use the same candidate pools, query decomposition, and retrieval budgets?
  4. Does the learned-reward improvement persist across multiple training seeds, and what explains its different effectiveness on H&M and Musical Instruments?
5 comments

We thank the reviewer for the careful reading. We address each weakness and question with new evidence: (i) two human annotation studies (seven annotators: four co-authors and three external annotators with no AI/RecSys background), (ii) a benchmark rebuilt with a different LLM family, (iii) a matched-candidate-pool comparison of LLM and non-LLM selection, and (iv) three-seed RL results on two backbones. Section and appendix numbers refer to the revised PDF.

W1 / Q1. SetHit@K and bundle quality

We agree that SetHit@K gives no credit to acceptable alternatives outside the reference set (Section 3.4). However, because of how the reference set is constructed, SetHit@K implicitly rewards complementarity (role coverage) and penalizes redundancy, and independent human judgments of bundle quality agree with it.

  1. Set quality is encoded in the reference set and the fixed set size KK. Reference sets come from logged positive interactions within a seven-day window and are kept only if their products serve distinct, useful roles toward one goal (Section 3.1). Because the report size is fixed at K=∣G∣K = |G|, recovering more reference items means covering more of these roles, and every redundant product occupies a slot that could have recovered a missing role.

  2. Independent evidence (Q1). On 50 held-out H&M queries, annotators compared two equal-size bundles produced by leaderboard agents, one with SetHit@K > 0 and one with SetHit@K = 0. They were blind to model identity, SetHit@K, and the reference set, and presentation order was balanced. They chose the bundle that better fits the request, checking (i) coverage of every requested piece (functional coverage), (ii) the stated colour, style, and occasion, (iii) whether the items work together as one outfit (complementarity/compatibility), and (iv) redundancy. Annotators preferred the bundle with positive SetHit@K in 81.0% of judgments (84.3% for external annotators), and SetHit@K agrees with individual annotators (κ = 0.62) more than annotators agree with each other (mean pairwise κ = 0.50).

Table R1. Human agreement with SetHit@K.

Agreement with SetHit@K [95% CI]Cohen's κ with SetHit@K
All annotators81.0% [74.0, 87.6]0.62
Co-authors78.1% [69.4, 86.2]0.56
External annotators84.3% [77.1, 91.1]0.69
Majority vote (ties excluded)88.6%—
Human–human (mean pairwise κ)—0.50
  1. Acceptable alternatives (Q1). 19.0% of judgments favoured the zero-hit bundle, but by majority vote only 5 of 50 queries favoured it. Given the moderate agreement between annotators, these cases reflect both annotator disagreement and acceptable alternatives outside the reference set. This is a standard limitation of behavior-based metrics such as purchase-based Recall@K, which also give no credit to unobserved but acceptable items. LLM judges can credit such alternatives, but in our experiments they do not track behavior: utility-aligned tools substantially increase SetHit@K with no statistically clear change in LLM-judge relevance (Section 4.2, Appendix G). We therefore report behavioral and semantic signals separately.

Revision. We state explicitly in Section 3.4 how SetHit@K rewards role coverage and penalizes redundancy, add the human study in Section 4.1 and as a new appendix, and expand the limitation on acceptable alternatives.

W2 / Q2. Query generation and bundle coherence

We agree that temporal proximity alone does not make a coherent bundle, which is why it is only the first filter. A mined 7-day set is kept only if at least 2 of 3 LLM coherence votes find one common goal, a useful role for every item, and no redundancy (Section 3.1). Each query must then pass deterministic leakage checks and an LLM query–set consistency check. Generating queries from the reference set is a deliberate design choice: public interaction datasets do not record the shopping request behind a multi-item purchase, so the request must be reconstructed from the purchased set. These construction steps are the only place where an LLM affects our main metric. LLMs reject candidate sets (never adding products) and write the queries, but SetHit@K, used for the leaderboard and all RL results, is computed by exact matching against logged items with no LLM at evaluation time. The LLM judge is a complementary signal reported separately.

We therefore validate the construction steps in two ways:

  • Human annotation (W2) checks whether the kept sets are coherent bundles and whether the generated queries match their sets, read as realistic, and avoid leaking items.
  • Independent regeneration with a different LLM family (Q2) checks whether our benchmark conclusions, in particular the leaderboard ranking, depend on the generator.
  1. Human verification. External and co-author annotators rated 50 random held-out H&M query–set pairs (Table R2). By majority vote, the set satisfies the request for 48/50 queries, the products form one coherent bundle for 49/50, and no request leaks an item (0/50). In 95.5% of judgments, the request reads as at least partly realistic.

Table R2. Human verification of queries and reference sets.

QuestionYesPartlyNoQueries with majority "Yes"
Does the set satisfy the request?86.2%13.0%0.8%48/50
Do the products make sense together as one bundle?85.4%12.2%2.4%49/50
Does the request read like a real shopper's request?70.7%24.8%4.5%46/50
Does the request leak the exact items? (lower is better)3.7%0.0%96.3%0/50
  1. Independently generated queries (Q2). We rebuilt the Musical Instruments benchmark with Gemini-Flash-3.8 replacing GPT-5.6 in every role (coherence filtering, query generation, and consistency checks), and all queries were regenerated. On the 3,189 mined candidate sets, the two filters agree on 77.9% of decisions (Cohen's κ = 0.57). 93.3% of the sets Gemini-Flash-3.8 keeps are also kept by GPT-5.6. The leaderboard ranking is unchanged in both thinking modes (Gemma 4 > Qwen3.8 > Nemotron 3.5 > GLM-4.7), and thinking improves every model, as in the original benchmark (Table R3).

Table R3. SetHit@K (%) on the Gemini-built Musical Instruments benchmark (1,446 held-out queries).

ModelThinking offThinking onGain (pp)
Gemma 4 31B IT13.7414.78+1.04
Qwen3.8 27B-FP811.1713.52+2.35
Nemotron 3.5 Lightning 30B-A3B8.0310.91+2.88
GLM-4.7 Flash7.929.06+1.13

Real-world queries (Q2). Evaluation on real shopper requests remains open, and this gap is precisely what motivates RecoAtlas: to our knowledge, no public dataset pairs real requests with the multi-item sets bought to satisfy them, so we reconstruct requests from logged behavior and validate them as above.

Revision. We detail the coherence voting and query checks in Section 3.1, report the human study and the Gemini-built benchmark in Section 4.1 and new appendices, and state evaluation on real shopper requests as a limitation.

AuthorsOpenReview ↗

Part III (Final)

W3 / Q3. LLM reasoning vs. retrieval

Answer to Q3: with identical candidate pools, 50–65% of the LLM agent's gain over the non-LLM baseline remains. To isolate selection from retrieval, we give the non-LLM selector the exact candidate pool that each LLM agent discovered through its tool calls. Discovery is therefore held fixed, and only the final selection changes.

Table R4. SetHit@K (%): non-LLM selector vs. LLM agent on the same candidate pool.

DatasetAgentNon-LLM, own retrievalNon-LLM, agent's poolLLM agentGain from LLM selection (pp)
Musical InstrumentsQwen3.8-27B-FP8 (thinking off)4.426.449.20+2.76
Musical InstrumentsGPT-5.6 Sol4.427.9914.56+6.57
H&MQwen3.8-27B-FP8 (thinking off)0.701.331.96+0.63
H&MGPT-5.6 Sol0.701.522.88+1.36

The LLM agent contributes in two ways, neither of which the tools provide alone:

  • Discovery. The LLM's query decomposition and tool use find better candidates: given the agent's pool, the same non-LLM selector improves on both datasets.
  • Selection. On the identical pool, the LLM recovers more reference items than the non-LLM selector.

Revision. We add this matched-pool comparison to Section 4.2 (Figure 2, Qwen3.8-27B-FP8) and report all four settings and the selector in Appendix E.2.

W4 / Q4. Multiple seeds and why H&M differs

We agree that a single run cannot support a general claim. We now report three seeds on two backbones, evaluated on the full held-out set (agent_hidden, 1,960 queries per domain; the original RL experiments used 406).

Table R5. Held-out SetHit@K (%) after RL, mean ± std over three seeds. Base = untrained model.

BackboneDomainBase policyDirect (SetHit@K) rewardLearned reward
Qwen3-4BMusical Instruments6.2010.77 ± 0.468.76 ± 0.38
Qwen3-4BH&M0.931.08 ± 0.171.87 ± 0.17
Gemma-4-E4B-itMusical Instruments1.218.92 ± 0.758.08 ± 0.39
Gemma-4-E4B-itH&M0.291.81 ± 0.112.27 ± 0.10

The H&M gain persists across seeds (Q4). On H&M, we observed that every learned-reward run beats every direct-reward run on both backbones. On Musical Instruments, direct optimization is better for both backbones. RL improves mean SetHit@K over the base policy in all eight settings. In three-seed cross-domain transfer with Gemma-4-E4B-it, both rewards also improve over the base policy in both directions.

Why the domains differ (Q4). GRPO learns from reward differences within a rollout group. When every report in a group is valid but recovers no reference item, the direct reward is constant and gives no learning signal, whereas the dense learned reward can still rank the reports. This matters more when recovery is sparse:

  • H&M is a sparser starting point. Base SetHit@K is 4–7× lower than on Musical Instruments.
  • The learned reward improves discovery where the direct reward stalls. On H&M, it raises discovery recall (the share of reference items surfaced by any tool call) on both backbones, while direct training barely moves it.

Revision. We revise Sections 1, 4.5, and 5 to state the domain-dependent result: the learned reward helps where recovery feedback is sparse (H&M), while direct optimization is better on Musical Instruments.

We are very grateful for the reviewer's time. We hope these additions address the reviewer's concerns, and we would be grateful if the reviewer would consider updating their score.

Reviewer_1D5KOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

To address the limitations of existing evaluation frameworks for recommendation agents—which primarily focus on ranking individual items or assessing semantic plausibility, while neglecting the utility of multi-item bundle recommendations—this paper introduces RecoAtlas, a benchmark and reinforcement learning environment. The framework constructs item bundles based on users' historical purchase or positive feedback data and employs large language models (LLMs) to generate the corresponding complex shopping needs. It further equips agents with tools for retrieving relevant items, discovering complementary products, and filtering out redundant items. Experiments conducted across three product categories from Amazon and a dataset of H&M apparel analyze how factors such as model scale, reasoning capability, tool quality, and reinforcement learning rewards influence recommendation performance.

Critical strengths
  1. This work focuses on multi-item bundle recommendation within complex shopping scenarios. It emphasizes that recommendations must go beyond simple relevance to individual items by also accounting for complementarity and diversity among items, representing a richer task formulation than traditional single-item recommendation.

  2. The framework integrates shopping intent generation, product retrieval tools, structured recommendation reporting, and result evaluation into a unified environment. It supports both testing and reinforcement learning-based training, offering notable scalability and practical engineering value.

  3. A variety of large language models are compared across four product domains, and the effects of model size, thinking patterns, tool types, and different reinforcement learning algorithms are examined, providing rich experimental results for analyzing the capabilities of recommendation agents.

Critical weaknesses
  1. The article takes Set-Level Utility as the core research objective, but the main evaluation metric SetHit@K is actually just the precise matching ratio between the recommended products and the historical reference product set. This metric fails to directly capture the complementarity, completeness, or diversity of the items within a set. For instance, if a user is shopping to "learn to play the guitar" and an agent recommends a different, yet equally suitable, guitar, the recommendation would not count as a "hit" simply because the item ID differs from the one in the historical record.

  2. The authors rely on GPT-5.6 to simultaneously perform item-set filtering, query generation, and semantic evaluation. The lack of independent human verification makes it difficult to ensure the authenticity and objectivity of the synthesized user needs and their corresponding reference item sets.

  3. Although the paper posits that a learned set-based reward function improves reinforcement learning performance, the experimental results are inconsistent. On the H&M dataset, the learned reward increased SetHit@K from 1.21% to 1.88%; however, on the Musical Instruments dataset, using SetHit@K directly as the reward yielded better results. Similarly, in cross-domain transfer experiments, a direct reward outperformed the learned reward (1.43% vs. 1.21%). These findings suggest that while a learned reward might be effective on certain datasets, they are insufficient to demonstrate a consistent training advantage or robust cross-domain generalization capability.

Questions for authors

Please see the weaknesses.

3 comments

We thank the reviewer very much for their careful reading. We address all three concerns with new evidence: (i) 2 human annotation studies (7 annotators: 4 co-authors and 3 external annotators with no AI/RecSys background), (ii) a benchmark rebuilt with a different LLM family (Gemini-Flash-3.8), and (iii) 3-seed RL results on two policy backbones (Qwen3-4B and Gemma-4-E4B-it), including cross-domain transfer in both directions. Section and appendix numbers refer to the revised PDF.

W1. Does SetHit@K capture set-level utility?

We agree that SetHit@K gives no credit to suitable alternatives outside the reference set; the paper notes this limitation (Section 3.4), and the reviewer's guitar example illustrates it well. However, because of how the reference set is constructed, SetHit@K still implicitly rewards role coverage and penalizes redundancy, and human judgments of bundle quality agree with it.

  1. Set quality is encoded in the reference set and the fixed set size KK. Reference sets come from logged positive interactions within a seven-day window and are kept only if their products serve distinct, useful roles toward one goal (Section 3.1). Because the report size is fixed at K=∣G∣K = |G|, recovering more reference items means covering more of these roles, and every redundant product occupies a slot that could have recovered a missing role.

  2. Independent evidence. On 50 held-out H&M queries, annotators compared two equal-size bundles produced by leaderboard agents, one with SetHit@K > 0 and one with SetHit@K = 0. They were blind to model identity, SetHit@K, and the reference set, and presentation order was balanced. They chose the bundle that better fits the request, checking (i) coverage of every requested piece (functional coverage), (ii) the stated colour, style, and occasion, (iii) whether the items work together as one outfit (complementarity/compatibility), and (iv) redundancy. Annotators preferred the bundle with positive SetHit@K in 81.0% of judgments (84.3% for external annotators), and SetHit@K agrees with individual annotators (Cohen's κ = 0.62) more than annotators agree with each other (mean pairwise Cohen's κ = 0.50).

Table R1. Human agreement with SetHit@K (95% CIs bootstrapped over queries).

Agreement with SetHit@K [95% CI]Cohen's κ with SetHit@K
All annotators81.0% [74.0, 87.6]0.62
Co-authors78.1% [69.4, 86.2]0.56
External annotators84.3% [77.1, 91.1]0.69
Majority vote (ties excluded)88.6%—
Human–human (mean pairwise κ)—0.50
  1. Exact matching reflects real-world feedback. As in standard offline recommender evaluation, the reference set records observed positive behavior (actual co-purchases or co-interactions), not every acceptable alternative. SetHit@K can therefore under-credit a good report, but it never credits products that users did not choose. LLM judges can credit unseen alternatives, but in our experiments they do not track behavior: utility-aligned tools substantially increase SetHit@K with no statistically clear change in LLM-judge relevance (Section 4.2, Appendix G).

Revision. We state explicitly in Section 3.4 how SetHit@K rewards role coverage and penalizes redundancy, add the human study in Section 4.1 and as a new appendix, and expand the limitation on acceptable alternatives.

W2. Dependence on GPT-5.6

GPT-5.6 plays three roles in RecoAtlas, but every reference item remains a real logged co-purchase or co-interaction:

  • Set filtering. GPT-5.6 only rejects candidate sets and never adds products.
  • Query generation. Each query passes deterministic leakage checks and an LLM query–set consistency check (Section 3.1). We now also verify the queries with human annotators and test robustness to the generator model (below).
  • Judging. The LLM judge is a secondary signal. Our main metric, SetHit@K, is computed by exact matching against logged items and uses no LLM at evaluation time; the leaderboard and all RL results are reported in SetHit@K.

Independent human verification. Annotators assessed 50 randomly sampled held-out H&M query–set pairs (Table R2). A majority of annotators judged that the set satisfies the request for 48/50 queries and that the products form one coherent bundle for 49/50. No query was judged by a majority to leak a reference item (3.7% of individual judgments), which is consistent with the leakage checks applied after generation. Realism is rated lower (70.7% "Yes", 24.8% "Partly"); we attribute most "Partly" answers to the queries being deliberately multi-facet, which makes them longer than typical atomic search queries.

Table R2. Human verification of queries and reference sets.

QuestionYesPartlyNoQueries with majority "Yes"
Does the set satisfy the request?86.2%13.0%0.8%48/50
Do the products make sense together as one bundle?85.4%12.2%2.4%49/50
Does the request read like a real shopper's request?70.7%24.8%4.5%46/50
Does the request leak any of the items? (lower is better)3.7%0.0%96.3%0/50

Robustness to the generator model. To test whether our conclusions depend on GPT-5.6, we rebuilt the Musical Instruments benchmark with Gemini-Flash-3.8 replacing GPT-5.6 in every role (set filtering, query generation, and query–set consistency checks). On the 3,189 mined candidate sets, the two filters (GPT-5.6 and Gemini-Flash-3.8) agree on 77.9% of decisions (Cohen's κ = 0.57). Gemini is stricter (1,446 vs. 1,957 sets kept), and 93.3% of the sets it keeps are also kept by GPT-5.6. All queries were regenerated by Gemini. When we re-evaluate the four open-weight agents with the same harness, the leaderboard ranking is unchanged (Gemma 4 > Qwen3.8 > Nemotron 3.5 > GLM-4.7), and thinking improves every model, as in the original benchmark (Table R3).

Table R3. SetHit@K on the Gemini-built Musical Instruments benchmark (1,446 held-out queries).

ModelThinking offThinking onGain
Gemma 4 31B IT13.74%14.78%+1.04 pp
Qwen3.8 27B-FP811.17%13.52%+2.35 pp
Nemotron 3.5 Lightning 30B-A3B8.03%10.91%+2.88 pp
GLM-4.7 Flash7.92%9.06%+1.13 pp

Revision. The revised paper separates the three roles of GPT-5.6 in Section 3, adds the human study and the Gemini-built benchmark as new appendices, and references them in the paper.

AuthorsOpenReview ↗

Part III (Final)

W3. Learned reward and cross-domain transfer

We agree that the original single-run evidence is not enough. We now report three training seeds on two backbones, evaluated on the full held-out set (agent_hidden).

Table R4. In-domain held-out SetHit@K (%), mean ± std over three seeds. Base = untrained model.

BackboneDomainBase policyDirect (SetHit@K) rewardLearned reward
Qwen3-4BMusical Instruments6.2010.77 ± 0.468.76 ± 0.38
Qwen3-4BH&M0.931.08 ± 0.171.87 ± 0.17
Gemma-4-E4B-itMusical Instruments1.218.92 ± 0.758.08 ± 0.39
Gemma-4-E4B-itH&M0.291.81 ± 0.112.27 ± 0.10

RL improves mean SetHit@K over the base policy in all eight settings. Which reward works better depends on the domain, and the pattern is the same for both backbones: the learned reward is better on H&M, and the direct reward is better on Musical Instruments.

Why might the learned reward help more on H&M? GRPO learns from reward differences within a rollout group. When every report in a group is valid but recovers no reference item, the direct reward is constant and gives no learning signal, whereas the dense learned reward can still rank the reports. This matters more when recovery is sparse, and base SetHit@K is 4–7× lower on H&M than on Musical Instruments. Consistent with this, the learned reward raises H&M discovery recall (the share of reference items surfaced by any tool call during the trajectory) on both backbones, while direct training barely changes it.

Cross-domain transfer. The original comparison came from a single Qwen3-4B run on 406 H&M queries. We added three-seed Gemma-4-E4B-it experiments in both directions, evaluated on the full held-out set (agent_hidden) of the target domain (Table R5). We ran this study on Gemma-4-E4B-it because all of its in-domain checkpoints were available; due to storage constraints, not all Qwen3-4B checkpoints were retained after their in-domain evaluation.

Table R5. Cross-domain transfer of Gemma-4-E4B-it, mean over three seeds (± std for SetHit@K). Base-policy SetHit@K is 0.28–0.29% on H&M and 1.18–1.23% on Musical Instruments.

Train → eval domainTraining rewardTransferred SetHit@K (%)In-domain SetHit@K (%), ref.Valid reports, base → transferredDiscovery recall, base → transferred
Musical Instruments → H&MDirect1.85 ± 0.131.81 ± 0.1118.84% → 99.98%3.19% → 3.40%
Musical Instruments → H&MLearned1.89 ± 0.072.27 ± 0.1019.27% → 99.93%3.24% → 3.66%
H&M → Musical InstrumentsDirect7.27 ± 0.218.92 ± 0.7517.23% → 99.95%12.32% → 12.13%
H&M → Musical InstrumentsLearned8.05 ± 0.038.08 ± 0.3917.59% → 98.11%12.36% → 15.40%

Report validity transfers almost fully in both directions (17–19% → 98–100%). Discovery recall improves more under the learned reward than under the direct reward, mirroring the in-domain pattern. Overall, transferred policies recover at least 78% of the in-domain SetHit@K gain over the base policy in all four settings.

Full results. The revised RL table (Section 4.5) also reports how RL changes the valid-report rate, discovery recall, and the share of discovered reference items kept in the final report; Appendix L gives the same metrics for cross-domain transfer.

Revision. We revise Sections 1, 4.5, and 5 to state the domain-dependent result, replace the single-run RL and transfer results with the three-seed results above, and report validity, discovery, and retention in the main RL table and Appendix L.

We are very grateful for the reviewer's time. We hope these additions address the reviewer's concerns, and we would be grateful if the reviewer would consider updating their score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Short summary of our rebuttal

Dear Reviewers and Area Chair,

Thank you for your careful and constructive reviews. You raised very interesting questions, and thus we tried to answer them comprehensively. As the discussion deadline is approaching, here is a short summary of how the revision addresses the main concerns:

1. Does SetHit@K reflect bundle quality? Yes. Human annotators preferred the positive-SetHit@K bundle in 81.0% of judgments.

2. Are the LLM-built sets and queries reliable? Yes. By majority vote of human annotators, 48/50 sets satisfy the request, 49/50 are coherent bundles, and 0/50 queries leak an item. Rebuilding the benchmark with a different LLM family (Gemini-Flash-3.8 instead of GPT-5.6) leaves the leaderboard ranking unchanged.

3. Is the RL evidence more than a single run? Yes, now. Across 3 seeds and 2 backbones, RL improves mean SetHit@K over the base policy in all 8 settings, and cross-domain transfer recovers at least 78% of the in-domain gain. The reward comparison is domain-dependent, with a likely cause: the direct SetHit@K reward gives very little signal when recovery is sparse, as it is on H&M, whereas the learned reward stays dense and therefore works better there.

4. Is the gain LLM reasoning or just better retrieval? Both. Of the LLM agent's gain over the non-LLM baseline, 35-50% comes from better retrieval (the agent's tool calls find better candidates) and 50-65% from better selection on the same candidate pool.

If any point remains unresolved, please let us know which one and we will gladly address it before the discussion closes. If your concerns are resolved, we would be grateful if you would consider re-evaluating our work.

Thank you again for your time,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.