- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes Agentic Generative Retrieval (AGR), which integrates external feature acquisition directly into hierarchical Semantic ID decoding. Instead of injecting all auxiliary features beforehand, AGR allows the model to emit tool tokens during decoding, retrieve the corresponding observations, and use them to guide subsequent SID generation. Tool-augmented trajectories are constructed based on feature applicability and discriminability along the SID hierarchy, followed by a two-stage training procedure.
The method is evaluated on one large industrial dataset and three public benchmarks, with additional ablations, serving-cost analysis, and an online A/B test. AGR consistently improves retrieval performance over the evaluated baselines and shows a favorable accuracy–cost trade-off compared with directly encoding long user histories.
Critical strengths
-
I find the formulation of feature access as an action inside SID decoding interesting. AGR provides a clean way to jointly model whether a feature should be acquired, which feature to use, and where it should affect hierarchical generation.
-
The empirical evaluation is fairly strong. The paper includes four datasets, recent GR baselines, several useful controlled ablations, serving-cost comparisons, and an online A/B test. In particular, the comparisons against static injection, query-guided gating, and fixed-position tool injection help isolate the benefit of adaptive feature interaction.
Critical weaknesses
- My main concern is the gap between how the tool policy is supervised and how it is used at inference time. The training trajectories are constructed using target-side agreement and golden SID prefixes: a tool is labeled as useful when its observation agrees with the positive target, and its insertion position is selected along the golden path. At inference time, however, the model must make these decisions from its own generated prefixes without access to the target item.
I do not view this as data leakage, but it makes the central claim somewhat weaker than the current framing suggests. AGR is effectively trained to imitate an oracle-constructed tool-use policy, and the paper does not quantify how well this policy transfers once decoding leaves the golden trajectory. Since deciding "when" and "where" to acquire evidence is the central contribution, I think this train–inference mismatch deserves more direct validation.
- I also find the novelty framing somewhat broader than the actual contribution. The Introduction motivates AGR largely against static feature injection, but the paper's own Related Work acknowledges that recent methods such as GLASS and DualGR already perform request- or state-dependent evidence selection. AGR's more specific novelty, as I understand it, is to make heterogeneous feature acquisition an explicit decoding-time action and to jointly learn tool identity and placement.
This is still useful, but it is a narrower claim. Moreover, the gains over the strongest recent baselines are considerably smaller than the headline improvements over TIGER, and on some metrics are quite modest. I therefore find the significance for ICLR less clear than the overall presentation suggests.
Questions for authors
-
The tool-augmented trajectories are constructed using target-side agreement and golden SID prefixes. Do the authors have any existing analysis showing how tool-selection and tool-placement behavior changes under model-generated prefixes, especially after an early SID error? This would materially affect my confidence that the learned policy generalizes beyond oracle-generated training trajectories.
-
Could the authors clarify the intended novelty claim relative to recent methods such as GLASS and DualGR? My reading is that these works already perform request-/state-dependent evidence selection, while AGR's main distinction is to formulate heterogeneous feature acquisition itself as a decoding-time action with adaptive placement. Is this the intended scope of the contribution?
-
The headline improvement of up to 14.04% is measured against TIGER, while the margins over stronger recent baselines are much smaller on several datasets. How do the authors view the practical and methodological significance of AGR relative to the strongest recent GR baselines rather than the basic TIGER baseline?
9 comments
2 W2, W3, Q2, and Q3. Novelty and gains over strong baselines
2.1 Core contribution
AGR introduces an agentic generative retrieval architecture for industrial search that allows the generative model to acquire heterogeneous features during decoding. Its original contribution is to formulate feature acquisition as an explicit decoding-time action and jointly optimize the tool-call policy and SID generation through post-training. Based on the current request and decoding state, the model decides whether to call a tool, which tool to call, and where in the generated sequence to call it.
-
Distinction from prior work. GLASS [1] and DualGR [2] focus on modeling historical behavior in generative recommendation. GLASS studies how long-term user interests can improve SID generation. DualGR studies how to coordinate short- and long-term interests and reduce noise from historical behaviors during fine-grained decoding. AGR extends the model’s decision scope beyond user history to the practical problem of injecting heterogeneous features at scale. It addresses when and where different features should be injected.
-
Architectural advantages. AGR changes the role of feature acquisition by letting the model act on information needs that arise during generation. In conventional approaches, predefined encoding and integration pipelines determine what information is available to the model at each stage. AGR lets the model decide how to acquire information. The model can issue a tool call based on what it has already generated and use the resulting information to continue generation. By retrieving features on demand, AGR provides a resource-efficient way to scale feature integration. This agentic design has the following advantages.
-
Extensibility. As discussed in the appendix of the new manuscript (lines 1240–1251), GLASS requires architectural changes to incorporate historical behavior features, which limits its ability to scale to a broad range of heterogeneous features. AGR wraps heterogeneous features in a common tool interface, making feature integration model-agnostic. AGR can therefore learn a tool-call policy through post-training while retaining the existing generative model architecture and incorporating more feature types through a common interface.
-
Adaptive tool use. Prior methods typically rely on more complex mechanisms to select features. In AGR, the generative model itself adaptively decides whether to call a tool, which tool to call, and where to call it. The entire process is optimized end to end.
-
A new paradigm for feature integration. AGR brings feature acquisition into the next-token prediction task itself. Tool calls and SID tokens are generated within the same autoregressive process, and the information returned by a tool conditions subsequent predictions. This formulation opens a broader research direction for generative retrieval.
-
2.2 Gains over the strongest baselines
As requested by the reviewer, we computed the relative improvements over the strongest baselines and updated Table 2 in the revised manuscript with these results.
2.2.1 Offline retrieval gains
Compared with the strongest baselines, AGR achieves relative improvements of up to 3.00% in recall and 2.89% in MRR. These are substantial gains in an industrial search system.
| Dataset | Recall@64 | Recall@50 | Recall@32 | MRR@64 | MRR@50 | MRR@32 |
|---|---|---|---|---|---|---|
| Industry | 0.3292 (+2.31%) | 0.3169 (+3.00%) | 0.2990 (+2.68%) | 0.2639 (+2.35%) | 0.2635 (+2.43%) | 0.2632 (+2.32%) |
| Qilin | 0.3467 (+2.42%) | 0.3267 (+1.74%) | 0.2874 (+1.74%) | 0.3071 (+0.16%) | 0.3068 (+0.03%) | 0.3061 (+0.13%) |
| JDSearch | 0.5663 (+0.12%) | 0.5440 (+1.49%) | 0.5079 (+1.65%) | 0.3399 (+1.00%) | 0.3335 (+2.85%) | 0.3326 (+2.89%) |
| KuaiSAR | 0.4655 (+2.79%) | 0.4483 (+2.21%) | 0.4293 (+0.98%) | 0.3447 (+1.74%) | 0.3312 (+1.69%) | 0.3288 (+1.82%) |
2.2.2 Online gains and practical significance
AGR’s improvements in online metrics are statistically significant at the 95% confidence level (Table 7 of the new manuscript). AGR has been deployed in an industrial search system operating at a scale of hundreds of millions. From a research perspective, these results show that formulating heterogeneous feature acquisition as a learnable decoding action can improve real-world search performance. From an industrial and business perspective, the online gains and full deployment together demonstrate the feasibility and practical value of the approach in a large-scale production environment.
3 References
[1] Cao S, She J, Liu J, et al. GLASS: A Generative Recommender for Long-sequence Modeling via SID-Tier and Semantic Search[J]. arXiv preprint arXiv:2602.05663, 2026.
[2] Yi Z, Feng K, Ma X, et al. DualGR: Generative Retrieval with Long and Short-Term Interests Modeling[C]//Proceedings of the ACM Web Conference 2026. 2026: 8329-8332.
Thanks for the update.
- After a SID1 miss, which tools does AGR call, and at which levels? Could you clarify the 69.08% in Table 15? I get 67.88% from the counts in the reply.
- For SID1 misses, could you compare relevance with and without the chosen tool, starting from the same request and generated prefix?
Response to Reviewer 4wSF
We thank the reviewer for the careful review and constructive suggestions. Our responses follow. For ease of reference, manuscript revisions will be highlighted in RED.
1 W1 and Q1. Tool-policy generalization at inference
1.1 Tool use under generated prefixes
AGR’s training trajectories are constructed using target-side agreement and golden SID prefixes. To analyze tool-call behavior when the model leaves the annotated path at inference time, we evaluated 10,000 test requests with a beam size of 50. We label a generated SID prefix a Prefix Hit if it exactly matches the target prefix and a Prefix Miss otherwise. We examine the tool-call rate (TCR) and the proportion of tool calls at each placement position (PR). Please refer to Table 15 and line 1149 of the new manuscript. \mathrm{TCR}_i=\frac{\\#\text{ beams with a tool call at depth }i}{\\#\text{ beams at depth }i}, \qquad \mathrm{PR}_i=\frac{\\#\text{ beams with a tool call at depth }i}{\\#\text{ beams with a tool call}}.
The results show how the model behaves after an SID prefix error.
Lower call rates after prefix errors. At the same SID depth, TCR is lower for the Prefix Miss group than for the Prefix Hit group. During training, we construct tool-call trajectories along the golden SID paths for positive examples that require tool calls. At inference time, once a branch has deviated from the prefixes of all annotated positives for a request, it can no longer retrieve those positives. However, clicks and consumption behavior are partly stochastic, so observed positives cover only some relevant items. These branches may therefore still lead to relevant results, and tool information may still be useful. We interpret the lower TCR as more conservative tool use. The model still makes tool calls after leaving the positive paths, but it is less likely to introduce additional evidence that could add noise.
Earlier placement after prefix errors. With the RQ-Kmeans-based SID construction studied in this paper, earlier SID levels capture coarser semantics and later levels capture finer-grained semantics. When the generated prefix contains an error, the model is less confident about calling a tool, and tool placement shifts toward earlier, coarser SID levels because the prediction has already gone off course at an early step.
1.2 Relevance analysis and case studies
We score videos retrieved under incorrect SID prefixes using a downstream relevance model. The table below shows a higher mean score with tool calls overall, with higher scores for GSU and Location.
We further illustrate two anonymized cases selected from the top 10 candidates by prediction score. Their predicted prefixes differ from the ground-truth labels. Neither video in either pair was clicked, yet both remain relevant to the query: