acceptodds
Under review as a conference paper at ICLR 2027

AGR: Agentic Generative Retrieval with Tool-Augmented Search

Abstract

Large-scale search and recommendation systems derive much of their retrieval quality from rich, heterogeneous features, yet existing generative retrieval (GR) methods struggle to exploit such feature ecosystems efficiently. Static feature injection either encodes all available signals or places them according to predetermined schemes, increasing serving cost and introducing irrelevant or conflicting evidence. More fundamentally, existing methods cannot adaptively determine when a feature is needed, which feature to select, or where it should influence hierarchical Semantic ID (SID) generation. We propose *Agentic Generative Retrieval (AGR)*, a tool-augmented paradigm that interleaves SID generation with on-demand feature acquisition. Conditioned on the request and decoded SID prefix, *AGR* decides whether to invoke a tool, which feature to acquire, and where its observation should influence subsequent decoding, thereby steering generation toward feature-consistent SID branches. To supervise this behavior, we construct tool-augmented trajectories based on feature applicability and information gain along the SID hierarchy, and train the model through SID pretraining followed by tool-augmented fine-tuning. Experiments on an industrial dataset and three public benchmarks, together with an online A/B test, consistently demonstrate the effectiveness of *AGR*.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
generative retrieval information retrieval tool-augmented generation agentic AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4wSFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Agentic Generative Retrieval (AGR), which integrates external feature acquisition directly into hierarchical Semantic ID decoding. Instead of injecting all auxiliary features beforehand, AGR allows the model to emit tool tokens during decoding, retrieve the corresponding observations, and use them to guide subsequent SID generation. Tool-augmented trajectories are constructed based on feature applicability and discriminability along the SID hierarchy, followed by a two-stage training procedure.

The method is evaluated on one large industrial dataset and three public benchmarks, with additional ablations, serving-cost analysis, and an online A/B test. AGR consistently improves retrieval performance over the evaluated baselines and shows a favorable accuracy–cost trade-off compared with directly encoding long user histories.

Critical strengths
  1. I find the formulation of feature access as an action inside SID decoding interesting. AGR provides a clean way to jointly model whether a feature should be acquired, which feature to use, and where it should affect hierarchical generation.

  2. The empirical evaluation is fairly strong. The paper includes four datasets, recent GR baselines, several useful controlled ablations, serving-cost comparisons, and an online A/B test. In particular, the comparisons against static injection, query-guided gating, and fixed-position tool injection help isolate the benefit of adaptive feature interaction.

Critical weaknesses
  1. My main concern is the gap between how the tool policy is supervised and how it is used at inference time. The training trajectories are constructed using target-side agreement and golden SID prefixes: a tool is labeled as useful when its observation agrees with the positive target, and its insertion position is selected along the golden path. At inference time, however, the model must make these decisions from its own generated prefixes without access to the target item.

I do not view this as data leakage, but it makes the central claim somewhat weaker than the current framing suggests. AGR is effectively trained to imitate an oracle-constructed tool-use policy, and the paper does not quantify how well this policy transfers once decoding leaves the golden trajectory. Since deciding "when" and "where" to acquire evidence is the central contribution, I think this train–inference mismatch deserves more direct validation.

  1. I also find the novelty framing somewhat broader than the actual contribution. The Introduction motivates AGR largely against static feature injection, but the paper's own Related Work acknowledges that recent methods such as GLASS and DualGR already perform request- or state-dependent evidence selection. AGR's more specific novelty, as I understand it, is to make heterogeneous feature acquisition an explicit decoding-time action and to jointly learn tool identity and placement.

This is still useful, but it is a narrower claim. Moreover, the gains over the strongest recent baselines are considerably smaller than the headline improvements over TIGER, and on some metrics are quite modest. I therefore find the significance for ICLR less clear than the overall presentation suggests.

Questions for authors
  1. The tool-augmented trajectories are constructed using target-side agreement and golden SID prefixes. Do the authors have any existing analysis showing how tool-selection and tool-placement behavior changes under model-generated prefixes, especially after an early SID error? This would materially affect my confidence that the learned policy generalizes beyond oracle-generated training trajectories.

  2. Could the authors clarify the intended novelty claim relative to recent methods such as GLASS and DualGR? My reading is that these works already perform request-/state-dependent evidence selection, while AGR's main distinction is to formulate heterogeneous feature acquisition itself as a decoding-time action with adaptive placement. Is this the intended scope of the contribution?

  3. The headline improvement of up to 14.04% is measured against TIGER, while the margins over stronger recent baselines are much smaller on several datasets. How do the authors view the practical and methodological significance of AGR relative to the strongest recent GR baselines rather than the basic TIGER baseline?

9 comments
AuthorsOpenReview ↗

Response to Reviewer 4wSF

We thank the reviewer for the careful review and constructive suggestions. Our responses follow. For ease of reference, manuscript revisions will be highlighted in RED\color{red}{\textbf{RED}}.

1 W1 and Q1. Tool-policy generalization at inference

1.1 Tool use under generated prefixes

AGR’s training trajectories are constructed using target-side agreement and golden SID prefixes. To analyze tool-call behavior when the model leaves the annotated path at inference time, we evaluated 10,000 test requests with a beam size of 50. We label a generated SID prefix a Prefix Hit if it exactly matches the target prefix and a Prefix Miss otherwise. We examine the tool-call rate (TCR) and the proportion of tool calls at each placement position (PR). Please refer to Table 15 and line 1149 of the new manuscript. \mathrm{TCR}_i=\frac{\\#\text{ beams with a tool call at depth }i}{\\#\text{ beams at depth }i}, \qquad \mathrm{PR}_i=\frac{\\#\text{ beams with a tool call at depth }i}{\\#\text{ beams with a tool call}}.

Call positionTCR, Prefix HitTCR, Prefix MissPR, Prefix HitPR, Prefix Miss
After SID127.01% (5,469 / 20,249)5.31% (27,433 / 516,985)69.08%78.30%
After SID210.09% (1,638 / 16,237)0.82% (4,042 / 494,937)19.39%11.69%
After SID35.96% (878 / 14,720)0.56% (3,030 / 536,760)10.65%8.71%
After SID40.54% (72 / 13,298)0.09% (461 / 515,913)0.87%1.29%

The results show how the model behaves after an SID prefix error.

  • Lower call rates after prefix errors. At the same SID depth, TCR is lower for the Prefix Miss group than for the Prefix Hit group. During training, we construct tool-call trajectories along the golden SID paths for positive examples that require tool calls. At inference time, once a branch has deviated from the prefixes of all annotated positives for a request, it can no longer retrieve those positives. However, clicks and consumption behavior are partly stochastic, so observed positives cover only some relevant items. These branches may therefore still lead to relevant results, and tool information may still be useful. We interpret the lower TCR as more conservative tool use. The model still makes tool calls after leaving the positive paths, but it is less likely to introduce additional evidence that could add noise.

  • Earlier placement after prefix errors. With the RQ-Kmeans-based SID construction studied in this paper, earlier SID levels capture coarser semantics and later levels capture finer-grained semantics. When the generated prefix contains an error, the model is less confident about calling a tool, and tool placement shifts toward earlier, coarser SID levels because the prediction has already gone off course at an early step.

1.2 Relevance analysis and case studies

We score videos retrieved under incorrect SID prefixes using a downstream relevance model. The table below shows a higher mean score with tool calls overall, with higher scores for GSU and Location.

Toolw/o tool callw/ tool callGSUIntentLocationFreshness
Mean relevance score0.86680.86880.87240.81560.87810.7772

We further illustrate two anonymized cases selected from the top 10 candidates by prediction score. Their predicted prefixes differ from the ground-truth labels. Neither video in either pair was clicked, yet both remain relevant to the query:

  • Case 1: Freshness-aware retrieval. For the beer-braised egg query, “autumn” suggests seasonality. The Freshness tool identifies a high freshness requirement in the user’s content-consumption intent, and the more recently uploaded video receives a higher relevance score.
  • Case 2: Historical behavior-based personalization. The GSU tool retrieves the user's previous clicks on videos about Baby White Fox uploaded by other creators. The candidate about the same work ranks higher and receives a higher relevance score.
QueryComparison itemVideo w/ tool callVideo w/o tool call
How do you make beer-braised eggs in autumn?Video titleMake braised eggs at home with a bottle of beerHomestyle beer-braised egg tutorial
Upload date2025-10-062020-08-21
Rank2nd4th
Relevance score0.92140.6508
Tool signalFreshness tool: high freshness requirement for content consumption—
Transmigrating into a foxVideo titleBaby White Fox: I Was Reborn as a Fox! (comic recap)How Did I Wake Up as a Little Fox Girl? (novel recommendation / comic recap)
Rank8th10th
Relevance score0.99470.5390
Tool signalGSU indicates that the user previously clicked videos about Baby White Fox uploaded by other creators—

2 W2, W3, Q2, and Q3. Novelty and gains over strong baselines

2.1 Core contribution

AGR introduces an agentic generative retrieval architecture for industrial search that allows the generative model to acquire heterogeneous features during decoding. Its original contribution is to formulate feature acquisition as an explicit decoding-time action and jointly optimize the tool-call policy and SID generation through post-training. Based on the current request and decoding state, the model decides whether to call a tool, which tool to call, and where in the generated sequence to call it.

  • Distinction from prior work. GLASS [1] and DualGR [2] focus on modeling historical behavior in generative recommendation. GLASS studies how long-term user interests can improve SID generation. DualGR studies how to coordinate short- and long-term interests and reduce noise from historical behaviors during fine-grained decoding. AGR extends the model’s decision scope beyond user history to the practical problem of injecting heterogeneous features at scale. It addresses when and where different features should be injected.

  • Architectural advantages. AGR changes the role of feature acquisition by letting the model act on information needs that arise during generation. In conventional approaches, predefined encoding and integration pipelines determine what information is available to the model at each stage. AGR lets the model decide how to acquire information. The model can issue a tool call based on what it has already generated and use the resulting information to continue generation. By retrieving features on demand, AGR provides a resource-efficient way to scale feature integration. This agentic design has the following advantages.

    • Extensibility. As discussed in the appendix of the new manuscript (lines 1240–1251), GLASS requires architectural changes to incorporate historical behavior features, which limits its ability to scale to a broad range of heterogeneous features. AGR wraps heterogeneous features in a common tool interface, making feature integration model-agnostic. AGR can therefore learn a tool-call policy through post-training while retaining the existing generative model architecture and incorporating more feature types through a common interface.

    • Adaptive tool use. Prior methods typically rely on more complex mechanisms to select features. In AGR, the generative model itself adaptively decides whether to call a tool, which tool to call, and where to call it. The entire process is optimized end to end.

    • A new paradigm for feature integration. AGR brings feature acquisition into the next-token prediction task itself. Tool calls and SID tokens are generated within the same autoregressive process, and the information returned by a tool conditions subsequent predictions. This formulation opens a broader research direction for generative retrieval.

2.2 Gains over the strongest baselines

As requested by the reviewer, we computed the relative improvements over the strongest baselines and updated Table 2 in the revised manuscript with these results.

2.2.1 Offline retrieval gains

Compared with the strongest baselines, AGR achieves relative improvements of up to 3.00% in recall and 2.89% in MRR. These are substantial gains in an industrial search system.

DatasetRecall@64Recall@50Recall@32MRR@64MRR@50MRR@32
Industry0.3292 (+2.31%)0.3169 (+3.00%)0.2990 (+2.68%)0.2639 (+2.35%)0.2635 (+2.43%)0.2632 (+2.32%)
Qilin0.3467 (+2.42%)0.3267 (+1.74%)0.2874 (+1.74%)0.3071 (+0.16%)0.3068 (+0.03%)0.3061 (+0.13%)
JDSearch0.5663 (+0.12%)0.5440 (+1.49%)0.5079 (+1.65%)0.3399 (+1.00%)0.3335 (+2.85%)0.3326 (+2.89%)
KuaiSAR0.4655 (+2.79%)0.4483 (+2.21%)0.4293 (+0.98%)0.3447 (+1.74%)0.3312 (+1.69%)0.3288 (+1.82%)
2.2.2 Online gains and practical significance

AGR’s improvements in online metrics are statistically significant at the 95% confidence level (Table 7 of the new manuscript). AGR has been deployed in an industrial search system operating at a scale of hundreds of millions. From a research perspective, these results show that formulating heterogeneous feature acquisition as a learnable decoding action can improve real-world search performance. From an industrial and business perspective, the online gains and full deployment together demonstrate the feasibility and practical value of the approach in a large-scale production environment.

3 References

[1] Cao S, She J, Liu J, et al. GLASS: A Generative Recommender for Long-sequence Modeling via SID-Tier and Semantic Search[J]. arXiv preprint arXiv:2602.05663, 2026.

[2] Yi Z, Feng K, Ma X, et al. DualGR: Generative Retrieval with Long and Short-Term Interests Modeling[C]//Proceedings of the ACM Web Conference 2026. 2026: 8329-8332.

Thanks for the update.

  1. After a SID1 miss, which tools does AGR call, and at which levels? Could you clarify the 69.08% in Table 15? I get 67.88% from the counts in the reply.
  2. For SID1 misses, could you compare relevance with and without the chosen tool, starting from the same request and generated prefix?
Reviewer_m2EMOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Agentic Generative Retrieval (AGR). During the Semantic ID (SID) generation, the model can call tools to obtain features such as query intent, location, freshness, or user history. The returned feature is inserted at a selected decoding position to guide subsequent SID generation. The model is trained using rule-constructed tool-use trajectories. Experiments on one industrial dataset, three public datasets, and an online A/B test show retrieval improvements.

Critical strengths
  • The paper addresses a practical problem: injecting all available features can be expensive and introduce irrelevant information.
  • Integrating feature acquisition directly into SID generation is useful and reasonably simple.
  • The evaluation is extensive, including public datasets, ablations, efficiency analysis, and an online A/B test.
Critical weaknesses
  • The presentation is unnecessarily difficult to follow. Excessive terminology, dense notation, and awkward expressions obscure a relatively simple idea.
  • The theoretical results are weak: some are direct consequences of definitions or of dynamic policies containing static policies as a special case.
  • AGR might approximately doubles serving cost relative to the feature-free baseline, despite being presented as efficient.
  • Reproducibility is limited because the main industrial dataset and implementation are unavailable.
Questions for authors
  1. How accurately does AGR decide when and which tool to call? Can the authors report tool-call precision and recall?
  2. How does AGR compare with a prefix-conditioned feature-gating model that accesses exactly the same features but does not generate tool tokens?
  3. Figure 3 appears to show multiple tool calls, while the default tool quota is M=1M=1. Are these calls on the same path or different beams?
4 comments
AuthorsOpenReview ↗

Response to Reviewer m2EM

We thank the reviewer for the careful review and helpful suggestions. For ease of reference, manuscript revisions will be highlighted in RED\color{red}{\textbf{RED}}.

1 Clarity and presentation (W1)

We thank the reviewer for the feedback on readability. AGR’s core idea is to let a generative model acquire the heterogeneous features it needs during decoding and use them to continue generation. The novelty of this design is that it turns feature acquisition, previously specified by the model architecture, into a learnable decoding decision. We would be happy to improve the writing further. If possible, could you point us to one or two places that were particularly difficult to follow, such as a term, an equation, or a paragraph? We would appreciate the opportunity to discuss those points and clarify any misunderstandings.

2 Scope of the theoretical results (W2)

We thank the reviewer for raising these theoretical concerns. This work proposes a new agentic architecture for generative retrieval in industrial search that lets the model actively acquire heterogeneous features during decoding. The primary area of this paper is “applications to computer vision, audio,\color{red}{\textbf{applications to computer vision, audio,}} language, and other modalities.\color{red}{\textbf{language, and other modalities.}}” Related generative retrieval work, including TIGER [3], GEMs [4], GLASS [1], and Unisearch [5] , likewise focuses on applications. We acknowledge that this work focuses on real-world applications rather than extensive theoretical analysis, and we accept the reviewer’s suggestion. In the new manuscript, we restate Theorem 2 as a corollary (line 1016). The ablation experiments in Section 5.3 (Table 3) of the new manuscript evaluate how much a dynamic policy improves performance in a practical retrieval system.

3 Retrieval quality and serving efficiency (W3)

We compare the serving resources required by two ways of improving retrieval performance through feature injection: extending the input feature sequence and using AGR to acquire features dynamically. Please refer to Table 4 and lines 426–429 of the new manuscript.

  • Longer input sequences. Gradually extending the sequence to 400 tokens raises Recall@50 to 0.3241. The GPU serving cost per thousand requests rises from 2.38 for the feature-free baseline to 8.33. Training time per step rises from 2.413 seconds for the 50-token configuration to 10.794 seconds.

  • On-demand feature acquisition. AGR uses a tool to retrieve relevant behaviors on demand from a sequence of 800 candidate tokens, achieving Recall@50 close to that of the 400-token configuration. The GPU serving cost per thousand requests is 4.76, a reduction of 42.9% relative to the 400-token configuration. With the same training batch size and configuration, training takes 1.969 seconds per step, a reduction of 81.8%.

AGR thus offers a less resource-intensive way to improve retrieval quality. The system can make fuller use of more and longer feature sequences while reducing the online GPU resources and training time required to achieve comparable retrieval quality. The former affects ongoing serving costs, and the latter affects the cost of model iteration.

4 Reproducibility (W4)

We thank the reviewer for raising reproducibility. The industrial dataset and implementation involve user interaction information and require a user privacy review before public release. We plan to release the industrial dataset and core implementation as soon as possible after the necessary review. To facilitate reproduction, we provide

  • We already have reported experiments against multiple baselines on three public datasets: Qilin, JDSearch, and KuaiSAR.
  • The paper also details the full training data construction procedure, supervised fine-tuning (SFT), the loss function, the evaluation setup, and the parameter settings (Section 5.1 and Appendix A).
  • The revised manuscript also provides further details on the engineering implementation of online deployment (please refer to new manuscript, lines 928–932).

Given the privacy considerations associated with the data and software services, we kindly ask for more time. We will release the core implementation details as soon as the review is complete and approval has been granted.

5 Tool-selection and call-timing accuracy (Q1)

To address the reviewer’s question, we evaluated two aspects of AGR’s tool-call decisions on the same set of 10,000 industrial search requests: which tool to call and when to call it. With a beam size of 50, we use the top-ranked beam to measure precision and all 50 retrieved beams to compute recall. The results show that AGR learns tool selection and call timing during decoding. Please refer to Table 16 and line 1176 of the new manuscript.

Match criterionTop-1 PrecisionRecall@50
Which tool to call61.79%84.99%
When to call69.63%86.66%

6 Prefix-guided gating (Q2)

We thank the reviewer for suggesting this comparison. We added Prefix-guided gating, a prefix-conditioned feature-gating baseline that accesses exactly the same tool features as AGR and does not generate tool tokens. It uses the attention module from Query-guided gating, takes the generated SID prefix as input, and fills positions that have yet to be generated with <pad>. At prespecified SID decoding positions, it uses the attention scores to weight and fuse the tool features. Please refer to line 388 and Table 3 of the new manuscript.

StrategiesRecall@50
(1) Static feature injection0.3159
(2) Query-guided gating0.3161
(3) Prefix-guided gating0.3173
AGR (ours)0.3264

Conditioning the gate on the decoding state (the SID prefix) improves Recall@50 from 0.3161 to 0.3173, but prefix-guided gating still underperforms AGR, which achieves 0.3264. One possible explanation is the difference in supervision. The gating baseline learns feature weights through the recommendation objective, whereas AGR receives explicit supervision on tool actions. Providing comparable action-level supervision for gating would require explicit gating targets and an auxiliary loss.

The two approaches also differ in how they incorporate new feature sources. In our gating baseline, adding a feature source requires updating the gating and fusion modules that consume its features. AGR can incorporate new feature sources through its shared tool-action interface during post-training, keeping the model architecture used for inference unchanged.

7 Q3. Tool-call quota and Figure 3

The two tool calls in Figure 3 occur on the same beam path. This example uses M=2M=2. AGR supports both calls to multiple tools and multiple rounds of tool invocation. We set M=1M=1 in the main offline and online experiments to meet the online inference resource budget and maintain a consistent experimental configuration. Beams that share a prefix inherit the number of calls made along that prefix and maintain separate counts after they branch. Once a beam reaches MM, subsequent tool-call scores are set to −∞-\infty, keeping the number of decoding steps for each path between LL and L+ML+M.

8 References

[1] Cao S, She J, Liu J, et al. GLASS: A Generative Recommender for Long-sequence Modeling via SID-Tier and Semantic Search[J]. arXiv preprint arXiv:2602.05663, 2026.

[2] Yi Z, Feng K, Ma X, et al. DualGR: Generative Retrieval with Long and Short-Term Interests Modeling[C]//Proceedings of the ACM Web Conference 2026. 2026: 8329-8332.

[3] Rajput S, Mehta N, Singh A, et al. Recommender systems with generative retrieval[J]. Advances in Neural Information Processing Systems, 2023, 36: 10299-10315.

[4] Zhou Y, Guo C, Cai K, et al. Gems: Breaking the long-sequence barrier in generative recommendation with a multi-stream decoder[J]. arXiv preprint arXiv:2602.13631, 2026.

[5] Chen J, Jiang X, Wang Z, et al. Unisearch: Rethinking search system with a unified generative architecture[J]. arXiv preprint arXiv:2509.06887, 2025.

I thank the authors for their response. However, I remain concerned about reproducibility, although the authors note that legal approval is currently in progress. Therefore, I maintain my original score.

AuthorsOpenReview ↗

Response to m2EM's remaining concern

Dear Reviewer,

Thank you for acknowledging the issues we have resolved and for sharing your remaining concern about reproducibility.

We have provided detailed methodology, experimental settings, and results on three public datasets to support reproduction. For a commercial platform serving billions of users, however, releasing core code and industrial data during the review period presents a significant challenge, as approval must account for user privacy and other reasons.

We have made every effort to advance this process internally and respectfully ask for more time, or the opportunity to provide further details addressing your specific concerns. We remain committed to releasing the core code and de-identified industrial data as soon as the necessary reviews and approvals are complete.

Best regards, The Authors

Reviewer_t1oSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces AGR, which integrates feature acquisition into trie-constrained semantic identifier (SID) decoding. Tool actions compete with SID tokens, and tool outputs are compressed into an observation token or embedding that conditions subsequent predictions. Training combines SID pretraining with tool-augmented fine-tuning using trajectories constructed from target-attribute agreement, history voting, and frequency changes along the target SID path. Experiments cover an industrial dataset, Qilin, JDSearch, and KuaiSAR, alongside production testing and ablations. Industry Recall@50 improves from the strongest reported baseline’s 0.3169 to 0.3264, approximately 3.00% relatively. The evidence supports practical gains, while their precise mechanism and full cost remain less established.

Critical strengths
  1. Meaningful empirical breadth and deployment evidence. Evaluation spans four datasets, with seven days of A/A testing followed by seven days of A/B testing. Table 2 reports paired bootstrap testing, and Table 6 provides online 95% confidence intervals. These are substantive strengths, although they do not establish training-seed stability or fully specify the production comparison.
  2. Useful controls support the contribution of actual observations. Table 3 compares static injection, query-guided gating, fixed-position tools, and AGR. In Table 11, shared observations obtain Recall@50 of 0.2901 versus 0.3264 for full AGR, indicating that observation content matters. Cross-backbone, fusion, and loss-scaling experiments further strengthen the empirical analysis.
Critical weaknesses
  1. The distinctive decision mechanism remains insufficiently isolated, and its placement justification is incomplete. GLASS already uses a generated SID prefix to retrieve relevant history, while learned tool invocation has established precedents. Table 3’s query-guided gate does not receive the same prefix information as AGR. A gate using the same decoder state, tools, observations, and budget is therefore an important missing control. Separately, Equation (9) measures target-side token frequencies, whereas Theorem 3 requires probabilities of actual tool outputs. Correct votes on selected training examples do not establish this equivalence, particularly with the reported training vote-hit ratio of 0.8. The current results support an effective heuristic more clearly than a maximum-gain placement principle.
  2. The acquisition-cost and quality–cost claims require a clearer execution protocol. Algorithm 2 executes tool branches before TopK pruning, and the quota is per beam path rather than per request. Retained-path tool rates therefore do not establish actual service-call counts. Deployment may use caching or deferred execution, but this is not sufficiently specified. Table 4’s GSU-only AGR saves 42.9% of the service GPU metric relative to 400-token input; against 50-token input, however, Recall@50 rises from 0.3195 to 0.3237 while that metric rises from 2.70 to 4.76, about 76.3%. Full AGR cannot automatically inherit this variant’s cost. The handling of multiple tool-call trajectories mapping to the same final SID needs clarification, particularly how their final ranking scores are determined and whether deduplication is followed by replenishment to K unique items.
Questions for authors
  1. Additional evidence needed for current claims: Does AGR retain its advantage over a prefix-conditioned gate using the same decoder state, feature access, observation representation, and call budget, and over a tuned fixed-position strategy?
  2. Additional evidence needed for current claims: Under what assumptions can Equation (9)’s target frequencies replace the actual observation probabilities in Theorem 3? Please either justify those assumptions for erroneous votes or narrow the theorem, and assess whether the frequency proxy ranks positions consistently with measured prediction gains.
  3. Clarification: Does deployment execute tools before or after branch pruning? Please reconcile Algorithm 2 with the implementation, distinguish candidate, executed, and retained calls per request, and state which external tool costs are included. Also clarify duplicate-SID aggregation and whether Table 4 measures the same variant used for the main accuracy claims.
13 comments
AuthorsOpenReview ↗

Response to Reviewer t1oS

We thank the reviewer for the suggestions. Our responses to the specific points follow. For ease of reference, manuscript revisions will be highlighted in RED\color{red}{\textbf{RED}}.

1. A/B test setup and seed stability (S1)

  • Online A/B test setup. We evaluated AGR on a production short-video search platform against a generative retrieval baseline that statically injects 50 GSU tokens and other features into the model input. Users were randomly assigned to control and treatment groups, each representing 10% of platform traffic. Both groups used the baseline during a seven-day A/A phase to check for systematic differences in online metrics. During the following seven-day A/B phase, the treatment group switched to AGR while the control group continued using the baseline. These experimental settings are clarified in the new manuscript, lines 505–511.

  • Training-seed stability. We additionally repeated training with 3 random seeds under the same configuration. The table below reports the mean ± standard deviation for each metric. AGR outperforms the baseline on both Recall@50 and MRR@50.

MetricBaselineAGRImprovement
Recall@500.3153 ± 0.00030.3260 ± 0.0004+0.0107 ± 0.0004
MRR@500.2563 ± 0.00040.2696 ± 0.0003+0.0133 ± 0.0003

2. Novelty, tool decisions, and placement (W1, Q1, and Q2)

2.1 Core contribution (W1)

AGR introduces an agentic generative retrieval architecture for industrial search that allows the generative model to acquire heterogeneous features during decoding. Its original contribution is to formulate feature acquisition as an explicit decoding-time action and jointly optimize the tool-call policy and SID generation through post-training. Based on the current request and decoding state, the model decides whether to call a tool, which tool to call, and where in the generated sequence to call it.

  • Distinction from prior work. GLASS [1] and DualGR [2] focus on modeling historical behavior in generative search and recommendation. GLASS studies how long-term user interests can improve SID generation. DualGR studies how to coordinate short- and long-term interests and reduce noise from historical behaviors during fine-grained decoding. AGR extends the model’s decision scope beyond user history to the practical problem of injecting heterogeneous features at scale. It addresses which features to incorporate and where to place them during SID generation.

  • Architectural advantages. AGR shifts feature acquisition from a predefined pipeline to a model-driven decision made during generation. In conventional approaches, predefined encoding and integration pipelines determine what information is available to the model at each stage. AGR lets the model itself decide how to acquire information. The model can issue a tool call based on what it has already generated and use the resulting information to continue generation. By retrieving features on demand, AGR provides a resource-efficient way to scale feature integration. This agentic design has the following advantages.

    • Extensibility. As discussed in the appendix of the new manuscript (lines 1240–1251), GLASS requires architectural changes to incorporate historical behavior features, which limits its ability to scale to a broad range of heterogeneous features. AGR wraps heterogeneous features in a common tool interface, making feature integration model-agnostic. AGR can therefore learn a tool-call policy through post-training while retaining the existing generative model architecture and incorporating more feature types through a common interface.

    • Adaptive tool use. Prior methods typically rely on more complex mechanisms to select features. \AGR instead makes tool use an integral part of generation: the model itself adaptively decides whether to call a tool, which tool to call, and where to call it. The entire process is optimized end to end.

    • A new paradigm for feature integration. AGR brings feature acquisition into the next-token prediction task itself. Tool calls and SID tokens are generated within the same autoregressive process, and the information returned by a tool conditions subsequent predictions. This formulation opens a broader research direction for generative retrieval.

2.2 Comparison with prefix-guided gating (W1 and Q1)

We thank the reviewer for suggesting a more comprehensive ablation to isolate the effects of the tool decision mechanism and tool placement. We added a tuned fixed-position gating strategy. It uses an attention module with the same architecture as query-guided gating and takes the SID decoding prefix as input, with <pad> tokens in positions that have yet to be generated. It uses attention scores to weight features from the tool types used in the main experiments. Please refer to line 388 and Table 3 of the new manuscript.

StrategiesRecall@50
(1) Static feature injection0.3159
(2) Query-guided gating0.3161
(3) Prefix-guided gating0.3173
AGR (ours)0.3264

Conditioning the gate on the decoding state (the SID prefix) improves Recall@50 from 0.3161 to 0.3173, but prefix-guided gating still underperforms AGR, which achieves 0.3264. One possible explanation is the difference in supervision. The gating baseline learns feature weights through the recommendation objective, whereas AGR receives explicit supervision on tool actions. Providing comparable action-level supervision for gating would require explicit gating targets and an auxiliary loss.

The two approaches also differ in how they incorporate new feature sources. In our gating baseline, adding a feature source requires updating the gating and fusion modules that consume its features. AGR can incorporate new feature sources through its shared tool-action interface during post-training, keeping the model architecture used for inference unchanged.

2.3 Adaptive versus fixed placement (W1 and Q2)

To clarify this point, we first observe that performance with fixed-position tool injection declines from coarse to fine SID levels (SID1 through SID4). Different queries and features may be associated with different SID levels, and rigid placement becomes more detrimental at finer levels. AGR learns where to inject tool information through joint training, which allows it to dynamically associate features with different SID levels for complex queries. Please refer to line 406 and Table 3 of the new manuscript.

StrategiesRecall@50
(4) Fixed-position tool injection after SID10.3174
(5) Fixed-position tool injection after SID20.3165
(6) Fixed-position tool injection after SID30.3159
(7) Fixed-position tool injection after SID40.3156
AGR (ours)0.3264

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.