acceptodds
Under review as a conference paper at ICLR 2027

It Takes Two to Match: Co-Evolving Generative Retriever with Reinforcement Learning

10 views

Abstract

Recent work increasingly trains large language models (LLMs) for retrieval. Yet existing methods predominantly optimize generation solely on the query side, leaving item representations fixed. This one-sided adaptation restricts mutual alignment and caps overall retrieval performance. We introduce , a co-evolving reinforcement learning framework that jointly trains both query-side and item-side LLM generators. Each generator produces a compact set of discrete keywords for retrieval, preserving full compatibility with existing keyword-based infrastructure. first employs supervised fine-tuning to establish a warm-started, aligned keyword space. It then applies co-evolving reinforcement learning, alternately optimizing each generator against the opposite side’s frozen index. Both sides optimize the same query-to-item objective: the query side receives directly, while the item side receives a counterfactual marginal reward designed to isolate each item's impact on retrieval performance. Evaluated against 10 representative sparse, dense, and generative retrieval baselines, achieves the strongest overall retrieval performance on both an internal APP Marketplace dataset and the public WANDS benchmark, improving over the strongest baseline by and , respectively. Further analysis demonstrates stable co-evolution dynamics and the emergence of more specific, balanced keyword representations.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Reinforcement Learning, Information Retrieval, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_aLpTOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes CoGR, a framework that jointly trains query-side and item-side keyword generators (LLMs) through alternating RL. The key idea of this paper is to let both sides adapt to each other, using retrieval F1 to train the query generator and rewarding the item generator when its changes (to the item keywords) improve retrieval performance.

Critical strengths
  • I think the motivation is great - jointly optimizing query and item representations is a natural and promising direction, and the proposed approach is simple and clean.

  • The reward design for the item-side RL is good.

  • The results look strong, with clear improvements over strong baselines. The ablations are also great.

Critical weaknesses
  • The evaluation is somehow limited - it only tests two datasets, and WANDS has only 50 validation queries. More datasets can improve this paper.

  • The method performs well on full-set F1, but its advantages on top-k ranking metrics are less consistent. This part worth a discussion.

  • Training two generators through multi-round RL is expensive. A more detailed cost analysis and compute-matched comparison are good to add.

Questions for authors

see weaknesses.

3 comments

We sincerely thank you for your positive assessment of our work and recommendation for acceptance. We greatly appreciate your encouraging feedback and constructive suggestions.

  • W1 (Additional Benchmark)

​ We are adding experiments on additional public benchmarks and hope to share the results soon.

  • W2 (Top-k Metrics)

    We acknowledge that our current top-k metrics are relatively less consistent. This is primarily because we use BM25 over generated keywords for ranking, which limits the ranking performance. To better illustrate this, we conduct two additional experiments: (1) using a dense retrieval baseline (ANCE-Qwen3-4B) to rerank the candidates retrieved by CoGR, and (2) applying Reciprocal Rank Fusion (RRF) to combine the ranking results of CoGR and ANCE-Qwen3-4B. Both approaches substantially improve the top-k metrics.

    MethodMRR@100NDCG@100P@100R@100F1@100
    ANCE-Qwen4B0.75720.48790.56910.08180.1312
    CoGR 4B0.7651 ± 0.00260.4931 ± 0.00080.5844 ± 0.00080.0843 ± 0.00020.1348 ± 0.0002
    CoGR 4B + ANCE rerank0.7845 ± 0.00120.5129 ± 0.00040.6008 ± 0.00060.0872 ± 0.00010.1391 ± 0.0002
    CoGR 4B + RRF0.7992 ± 0.00180.5319 ± 0.00030.6153 ± 0.00040.0899 ± 0.00010.1433 ± 0.0001
  • W3 (Computation Cost)

    We acknowledge that training two generators through multi-round RL incurs substantial computational costs. Our primary focus is on improving retrieval effectiveness rather than minimizing training compute. However, since training is performed infrequently in practice, its cost can be amortized over a large volume of retrieval requests in industrial applications. Below, we provide a detailed breakdown of the training cost.

    Training cost. Training is conducted on a node with 8 B200 GPUs. SFT takes 0.13 hours, while each round of query-side and item-side RL takes 3.95 and 4.20 hours, respectively. The five rounds of RL optimization take approximately 40.8 hours in total.

    We also provide the indexing, inference, and retrieval costs below to offer a more comprehensive view.

    Indexing cost. Indexing the item corpus of 39.6K items takes 3.3 minutes on one B200 GPU. Since item keyword generation can be performed independently, this process is naturally parallelizable and can be accelerated with parallel sampling.

    Inference cost. CoGR employs autoregressive generation, with an average query-side inference latency of 128 ms. In industrial applications, frequently occurring queries can be cached, reducing the need for online inference. For new queries, techniques such as speculative decoding or replacing the generator with a diffusion language model could reduce latency.

    Retrieval cost. CoGR benefit from reverted index thus achieves low retrieval latency and a compact index compared with the dense baseline as summarized below.

    MetricCoGRQwen3-4B Emb
    Index Storage2.3Mb193Mb
    Retrieval Latency0.25 ms0.95 ms
AuthorsOpenReview ↗

Results on new dataset

Thank you again for your positive assessment of our work. Your support means a lot to us.

As promised in our previous response, we are pleased to share that we have now completed additional experiments on NFCorpus, a publicly available retrieval benchmark, including comparisons with CoAugRetriever and ANCE-Qwen4B. The results are summarized below:

MethodMRR@100NDCG@100F1@100PRF1
CoAugRetriever-4B0.57060.33110.10550.20760.22210.1470
ANCE-4B0.60100.47700.17900.30900.32300.2300
CoGR-4B0.57100.41200.24500.32800.33700.2460

CoGR achieves the highest F1@100 and overall precision, recall, and F1. These results extend our evaluation to a third benchmark and provide further evidence of CoGR's effectiveness beyond the original two datasets.

We are extremely grateful for your positive score and your supportive feedback on our work. While we hesitate to ask for more, if these additional results further strengthen your confidence, we would be truly grateful if you would consider raising your score.

Thanks for your response. I will keep my score.

Reviewer_vBhEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  1. Generative retrieval typically trains only a query-side generator to match queries against item identifiers. CoGR trains an item-side generator as well to adapt the item identifiers / keywords.
  2. For item identifier generation, CoGR uses a counterfactual marginal reward based on F1 scores in order to incentivize the generation of keyword sets for items that increase the chances of those items being retrieved for relevant queries and being skipped for irrelevant queries.
  3. CoGR demonstrates that the combination of training objectives selected for query and item identifier generation yields a stable iterative optimization algorithm that can adapt not only the query identifier generation but also the item identifier generation and improve overall performance.
Critical strengths
  1. CoGR is a general purpose algorithm that could be easily applicable to a wide variety of metrics beyond F1, and its approach to calibrate item-side rewards seems to be quite stable.
  2. CoGR is not fully tied to the initial set of keywords linked to a query or item, as it will evolve over the course of optimization. In other generative retrieval papers, the choice of document identifiers is a crucial design choice that can outstrip most algorithmic innovations.
  3. Well written and easily understandable paper with good presentation.
Critical weaknesses
  1. CoGR mentions generative retrieval baselines like GLEN and NOVO which learn lexical identifiers, and more recently there are papers such as CoAugRetriever (query–document co-augmentation with RL [Liu et al]) and ETEGRec (SIGIR 2025, recommendation) which learn to adapt item representations over the course of training. Comparisons with at least one such generative retrieval paper can help strengthen the case for the co-evolving identifiers.
  2. This paper uses an internal evaluation benchmark and WANDS, which offer two useful points of comparison. However, WANDS has only 50 samples in the validation set. And given that most generative retrieval papers choose to evaluate on MSMARCO or NQ, which are two large scale passage retrieval benchmarks, including one of those would give me more reassurance about the strength and reproducibility of the compared baselines. (Although both of these technically have one ground truth passage, a slightly weaker definition of relevance will yield many relevant passages.) If NQ or MSMARCO are inappropriate, Amazon ESCI is a good alternative as it has many relevant items.
Questions for authors
  1. While CoGR is not fully tied to the initial choice of keywords, if poorly chosen final performance may be limited. How crucial is the initial prompt to generate item keywords?
  2. It seems like a burden to train the model to generate keywords in one particular order, when the order doesn’t matter for final performance. The SFT stage would include one single ordering I imagine, even if the RL stage can allow for multiple orderings. Also, it seems to be potentially useful to allow the query-side keyword generator to be able to generate a variable number of keywords. Can the keyword budget be adjusted at inference time to to increase recall (without retraining)?
2 comments

We sincerely thank the reviewer for recognizing the generality of our framework, the merits of our item-side reward design, and the clarity of our presentation. We address your concerns point by point below.

  • W1 (Additional baseline)

We have added CoAugRetriever as a baseline, as it is more closely related to CoGR, and discussed the key methodological differences and advantages of our approach (see our response to Reviewer 6H3A). The other work adopts semantic ID based rather than LLM-based approach, making it more appropriate for the related work discussion alongside GLEN and NOVO in the appendix. Nevertheless, we will include it in the related work section and would appreciate any suggestions for additional relevant references we may have missed.

We listed the performance of CoAugRetriever on both datasets for your reference:

DatasetMethodMRR@10NDCG@10F1@10PRF1
InternalCoAugRetriever 4B0.65550.36280.01580.19670.22570.1854
InternalCoGR 4B0.76430.41980.02020.39760.45690.3963
WANDSCoAugRetriever 4B0.80650.69150.09440.49290.52380.4169
WANDSCoGR 4B0.92090.75370.11620.79400.68850.6819
  • W2 (Additional benchmark)

We agree that evaluation on an additional public benchmark would strengthen our empirical evidence. However, we would like to note that Amazon ESCI has only about 4 relevant items per query, making it a sparsely labeled setting. We are currently exploring other suitable benchmarks, and hope to have additional results later.

  • Q1 (Initial keyword)

We agree that the quality of the initial item keywords can affect final performance. However, CoGR is relatively robust to simple changes in prompt as long as it is generating in the right direction.

We evaluated two additional initialization settings: (1) Rephrased, which rephrases the original prompt while preserving its intent, and (2) Bad, which deliberately initializes items with unrelated keywords. The table below reports final retrieval performance after CoGR training.

InitializationPRF1
Original0.3970.4560.396
Rephrased0.3840.4490.384
Bad0.1480.4110.202

The rephrased prompt yields performance comparable to the original, while initialization with unrelated keywords substantially degrades final performance. These results suggest that meaningful initial item keywords are important, while the simple prompt rephrasing tested here has little effect.

  • Q2 (Ordering of keyword)

We thank the reviewer for this question. Since LLMs are autoregressive models, generating keywords sequentially is a natural choice. We use a single ordering SFT for simplicity. While SFT with multiple permutations is possible, it could introduce multiple targets for the same prompt, potentially increasing complexity without clear benefits. Moreover, the goal of SFT is to provide a reasonable initialization for RL with sufficient query-side recall (thus positive reward), and SFT with a single ordering is sufficient for this purpose.

  • Q3 (Adjustable Keyword budget)

We thank the reviewer for this suggestion. Our framework supports this flexibility without retraining. A simple approach is to sample multiple times (or beam search) from the same query and take their union. To better illustrate this, we conduct further experiment with the results as follows and it can be seen.

Sampling TimesPRF1
1 (Orginal)0.39760.45690.3963
40.38740.47230.3977
80.38380.47800.3981
AuthorsOpenReview ↗

The prompts used in response to Q1

Rephrased

App title: {appname}

App description: {description}

Write the search terms that best describe this app for keyword-based retrieval. Return up to 10 keywords, comma-separated, with no other text.

Bad:

App title: {appname}

App description: {description}

List keywords that are IRRELEVANT to this app: terms a user looking for this app would never search for, drawn from unrelated topics. Do not reuse any word from the app information above. Output at most 10 keywords as a comma-separated list, no explanation.

Reviewer_ndMQOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+2)
Read review
Paper summary

This paper proposes a retrieval framework that trains two separate LLM-based keyword generators (one for queries and another for items), in such a way that both sides adapt to each other rather than solely optimizing the query side. After a supervised warm-start, the two generators are alternately optimized with reinforcement learning under a shared retrieval objective, each against the other side's frozen index. Experiments on an internal app-marketplace dataset and WANDS benchmark show F1 improvement over sparse, dense, and generative retrieval baselines.

Critical strengths
  1. The motivation that most LLM-based retrieval methods focus only on the query side is well-targeted.
  2. The reward design for the item side is interesting. Defining the reward as the marginal change in aggregate quey-side F1 when a single item's keywords are swapped is a sensible way to attribute credit to item representations.
  3. The additional analyses together provide a coherent picture of the proposed method.
Critical weaknesses
  1. One of the two benchmarks is internal and cannot be examined or reproduced, and WANDS is very small, with only 50 validation queries. No variance, confidence intervals, or multi-seed results are reported, which is expecially concerning given these two facts. In addition, while the motivation of jointly optimizing the query and item side is general, the evaluation is confined to a specific regime. So it is unclear how the method behaves on other standard benchmarks

  2. The authors emphasize the notion of co-evolution over multiple rounds as the key element of the method, but Figure 3 shows that most of the gain comes from the first round, with later rounds contributing marginally. Additional experments such as ablation on the number of rounds or a comparison with a single round trained longer are needed to show more clearly about the benefit from the iterations.

  3. The paper is motivated by industrial needs, but it does not show whether the method would practically scale to the industry-level item universe. A more comprehensive report on training cose, re-indexing cost or inference latency etc is required. The paper should also discuss how new items are handled between item-side updates since they would be represented by a model that is no longer aligned with the current query side.

Questions for authors

See weaknesses

5 comments

We sincerely thank the reviewer for recognizing the merits of our item-side reward design and the additional analyses. We hope our responses can address your concerns.

  • W1 (Significance of results)

To address the concern about statistical variability, we additionally report two complementary measures: query-bootstrap std, which quantifies uncertainty due to the finite evaluation query set, and inference-seed std, which measures variability across different LLM inference seeds.

DatasetF1 $\pm$ Query StdF1 $\pm$ Seed Std
Internal0.3961 $\pm$ 0.00550.3961 $\pm$ 0.0005
WANDS0.5736 $\pm$ 0.04160.5736 $\pm$ 0.0012

The results show that inference variability is minimal. WANDS exhibits larger query-level uncertainty due to its smaller size but the advantage of CoGR is still significant.

We are also adding experiments on additional public benchmark and hope to share the results soon.

  • W2(Longer single round)

We agree that the largest gain occurs in the first round. We would like to kindly point out that even a single involves optimizing both sides, demonstrating the key benefit of CoGR over one-sided methods. Moreover, subsequent rounds provide 5 additional points, which is non trivial and distinguishes CoGR from competing baselines.

To separate the benefit of iterative rounds from longer training, we follow your advice and conduct a single round experiment that extends training epoch of both sides by 5×, matching the total training budget of five-round CoGR. As shown below, this variant is worse, demonstrating the importance of repeated adaptation between the two sides.

MethodPRF1
Original0.3970.4560.396
One round0.3560.4510.362
  • W3 (Computation cost)

We provide a detailed breakdown of the training, indexing, inference, and retrieval costs.

Training cost. Training is conducted on a node with 8*B200s. SFT takes 0.13 hours, while each round of query-side and item-side RL takes 3.95 and 4.20 hours, respectively. The five rounds of RL optimization takes ~40.8 hours. While the training cost is substantial, training is performed infrequently in practice. Therefore, this cost can be amortized over a large volume of retrieval requests in industrial applications.

Indexing cost. Indexing the item corpus of 39.6K items takes 3.3 minutes on one B200 GPU. Since item keyword generation can be performed independently, this process is naturally parallelizable and can be accelerated with parallel sampling.

Inference cost. CoGR employs autoregressive generation, with an average query-side inference latency of 128 ms. In industrial applications, frequently occurring queries can be cached, reducing the need for online inference. For new queries, techniques such as speculative decoding or replacing the generator with a diffusion language model could reduce latency.

Retrieval cost. CoGR benefit from reverted index thus achieves low retrieval latency and a compact index compared with the dense baseline as summarized below.

MetricCoGRQwen3-4B Emb
Index Storage2.3Mb193Mb
Retrieval Latency0.25 ms0.95 ms
  • W3 (Alignment between Item updates)

We would like to clarify that CoGR's alternating optimization actually maintains alignment between query and item generators.

At round t, query-side RL optimizes $G_t^Q$ against frozen index produced by $G_{t-1}^I$. Thus, the updated query generator $G_{t+1}^Q$ aligns with $G_{t-1}^I$. During the subsequent item-side RL, item-side model is initialized from the checkpoint of $G_{t-1}^I$, so it remain aligned with index generated by $G_{t+1}^Q$. Therefore there is no misalignment between item-side updates.

Thank you for the efforts to address my questions. However, I still have some concerns that are not fully addressed.

First, regarding the statistical analysis, I could not find the reported WANDS F1 score of 0.5736 in the main paper. Could you clarify where this number comes from and provide more details about the experimental setup? Also, the query-bootstrap standard deviation on WANDS seems relatively large, especially considering the performance differences between CoGR and the baselines. To support the claim of statistical significance, I believe the analysis should directly compare CoGR against the strongest baselines. In addition, variance across different training seeds would be important as inference-seed variance alone does not capture training randomness.

Secondly, regarding computational scalability, while the additional analysis is helpful, it remains unclear how well the method would scale to a substantially larger item universe in practice. In particular, generating keywords for all items would need a large amount of computation. It would be helpful to provide more details on this cost and how it scales with the number of items.

Lastly, regarding the alignment between item updates, my question was more about the deployment setting rather than the alternating training procedure itself. Specifically, how would new items be incorporated into an existing index, and how well would the item-side generator generalize to them?

Thank you for your prompt response. We hope you are having a pleasant weekend! We have provided additional responses and hope they can address your concerns.

  1. The reported score of 0.5736 was a copy-paste error. The correct mean F1 across nine decoding seeds is 0.6792, whereas the paper reports a single-run F1 of 0.6819. We apologize for the confusion and clarify how the reported statistics are computed:

    • Inference-seed standard deviation: For each evaluation query, we generate keywords using 9 different decoding seeds and compute the macro-averaged F1 score for each seed. We report the standard deviation of these 9 F1 scores.
    • Query-bootstrap standard deviation: We first average the F1 scores across the 9 decoding seeds for each evaluation query. We then perform 10,000 bootstrap resamples, each drawing the same number of queries as the original evaluation set, with replacement, and recomputing the mean F1 score. We report the standard deviation of these bootstrap estimates.

    For WANDS, we re-calculated the 95% query bootstrap CI of both CoGR-4B and the strongest baseline ANCE-Qwen4B. These intervals do not overlap, providing strong evidence that CoGR-4B outperforms ANCE-Qwen4B with respect to query-sampling uncertainty.

    CoGR-4BANCE-Qwen4B
    95% CI of F10.6792 [0.5977, 0.7607]0.5012 [0.4353, 0.5655]

    We agree that multi-training-seed results would provide a more comprehensive assessment. However, given the training cost of CoGR and limited time and resources, we have to prioritize experiments on additional benchmarks. We will try to provide the results if time permits.

  2. Regarding your second point, the indexing cost scales approximately linearly with the number of items. Based on our measured runtime, indexing 1 million items would require approximately 83 minutes on a single B200 GPU (1.39 GPU-hours). Since indexing can be parallelized across GPUs, this translates to roughly 10 minutes of wall-clock time on 8 B200 GPUs.

  3. When a new item arrives, we can directly use the trained item-side generator to generate its keywords and update the keyword index accordingly, without retraining. Our results also suggest that the item-side generator generalizes well to unseen items.

    Specifically, in our response to Reviewer 6H3A, we collected 230 new apps and annotated their relevance to the evaluation queries. We then evaluated CoGR checkpoints on these unseen apps by replacing the original app universe (39K apps) with the 230 new apps. As shown in the table, retrieval performance on these unseen apps improves alongside performance on the original item set throughout training, suggesting that the learned keyword generation capability generalizes well to unseen items.

    F1 Before RLF1 After RL
    Held-out Query & Seen Apps0.2090.305 (+0.096)
    Held-out Query & New Apps0.2620.396 (+0.135)

    Moreover, the generated keywords for unseen items are semantically meaningful. For example, for the unseen app "I Have Never: group games adult," the item-side generator produces keywords "truth or dare, dare game, party game, …" after RL. This allows the app to be successfully retrieved by its relevant evaluation query "whos most likely to" through the shared keyword "party game".

Reviewer_6H3AOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper proposes CoGR, a framework that trains separate LLMs to generate retrieval keywords for both queries and items, motivated by the observation that most existing LLM-based retrieval methods mainly optimize the query side. The two generators are warm-started with SFT and then alternately optimized with RL against each other’s frozen index, with a counterfactual marginal reward on the item side. On an internal APP Marketplace dataset and the public WANDS benchmark, the authors report F1 improvements of 10.9% and 36.1% over the strongest baseline. Overall, the paper is clearly written, and the idea of co-adapting both sides is well motivated and of practical relevance.

Critical strengths
  1. The method design has merits. Representing both queries and items as discrete keywords keeps the method compatible with existing inverted-index infrastructure. The counterfactual marginal reward is a simple way to assign credit to individual items, and Appendix C.3 shows how it can be computed incrementally.

  2. The full model outperforms its frozen-item-side variant on both datasets (F1 0.396 vs. 0.262 and 0.682 vs. 0.466), and the keyword analysis shows a shift from unigrams to multi-word keywords over training.

Critical weaknesses
  1. The co-augmentation for LLM retriever has been studied in [1], while is not cited and discussed in the paper. It is unclear about the additional contribution compared with it.

[1] https://arxiv.org/abs/2506.18670

  1. The experimental justification can be improved:
  • 2.1. Limited evaluation of the item side. The data are split only by query, so the item generator is trained and evaluated on the same items (Section 3.1). Since item-side training is the main difference from query-side-only methods, it remains unclear whether its gains transfer to unseen items. Moreover, because the item-side reward (Eq. 3) only reflects training queries, items irrelevant to all training queries may be pushed toward not being retrieved, which could also affect performance on unseen queries.

  • 2.2. Unclear source of the F1 gains. CoGR determines how many items to retrieve for each query through keyword overlap, whereas for the ranked baselines the authors “select the retrieval cutoff that maximizes training F1 and apply it unchanged” to the validation set (Section 4.1), so every query receives the same number of items. As a result, the F1 comparison in Table 2 reflects both the quality of the retrieved items and the choice of set size. When all methods retrieve the same number of items, the gain becomes much smaller: in the F1@100 column of Table 2, CoGR is close to ANCE-Qwen4B on both datasets (0.1349 vs. 0.1312 on Internal and 0.4778 vs. 0.4717 on WANDS). The main results may therefore overstate the benefit of the co-evolved keyword representations, which is the central claim of the paper.

  1. Unverified reliability of the Internal labels. The Internal dataset relies on relevance labels produced by an internal LLM judge (Appendix B), and these labels serve as the ground truth for both training and evaluation. However, the paper does not report whether the judge’s labels were checked against human annotation or validated in any other way.
Questions for authors
  1. Please justify the contribution beyond the existing work [1] as pointed out in Weakness 1.

  2. How was the reliability of the LLM-judged labels verified? Could the authors report the agreement between the LLM judge and human annotators on a sample of query–item pairs?

  3. How does CoGR perform on items held out from item-side training? What fraction of items have no relevant training queries, and how are these items retrieved after training?

  4. Could the authors compare CoGR’s full retrieved set with each baseline’s ranked list truncated to the same number of items for each validation query? Unlike F1@100, this comparison does not depend on CoGR’s BM25 ranking, and it would show how much of the gain comes from the keyword representations rather than the adaptive set size.

5 comments

First we sincerely thank the reviewer for recognizing the method design of our method, and especially for the thoughtful and constructive suggestions. These suggestions motivated several valuable new experiments that strengthen our paper.

  • Q1 (Compare with CoAugRetriever)

We thank the reviewer for pointing out CoAugRetriever. While both approaches explore co-augmentation for retrieval, our method offers two key contributions: better alignment with the retrieval objective and more stable training dynamics.

  1. Training logic. CoAugRetriever optimizes query and document augmentation using DPR-style mini-batches with in-batch negatives. As acknowledged by the authors, this approach is not suitable for densely labeled retrieval settings, which are common in industrial applications. Moreover, jointly updating both sides introduces a moving-target challenge that may affect training stability, whereas CoGR alternates optimization with one side frozen.

  2. Reward design. Due to its mini-batch training setup, CoAugRetriever uses mini-batch NDCG@10 as the reward, which is a biased proxy for full-corpus NDCG@10 even in expectation. EB[NDCG@10(q;aq,aB)]≠NDCG@10(q;aq,aI)\mathbb{E}_{\mathcal B}[\mathrm{NDCG@10}(q;a_q,a_{\mathcal B})] \neq \mathrm{NDCG@10}(q;a_q,a_{\mathcal I})

    In contrast, our COGR uses F1 computed over the entire item universe as the query-side reward and a counterfactual reward as item-side reward. Both rewards are mathematically the same as full-corpus F1 optimization objective.

Moreover, we compare COGR empirically with CoAugRetriever and the results are as follows.

DatasetMethodMRR@10NDCG@10F1@10PRF1
InternalCoAugRetriever 4B0.65550.36280.01580.19670.22570.1854
InternalCoGR 4B0.76430.41980.02020.39760.45690.3963
WANDSCoAugRetriever 4B0.80650.69150.09440.49290.52380.4169
WANDSCoGR 4B0.92090.75370.11620.79400.68850.6819
  • Q2 (LLM-judged labels)

Thank you for raising this important question. The LLM judge was developed and evaluated by another internal team. According to the results shared with us, it achieved approximately 90% agreement with annotators on a sample of a few hundred query–item pairs. While further details on the evaluation and judge model cannot be disclosed due to confidentiality constraints, we do hope these additional information provide supporting evidence for the reliability of the LLM-generated relevance labels.

  • Q3 (Held out Item)

We acknowledge that items without positive training queries may be pushed away. However, this is a general challenge in retrieval rather than an issue unique to CoGR. Importantly, CoGR encourages items with positive training queries to generate keywords from a shared vocabulary. This learned capability can generalize to unseen items and those without positive training queries, reducing their likelihood of generating unretrievable keywords.

For Items with no positive training queries:

In the internal dataset, 815 out of 39,595 apps have no relevant training queries. Many of their generated keywords remain semantically meaningful, allowing them to be retrieved by relevant evaluation queries. Specifically, 23 of these apps have relevant queries in the validation set, and 9 are successfully retrieved.

For example:

The app Alcohol-free tracker has no positive training query. After RL, it gives keyword “ sobriety, sobriety streak … ” and is successfully retrieved by the relevant query “alcohol tracker” in evaluation set via keyword "sobriety"

For unseen items:

We collected and annotated an additional 230 unseen apps. As shown in the table below, retrieval performance on these unseen apps improves alongside the original item set throughout training.

F1 Before RLF1 After RL
Held-out Query & Seen Apps0.2090.305 (+0.096)
Held-out Query & Held-out Apps0.2620.396 (+0.135)
  • Q4 (F1 calculation protocol)

Thank you for this constructive suggestion. We conducted additional experiments where ANCE-Qwen4B's ranked list is truncated to match the number of items retrieved by CoGR (Matched). The results are summarized below.

DatasetMethodPRF1
InternalCoGR-4B0.39760.45690.3963
ANCE-Qwen4B (Original)0.37560.43540.3575
ANCE-Qwen4B (Matched)0.38860.44840.3871
WANDSCoGR-4B0.79400.68850.6819
ANCE-Qwen4B (Original)0.61360.60920.5012
ANCE-Qwen4B (Matched)0.72460.59060.5634

We highlight two key observations:

First, we want to kindly point out that this matched-cutoff evaluation favors ANCE-Qwen4B. Specifically, adaptively determining the retrieval set size is itself an important advantage of CoGR, which is not naturally available to ANCE. While useful as a diagnostic comparison, it does not represent a fair standalone evaluation.

Second, CoGR outperforms the strongest baseline even under the matched-cutoff setting. This demonstrates that CoGR's performance gains are attributable not only to its ability to adaptively determine retrieval set sizes, but also to its retrieval quality.

Thanks for the detailed responses. I appreciate the new experimental results. However, the comparison against [1] is not sufficiently convincing due to the limitation in the choice of benchmarks. I share the similar concern with ndMQ that the evaluation under the two benchmarks is not sufficient. To soundly justify the benefit of the proposed approach, evaluations under more retrieval benchmarks are necessary. Due to this reason, my overall score remains unchanged.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Updated Manuscript and Sincere Thanks to All Reviewers

We have include additional experiments and analyses conducted during the rebuttal period into our updated manuscript, with all changes highlighted in red.

We would also like to take this opportunity to sincerely thank all the reviewers once again. We feel truly fortunate to have received such thoughtful and dedicated reviews. Each of you has provided valuable suggestions and actively engaged in the discussions. Through these constructive exchanges, we have worked together to make our paper more rigorous, solid, and compelling. We are genuinely grateful for your time and effort in improving our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.