acceptodds
Under review as a conference paper at ICLR 2027

Improving Retrieval Embeddings on Conjunctive Queries with Partial Credit

Abstract

As users refine search queries, especially in e-commerce, they tend to add requirements as conjunctions, making each query more specific than the last. One limitation in existing training paradigms is the binary training objective; hard negatives that subtly conflict with the query are considered just as incorrect as easy, obviously irrelevant negatives. To leverage the wasted information, we present the Graded Objective for Learning Distinctions, or GOLD mining, a hard negative mining strategy that generates hard negatives with known violations of query conditions. Additionally, we contribute GOLD reweighting, a loss function modification strategy that assigns partial credit to hard negatives that nearly satisfy the query. We demonstrate that both GOLD mining and GOLD reweighting are compatible with four common existing loss function families. Applying both generally improves win rate and achieves state-of-the-art recall. Our theoretical analysis establishes that under reasonable assumptions, GOLD Margin-MSE gains the guarantee that uniformly low loss on the objective implies approximate affine recoverability of attributes.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
embeddings, information retrieval, credit assignment, ecommerce

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_eVSAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper targets dense retrieval for conjunctive queries ("has a, b, c and does not have x, y"). It proposes GOLD mining — synthesising a natural-language query from a (positive, hard-negative) document pair with Gemini 2.5 Flash so that the negative is known to violate exactly d of the query's conditions — and GOLD reweighting, which modifies Margin-MSE, InfoNCE, SigLIP and CoSENT so that hard negatives receive partial credit as a function of d/V. Datasets are built from Amazon ESCI (text, 40k triples) and DeepFashion In-Shop (images, 26k triples), with 133 / 83 human-written test queries. all-mpnet-base-v2 and CLIP ViT-B/32 are fine-tuned for one epoch over three seeds, compared against the unmodified losses and NV-Retriever mining. Appendix A shows that, for Margin-MSE without easy negatives, low GOLD loss implies attributes are approximately affinely recoverable from the embeddings and vice versa, that binary Margin-MSE gives no such guarantee, and gives a capacity construction adapted from Garg et al. (2026). The paper reports that mining plus reweighting improves recall over the baseline for every loss and modality, achieves "state-of-the-art recall," and that mining raises hard-negative win rate while reweighting trades win rate for recall.

Critical strengths
  • A cheap, generic source of graded supervision, applied uniformly across four loss families. Deriving a violation count from LLM-extracted attributes and plugging it into Margin-MSE, InfoNCE, SigLIP and CoSENT is a clean idea, and testing it on both text and image modalities with the same recipe is the right way to show it is loss-agnostic. The construction pipeline, prompts (Appendix E) and annotator instructions (Appendix F) are released.

  • Honest reporting of negative and null results, with paired statistics. The paper states that mining alone reduces text recall (L408–409), that reweighting lowers hard-negative win rate (L418–424), and that the InfoNCE improvement is not significant (L412–414); Table 3 gives per-query paired permutation tests over three seeds for every comparison, and Table 2 reports seed ranges. The theorems in Appendix A are stated carefully. Overall write up is easy to follow.

Critical weaknesses
  • The results do not support the headline claim, and reweighting moves the motivating metric the wrong way.

    • On the strongest loss, InfoNCE, the baseline already matches GOLD: +0.001 recall in both modalities, not significant (Table 2, Table 3). The significant gains (SigLIP, Margin-MSE, CoSENT) bring weaker losses up to roughly InfoNCE's level, and NV-Retriever does the same on text for two of them. "State-of-the-art recall" is not shown; "lifts weaker losses" is. The paper should say why GOLD is needed if plain InfoNCE does as well.
    • Win rate against the paired hard negative is the conjunctive-discrimination metric, and reweighting lowers it (Figure 4, L418–424) while raising Recall@K over mostly easy negatives. That questions the design: partial credit reduces pressure on exactly the near-miss negatives the paper is about. The CoSENT variant drops d entirely (all hard negatives get rank ½, L267–268) yet has the largest text gain (+.044), so the gain there comes from a hard/easy split, not from counting violations. No graded metric (nDCG with a d-based gain, or precision over graded negatives) is reported, though d is available for every test triple.
    • On human image queries, reweighting lowers recall for three of four losses (up to −.060, Table 2); the main text does not mention it, and the synthetic-to-human correlation in Figure 5 is driven by differences between losses, not by the method effects being claimed.
  • The theory is not tested, and the contribution is not positioned against the relevant literature.

    • Appendix A proves that low GOLD loss implies attributes are linearly recoverable by a probe, but no probe is ever trained on the paper's own embeddings before or after GOLD training. Section 4.3 and Appendix A contain statements and proofs only. The theory covers Margin-MSE, while the best empirical loss is InfoNCE, and its condition on V is unrelated to the tuned V = 20.
    • Graded relevance is standard in learning to rank (RankNet/LambdaRank; LambdaLoss, CIKM 2018) and in contrastive learning (RINCE, AAAI 2022); ESCI itself ships four-level labels that the paper discards without running a graded-label baseline. Hard-vs-easy negative treatment is established in retrieval and in recommender systems. The abstract's "binary training objective" framing does not hold against this. It will be helpful to show some other approaches which have used graded judgement for improving retrieval.
    • Conjunctive queries have also been handled through a standard non-embedding route — Boolean retrieval, LLM-formulated Boolean clauses (SIGIR 2023), filtered dense retrieval (WWW 2023). The paper quotes filtering in one sentence ("ontologies are fixed"), yet its own pipeline extracts open-vocabulary attributes per document and composes queries with explicit AND/NOT — the pieces of a parse-then-filter baseline it never runs. By Section 3, what is new beyond attaching a counted label to standard LLM query synthesis and BM25 / same-category negatives is hard to identify.

Overall paper mentions multiple relevant IR and Recommender Systems directions that were developed over time. However, the baselines used, and other aspects fall short of truly standing out. Refer to questions for further discussion.

Questions for authors

Clarification:

  • Why is GOLD needed when the unmodified InfoNCE baseline matches it (Table 2)? Is there any setting — base model, dataset, cutoff — where GOLD-InfoNCE separates from InfoNCE outside seed noise?
  • Please report a graded metric (nDCG with gain derived from d, or precision over graded negatives) alongside Recall@K and win rate, so the trade-off in Figure 4 can be evaluated on the quantity the paper is about.
  • Were linear probes trained on the baseline and GOLD-trained embeddings to test affine recoverability of attributes? If not, can this be added — it is the experiment that connects Appendix A to Section 5.
  • Is the Main test set generated by the GOLD pipeline (queries conditioned on the negative) while baseline training queries are not (L216–217)? If so, how much of the gap is train/test distribution match?
  • What is the retrieval corpus for Recall@5 / Recall@20, and how many unique test products are there (Table 3 notes text test queries share products)? Why @5 for text and @20 for images?
  • CoSENT: why collapse all hard negatives to rank ½ rather than use rank k − d, given the loss only needs an ordering? Does the +.044 text gain survive with graded ranks, and does it survive with a binary hard/easy flag in the other losses (i.e. is d itself doing work)?
  • Eq. 5: the SigLIP target T = e^{−s·d/V} uses the learned scale s. Is s detached in the target? If not, the regression target is non-stationary.
  • L280–282 require V ≥ max(η, K*), but the sweep (L371–373) includes V = 20 with η = 40. Were those cells excluded, or is the constraint not enforced? Please also report sensitivity to V rather than only the selected value.
  • Was the accuracy of Gemini's attribute buckets (A-only / B-only / both / neither) ever checked? d is only correct if those buckets are, and the method rests on d.
  • Human queries (L219–221): how many annotators, how recruited and compensated, what agreement, and how exactly were 133 / 83 queries used? Appendix F gives instructions only.
  • GOLD-reweighted InfoNCE on human image (0.574 [.542–.590]) is identical to NV-Retriever InfoNCE on the same column, including seed range. Is this a copy error?
  • SigLIP is cited as Tschannen et al. 2025 in Section 4 and defined from Zhai et al. 2023 in Appendix C; which was used?

Suggestions:

  • Run a parse-then-filter baseline (LLM clauses over the extracted attributes, optionally reranked by the dense model) and a graded-label baseline on ESCI's native labels (LambdaLoss or RINCE), and reposition the contribution against LTR, Boolean/filtered retrieval, and graded contrastive losses.
  • Test semantic hard negatives for GOLD in addition to BM25 (text currently uses lexical negatives only, L190–192), and grade in-batch "easy" negatives by attribute distance rather than treating all as maximally wrong — the extracted attributes make this cheap.
  • Address how the pipeline extends when labelled positives are unavailable or new documents arrive; text positives are drawn from ESCI's labelled pairs (L188–190), and every query derives from a single seed positive. More importantly, the pipeline is highly dependent on the attributes of positive example, and can face topic/attribute drift based problems.
  • Use a second attribute extractor or a human audit sample for d; use a stronger, current base embedder for at least one configuration.
  • Notation: ℓ is used for elements of S ⊆ D (Eq. 8, L353) while L = Q ∪ D (L322) and Appendix A uses x ∈ S; pick one. Clarify L345: z_i on a query means "requires attribute i," on a document "has attribute i." Define "Main" once and use it consistently (Table 1 says "Synthetic," Table 2 and Figure 5 say "Main").
  • Add the reproducibility and ethics checklist after the AI-use statement, and drop the "to yell at when frustrated" line from that statement (I admit that it mildly amused me).
4 comments

Response to Reviewer eVSA

We thank the reviewer for their thoughtful feedback. It is clear the reviewer is well-read in this field and thought deeply about the paper, and the suggestions will surely improve our research going forward.

Critical Weaknesses:

The results do not support the headline claim, and reweighting moves the motivating metric the wrong way.

On the strongest loss, InfoNCE, the baseline already matches GOLD: +0.001 recall in both modalities, not significant (Table 2, Table 3).

This is a fair weakness. We will update the main framing in the paper to reflect it.

Win rate against the paired hard negative is the conjunctive-discrimination metric, and reweighting lowers it (Figure 4, L418–424) while raising Recall@K over mostly easy negatives.

Can the reviewer clarify the purpose of precision over graded negatives? With only one graded negative per triple, the value of this metric is not clear, and we find defining precision among easy and hard negatives to be ambiguous.

We report nDCG@2 below and will add it to the paper.

LossGroupText nDCG@2Image nDCG@2
–Untrained0.94140.9351
InfoNCEBaseline0.9840 [0.9835–0.9847]0.9655 [0.9650–0.9658]
InfoNCENV-Retriever0.9787 [0.9781–0.9795]0.9658 [0.9648–0.9663]
InfoNCEGOLD mined0.9856 [0.9853–0.9858]0.9671 [0.9661–0.9678]
InfoNCEGOLD reweighted (ours)0.9834 [0.9827–0.9842]0.9657 [0.9643–0.9669]
SigLIPBaseline0.9832 [0.9830–0.9836]0.9655 [0.9650–0.9663]
SigLIPNV-Retriever0.9779 [0.9777–0.9782]0.9631 [0.9626–0.9641]
SigLIPGOLD mined0.9846 [0.9841–0.9850]0.9656 [0.9652–0.9663]
SigLIPGOLD reweighted (ours)0.9825 [0.9821–0.9830]0.9633 [0.9621–0.9643]
Margin-MSEBaseline0.9820 [0.9819–0.9822]0.9660 [0.9651–0.9670]
Margin-MSENV-Retriever0.9788 [0.9787–0.9790]0.9624 [0.9609–0.9633]
Margin-MSEGOLD mined0.9829 [0.9824–0.9832]0.9656 [0.9645–0.9668]
Margin-MSEGOLD reweighted (ours)0.9801 [0.9794–0.9804]0.9629 [0.9627–0.9634]
CoSENTBaseline0.9839 [0.9831–0.9848]0.9638 [0.9628–0.9648]
CoSENTNV-Retriever0.9807 [0.9805–0.9809]0.9594 [0.9573–0.9609]
CoSENTGOLD mined0.9865 [0.9860–0.9872]0.9643 [0.9630–0.9660]
CoSENTGOLD reweighted (ours)0.9835 [0.9830–0.9842]0.9597 [0.9576–0.9620]

We would like to note that CoSENT is unique among loss functions in that it uses ordinal rankings. It is possible that with access to multiple ranked hard negatives, the gain would be even larger, but we did not want to dilute the focus by treating it specially. Being unique in this regard, we emphasize that this behavior may not apply to other loss strategies.

On human image queries, reweighting lowers recall for three of four losses (up to −.060, Table 2); the main text does not mention it, and the synthetic-to-human correlation in Figure 5 is driven by differences between losses, not by the method effects being claimed.

We will update the text to emphasize the human trial results. When correlations are taken within loss type groups then averaged, the text correlation remains high but image does not:

LossText Pearson rText Spearman ρImage Pearson rImage Spearman ρ
InfoNCE0.3060.3100.3540.421
SigLIP0.6250.756−0.547−0.422
Margin-MSE0.9110.844−0.176−0.230
CoSENT0.6370.4840.7930.727
Average across losses0.6200.5980.1060.124
Pooled (Figure 5)0.7290.7210.7870.653

The theory is not tested, and the contribution is not positioned against the relevant literature.

Appendix A proves that low GOLD loss implies attributes are linearly recoverable by a probe, but no probe is ever trained on the paper's own embeddings before or after GOLD training.

We attempted to train these probes but found no significant difference between baseline and ours with respect to linear recoverability. We will remove the proof.

Graded relevance is standard in learning to rank (RankNet/LambdaRank; LambdaLoss, CIKM 2018) and in contrastive learning (RINCE, AAAI 2022); ESCI itself ships four-level labels that the paper discards without running a graded-label baseline.

Numerous works including those leverage ranking to improve retrieval. GOLD differs from the listed works in that GOLD scores can be calculated singularly, i.e. in the absence of a paired example, although we do not do so in this paper. This distinction makes the training independent of the pair selection algorithm, which may be difficult or biased in some contexts.

ESCI labels are tied to specific queries. Having rewritten the queries in our dataset creation, the four-level labels no longer apply, so would not be useful to use. We also decline to compare to them because they are human generated. Moreover, the comparison would show whether BM25/GOLD-mining are superior or inferior to human annotators under a particular regime, which we consider out of scope for our contributions.

Conjunctive queries have also been handled through a standard non-embedding route — Boolean retrieval, LLM-formulated Boolean clauses (SIGIR 2023), filtered dense retrieval (WWW 2023).

The parse-then-filter (we understand this to mean applying an independent embedding lookup for each parsed attribute, then intersecting results) is awkward in production environments because it requires at least one generative model call, followed by arbitrarily many embedding model calls. Compared to a single, unified embedding call, this is slow, expensive, and inconsistent in resource use. Moreover, it cannot be cleanly swapped with a traditional embedding system.

Questions For Authors:

Clarification:

Why is GOLD needed when the unmodified InfoNCE baseline matches it (Table 2)? Is there any setting — base model, dataset, cutoff — where GOLD-InfoNCE separates from InfoNCE outside seed noise?

No.

Please report a graded metric (nDCG with gain derived from d, or precision over graded negatives) alongside Recall@K and win rate, so the trade-off in Figure 4 can be evaluated on the quantity the paper is about.

Please see above.

Were linear probes trained on the baseline and GOLD-trained embeddings to test affine recoverability of attributes? If not, can this be added — it is the experiment that connects Appendix A to Section 5.

After training the probes, we find that the linear recoverability does not increase wth GOLD. We will remove the proof from the paper.

Is the Main test set generated by the GOLD pipeline (queries conditioned on the negative) while baseline training queries are not (L216–217)? If so, how much of the gap is train/test distribution match?

Correct. To address this concern, we rerun the experiment using an evaluation set composed of 50/50 baseline-selected and gold-selected queries. The results do not differ significantly.

New candidate corpora: Text Recall@5

LossGroupGOLD-selectedBaseline-selected50/50 mixed
–Untrained0.6920.6890.689
InfoNCEBaseline0.886 [0.885–0.888]0.874 [0.871–0.876]0.879 [0.877–0.882]
InfoNCENV-Retriever0.883 [0.882–0.884]0.872 [0.870–0.873]0.876 [0.875–0.878]
InfoNCEGOLD mined0.884 [0.882–0.886]0.874 [0.872–0.877]0.876 [0.874–0.879]
InfoNCEGOLD reweighted (ours)0.888 [0.885–0.890]0.877 [0.874–0.879]0.881 [0.878–0.884]
SigLIPBaseline0.847 [0.845–0.849]0.836 [0.834–0.839]0.840 [0.838–0.842]
SigLIPNV-Retriever0.853 [0.852–0.853]0.846 [0.846–0.847]0.847 [0.845–0.849]
SigLIPGOLD mined0.840 [0.835–0.844]0.829 [0.825–0.831]0.834 [0.829–0.839]
SigLIPGOLD reweighted (ours)0.866 [0.865–0.867]0.857 [0.855–0.859]0.860 [0.858–0.862]
Margin-MSEBaseline0.849 [0.847–0.852]0.837 [0.834–0.843]0.841 [0.838–0.845]
Margin-MSENV-Retriever0.868 [0.864–0.870]0.857 [0.856–0.859]0.860 [0.858–0.861]
Margin-MSEGOLD mined0.839 [0.838–0.841]0.829 [0.826–0.830]0.831 [0.829–0.833]
Margin-MSEGOLD reweighted (ours)0.862 [0.861–0.863]0.856 [0.854–0.857]0.857 [0.856–0.859]
CoSENTBaseline0.817 [0.812–0.822]0.811 [0.806–0.814]0.815 [0.812–0.818]
CoSENTNV-Retriever0.863 [0.861–0.865]0.856 [0.855–0.857]0.857 [0.855–0.858]
CoSENTGOLD mined0.809 [0.802–0.814]0.797 [0.786–0.804]0.802 [0.795–0.807]
CoSENTGOLD reweighted (ours)0.861 [0.861–0.861]0.852 [0.849–0.853]0.857 [0.857–0.857]

New candidate corpora: Image Recall@20

LossGroupGOLD-selectedBaseline-selected50/50 mixed
–Untrained0.2460.2610.255
InfoNCEBaseline0.492 [0.480–0.499]0.505 [0.492–0.514]0.501 [0.487–0.511]
InfoNCENV-Retriever0.487 [0.485–0.490]0.501 [0.499–0.503]0.495 [0.493–0.495]
InfoNCEGOLD mined0.493 [0.491–0.498]0.505 [0.503–0.508]0.501 [0.496–0.505]
InfoNCEGOLD reweighted (ours)0.493 [0.492–0.494]0.504 [0.503–0.507]0.499 [0.496–0.501]
SigLIPBaseline0.446 [0.440–0.454]0.461 [0.455–0.472]0.455 [0.450–0.463]
SigLIPNV-Retriever0.441 [0.433–0.451]0.457 [0.451–0.465]0.450 [0.443–0.458]
SigLIPGOLD mined0.448 [0.443–0.453]0.465 [0.462–0.467]0.456 [0.453–0.460]
SigLIPGOLD reweighted (ours)0.465 [0.451–0.473]0.479 [0.466–0.485]0.472 [0.460–0.480]
Margin-MSEBaseline0.431 [0.428–0.436]0.442 [0.438–0.448]0.437 [0.434–0.440]
Margin-MSENV-Retriever0.413 [0.412–0.415]0.425 [0.423–0.427]0.420 [0.418–0.422]
Margin-MSEGOLD mined0.437 [0.432–0.442]0.447 [0.439–0.454]0.442 [0.435–0.448]
Margin-MSEGOLD reweighted (ours)0.446 [0.444–0.448]0.460 [0.456–0.462]0.454 [0.452–0.456]
CoSENTBaseline0.379 [0.375–0.385]0.388 [0.381–0.395]0.384 [0.378–0.394]
CoSENTNV-Retriever0.335 [0.330–0.344]0.347 [0.340–0.356]0.342 [0.335–0.350]
CoSENTGOLD mined0.361 [0.350–0.368]0.375 [0.362–0.382]0.371 [0.359–0.378]
CoSENTGOLD reweighted (ours)0.381 [0.368–0.390]0.398 [0.383–0.406]0.391 [0.375–0.400]

What is the retrieval corpus for Recall@5 / Recall@20, and how many unique test products are there (Table 3 notes text test queries share products)? Why @5 for text and @20 for images?

The original Main evaluation ranks each query against a shared held-out corpus of 3,173 text documents or 2,318 image files.

The corpus is the deduplicated union of all test positives, hard negatives, and random easy negatives. Training and validation documents are excluded. Every model uses the same corpus within a modality.

Text retrieval approaches saturation at larger K, so reducing recall@k to 5 separates high-performing strategies better.

CoSENT: why collapse all hard negatives to rank ½ rather than use rank k − d, given the loss only needs an ordering? Does the +.044 text gain survive with graded ranks, and does it survive with a binary hard/easy flag in the other losses (i.e. is d itself doing work)?

Ordering based on the number of attributes violated slightly decreases recall on both text and images compared to the collapsed 1/2 case. We stress that this result does not necessarily translate to other loss functions.

CoSENT supervisionText Recall@5Image Recall@20
GOLD-mined0.81510.3633
Hard negatives rank ½0.86630.3839
Graded ranks: positive 11, hard 11 − d, easy 00.86080.3781

Eq. 5: the SigLIP target T = e^{−s·d/V} uses the learned scale s. Is s detached in the target? If not, the regression target is non-stationary.

We accidentally used s for two distinct scales. The target uses a fixed α = 20; only the logit scale s is learned. Thus targets remain constant, and no detachment is needed. We will correct Eq. 5.

L280–282 require V ≥ max(η, K), but the sweep (L371–373) includes V = 20 with η = 40. Were those cells excluded, or is the constraint not enforced? Please also report sensitivity to V rather than only the selected value.*

The V = 20, η = 40 cells were included. Our implementation clips normalized distances to [0, 1], so the easy-negative margin is min(η/V, 1). Thus this cell uses a margin of 1 rather than 2. The paper omitted the clipping operation; we will correct the equation and clarify that V ≥ max(η, K*) is sufficient to avoid clipping, rather than an enforced constraint.

The sweep sizes differ because Margin-MSE has two hyperparameters. It uses V ∈ {20, 40, 80} and η ∈ {10, 20, 40}, giving nine configurations per modality, or 18 across text and images. InfoNCE and SigLIP each use V ∈ {10, 20, 40, 80}, giving four configurations per modality, or eight across both modalities. Thus “eight versus nine” compares different scopes; across both modalities, the comparison is eight versus 18. CoSENT uses neither parameter.

The following tables report validation recall from seed 42: Recall@5 for text and Recall@20 for images.

Loss / modalityV = 10V = 20V = 40V = 80
InfoNCE / text.8913.8920.8867.8875
InfoNCE / image.5292.5370.5370.5316
SigLIP / text.8433.8456.8502.8593
SigLIP / image.4778.4817.4887.5027

For Margin-MSE, the full grids are:

Text: ηV = 20V = 40V = 80
10.8707.8730.8259
20.7711.8639.8510
40.7711.7612.8411
Image: ηV = 20V = 40V = 80
10.4856.4856.4832
20.4123.4762.4754
40.4123.3882.4606

InfoNCE varies little across the tested V values. SigLIP improves through V = 80 in both modalities, so its selected value is at the grid boundary. Margin-MSE depends on both V and η: at η = 10, image recall changes little, while text recall falls at V = 80. Clipping makes η = 20 and η = 40 equivalent at V = 20, explaining their identical scores. These single-seed sweeps describe validation sensitivity; they do not establish statistical significance.

Was the accuracy of Gemini's attribute buckets (A-only / B-only / both / neither) ever checked? d is only correct if those buckets are, and the method rests on d.

We annotated 50 examples on each dataset and find that attributes are correct on text 97.3% and on images 98.4%.

Human queries (L219–221): how many annotators, how recruited and compensated, what agreement, and how exactly were 133 / 83 queries used? Appendix F gives instructions only.

We recruited 11 annotators from our university. Annotators were compensated in the form of reciprocal annotation by the authors on unrelated research. The queries are used as the queries in the human datasets.

GOLD-reweighted InfoNCE on human image (0.574 [.542–.590]) is identical to NV-Retriever InfoNCE on the same column, including seed range. Is this a copy error?

It is not. We verified that although summary results are identical, the models differ in which examples are inferred correctly.

SigLIP is cited as Tschannen et al. 2025 in Section 4 and defined from Zhai et al. 2023 in Appendix C; which was used?

We will correct both to use Zhai et al. 2023

Reviewer_EShkOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper targets (product) retrieval with conjunctive queries ("has a, b, c; does not have x, y"). It primarily proposes the following ideas:

  1. GOLD mining: an LLM (Gemini 2.5 Flash) extracts attributes from a (positive, hard negative) pair. A query is composed from them, so the hard negative violates a known number d of the query's conditions.
  2. GOLD reweighting: each loss term is modified to give partial credit to hard negatives according to d. The strategy is tried with Margin-MSE, InfoNCE, SigLIP and CoSENT.

The paper uses/builds text (ESCI) and image (DeepFashion) datasets of synthetic triples + 133 (text) and 83 (image) human-written queries. Results are reported over all-mpnet-base-v2 and CLIP ViT-B/32 encoders under four losses. The proposed method is also compared against binary baselines and NV-Retriever.

-- Updated score post rebuttal.

Critical strengths
  1. Conjunctive and negated queries are a known weakness of bi-encoders, and the paper explains why binary negatives discard graded information.
  2. The proposed modifications applied for various loss families including Margin-MSE, InfoNCE, SigLIP and CoSENT.
  3. The human set tests transfer to non-LLM queries, and the win-rate diagnostic (Fig. 4) separates hard-negative from easy-negative behavior.
  4. The experimental details including hyper-parameters, prompts, dataset details are clearly stated.
Critical weaknesses
  1. Evaluation and the results:

(i) InfoNCE is the best loss in both modalities, and for InfoNCE the gain from GOLD reweighting over the plain baseline is negligible: +0.001 on synthetic text and images. The significant improvements appear only for SigLIP, Margin-MSE and CoSENT, which are weaker starting points, and even after those gains none of them reaches the InfoNCE baseline (e.g., CoSENT text 0.866 vs. 0.890). GOLD mining alone lowers text recall for all four losses.

(ii) Results against NV-Retriever are mixed: GOLD loses on Margin-MSE text (0.865 vs. 0.870) and CoSENT text (0.866 vs. 0.867), and the InfoNCE image comparison is not significant (p = .093).

(iii) Two small backbones (all-mpnet-base-v2 for text and clip-ViT-B-32 for image) and one epoch are used.

(iv) There is no evaluation on an existing conjunctive/negation benchmark (e.g., MultiConIR, and NevIR, ExcluIR, and WANDS).

(v) General retrieval quality after fine-tuning is not reported, so a regression is possible.

(vi) Zero-shot recall is also missing.

  1. The abstract & introduction emphasize discriminating queries whose hard negatives subtly violate conditions. Reweighting explicitly penalizes hard-negative errors less, and the authors show it lowers hard-negative win rate while raising recall. It could be framed better for clarity.
  2. Training, test and the rephrasing all use the same LLM. Queries have about 10 conditions, which is long compared with real e-commerce queries. Please comment on biases.
  3. The theory covers Margin-MSE. However, empirically Margin-MSE is not the best loss, and the gains and the best results come from InfoNCE, SigLIP and CoSENT.
  4. The CoSENT variant ignores d and just assigns rank 1/2 to all hard negatives, yet it shows the largest gain (+0.044 on text). It is worth explaining.
  5. Missing inference details. Please comment on whether it requires any change (query re-writing using Gemini?) to the inference pipeline?
  6. Missing training and inference costs (time benchmarking). The proposed method concerns real-world application where inference time can be an important factor.
Questions for authors
  1. How does GOLD behave with constant partial credit, shuffled d, or a down-weighted binary loss? Do the CoSENT results (no d) show that d is not needed?
  2. Which test set (mined or non-mined) is each model evaluated on, and is the split by product?
  3. What is the retrieval corpus for Recall@5/@20, and what are results restricted to hard negatives or to a corpus with many near-duplicate products?
  4. Why does GOLD reweighting hurt human image recall for three of four losses, and what is the within-loss correlation between synthetic and human results?
  5. How accurate are the LLM-derived d labels on a human-audited sample?
  6. Please consider reporting numbers on existing datasets such as MultiConIR.
  7. Please comment on inference and latency.
3 comments

Response to Reviewer EShk

Critical Weaknesses:

Evaluation and the results:

(i) InfoNCE is the best loss in both modalities, and for InfoNCE the gain from GOLD reweighting over the plain baseline is negligible: +0.001 on synthetic text and images.

The primary focus for our paper is for GOLD reweighting, which requires GOLD mining. The GOLD mining result itself should be viewed as an ablation rather than the primary result. We will adjust the framing throughout the paper to make this distinction more clear.

(ii) Results against NV-Retriever are mixed: GOLD loses on Margin-MSE text (0.865 vs. 0.870) and CoSENT text (0.866 vs. 0.867), and the InfoNCE image comparison is not significant (p = .093).

It is correct that in some cases the results are mixed. However, GOLD achieves the strongest result in both the Image and Text domain and consistently improves across all loss functions in the image domain. Thus we conclude that GOLD is the optimal strategy in many cases, but report full results for completeness.

(iii) Two small backbones (all-mpnet-base-v2 for text and clip-ViT-B-32 for image) and one epoch are used.

We surveyed all papers accepted to ICLR 2026 pertaining to encoder research. Of these, only 36% (32/89) evaluate multiple model families. Even fewer (14.6%, 13/89) studied multiple model sizes within the same family. We are unaware of a theoretical reason why some architectures would struggle more or less with this type of fine-tuning. Therefore, due to resource limitations and the large number of training runs necessary (96), we chose the most popular models in the 50M-500M parameter range.

We train for one epoch because, with an abundance of synthetic data, training converges after one epoch for all models. Further training would likely merely overfit the models at additional cost. Regardless, we are unclear how the existence of additional hyperparameter choices would invalidate the contributions presented.

(iv) There is no evaluation on an existing conjunctive/negation benchmark (e.g., MultiConIR, and NevIR, ExcluIR, and WANDS).

MultiConIR is an evaluation-only dataset, so we cannot evaluate our training method on it without suffering severe distribution shift. The domains are sufficiently different that we do not feel this result would be useful.

WANDS queries are not conjunctive/negation. Nothing in the paper suggests they encouraged or enforced this property during query generation.

NevIR and ExcluIR are derived from Wikipedia. We could not think of a realistic reason why users would seek out Wikipedia pages with extended conjunctive queries. E-commerce products and images can be reasonably well-specified by a list of included and excluded attributes, whereas an article about Avengers: Endgame (ExcluIR Figure 1) cannot be. Therefore we intentionally excluded these datasets in favor of more realistic scenarios.

We thank the reviewer for highlighting this gap in the literature and emphasizing our contribution.

(v) General retrieval quality after fine-tuning is not reported, so a regression is possible.

General retrieval quality after fine-tuning is reported in Figure 1, Figure 4, and Table 2 (Appendix B)

(vi) Zero-shot recall is also missing.

Untrained checkpointMetricMainHuman (matched)
MPNet — textRecall@50.69340.9023
CLIP — imageRecall@200.24670.3735

Fine-tuned recall exceeds zero-shot recall in all cases; regression did not occur.

The abstract & introduction emphasize discriminating queries whose hard negatives subtly violate conditions. Reweighting explicitly penalizes hard-negative errors less, and the authors show it lowers hard-negative win rate while raising recall. It could be framed better for clarity.

We thank the reviewer for this point and agree. We will update the paper accordingly, throughout. Win rate and recall are indeed separate objectives and we feel that some existing work conflates these. We believe our work is among the first that explicitly addresses this tradeoff, at least in the conjunctive query domain.

Training, test and the rephrasing all use the same LLM. Queries have about 10 conditions, which is long compared with real e-commerce queries. Please comment on biases.

We are confused by this comment. Is the reviewer suggesting that we should use different LLMs for different stages of the dataset generation pipeline? We found that Gemini Flash produced qualitatively high quality results for all pipeline stages at reasonable cost. These tasks are not very hard for state-of-the-art generative models, so likely most large models would work. More importantly, we find that results on this particular data synthesis pipeline correlate strongly with results on human queries, validating the approach end-to-end (Figure 5).

The theory covers Margin-MSE. However, empirically Margin-MSE is not the best loss, and the gains and the best results come from InfoNCE, SigLIP and CoSENT.

We chose to prove the theory in Margin-MSE because we found it the most conceptually intuitive. At this time, we are not certain whether a similar result can be proved for InfoNCE and SigLIP. We are confident the result would not hold for CoSENT because it operates on ordinal rankings. However, after training probes, we did not find better linear recoverability of attributes in our model vs. baseline, so we will remove the proof.

The CoSENT variant ignores d and just assigns rank 1/2 to all hard negatives, yet it shows the largest gain (+0.044 on text). It is worth explaining.

CoSENT relies on ordering in its loss function. By creating the rank 1/2, we increase the resolution of the training objective from two ranks to three, which we believe to be quite significant.

Missing inference details. Please comment on whether it requires any change (query re-writing using Gemini?) to the inference pipeline?

Query rewriting does not occur during inference, only during dataset creation (Section 3). Therefore there are no changes to the inference pipeline.

Missing training and inference costs (time benchmarking). The proposed method concerns real-world application where inference time can be an important factor.

The training costs are essentially identical between all models on the same task. The computational change to each loss function is imperceptibly small, and the computational difference between loss functions is also negligible.

The inference cost and speed for all models is exactly identical. Only the weights differ.

Questions For Authors:

How does GOLD behave with constant partial credit, shuffled d, or a down-weighted binary loss? Do the CoSENT results (no d) show that d is not needed?

Surprisingly, yes. We find that Introducing d to produce finer-grained CoSENT ordering reduces recall by roughly 0.6 percentage points.

Which test set (mined or non-mined) is each model evaluated on, and is the split by product?

Models are evaluated on queries from the mined dataset. We will clarify this in the paper. Splits are chosen to ensure that no product nor query appears in multiple sets.

What is the retrieval corpus for Recall@5/@20, and what are results restricted to hard negatives or to a corpus with many near-duplicate products?

For Main Recall@5/@20, the retrieval corpus contains all unique positive and negative documents appearing in the test split. Human-query evaluation uses this corpus plus the human queries’ positive documents. We do not restrict to hard negatives; rather our corpus matches the original dataset test distribution of products, as one would expect in a production-like scenario.

Why does GOLD reweighting hurt human image recall for three of four losses, and what is the within-loss correlation between synthetic and human results?

We do not have a specific explanation for that result except that, of the four datasets, Human Image has the lowest sample size, so the probability of a noisy result is the greatest.

Within-loss correlations are:

ScopeText rText ρImage rImage ρ
All losses0.7290.7210.7870.653
InfoNCE0.3060.3100.3540.421
SigLIP0.6250.756−0.547−0.422
Margin-MSE0.9110.844−0.176−0.230
CoSENT0.6370.4840.7930.727

How accurate are the LLM-derived d labels on a human-audited sample?

We annotated 50 examples on each dataset and find that attributes are correct on text 97.3% and on images 98.4%.

Please consider reporting numbers on existing datasets such as MultiConIR.

MultiConIR does not include a training dataset, nor sufficient raw data to create one. We feel that collecting additional data for the domains in MultiConIR is beyond the scope of this paper. We believe the distribution shift between our dataset and MultiConIR is sufficently different that the out-of-distribution evaluation would not be meaningful.

Please comment on inference and latency.

Inference latency is identical for all models on each dataset.

Reviewer_EShkOpenReview ↗

Official response

Thanks for the detailed response. Some of the queries have been addressed, and I have updated the score accordingly.

  1. The dataset construction involves a model (Gemini Flash). If the same model is used as a part of training a ML pipeline, then there is a chance that some bias may creep in -- it is worth investigating. Thanks for clarifying that the inference pipeline does not involve rewriting.
  2. On InfoNCE and overall accuracy: The theoretical angle is already addressed (proofs with Margin-MSE are fair). In practice, the proposed algorithm should be compared against the strong baseline. Gains over a weaker baseline limits the scope of applying the proposed approach in real world, i.e., eCommerce.
  3. On inference cost: Reporting cost only strengthen the paper -- provides a clear picture to a reader. Fair enough either way.
  4. On architecture and size: It is fair to stick with all-mpnet-base-v2 and clip-ViT-B-32 due to time and other factors. However, results with additional architectures may help in empirically showing generalization ability of the proposed approach. Finally, the behavior of mining approach may change with the size (capacity) of model -- a random negative mining strategy may perform differently as compared to hard negative mining strategy with different model size under a limited training budget & data availability. https://arxiv.org/pdf/2607.10096 touches upon it a bit.
Reviewer_sPBqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies dense retrieval for conjunctive queries containing multiple required and forbidden attributes. It introduces GOLD mining, which constructs queries and hard negatives with known numbers of violated conditions, and GOLD reweighting, which incorporates partial credit into four retrieval objectives. Experiments cover text-based product retrieval and apparel image retrieval, with additional evaluation on human-written queries. The paper also provides a theoretical connection between low GOLD Margin-MSE loss and affine recoverability of attributes under explicit assumptions.

Critical strengths
  1. A practically relevant problem and an intuitive supervision strategy. Conjunctive queries require retrieval models to distinguish between candidates that satisfy different subsets of the requested conditions. The paper makes useful distinctions among negatives explicit by constructing examples with known attribute violations. The combination of targeted data construction and partial-credit supervision is straightforward to understand and potentially useful for product retrieval.

  2. Evaluation across objectives and modalities, supported by theoretical analysis. The authors investigate four loss families in both text and image retrieval, compare mining alone with mining plus reweighting, and report multiple seeds and paired statistical tests. The distinction between hard-negative win rate and retrieval recall provides an informative perspective on the effects of the proposed modifications. The theoretical analysis offers an additional motivation by showing how count-based supervision can preserve affinely recoverable attribute information under stated assumptions.

Critical weaknesses
  1. The benefit of accurate violation counts could be isolated more clearly. The comparison between GOLD mining and GOLD reweighting supports the usefulness of modifying supervision, but leaves open how much improvement requires accurate violation counts rather than a fixed amount of partial credit for hard negatives. This distinction is relevant because the CoSENT variant assigns all hard negatives the same intermediate rank. On the same GOLD training data, comparisons against fixed intermediate targets and shuffled violation counts would help establish the specific contribution of count information. I consider this a limitation of the current mechanistic evidence rather than a fundamental problem with the approach.

  2. The empirical gains and their transfer to human queries require a more qualified interpretation. Improvements over the strongest InfoNCE baseline on the synthetic evaluations are small and are not statistically significant in the reported comparisons. On human-written image queries, Table 2 also shows lower mean recall for the reweighted variants of InfoNCE, SigLIP, and Margin-MSE relative to their respective baselines. The small human evaluation set limits firm conclusions, but the cross-model correlation in Figure 5 does not by itself establish that the gains from a particular training modification transfer to human queries. A direct discussion of these differences, with uncertainty estimates, would clarify the scope of the results. A small manual audit of attribute extraction, violation counts, and query rewriting would also strengthen confidence in the supervision pipeline.

Questions for authors
  1. Contribution of violation counts. Have you compared the measured violation counts with a fixed intermediate target or shuffled counts on the same GOLD training data? Such a comparison would help distinguish the benefit of accurate counts from that of giving hard negatives partial credit more generally.

  2. Generalization to human-written queries. How do you interpret the lower mean recall on human-written image queries for several reweighted objectives in Table 2? Could you provide uncertainty estimates for these differences and discuss whether particular query characteristics help explain the results?

  3. Quality of synthetic supervision. Was a sample of the generated attributes and rewritten queries manually checked? It would be helpful to know how often positives satisfy all conditions, the assigned violation counts are accurate, and rewriting preserves the original constraints.

  4. Connection to the theoretical analysis. Have you evaluated attribute recoverability in the trained embeddings using affine probes? This could help connect the theoretical results to the representations learned in practice.

1 comment

Response to Reviewer sPBq

Critical Weaknesses:

The benefit of accurate violation counts could be isolated more clearly.

We agree that the distinction is important. We tested finer-grained CoSENT ranks based on d and found slightly lower recall than with a fixed intermediate rank. We report the comparison below.

The empirical gains and their transfer to human queries require a more qualified interpretation.

This is a fair weakness. We will update the framing to reflect the small InfoNCE gains and emphasize the human image results. Within-loss correlations remain high on average for text, but not for images.

Questions For Authors:

Contribution of violation counts. Have you compared the measured violation counts with a fixed intermediate target or shuffled counts on the same GOLD training data?

We did not run these experiments.

Generalization to human-written queries. How do you interpret the lower mean recall on human-written image queries for several reweighted objectives in Table 2?

We do not have a specific explanation beyond the somewhat small Human Image sample size.

Quality of synthetic supervision. Was a sample of the generated attributes and rewritten queries manually checked?

We annotated 50 examples on each dataset and find that attributes are correct on text 97.3% and on images 98.4%.

Connection to the theoretical analysis. Have you evaluated attribute recoverability in the trained embeddings using affine probes?

Yes. After training the probes, we find that linear recoverability does not increase with GOLD relative to the baseline. We will remove the proof from the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.