- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes GDER, a method that refines dense-retrieval query embeddings using signals derived from LLM generation gradients. In an offline stage, it computes the answer cross-entropy loss given top-5 retrieved documents and backpropagates it to the LLM input embeddings, averaging over answer positions. It then scores each query token by the inner product of its LLM input embedding with this aggregated gradient. The top-K tokens are re-encoded with BGE, softmax-weighted into a direction d_grad, and added to the query embedding. The resulting shift e*_q − e_q is kept as a pseudo-label only if re-retrieval reduces the generation loss by at least τ; otherwise the target is zero. A small MLP is trained to predict this offset from the LLM’s final-layer hidden state at the query position, using a cosine loss for positive samples and an L2-norm penalty for zero samples. At inference, the predicted offset’s norm gates whether to re-retrieve. Training data comes from 10k HotpotQA samples with Qwen2.5-7B. The method is evaluated on HotpotQA, 2WikiMultiHopQA, MuSiQue and StrategyQA with three generator LLMs, reporting gains of about 1–3 points over Naive RAG and over prompt-rewriting, LeReT, DeepRetrieval, Dense PRF and HyDE. It also reports retrieval metrics, ablations (random perturbation, no gradient, learned space alignment), and lower latency than autoregressive rewriting methods.
Critical strengths
-
GDER modifies neither the retriever nor the generator, requires no autoregressive rewriting, and adds about 233 ms on average with a small MLP. For deployed RAG systems, an embedding-space correction that is learned once offline and gated per query is an attractive engineering point compared with RL-trained query rewriters.
-
Turning answer-level loss into retrieval-space pseudo-labels, verified by an actual loss reduction after re-retrieval, is a reasonable and cheap way to obtain supervision. The offline cost is modest.
-
The random-perturbation and no-gradient ablations, the gate-threshold sweep, and the triggered/skipped subset breakdown help show where the gains come from. The method also transfers from HotpotQA-only training to other datasets and generator backbones.
Critical weaknesses
-
The paper claims gradients “point toward missing semantic information.” However, d_grad is a weighted combination of BGE embeddings of tokens already in the query, so the method can only re-emphasize existing query terms, not add missing content. In effect, it is gradient-saliency-based term reweighting, a much narrower contribution than the framing suggests, and it is not compared to simple term-weighting baselines.
-
The paper does not state how the gold and document representations are computed, in which space the cosine is measured, or on how many samples. Random documents show higher alignment than top-2 retrieval, which would be expected trivially because random documents are farther from gold. The figure therefore does not establish that gradients encode useful retrieval directions.
-
On HotpotQA, Recall@5 rises from 70.00 to 79.77 overall, yet only about 16% of queries trigger refinement and only about 14% of documents change in Table 8. This implies roughly a 60-point recall gain on the triggered subset. Meanwhile, the QA accuracy gain is only 2.1 points. This needs explanation and verification.
-
Improvements are typically 1–3 points from a single run, with no variance or significance tests. The paper does not compare with retriever fine-tuning from LM feedback (e.g., REPLUG-LSR, Atlas-style training), rerankers, the closely related gradient-based GRO-RAG (Chen et al., 2026), or iterative multi-hop retrieval methods (e.g., IRCoT), even though three of the four benchmarks are multi-hop.
Questions for authors
-
Can GDER ever introduce information absent from the query text? How does it compare with a simple baseline that reweights query tokens, for example by IDF, by LLM attention, or by the best K-token subset found through brute-force search with the same loss check?
-
Please clarify the scoring in Eq. (3): why dot answer-position gradients with query-token input embeddings, and why select the maximum of ⟨E, g⟩ rather than of ⟨E, −g⟩? What happens if the sign is flipped?
-
How was Figure 1(c) computed (spaces, representations, sample size)? What does the same plot look like for a random-direction or random-token baseline?
-
Can you explain the Recall@5 gain of +9.77 on HotpotQA given a 16% trigger rate and 14% document change rate? Please report retrieval metrics separately for the triggered subset.
Response to Authors
Thank authors for the response. I still have a few concerns remaining: