- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PIQE, which replaces an external query encoder with a linear projection of the policy's search-token hidden states. It jointly optimizes token and passage actions against a fixed passage index. A matched Search-R1 comparison shows small average accuracy losses, and retrieval diagnostics examine the remaining gap.
Critical strengths
-
The architectural question is clear, and a 6.55M-parameter projection retains most of the matched baseline's accuracy without updating the passage index.
-
The paper reports three seeds, paired tests, and extensive retrieval diagnostics, while clearly stating the limitations of the prototype.
Critical weaknesses
-
The prototype requires an additional policy forward pass to recover hidden states and demonstrates no actual speedup. Removing a separate encoder therefore does not yet establish a practical efficiency advantage.
-
The baseline uses a 4B query encoder, whereas the original Search-R1 uses the much smaller e5-base-v2. A smaller-encoder comparison is needed to assess the accuracy-resource trade-off.
-
The benefit of joint passage-level RL remains unclear. In Table 4, PIQE does not improve ID macro EM over the frozen-projection control (35.73 vs. 35.76), and the controls also differ in sampling and gradient flow.
-
Experiments cover only one policy family, one embedding space, and one corpus. It is unclear whether the replacement remains effective with different policy-retriever combinations.",
Questions for authors
See the weaknesses above.
Thank you for the careful review and for recognizing the clear architectural question and the multi-seed, paired evaluation. All new results below are from post-submission runs on Cluster B (4× H100), kept separate from the submitted Cluster A results.
1. Extra forward pass and practical efficiency
We agree. We timed one greedy Dev1200 pass (1,200 questions) per system on the same node with identical vLLM and batch settings.
The extra forward costs about 4.7 times the encoding it replaces, and the prototype fetches text for all 20 candidates although only the 5 selected enter the context. Excluding text lookup, a pass takes ≈331 s for Search-R1 and ≈386 s for PIQE. PIQE times use a lossless binary transfer (identical outputs; 1,326 s with the JSON transfer of the submitted runs), and a training step takes 1.46× longer (JSON transfer, as in Table 14). Removing the online encoder saves 7.5 GiB on the GPU hosting it. The current prototype does not reduce latency; reusing hidden states from generation and fetching only the selected passages are implementation improvements that remain untested.
2. Smaller query encoder
We trained Search-R1 with e5-base-v2, as in the original Search-R1, with its own index of the same corpus on all three seeds. On the 11,262-question reporting partition (averaging steps 180/190/200 rather than the submitted selection rule), micro EM is 42.97 ± 0.73 for the 4B baseline, 42.23 ± 0.42 for PIQE, and 41.08 ± 0.83 for e5 (mean ± s.d.). Paired micro EM differences (McNemar):
PIQE has higher micro EM than e5 on all three seeds, significantly on two. Both remove the 4B encoder from the online path: PIQE keeps the index built offline by that encoder, whereas e5 replaces both the encoder and the index. e5 is the fastest and lightest option (≈58 vs. ≈76 GiB peak on every GPU, mainly from its smaller 768-d index) but the least accurate. We will add this trade-off to the paper.
3. Benefit of joint passage-level RL
We agree that Table 4 does not show a benefit and that its controls differ in sampling and gradient flow. As a more direct test, we kept a trained actor fixed and swapped only the projection. At the step-100 checkpoints of seeds 13 and 21, replacing the RL-adapted projection with its warm-started initialization changes micro EM on Dev1200 + reserve (11,136 questions) by +0.31 [−0.15, +0.77] and +0.23 [−0.25, +0.71] (paired bootstrap 95% CI). The projection changes during RL (≈1.0% relative weight change by step 200), but we find no evidence that this improves accuracy. This test isolates projection adaptation under a fixed actor, not the passage objective's effect on actor learning.
We will not claim an accuracy gain from joint passage-level training. Instead, we will emphasize the hidden-state query interface, which queries the fixed external index without an online encoder, together with the measured accuracy–memory trade-off and the remaining latency overhead.
4. One policy family, embedding space, and corpus
We agree that this limits generality, as our Limitations section states. We have not evaluated other policy-retriever pairs or corpora and will keep the claims scoped to the tested configuration.