Policy-Integrated Query Encoding for Fixed-Index Agentic RAG: Feasibility and Limits

Dense-retrieval agentic RAG usually runs two neural models at inference time: a policy language model that writes search queries, and a separate dense encoder that embeds every query for the passage index. We ask whether the policy’s own search-token representations can act as the query-side interface to a fixed pas- sage index, so that the external encoder is not needed online. We mean-pool the search-token hidden states, map them into the passage-embedding space with a 6.55M-parameter linear projection, retrieve a top-M candidate set with the pro- jected query, define a stochastic policy over that candidate set, and optimize token and selected-passage log-probabilities jointly under the same final-answer reward. We refer to this architecture as policy-integrated query encoding (PIQE). PIQE removes the external 4B query encoder from the online retrieval path while the corpus index stays fixed. Under a controlled comparison with a matched Search- R1 baseline over three seeds, PIQE reaches 36.11 ± 0.34 macro and 42.36 micro exact match on the frozen reporting partition against 36.77 ± 0.72 and 43.16 for the baseline, and 56.55 against 57.17 micro exact match out of domain. PIQE is within one exact-match point of the baseline on each aggregate, but remains below it on average. Retrieval diagnostics associate the deficit with candidate generation: the hidden-state interface puts an answer-containing passage into the candidate set less often than the text-query interface (Pool@20 45.67 against 47.14), and passage-level credit assignment does not compensate through better selection in- side the candidate set. PIQE is thus architecturally feasible but not lossless, and the results motivate learning signals that improve candidate generation without establishing that better selection could not close the gap.