- Rating
- 3
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper studies behavioral homogenization in agentic RAG trained with RLVR, query-side search-depth homogenization and document-side integration-selectivity homogenization. It proposes Query-and-Document Trajectory Steering (QDTS), which injects a pre-rollout context based on query depth and a mid-rollout context based on retrieval state, optimized before RL with GEPA. Across six QA benchmarks, QDTS reports an average EM gain of 3.7 points over the strongest baseline.
Critical strengths
- The author gives a clear formalization of two homogenization modes and their metrics, depth slope, over-/under-search, and selectivity.
- Prospective context steering is lightweight and does not change the reward or rollout budget.
- The author conducts extensive experiments, including 6 QA benchmarks, multiple backbones, and ablations.
Critical weaknesses
- The theoretical bound assumes independent per-hop retrieval success and does not empirically estimate .
- GEPA context-optimization cost is not reported in the efficiency table. It looks like there is no direct evidence found in the manuscript for its total training overhead.
- Mis-triggered context cost is not quantified, Fig. 5 shows the agent can ignore context, but no aggregate failure rate is given.
Questions for authors
- How is the per-hop success probability in Eqn. 4 & 13 estimated or validated?
- What is the wall-clock and token cost of GEPA context optimization, and is it included in Tab. 3 training time?
- Suggestions for future work: Report context mis-trigger rates and the over-search/under-search trade-off across more retrievers and larger backbones.
4 comments
Response to Reviewer rHKq (2/2)
W3 / Q3. Mis-triggered context cost is not quantified; Figure 5 shows the agent can ignore context, but no aggregate rate is given; report context mis-trigger rates.
We now measure two new quantities, which are the rate of mis-triggered contexts and the aggregate failure rate of the behavior in Figure 5, where the policy ignores a context. We call this behavior a policy override, because the policy receives a context and does not follow it, and Table R20 shows that it does not lower accuracy. The aggregate failure rate that the reviewer asks for is the share of test questions with at least one override. We first quantify the mis-triggers of each stage and then the overrides. The three analyses use the per-question outputs of a single seed.
Mis-triggered pre-rollout context (Table R18). A pre-rollout context () is mis-triggered when the parser assigns the wrong class, which happens for only 13.5% of test questions. QDTS still outperforms GRPO on these questions, so a wrong class reduces the gain but does not turn it into a loss. GRPO, which never sees the class, also drops on this subset, which shows that the parser fails mostly on questions that are difficult in themselves.
Table R18: Mis-triggered : test EM stratified by parser correctness (Qwen2.5-7B-Instruct, E5; average over four multi-hop benchmarks; single seed).
| Subset | Share of test set | GRPO EM | QDTS EM | Δ |
|---|---|---|---|---|
| Parser correct | 86.5% | 37.5 | 44.8 | +7.3 |
| Parser wrong ( mis-triggered) | 13.5% | 26.4 | 30.6 | +4.2 |
| All | 100% | 36.0 | 42.9 | +6.9 |
Mis-triggered mid-rollout context (Table R19). A mid-rollout context () is mis-triggered when it is emitted although the gold passage is already among the retrieved passages. This is rare, and QDTS still outperforms GRPO on the affected questions, because the context describes a state and leaves the decision to the policy.
Table R19: Mis-triggered (QDTS, Qwen2.5-7B-Instruct, E5; average over four multi-hop benchmarks; single seed). EM is computed on the questions that received a mis-triggered context of the given state; GRPO is evaluated on the same questions.
| State | Mis-trigger share of test set | GRPO EM | QDTS EM | Δ |
|---|---|---|---|---|
| Distant Query-Doc | 1.6% | 37.4 | 40.7 | +3.3 |
| Dense Docs | 4.6% | 34.5 | 38.6 | +4.1 |
Policy overrides of mid-rollout contexts (Table R20). An override, which is the behavior shown in Figure 5, happens in only 7.8% of test questions. These questions are relatively easy, and GRPO is also more accurate on them than on the rest. The retrieved passages already suffice there, and the policy answers instead of searching again. An override is therefore not always harmful, and QDTS remains ahead of GRPO on these questions. For , not following the context means falling back to a query-independent depth, which is the homogenization that Table 2 measures.
Table R20: Policy overrides of (Qwen2.5-7B-Instruct, E5; average over four multi-hop benchmarks; single seed). An override is a mid-rollout context that the policy receives and does not follow; GRPO is evaluated on the same questions.
| Subset | Share of test set | GRPO EM | QDTS EM | Δ |
|---|---|---|---|---|
| At least one override of | 7.8% | 40.6 | 42.2 | +1.6 |
| No override | 92.2% | 35.6 | 43.0 | +7.4 |
Suggestion. Report the over-/under-search trade-off across more retrievers and larger backbones.
We thank the reviewer for this valuable suggestion. We will add the query-side metrics (depth slope, over-search, under-search) for the BGE-M3 retriever and the Qwen3-8B and Llama-3.1-8B backbones to Appendix M in the final version, and extend to larger backbones and additional retrievers in future work, where the over-search term in particular deserves study since part of it reflects legitimate re-searches after a missed retrieval rather than homogenization.
Overall, we have incorporated these responses and additional results into the revised paper. Please refer to the uploaded PDF, where the changes are highlighted in red. We hope that most of your concerns have been addressed and would be glad to engage in further discussion if needed.
I thank the authors for their detailed rebuttal, which addresses most review comments. Particularly, the derivations relaxing the independence assumption, empirical success rates for step-by-step retrieval, and quantified GEPA optimization overhead resolve most of my concerns. Consequently, I maintain my positive stance on the paper and have raised my confidence.
We are very glad that our rebuttal strengthened your confidence in your positive evaluation. Thank you again for your positive assessment of our work. Your comments were of great help in improving our work further!
Response to Reviewer rHKq (1/2)
We sincerely appreciate the positive feedback on our work and the suggestions for strengthening the empirical side of the analysis. We hope that our rebuttal addresses all major concerns and clarifies any remaining questions.
We thank the reviewer for this careful observation; it is a good catch. Independence is not essential to the argument. We assumed it in the submission only for simplicity of exposition, so that the product over hops could be written with a single p. The submission did not estimate p, because the argument uses only p<1. Below, we show that the bound holds without independence, and we estimate p empirically.
The bound holds without independence.Let Eh denote the event that the search at hop h retrieves the passage required for that hop. A trajectory completes a query of depth H∗(x) only if all of E1,…,EH∗ occur, and by the chain ruleP(⋂h=1H∗Eh)=∏h=1H∗P(Eh∣E1,…,Eh−1)=∏h=1H∗ph≤pH∗,p=maxhph,
where ph is the success probability of hop h conditional on the earlier hops having succeeded. This is an identity followed by a maximum, so it holds for any dependence between hops; the bound itself needs only p<1. We have updated Equations 4 and 13 in this conditional form and define p as maxhph in the revision.
Empirical estimate of hop success probability.We also estimated hop success probability (ph) directly from rollouts of the base policy θ0 on the 3-hop+ questions of MuSiQue, the deepest bucket in our evaluation (Table R16): p^h is the fraction of search steps at hop h whose top-3 passages contain the gold passage of that hop, computed on trajectories whose earlier hops succeeded. Two observations follow. First, every p^h is far below 1, so the condition p<1 required by the bound holds with a wide margin. Second, ph decreases as the hops proceed, from 0.244 at the first hop to 0.157 at the second and 0.089 at the third. As a result, the mass of full-chain trajectories ∏hp^h=0.0034 decays faster than the constant-p bound pH∗=0.0145 of Equation 4, by a factor of 4.3. The bound of Equation 4 is therefore conservative, and the depth-dependent decay that motivates QDTS is stronger in practice than the bound states.Table R16: Estimated per-hop retrieval success of the base policy θ0 on MuSiQue 3-hop+ questions (Qwen2.5-7B-Instruct, E5, top-3; 400 questions, 8 rollouts each). p^h is conditional on the earlier hops having succeeded. "Bound" is the constant-p bound pH∗ with p=maxhp^h=p^1, and "Estimated" is the product ∏hp^h.
It was not included in Table 3, and we agree that it should be reported.
Wall-clock time.On the same 4×H100 used for training, context optimization takes only 38.8 minutes in total (17.7 min for Cpre and 21.1 min for Cmid), that is, 2.9% of the total training cost, with 1,731 rollouts and 121 reflection calls (Table R17). The dominant cost of the method is the RL rollouts, as for the baselines. We have added this time to the training time of QDTS in Table 3, which becomes 22.1 h and still remains below that of GRPO + MGS (24.7 h) and AutoSearch (22.4 h).Token cost.In tokens, the optimization processes 4.1M input tokens and 256k output tokens, against 35M and 2.8M for the rollouts of RL training. Its share of the tokens (about 10%) is larger than its share of the time, because RL training also spends time on the gradient updates, which the token counts do not include. All tokens are processed by θ0 on the same GPUs, and no external API is called.Table R17: Cost of context optimization and RL training (4×H100).