- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Recursive Dense Retrieval (RDR). The authors fine-tune Qwen3-Embedding-8B on 4.75 million training data from Recursive Retrieval Worlds (RRW). During inference, RDR constructs a retrieval tree and combines document scores across search states using a level-balanced global readout. The method is evaluated on a range of single-hop and multi-hop retrieval benchmarks.
Critical strengths
-
RDR outperforms GRITHopper's native retrieval procedure on all four multi-hop benchmarks. It also achieves the highest five-suite single-hop mean among the models evaluated in Table 2. These results support the practical value of a unified framework across different retrieval tasks.
-
The paper compares the initialization and fine-tuned encoder under the same search procedure. It also compares readout methods on fixed traversals. These controls help identify the sources of the performance gains and strengthen the empirical findings.
Critical weaknesses
-
The proposed method combines existing ideas in context-conditioned recursive retrieval and tree-based score aggregation. RDR's main difference is that it aggregates document scores across learned, sequentially conditioned search states rather than only along selected retrieval paths. The experiments show that this combination is effective, but the methodological contribution appears incremental.
-
RRW is presented as a major contribution, but the effect of its specific data construction is not sufficiently isolated. The current comparisons do not distinguish this effect from that of additional fine-tuning on conventional retrieval data.
Questions for authors
- The paper compares RDR-8B with the original Qwen3-Embedding-8B, but not with a model fine-tuned on the same amount of conventional retrieval data. How can the authors distinguish the contribution of RRW's data construction from the effect of additional fine-tuning?
- How does the proposed readout, which assigns equal weight to each search level, compare with assigning equal weight to every search node? Using the same retrieval tree and per-node scores would help isolate the effect of level balancing.
4 comments
Rebuttal by Authors (Part 2/3)
Comment: RRW is presented as a major contribution, but the effect of its specific data construction is not sufficiently isolated. [...] How can the authors distinguish the contribution of RRW's data construction from the effect of additional fine-tuning?
Thank you for this question. Full training runs require thousands of A100 GPU-hours, so a further matched-size conventional-data run is not feasible within the rebuttal period and our available compute budget (revised Appendix A). We view Qwen3-Embedding-8B (Zhang et al., 2025), the checkpoint we train from (Section 4.4, lines 303–310), as a relevant conventional-retrieval baseline: it has already been trained on large-scale retrieval data, including synthetic supervision. The targeted recovery ablation below is the closest controlled training comparison we can provide within this budget. Started before and completed during the rebuttal period, it uses the same recipe for one epoch on RRW without recovery samples.
Table R1. New recovery-data ablation, Recall@5, RDR(4,3). Continuation is the primary readout; recovery re-encodes the same traversal. Full RDR rows: submitted Tables 1 and 20. Complete comparison: revised Appendix E.2, Table 22.
| Training | Instruction | MuSiQue | BC+ | FanOutQA | FRAMES | Mean |
|---|---|---|---|---|---|---|
| Qwen3-Embedding-8B (no RRW) | continuation | 62.68 | 16.91 | 27.65 | 68.25 | 43.87 |
| recovery | 59.75 | 15.61 | 29.17 | 67.18 | 42.93 | |
| RRW without recovery samples | continuation | 77.77 | 21.58 | 39.47 | 76.86 | 53.92 |
| recovery | 71.38 | 20.54 | 36.78 | 71.67 | 50.09 | |
| RRW (RDR-8B) | continuation | 80.14 | 25.28 | 37.78 | 77.41 | 55.15 |
| recovery | 80.47 | 24.73 | 40.31 | 76.22 | 55.43 |
Removing recovery samples reduces mean Recall@5 by 1.23 under continuation and 5.34 under recovery. Recovery then falls 3.83 points below the ablation’s own continuation ranking. This supports recovery-specific supervision, one component of RRW’s construction. Unlike GRITHopper (Erker et al., 2026), which has no explicitly trained recovery instruction, RDR selects this learned mode with unchanged encoder weights (Section 5.1, lines 359–365; Appendix K). We add these results to the revised Appendix E.2, Table 22 in the attached diff.
Revised Text — Appendix E.2, Table 22
We evaluate a run started before and completed during the rebuttal period, trained with RDR-8B’s recipe for one epoch on RRW without its 1,043,474 recovery samples (3,706,250 samples, 1,809 steps). Evaluation follows Table 21. Full RRW training adds 1.23 mean Recall@5 under continuation and 5.34 under recovery (Table 22). Without recovery supervision, recovery scores below continuation (50.09 vs. 53.92); with full RRW, both instructions perform similarly (55.43 vs. 55.15). This supports recovery-specific supervision. GRITHopper (Erker et al., 2026) has no explicitly trained recovery instruction; RDR invokes this learned mode with unchanged encoder weights.
Revised Text — Section 6
A recovery-data ablation supports instruction-conditioned recovery behavior (Appendix E.2).
Revised Text — Appendix A — training cost
A full training run requires thousands of A100 GPU-hours. We report one full RRW run and one ablation without recovery supervision (Appendix E.2); multi-seed estimates and further data ablations remain future work.
We continue with the rebuttal in the next comment.
Rebuttal by Authors (Part 3/3)
Comment: How does the proposed readout, which assigns equal weight to each search level, compare with assigning equal weight to every search node? Using the same retrieval tree and per-node scores would help isolate the effect of level balancing.
Thank you for this suggestion; we add this new comparison to revised Appendix F (Table 23), marked in the attached diff. On identical traversals and per-state scores, level balancing performs better on average: 55.15 vs. 52.96 mean Recall@5 on multi-hop, and 59.34 vs. 58.90 mean nDCG@10 on single-hop (difference 0.44, 95% CI [0.26, 0.62]).
Table R2. New node-versus-level comparison on identical traversals; paired bootstrap, 2,000 resamples. Differences are node minus level.
| Suite | Metric | Equal weight per level | Equal weight per node | Difference [95% CI] |
|---|---|---|---|---|
| MuSiQue | Recall@5 | 80.14 | 77.17 | −2.97 [−3.82, −2.12] |
| BrowseComp+ | Recall@5 | 25.28 | 24.94 | −0.34 [−1.00, 0.33] |
| FanOutQA | Recall@5 | 37.78 | 38.62 | +0.84 [−0.59, 2.25] |
| FRAMES | Recall@5 | 77.41 | 71.12 | −6.29 [−7.57, −5.10] |
| Multi-hop mean | Recall@5 | 55.15 | 52.96 | −2.19 |
| NanoBEIR | nDCG@10 | 68.13 | 67.86 | −0.26 [−0.75, 0.21] |
| BRIGHT | nDCG@10 | 30.81 | 31.05 | +0.24 [−0.06, 0.55] |
| TopiOCQA | nDCG@10 | 70.79 | 69.72 | −1.07 [−1.52, −0.60] |
| ToolRet | nDCG@10 | 48.11 | 47.89 | −0.22 [−0.64, 0.17] |
| CoIR (sample) | nDCG@10 | 78.87 | 77.98 | −0.89 [−1.20, −0.57] |
| Single-hop mean | nDCG@10 | 59.34 | 58.90 | −0.44 [−0.62, −0.26] |
Multi-hop retrieval needs the complete evidence chain: finding only hops three and four still leaves hops one and two missing. With 1, 3, 9, and 27 states per level, node weighting gives the deepest level 67.5% of the mass and the root 2.5%; level balancing gives each depth 25%. Of the gold documents lost from the top 5 under node weighting, 62% on MuSiQue and 92% on FRAMES were in the root’s top 5 (new analysis in revised Appendix F).
Single-hop differences are smaller: node weighting slightly improves the reasoning-intensive BRIGHT benchmark (Su et al., 2025) (+0.24 nDCG@10), while the other four suites decrease by 0.22–1.07 points. Level balancing remains better on average.
Revised Text — Appendix F, Table 23 — node weighting and interpretation
A multi-hop question needs the complete evidence chain. Equal weight per node shifts 67.5% of the mass to the deepest level and 2.5% to the root, versus 25% per level. On MuSiQue and FRAMES, 62% and 92% of gold documents lost from the top 5 were in the root’s top 5. Single-hop differences are smaller: node weighting slightly improves BRIGHT (Su et al., 2025) (+0.24), while the other four suites decrease by 0.22–1.07 nDCG@10; level balancing remains better on average (59.34 vs. 58.90).
Thank you again; the two analyses you suggested strengthen the paper. Please also see the uploaded paper diff, which highlights the additions in blue and shows the revisions proposed for the camera-ready version.
Follow-up on the training and level-balancing ablations
Dear Reviewer YquP,
thank you again for your careful review and initial weak-accept recommendation. We appreciate your recognition of the practical value of unified retrieval and the evidence provided by our matched-search and fixed-traversal controls.
Our responses clarify the methodological distinction, present the recovery-data ablation, and add your suggested node-versus-level comparison on identical traversals and scores, together with an explanation of why level balancing helps retain the complete evidence chain. The corresponding revisions are highlighted in the uploaded paper diff.
We hope our responses address your concerns. With less than 48 hours remaining in the discussion period, we would be grateful if you could let us know whether any questions or concerns remain, so that we have time to address them before the deadline.
Thank you again for your time and consideration.
Best,
The authors
Rebuttal by Authors (Part 1/3)
Dear Reviewer YquP,
Thank you for your review. We are glad that you see the practical value of a unified framework across retrieval tasks and that the matched-search and fixed-traversal controls help identify the sources of the gains.
Line numbers below refer to the submitted paper. New material is identified separately and marked in the attached revised-paper diff.
Comment:
The proposed method combines existing ideas in context-conditioned recursive retrieval and tree-based score aggregation. [...] the methodological contribution appears incremental.RDR combines a learned state operator with global readout. MDR (Xiong et al., 2021), BeamDR (Zhao et al., 2021), and GRITHopper (Erker et al., 2026) score selected retrieval paths; COR (Park et al., 2025) aggregates a tree of document-derived query views rather than learned state transitions (Section 2, lines 141–157; Appendix K, lines 1749–1755).
Both components matter: replacing learned states with document-as-query states lowers global-readout performance from 55.15 to 40.41 mean Recall@5 (Section 5.4, Table 3, lines 449–463). The same tree with Qwen3-Embedding-8B (Zhang et al., 2025) before RRW fine-tuning gains little (Section 5.1, Table 1, lines 331–358). The new recovery ablation further demonstrates the importance of explicitly training the recovery instruction (Table R1 in our next response below, also reported to ttJR; revised Appendix E.2, Table 22 in the attached diff).
Revised Text — Section 3; existing comparison in Section 2 / Appendix K
For both task families, the encoder weights, tree configuration, and level-balanced readout remain fixed; only the task input (instruction and query) changes.
We continue with the Rebuttal in the next comment.