- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper introduces a two-stage framework consisting of CS-OPD and RD-DPO to transfer ranking knowledge from teacher model to a compact dense retriever. CS-OPD aligns the student's retrieval score distribution with the frozen teacher over student-induced Top-K candidate sets via reliability-weighted KL divergence to capture listwise competition, while RD-DPO mines teacher-validated positive-negative ranking inversions and applies a score-level DPO objective for targeted ranking correction.
Critical strengths
- CS-OPD avoids the limitation of static offline candidate pools by letting the student dynamically generate is own candidate support during distillation.
- RD-DPO successfully targets and resolves residual local ranking errors near the top of the list.
- Experiments on ToolRet and SkillRet benchmarks show that the framework significantly outperforms offline distillation and contrastive baselines across multiple student scales.
Critical weaknesses
- The two-stage framework requires careful hyperparameter tuning, which makes complex to implement than standard contrastive fine-tuning.
- Training relies heavily on a powerful teacher model to compute candidate score distributions and validate preference directions, while experiments on computational and training efficiency are missing.
Questions for authors
- What is the theoretical justification for mapping sequence-level preference optimization directly to score margins, and why doesn't direct margin regression achieve the same targeted correction?
- Could the authors provide an efficiency analysis detailing the training cost breakdown between Stage 1 and Stage 2?
- Does the two-stage framework generalize zero-shot to broader, traditional passage retrieval domains, or are CS-OPD and RD-DPO structurally specialized for tasks where candidates are highly categorical and densely clustered?
4 comments
Regarding W2 and Q2: teacher dependence and training efficiency
We thank the reviewer for this important point. We agree that teacher quality affects the reliability of both relevance distributions and preference directions. This dependence is shared by existing retrieval-distillation methods; some approaches, including the original Rabtriever, additionally rely on computationally expensive generative rerankers. Our framework directly transfers knowledge from an embedding teacher to an embedding student, without requiring a generative reranker or an additional predictor. Under the same frozen 8B embedding teacher, our framework outperforms the evaluated distillation baselines on ToolRet and SkillRet, including MarginMSE, Score-Only KD, and the adapted Rabtriever baseline.
Our teacher-size comparison in Appendix Tables 5–6 also evaluates a smaller teacher. For the 0.6B student, a 4B teacher yields NDCG@10 scores of 50.50 on ToolRet and 81.68 on SkillRet, exceeding the corresponding InfoNCE-trained baselines by 2.86 and 2.00 percentage points, respectively. Using the 8B teacher further improves these scores to 51.42 and 82.08. These results show that the framework improves upon contrastive fine-tuning with a smaller teacher while benefiting further from the larger teacher.
We separate training cost into Stage-I distillation, Stage-II data preparation, and DPO optimization. All reported times are elapsed wall-clock times measured using the eight NVIDIA A800 GPUs. For the 0.6B student, CS-OPD takes approximately 90 minutes, followed by 10 minutes of data selection and 30 minutes of DPO training, totaling 130 minutes. For the 4B student, Stage-I distillation takes approximately 330 minutes, data preparation takes 20 minutes, and DPO optimization takes 60 minutes, totaling 410 minutes.
Regarding Q3: zero-shot generalization
We thank the reviewer for raising this concern. CS-OPD constructs candidate support using student retrieval scores, while RD-DPO selects residual errors using relevance annotations and teacher–student score margins. Neither stage requires tool-category labels or explicit cluster assignments. To assess generalization beyond tool and skill catalogs, we evaluated the 0.6B student on selected retrieval tasks from BEIR and MTEB: HotpotQA for multi-hop question answering, SciFact for scientific evidence retrieval, and ArguAna for counterargument retrieval. We compared the same student before and after our two-stage training, without additional training on these datasets.
| Dataset | Model | NDCG@10 ↑ | Recall@10 ↑ | MRR@10 ↑ |
|---|---|---|---|---|
| HotpotQA | Original student | 66.28 | 67.83 | 83.44 |
| HotpotQA | CS-OPD + RD-DPO | 66.85 | 68.53 | 83.92 |
| SciFact | Original student | 70.26 | 84.10 | 66.39 |
| SciFact | CS-OPD + RD-DPO | 70.84 | 84.52 | 66.95 |
| ArguAna | Original student | 67.47 | 92.89 | 59.87 |
| ArguAna | CS-OPD + RD-DPO | 67.49 | 92.96 | 59.91 |
Our method improves all three reported metrics across HotpotQA, SciFact, and ArguAna. These results provide evidence that the two-stage framework can transfer beyond tool and skill catalogs to different text-retrieval tasks.
A gentle discussion-phase reminder
Dear Reviewer 1XRA,
Thank you again for your constructive feedback. As the discussion phase draws to a close, we would appreciate any further thoughts on whether our responses and additional results address your concerns.
Best regards,
The Authors
Dear Reviewer 1XRA,
As the discussion period is coming to a close, we would appreciate your feedback on whether our responses have addressed your concerns. Please let us know if any further clarification is needed.
Best regards, The Authors
We thank the reviewer for the constructive feedback and important questions.
Regarding W1: hyperparameter tuning and implementation complexity
We thank the reviewer for this helpful comment. Our experiments use largely shared hyperparameters across ToolRet and SkillRet. In Stage I, the only dataset-specific setting is the candidate-set size: Top-10 for ToolRet and Top-30 for SkillRet. All other Stage-I hyperparameters are shared, and Stage II uses the same configuration on both benchmarks. Learning rates and other general training settings were kept fixed across the two datasets rather than extensively tuned for each benchmark. Section 4.5 and Figure 4 also evaluate Stage-I sensitivity to candidate-set size on ToolRet, with NDCG@10 ranging from 50.80 to 51.22 across the tested values of K.
The framework introduces additional retrieval and preference-pair construction steps compared with InfoNCE fine-tuning, while achieving better retrieval performance, particularly at the first rank. On ToolRet, our 4B student outperforms the InfoNCE-trained Tool-Embed (Tool-DE)-4B baseline by 5.26 percentage points in NDCG@1 and 3.41 points in Recall@1.
Regarding Q1: theoretical justification and comparison with margin regression
We thank the reviewer for this important question. Our score-level formulation follows from defining a local retrieval policy over a positive–negative pair:
πθ(c∣q,{p,n})=exp(sθ(q,p)/τs)+exp(sθ(q,n)/τs)exp(sθ(q,c)/τs),τs>0.
Its log-odds satisfy the exact identity
logπθ(n∣q)πθ(p∣q)=τssθ(q,p)−sθ(q,n)=Δθ.
Applying this identity to both the student and reference policies yields the per-pair DPO loss:
LDPO=−logσ[β(Δθ−Δref)],
where the reference score margin is defined analogously using the frozen Stage-I checkpoint. Thus, score-level DPO is standard DPO applied to the score-induced local retrieval policy. This derivation is presented in Section 3.3 (Eqs. 12–15) of the main text.
Direct margin regression can also correct the selected inversions. MarginMSE minimizes
(mθ−mT)2,
where the unscaled student and teacher margins are
mθ=sθ(q,p)−sθ(q,n),mT=sϕ(q,p)−sϕ(q,n).
It therefore requires reproducing the teacher’s numerical margin, introducing a calibration requirement beyond recovering the correct ordering. In RD-DPO, the teacher validates the preference direction, while the frozen Stage-I reference defines the starting preference. DPO encourages reference-relative preference improvement without requiring reconstruction of the teacher’s margin magnitude. The targeted supervision comes from reliable-disagreement pair selection; the objective choice is evaluated separately below.
For the 0.6B student, we compared both objectives from the same Stage-I checkpoint, using identical reliable-disagreement pairs and matched optimizer-update budgets:
Both objectives improve all three @10 metrics over the Stage-I checkpoint. DPO exceeds MarginMSE by 0.31/0.41/0.52 percentage points on ToolRet and 0.44/1.26/2.09 points on SkillRet. These comparisons support DPO as the correction objective in our evaluated setting.