acceptodds
Under review as a conference paper at ICLR 2027

AdaTutoRank: Learning to Rerank Document Sets via Adaptive Tutoring Optimization for RAG and Deep Research

15 views

Abstract

Document rerankers determine what evidence reaches the downstream model in RAG and deep research, yet mainstream rerankers select by relevance matching, and individually relevant documents rarely constitute the complete, complementary, non-redundant set a complex information need demands. Prior work rewards a set by its aggregate rubric score, shifting the objective from ranking documents to composing sets. Yet that score is one scalar shared by every document in the set, so the supervision is sparse: a redundant document is rewarded with the rest whenever the set scores well, and a decisive one penalized with the rest whenever it does not; credit assignment leaves contributors indistinguishable from free riders. On-policy distillation could densify this supervision, but existing methods give every rollout the same fixed guidance, too prescriptive for strong rollouts and too abstract for weak ones. We therefore propose AdaTutoRank, a setwise reranker trained with Adaptive Tutoring Optimization (ATO) under a three-level hierarchy of nine rubric dimensions, which supplies silver labels for the cold start, rewards for reinforcement learning, and hints for distillation. ATO draws three hint forms of increasing specificity from the policy's own frozen snapshot: the rubrics alone, a self-selector's sibling-set chosen under rubrics, and a self-reflector's reflection contrasting the rollout with that sibling-set; each rollout receives the form matched to its quality. Re-scoring that rollout under the hint-conditioned frozen teacher and the hint-free snapshot distills the hint's effect into a token-level advantage that complements the group-relative outcome advantage. Across ten benchmarks spanning RAG, deep research, and setwise evaluation, AdaTutoRank attains the best overall performance while issuing fewer retrieval calls.

Primary area:
foundation or frontier models, including LLMs
Keywords:
RAG, Deep Research, Setwise, Rubric-oriented, Reinforcement Learning, On-Policy Self-Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nu8QOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper argues that mainstream rerankers select by relevance, yielding individually relevant but collectively redundant or incomplete sets. It also identifies a key limitation of the rubric-reward approach: the reward is a single scalar shared by every document, making contributors indistinguishable from free riders. To address this, the paper extends RubricReranker and proposes AdaTutoRank, a setwise reranker trained with Adaptive Tutoring Optimization (ATO) under a three-level rubric hierarchy of nine dimensions that covers document-level, set-level, and global set properties. These rubrics are carried through the entire training pipeline, serving as silver labels for cold-start SFT, as rewards for reinforcement learning, and as hints for distillation, which makes multi-dimensional set quality explicitly optimizable at every stage. The method achieves the best overall performance across ten benchmarks while issuing fewer retrieval calls.

Critical strengths
  • The paper is well-organized and easy to follow. The appendices are extensive and genuinely useful for understanding the method.
  • The proposed three-level, nine-dimension rubric hierarchy is a thoughtful design. Carrying the same rubrics through silver-label generation, RL reward computation, and distillation hints ties everything together, and it clearly contributes to the strong performance.
  • The proposed Adaptive Tutoring Optimization (ATO) is a novel design. The idea of matching hint specificity to rollout quality is simple, intuitive, yet effective.
  • Extensive experiments demonstrate the effectiveness of the proposed method. It consistently outperforms existing setwise and pointwise rerankers.
Critical weaknesses
  • The novelty is limited. AdaTutoRank builds on RubricRanker's rubric-as-reward formulation and essentially extends it with a richer rubric hierarchy and a token-level distillation advantage on top of the scalar RL reward.
  • BGE’s parameter counts should be reported in Table 1. BGE-Reranker-Large is only 0.6B yet remains highly competitive, beating several setwise approaches on the deep research benchmarks. Given this, I suggest comparing against stronger LLM-based pointwise rerankers such as ProRank (0.5B and 1.5B) [1] and Qwen3-Reranker [2] to test whether the setwise formulation consistently outperforms them.
  • An efficiency comparison between pointwise and setwise rerankers at the same model scale would be valuable — for example, Qwen3-Reranker-8B (pointwise) versus AdaTutoRank (8B) or other 8B setwise models. This would clarify the practical trade-offs between the two paradigms.
  • Real-world applications typically retrieve 100–1000 candidates at the first retrieval stage, but this paper only tests the top-20. It would be interesting to see whether the method supports larger candidate pools and how performance changes as k increases. Such an experiment would significantly strengthen the work.

Reference:

[1] Li, X., Shakir, A., Huang, R., Lipp, J., Clavié, B., & Li, J. (2026, July). Prorank: Prompt warmup via reinforcement learning for small language models reranking. In Findings of the Association for Computational Linguistics: ACL 2026 (pp. 1026-1037).

[2] https://huggingface.co/collections/Qwen/qwen3-reranker

Questions for authors

The candidate pool size is fixed at the top 20 across all main experiments. Would it support larger pools, say 500 or 1000 candidates? If not, what design changes (e.g., two-stage selection, chunking, or a longer-context backbone) would be needed to handle such scenarios?

Reviewer_Kpd7OpenReview ↗
Rating
1
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AdaTutoRank, a setwise document reranker for RAG and deep research. The method uses nine rubric dimensions to evaluate document sets and trains the model in two stages. It first uses silver labels for supervised fine-tuning, followed by Adaptive Tutoring Optimization (ATO). ATO combines an RL reward with a token-level distillation signal. Depending on the quality of each rollout, it provides one of three types of hints: rubrics, corrective reflections, or sibling sets. Experiments on RAG, deep research, and setwise evaluation benchmarks show improvements over existing rerankers.

Critical strengths
  1. The paper addresses an important problem. Selecting a useful document set rather than simply ranking documents by relevance is valuable for RAG and deep research, where redundancy and missing evidence can affect the final answer.
  2. The overall framework is clear and easy to follow. The idea of using different hints for rollouts of different quality is interesting, and the method combines rubric-based rewards with distillation in a reasonable way.
  3. The experiments cover several tasks and include comparisons with both traditional and setwise rerankers. The authors also provide detailed ablation results and additional analysis in the appendix.
Critical weaknesses
  1. The benefit of adaptive tutoring seems quite limited. My main concern is whether adaptive tutoring actually provides a meaningful improvement over using a fixed hint. In Table 3, the authors report a gain of 1.57 points over Only Reflection (49.19 vs. 47.62) on four benchmarks. However, Table 7 in the appendix shows that the gain is only 0.23 points (45.28 vs. 45.05) when all nine benchmarks are considered. On WebWalkerQA, adaptive tutoring even performs much worse than Only Reflection (41 vs. 47). These results make me question how useful the adaptive strategy really is. Since this is the main contribution of the paper, I would expect stronger evidence that choosing different hints based on rollout quality is better than simply using a fixed hint.
  2. Token-level advantages do not necessarily solve document-level credit assignment. The paper repeatedly motivates ATO as a way to address document-level credit assignment. However, Eq. 9 computes the advantage over tokens in the generated document ID sequence, rather than measuring the contribution of each document to the selected set. For example, selecting [2] [5] and [5] [2] gives the same document set and the same set-level reward, but their token-level advantages can be different because the generation order changes. Also, changes in the probabilities of tokens such as numbers and brackets do not directly reflect the usefulness of the corresponding documents. I agree that the proposed approach provides a denser training signal, but it is not clear how this translates into better credit assignment at the document level.
  3. It is unclear how much of the improvement comes from the hints themselves. In Eq. 9, the distillation advantage compares a hint-conditioned frozen SFT teacher with the current policy snapshot without hints. Therefore, the log-probability difference contains not only the effect of the hint but also the difference between the two models. In fact, the authors show in Appendix A that the objective includes a reverse-KL term that pulls the policy toward the SFT checkpoint. This makes it difficult to tell whether the improvements mainly come from adaptive tutoring or from regularization toward the SFT model. I think an important missing baseline is distillation from the frozen SFT teacher without any hints.
  4. Some comparisons do not fully separate the effect of ATO from other design choices. AdaTutoRank uses nine rubric dimensions and silver-label SFT before ATO, while the closest baseline, RubricRanker, uses a different rubric design and training setup. It is therefore difficult to know how much of the improvement over RubricRanker comes from ATO itself. A stronger comparison would use the same nine rubrics, training data, and SFT initialization, but train with standard GRPO instead of ATO. In addition, Appendix B.2 states that traditional rerankers return the top five documents, while AdaTutoRank can choose a variable number of documents. This makes the comparisons less controlled, since the methods may provide different amounts of context to the downstream model.
  5. The reward scores may not be directly comparable across different selected sets. As described in Appendix C.1, when a rubric is not applicable, the judge returns null, and that rubric is removed from both the numerator and denominator of the reward calculation. As a result, different selected sets may be evaluated using different combinations of rubrics. For example, a set with only one relevant document may avoid being evaluated on conflict or complementarity, while another set is evaluated on all dimensions. This raises a concern about whether the resulting rewards are comparable, especially when they are used to compute group-relative advantages in GRPO. It may also create incentives for the model to avoid certain evaluation dimensions.
  6. The reward judge prompt contains inconsistent scoring instructions. I noticed an inconsistency in Appendix F.5. The judge instructions state that Complementarity should be null when fewer than two relevant documents contribute different required information points. However, in the provided scoring example, Doc A already covers all required points, Doc B does not add a new required point, and Doc C is irrelevant. The example still assigns a Complementarity score of 3 instead of null. This appears to contradict the stated scoring rule. Since the reward is central to both the RL training and adaptive hint selection, such inconsistencies may affect the reliability of the supervision.
Questions for authors
  1. In Table 7, adaptive tutoring only improves the average score by 0.23 over Only Reflection, and performs 6 points worse on WebWalkerQA. Why is the improvement so limited compared to Table 3?

  2. How does the token-level advantage reflect the contribution of individual documents? Would changing the order of document IDs affect the advantage for the same selected set?

  3. Have the authors tried distillation from the frozen SFT teacher without hints? How much of the gain actually comes from adaptive hints rather than the regularization effect?

  4. Can the authors clarify the Complementarity scoring rule in Appendix F.5? The instructions require a null score in certain cases, but the provided example assigns a score of 3.

Reviewer_7e9iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose the AdaTutoRank, a Qwen3-8B document-set selector for RAG and deep research. It combines rubric-supervised SFT with Adaptive Tutoring Optimization, which assigns rubric, reflection, or sibling-set hints according to rollout quality and combines token-level distillation with GRPO. Experiments span five RAG datasets, four deep-research benchmarks, and SetwiseEvalKit, reporting improved overall answer and set quality with fewer search calls.

Critical strengths
  1. The method addresses an important limitation of relevance-based reranking: individually relevant documents need not form sufficient, complementary evidence. Combining set-level rewards with token-level distillation provides a plausible approach, while keeping privileged hints confined to training.
  2. The evaluation connects evidence selection to downstream performance across both RAG and research agents. With the generator or agent held fixed, AdaTutoRank improves the answer-level average over RubricRanker from 43.84 to 45.28 and achieves the best overall scores in both setwise evaluation scenarios.
Critical weaknesses
  1. The relevance-ranking baselines omit Qwen3-Reranker, particularly the 8B variant, which matches AdaTutoRank’s model family and parameter scale. This is a strong, directly relevant comparator. Without it, the advantage over contemporary relevance-based reranking remains insufficiently established. Comparisons should also control the retained-context budget, since relevance baselines use top-5 documents while AdaTutoRank selects variable-size subsets.
  2. The evidence for adaptive hint assignment, the central methodological contribution, is limited. Across all nine answer-level benchmarks, it improves over fixed reflection by only 0.23 points (45.28 vs. 45.05, Table 7). No uncertainty estimates or control using the same hint proportions without reward-based routing are reported. The results therefore do not establish whether adaptive assignment provides a reliable benefit beyond the available hint types.
Questions for authors
  1. How does AdaTutoRank compare with Qwen3-Reranker-8B using identical retrieved candidates, document truncation, and matched retained-context budgets?
  2. Additional evidence: Does adaptive routing consistently outperform fixed reflection and randomized routing with matched hint proportions across repeated runs?

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers,

We sincerely appreciate your time and insightful feedback. We are glad that reviewers R1\text{R1}, R2\text{R2}, and R3\text{R3} recognized that our work tackles an important problem beyond relevance-based reranking (R1, R2\text{R1, R2}), that the nine-dimension rubric hierarchy and Adaptive Tutoring Optimization (ATO) form a well-motivated and novel design (R1, R2, R3\text{R1, R2, R3}), that our evaluation is extensive across RAG and deep research (R1, R2, R3\text{R1, R2, R3}), and that the paper is clear and well-organized (R2, R3\text{R2, R3}).

The main concerns were whether adaptive tutoring gives a reliable gain beyond fixed hints and other design choices, how AdaTutoRank compares with stronger pointwise rerankers under matched conditions, and several design details. We addressed each with new experiments and clarifications, summarized below. Point-by-point responses are posted under each review.

  • New experiments:
    • Per-benchmark comparison with Only Reflection, and repeated runs on WebWalkerQA: AdaTutoRank leads on 7 of 9 benchmarks, and the WebWalkerQA deficit shrinks substantially over repeated runs. (R1, R2\text{R1, R2})
    • Matched-ratio random hint routing without reward-based routing: below AdaTutoRank on all 9 benchmarks. (R1, R2\text{R1, R2})
    • Hint-free distillation teacher: below AdaTutoRank on 8 of 9 benchmarks, so the gain comes from the hint itself rather than SFT regularization. (R2\text{R2})
    • Standard GRPO with the same rubrics, data, and SFT initialization: below AdaTutoRank on 8 of 9 benchmarks. (R2\text{R2})
    • Stronger pointwise baselines (Qwen3-Reranker, ProRank), including a per-query matched context budget: AdaTutoRank leads on every benchmark. (R1, R2, R3\text{R1, R2, R3})
    • Efficiency comparison with pointwise rerankers: AdaTutoRank is faster than Qwen3-Reranker-4B and 8B. (R3\text{R3})
  • Clarifications and revisions:
    • Document-level credit and order sensitivity of the token-level advantage. (R2\text{R2})
    • Reward comparability under null-gating: the model has no incentive to under-select, since single-document selections are only 0.89% of training rollouts. (R2\text{R2})
    • Complementarity example in Appendix F.5: we will correct it in the revision. (R2\text{R2})
    • Difference from RubricRanker: the rubric hierarchy guides three training stages, and ATO provides token-level credit that a scalar reward cannot. (R3\text{R3})
    • Candidate-pool size: AdaTutoRank stays ahead at pool sizes 10–50, and we discuss a sliding-window extension for larger pools. (R3\text{R3})
    • We will report BGE-Reranker's parameter count in Tables 1 and 2. (R3\text{R3})

We warmly encourage you to review our responses. We hope the new results and clarifications address your concerns, and we are happy to clarify any remaining questions.

Best regards,

Authors of Submission 20037

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.