acceptodds
Under review as a conference paper at ICLR 2027

Beyond Oracle Gains: Measuring the Learnability of Adaptive RAG Routing

Abstract

Retrieval-augmented generation (RAG) exposes multiple inference-time compu- tational choices, including retrieval depth, generator size, and numerical preci- sion. Oracle analyses suggest that selecting these configurations separately for each query can yield substantially better quality–cost trade-offs than using a sin- gle fixed configuration. However, an oracle observes the realized outcome of every candidate configuration and therefore does not establish whether the correspond- ing routing decisions are predictable before execution. We study the distinction between oracle opportunity and routing learnability in adaptive RAG. Our analysis considers retrieval depth, generator capacity, and in- ference precision across four multi-hop question-answering datasets. Within a targeted four-configuration space, a retrospective oracle achieves 0.423 F1 at an average estimated cost of 36.57, while individual fixed configurations obtain only 0.203–0.258 F1. A learned router provides useful low-cost operating points but reaches only 0.249 F1 even at an average cost of 57.59. To determine whether this gap is primarily caused by insufficient supervision, we construct 2,000 additional train-only queries. Increasing routing supervision from 1,421 to 3,421 queries improves retrieval-depth prediction by 0.072 ROC-AUC and model-size prediction by 0.042, but decreases precision-routing ROC-AUC by 0.033. These effects also vary substantially across datasets. Our results show that large oracle headroom does not necessarily translate into an equally learnable routing policy. We argue that adaptive RAG should therefore be evaluated in terms of both available oracle opportunity and the fraction of that op- portunity that can be predicted and recovered before paying the cost of candidate configurations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
retrieval-augmented generation, adaptive RAG, model routing, adaptive computation, inference efficiency, retrieval depth, quantization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_5HjDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the distinction between retrospective oracle gains and practically learnable routing policies in adaptive RAG. It evaluates query-dependent selection of retrieval depth, generator size, and inference precision across four multi-hop QA datasets. The experiments demonstrate a substantial gap between oracle performance and learned routing, while increasing training supervision produces different effects across the three configuration dimensions.

Critical strengths
  1. Important evaluation distinction. Separating oracle opportunity from achievable routing performance addresses a meaningful issue in adaptive inference research.
  2. Controlled supervision-scaling analysis. The experiments demonstrate that additional training data improves some routing decisions but not others, highlighting differences in predictability across computational dimensions.
Critical weaknesses
  1. Limited methodological novelty. The primary contribution is an empirical characterization of the oracle–router gap rather than a new routing method or general theoretical framework. The conceptual distinction is intuitive, and the paper would benefit from stronger methodological or explanatory contributions.
  2. Limited evidence for general routing learnability. The study relies on a restricted four-configuration setting, a relatively small frozen test set of 287 queries, and estimated computational costs. The observed gap may partly reflect the selected router architecture and available features rather than an intrinsic limitation of adaptive RAG.
Questions for authors
  1. Does the oracle–router gap persist when using stronger routing architectures or retrieval-aware features, particularly features derived from inexpensive retrieval signals?
  2. How sensitive are the reported quality–cost conclusions to actual measured inference latency and a larger configuration search space?
Reviewer_hgsnOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how much of the oracle gain in adaptive RAG can be recovered by a learned router. It considers retrieval depth, model size, and numerical precision, and tests how additional training data affects routing performance.

Critical strengths

The paper asks a useful practical question: whether gains from an oracle can translate into gains from an actual router.

Critical weaknesses
  1. The writing needs substantial revision. The abstract spreads a simple message over four paragraphs. The introduction repeats experimental results and presents individual scores and configuration changes as numbered equations, without making the contribution clear.

  2. The experiments do not support the claims about learnability. The main comparison uses only four configurations, XGBoost routers, and 287 test questions, with no stronger routing baselines or retrieval-aware features. The data-scaling study reports classification AUC without showing whether end-to-end routing improves. A weak router is not enough evidence that the task is hard to learn.

Questions for authors
  1. Does the oracle–router gap remain with stronger routers and features based on retrieved passages?
  2. How does additional training data affect end-to-end answer quality and cost?
Reviewer_PmBeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether the benefits of additional computation in RAG can be predicted before generation, and how much of the improvement available through oracle configuration selection can be recovered by a learned router. Experiments on four datasets reveal a gap between the evaluated router and the retrospective oracle. The authors also conduct additional experiments to examine how routing performance changes with the amount of training supervision.

Critical strengths

The motivation is clearly presented, and the presentation of the experiments is easy to follow.

Critical weaknesses
  1. Insufficiently established novelty. The observation that learned routers fall short of oracle performance is not new. Previous works already compared learned routing with oracle routing, and some others have studied the benefits of model selection in RAG [1,2]. The paper does not sufficiently clarify what new insights its analysis provides beyond these studies.

  2. Limited experimental scope. The primary experiment evaluates how much oracle headroom a particular router recovers within a four-configuration space. Each computational dimension—model size, numerical precision, and retrieval depth—includes only one local upgrade from the anchor configuration, and combinations of upgrades are not evaluated in this experiment. This restricts the scope of the conclusions and the overall contribution.

  3. Insufficient discussion of prior work. The related-work discussion largely focuses on studies from 2024 or earlier and does not adequately address more recent RAG-routing research. In particular, the “Oracle-Based Adaptive Inference” paragraph makes general claims about prior work without citations.

[1] Soyeong Jeong et al. Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity. NAACL 2024, pp. 7036–7050. [2] Jiarui Zhang et al. RAGRouter: Learning to Route Queries to Multiple Retrieval-Augmented Language Models. NeurIPS 2025, pp. 17745–17773.

Questions for authors
  1. Beyond increasing training data, can richer input features or alternative routing models consistently improve performance and close the oracle–router gap?

  2. How sensitive are the reported findings to the choice of generator family? Would they hold for other model families?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.