Beyond Oracle Gains: Measuring the Learnability of Adaptive RAG Routing

Retrieval-augmented generation (RAG) exposes multiple inference-time compu- tational choices, including retrieval depth, generator size, and numerical preci- sion. Oracle analyses suggest that selecting these configurations separately for each query can yield substantially better quality–cost trade-offs than using a sin- gle fixed configuration. However, an oracle observes the realized outcome of every candidate configuration and therefore does not establish whether the correspond- ing routing decisions are predictable before execution. We study the distinction between oracle opportunity and routing learnability in adaptive RAG. Our analysis considers retrieval depth, generator capacity, and in- ference precision across four multi-hop question-answering datasets. Within a targeted four-configuration space, a retrospective oracle achieves 0.423 F1 at an average estimated cost of 36.57, while individual fixed configurations obtain only 0.203–0.258 F1. A learned router provides useful low-cost operating points but reaches only 0.249 F1 even at an average cost of 57.59. To determine whether this gap is primarily caused by insufficient supervision, we construct 2,000 additional train-only queries. Increasing routing supervision from 1,421 to 3,421 queries improves retrieval-depth prediction by 0.072 ROC-AUC and model-size prediction by 0.042, but decreases precision-routing ROC-AUC by 0.033. These effects also vary substantially across datasets. Our results show that large oracle headroom does not necessarily translate into an equally learnable routing policy. We argue that adaptive RAG should therefore be evaluated in terms of both available oracle opportunity and the fraction of that op- portunity that can be predicted and recovered before paying the cost of candidate configurations.