Block-Level Recursion: Adaptive Test-Time Routing in Large Language Models

Test-time routing improves frozen large language models (LLMs) by taking non-linear paths through their layers, without modifying weights or generating extra tokens. Existing approaches define route spaces that grow exponentially with depth, making them costly to search and hard to learn from. We therefore introduce $\text{Block-Level Recursion}$ ($\text{BLR}$), a restricted route family that repeats a single contiguous block of transformer layers once. This reduces the number of routes from exponential to quadratic in the number of layers, making exhaustive per-instance evaluation tractable and the oracle upper bound directly measurable. Despite this restriction, $\text{BLR}$ yields large accuracy gains while keeping route selection practical. Across six model families and ten reasoning benchmarks, the optimal block varies across models, tasks, and individual inputs, with per-instance oracle gains of $+59.1\%$ overall and up to $+75.8\%$ on individual tasks. $\text{BLR}$ also supports two practical policies: a single train-selected block ($\text{sBLR}$) that requires no router or per-input overhead, and a learned global router ($\text{aBLR}$) trained from dense per-instance rewards over all routes. $\text{sBLR}$ already improves accuracy substantially, while $\text{aBLR}$ improves further by selecting routes per input. With a frozen Qwen2.5-0.5B backbone, $\text{aBLR}$ achieves higher accuracy than the unrouted Qwen2.5-7B model at lower FLOPs.