acceptodds
Withdrawn from ICLR 2027

FORGE: Form-Optimal Routing of Grounded Evidence for Frozen LLM Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

In agentic AI systems, frozen foundation models are increasingly deployed as closed-weight API endpoints, making downstream adaptation possible only through the inputs and inference procedures surrounding the model. As a result, for each input query, two coupled decisions largely determine both answer quality and token cost: what evidence to provide and how much reasoning budget to allocate. Fixed defaults along these axes are often suboptimal, misallocating support form or reasoning depth on roughly 80% of queries in our analysis. To address this challenge, we propose \ours, a unified framework for adapting frozen models through per-query routing over a joint action space that spans both support form and thinking depth. Under an entropy-regularized, cost-aware utility objective, we derive a closed-form Boltzmann routing target and instantiate the policy as a lightweight 269K-parameter factorized router. The routing policy is trained around the frozen host, without any weight access, through a three-stage pipeline: offline arm enumeration, supervised Kullback-Leibler (KL) distillation from the Boltzmann target, and Group Relative Policy Optimization (GRPO) refinement with host feedback. Across 5 knowledge-intensive benchmarks and 8 frozen backbones ranging from 7B to 671B parameters, Forge improves accuracy at 42–45% lower token cost on both main hosts, transfers zero-shot across hosts at lower token cost, and composes with intrinsic thinking budgets where available.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, agentic AI

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_YBP7OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FORGE, a framework that dynamically selects the form of external evidence provided to a frozen LLM when answering knowledge-intensive questions. The framework routes each query among predefined support forms, including Direct, Summary, and Raw, while also considering different reasoning budgets, with the goal of maximizing answer quality while minimizing token cost.

The router is trained through KL distillation from a utility-based Boltzmann distribution, followed by GRPO-based refinement using feedback from the frozen host model. Experiments are conducted on five knowledge-intensive QA and fact-verification benchmarks across eight frozen LLM backbones. The results demonstrate improvements in QA accuracy and token efficiency compared with fixed inference configurations.

Critical strengths
  1. This paper addresses a practically valuable problem: how to maximize QA accuracy and minimize token cost by dynamically switching between several RAG configurations. It also proposes a "utility calculation method." The proposed framework is practical by offering a lightweight mechanism, and the empirical results indicate that query-dependent routing can improve the quality-cost tradeoff compared with fixed inference configurations.

  2. The evaluation covers a broad range of model families by testing eight frozen LLM backbones, ranging from open-weight models to large API-served models. This demonstrates FORGE's transferability and generalization capabilities. The authors also provide ablations separating action-space expansion from GRPO refinement, cross-host transfer experiments, and measurements of fresh-online inference costs. These analyses strengthen the empirical evaluation and help clarify the conditions under which the method is practically useful.

Critical weaknesses
  1. Limited scientific insight into the underlying routing problem and insufficient methodological novelty.

The central contribution of FORGE is a lightweight router that maps query features to predefined combinations of evidence support forms and reasoning configurations. While this is a reasonable engineering approach, the paper provides limited insight into the fundamental question motivating the method: what properties of a query determine whether external evidence is necessary, whether compressed evidence is sufficient, and how much reasoning is required?

Although a router is obtained through KL distillation and reinforcement learning exploration, the paper provides no insight into why a particular question is better answered directly by the model, with reference to raw text, or with reference to a summary. This may be the scientific question worth exploring. Otherwise, the method appears to be a classifier that distinguishes between knowledge already possessed by the current model, knowledge that can be summarized from the dataset, and knowledge contained in the raw text of the database.

  1. Narrow task generality and limited evidence for broader agent adaptation.

Although the paper evaluates five benchmarks, they all belong to a relatively narrow family of knowledge-intensive question answering and fact verification. The method operates as a single-step, pre-answer router over three predefined evidence forms, rather than performing adaptive retrieval, iterative search, or sequential agent decision-making.

Although the paper focuses on QA-type benchmarks, today's QA benchmarks have also been widely synthesized and extended, for example, to involve multi-hop reasoning or parallel exploration of multiple questions. Despite conducting many ablation experiments, the paper falls short of proposing a method that can be more broadly applied to QA-type tasks.

Questions for authors

Q1. What generalizable query properties does FORGE actually learn?

Q2. What is the specific advantage of joint evidence and reasoning optimization?

Reviewer_rWkfOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

FORGE trains a small router for frozen language models, which selects both the form of evidence and the reasoning configuration for a given question. The method first learns from the results of offline usage, then optimizes its strategy based on answer quality and token costs. Experiments demonstrate that Forge improves the trade-off between quality and cost in Q&A tasks.

Critical strengths
  1. The experiments are quite comprehensive. The experimenal results demonstrate evidence supporting the joint selection of evidence and reasoning configurations.
  2. The cost report provides a clear breakdown, distinguishing between the overhead of cache hits and that of new queries. The Lite Forge router retains some of performance gains even after the removal of additional probes.
Critical weaknesses
  1. The novelty is limited. Existing work has already explored training routers to select evidence or allocate computational resources optimally. This paper primarily focuses on jointly selecting the form of evidence and the reasoning configuration. I believe this modification does not offer a sufficiently significant new contribution.
  2. Its practical value is unclear. The paper does not fully state what specific advantages the training of this "router" offers over approaches like direct retrieval or the utilization of memory.
  3. Significant preparation and cost is required before use. Users must curate representative datasets of questions and evaluations, and also bear the costs associated with offline training. As the appendix says, the cost for training Forge is expensive. When adding this part to the cost, the advanatage of optimizing cost is minimal.
  4. How subsequent updates should be handled? Existing routing strategies may no longer be suitable for the new information or requirements. The paper does not clearly specify when updates are necessary, how they should be performed, or the associated maintenance costs. If retraining the router frequently, the cost will be significantly increase. So, the assumption of the Forge is that they do not update information, which I believe is not practical.
Questions for authors
  1. Compared to existing methods for evidence routing and reasoning budget allocation, what is the primary new contribution of FORGE?
  2. Given a set of Q&A, why choose to train a router using this data rather than considering them as existing data to be retrieved when encountering new questions?
  3. How to start using Forge for a new application that lacks historical queries or ground-truth answers?
  4. Given the investment required for offline data collection and training, for what use cases is FORGE best suited?
  5. When the knowledge base or user requirements change, in in which cases is it necessary to retrain the router?
Reviewer_U7yYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper adapts frozen, closed-weight LLMs without weight access. A ~269K-parameter router jointly selects, per query, a support form and a thinking depth before the host answers. The utility is F1 minus token cost; entropy regularization gives a Boltzmann soft target. Training proceeds in three stages: offline enumeration, KL distillation, and GRPO refinement. On Qwen3-8B and Mistral-7B, FORGE beats Always-Raw by 2-3 F1 at 42-45% lower cached token cost, and it transfers zero-shot to larger hosts.

Critical strengths
  • The problem is practical and well motivated; the Figure 2 probe is convincing.
  • Cost accounting is careful: Cached and Fresh Online costs are separated, and latency is reported.
  • A matched six-arm Stage-1 control isolates GRPO’s contribution, with confidence intervals.
  • The proxy-reward experiment shows most gains survive without gold labels.
Critical weaknesses
  • Limited novelty. Both axes are well studied (Adaptive-RAG, TierMem, RECOMP, AdaReasoner), and the paper mainly combines them. The Boltzmann solution and Propositions 1–3 are standard results. Related work omits RECOMP and similar selective-compression methods.
  • Missing key baselines. The thinking axis is compared only against a fixed maximum budget, not against the host’s native adaptive reasoning. There is also no comparison against two independently trained routers composed together, so the need for a joint decision is untested.
  • Questionable Stage 2 gains. At τ=1 the soft target is nearly uniform, and rewards are deterministic. GRPO uses 1.28M calls where full enumeration needs ~9K. A τ-tuned Stage 1 control is needed.
  • Consistency issues. Stage 2 on Claude and DeepSeek-R1 appears infeasible under the API rate limit. The 6-arm results on Qwen3-8B-Thinking are identical to the non-thinking host, yet the Raw baselines differ. Possibly per-task routing. A best-fixed-arm-per-task baseline and leave-one-task-out evaluation are needed.
  • Misleading cost reporting. The abstract’s 42–45% savings use cached accounting; under Fresh Online accounting, savings drop to ~11% and latency doubles.
Questions for authors
  • How were the 12-arm routers for Claude and DeepSeek-R1 trained under the API rate limit? Was Stage 2 run on these hosts, or transferred from Qwen3-8B-Thinking?
  • How do a τ-tuned Stage 1 and hard-label distillation compare with Full FORGE?
  • Why are the 6-arm results on Qwen3-8B-Thinking identical to the non-thinking host when the Raw baselines differ?
  • How much does the 12-arm router gain over the host’s native adaptive reasoning (no forced budget)?
  • What do a best-fixed-arm-per-task baseline and leave-one-task-out evaluation give?
  • How is the CoT-Prompt answer extracted under the 64-token output cap, and how often is it truncated?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.