- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies per-query allocation of reasoning effort and retrieval depth in single-turn RAG under a single expected token budget. PACO-PO is a ~0.2M-parameter controller (base LLM Llama-3.1-8B-Instruct and BGE retriever both frozen) that picks one of 12 (effort, depth) cells: reasoning caps of 128/1024/4096 tokens x k {0, 3, 5, 10} passages. It reads a MiniLM query embedding plus a "retrieval preview" (top-10 retriever score statistics and passage lengths) that costs no LLM tokens. The policy is factorized as p(e)·p(k|e), framed as a one-step constrained MDP, and trained with GRPO, where a single Lagrangian dual variable prices the budget (B = 0.65x the cost of the best fixed cell). On HotpotQA, NQ and MATH-500 (with retrieval over MATH training solutions), PACO-PO reaches 46.9% macro accuracy at 0.64 of the best fixed cell's cost (47.6%). It also beats a confidence cascade, a difficulty router and a LinUCB bandit (44.4–45.5%), which spend more (0.71–0.76). Ablations cover single-axis controllers, a flat 12-way head, removing the preview, a frozen price, a tuned scalarization weight, and transfer to Qwen2.5-7B.
Critical strengths
- The problem is of interest to atleast one sub-field of researchers as reasoning tokens and retrieved context draw on the same bill, and letting the deployer set a budget instead of an exchange-rate weight is a sensible interface.
- Each component (factorized head, score-profile features, primal–dual pricing on top of GRPO) is simple, clearly motivated and cheap with the controller having ~0.2M parameters and the base LLM and retriever staying frozen.
- PACO-PO uses ~36% fewer tokens than the best fixed cell for a 0.7-point macro accuracy drop. On HotpotQA it is 4.8 points above the best budget-matched random mix of fixed cells (Table 2). The savings over the simplest adaptive baseline are smaller, though (see Weakness 3).
- Well organized ablations. Table 3 changes one thing per row at the same budget. The frozen-λ row shows that budget tracking depends on the price moving during training, not only on where it ends up.
Critical weaknesses
- The methodological contribution is incremental. Primal–dual Lagrangian relaxation for constrained RL is standard. GRPO is used off the shelf. The novelty therefore lies mainly in applying these tools to a two-dimensional action grid.
- The experiments are too narrow. Three datasets seem too few for claims about general per-query allocation, and two of them (HotpotQA, NQ) are Wikipedia QA over the same index.
- Each baseline is shown at a single operating point that costs more than PACO-PO (0.71–0.76 vs. 0.64). Sweeping each baseline's knob (cascade threshold, router mapping) and comparing Pareto curves, or comparing at matched cost, would be fairer. It is also unclear what the cascade threshold and router mapping were tuned for on the probe split.
- PACO-PO adds components to the pipeline including a retriever call on every query (including queries that end up at k = 0), a MiniLM encoder and the controller. End-to-end serving latency should therefore be measured, and I could not find it in the paper.
- No measure of statistical uncertainty. The ablations are also single-seed.
- The paper also overstates the claim that prior work treats the two axes separately. Yue et al. (Inference Scaling for Long-Context Retrieval Augmented Generation) jointly allocate the number of retrieved documents and the inference iterations under a compute budget, using a computation-allocation model.
Questions for authors
- The controller trains on MATH training problems, and the retrieval corpus contains MATH training solutions. Unless self-matches are removed, retrieval during training returns each query's own solution, which cannot happen on MATH-500 at evaluation. Please clarify.
- Normalization, budget and training pool all appear to be set per benchmark. Real deployments see a mixed query stream. Training a single controller on the union and evaluating on a mixture, or across datasets, would test the routing claim much better. More multi-hop or knowledge-intensive sets (e.g., 2WikiMultiHopQA, MuSiQue, PopQA) and a non-math reasoning set will also help.
- Every experiment uses B = 0.65. The method's interface is "the deployer sets a budget", so it needs accuracy–cost curves across several budgets, with baselines at matched cost. All experiments use 7–8B instruct models, with reasoning effort set by a prompt instruction plus a hard stop. Experiments with broader set of reasoning models with reasoning effort controls will help.
- Please give the split of savings into input vs. output tokens, plus p50/p95 latency and throughput under batched serving (vLLM is already in the stack). This would considerably strengthen the practical claim, and it could favor PACO-PO over the cascade, which needs several LLM calls.
5 comments
(Continued)
Summarized W4: Serving latency and the input/output split (Weakness 4, Question 4)
A4: As you suggested, we further report the input and output tokens per query and a modeled token-processing latency at batch 1. Table R3 shows that the savings come from both input and output tokens, as PACO-PO saves 474 input and 87 output tokens per query against the best fixed cell. The modeled latency covers token processing and the controller overhead, while the retriever call, which is part of every action with k > 0 and of every retrieving baseline and adds a call for the queries routed to k = 0, the MiniLM encoding and the cascade round trips come on top of it, and end-to-end latency, latency percentiles and batched throughput are not reported.
Table R3: HotpotQA mean input and output tokens per query and modeled token-processing latency per query at batch 1.
| Method | Input tokens | Output tokens | Modeled latency (s) |
|---|---|---|---|
| Best fixed cell (HIGH, 10) | 1301 | 314 | 8.5 |
| Confidence cascade | 764 | 448 | 11.9 |
| PACO-PO (Ours) | 827 (-474) | 227 (-87) | 6.0 |
Following your suggestion, we have added these results to Appendix A.9 as Table 21 (Lines 1747–1753).
Summarized W5: Statistical uncertainty (Weakness 5)
A5: We respectfully clarify that an upper bound on the binomial standard error of a single accuracy estimate, which we compute for this response, is 1.58 points per QA benchmark, 2.24 points on MATH-500 and 1.05 points for the macro. Following your suggestion, we further report the across-seed standard deviations of the main comparison in Table R4, where the macro accuracy of PACO-PO is 47.1 ± 0.8, and of the ablation rows in Table R5, whose learned rows are three-seed means in the revised version.
Table R4: Accuracy (%) of the five methods of Table 1 over three seeds as mean ± standard deviation.
| Method | HotpotQA | NQ | MATH-500 | Macro | Macro diff. to PACO-PO |
|---|---|---|---|---|---|
| PACO-PO (Ours) | 47.6 ± 2.0 | 42.5 ± 2.2 | 51.1 ± 3.9 | 47.1 ± 0.8 | (ref.) |
| Best fixed cell | 48.4 ± 0.0 | 42.7 ± 0.0 | 52.0 ± 0.0 | 47.7 ± 0.0 | 0.6 |
| Confidence cascade | 47.4 ± 2.1 | 41.6 ± 1.1 | 45.9 ± 1.6 | 45.0 ± 1.2 | -2.1 |
| Difficulty router | 46.5 ± 1.2 | 39.2 ± 1.7 | 50.8 ± 1.2 | 45.5 ± 0.8 | -1.6 |
| Bandit (LinUCB [2]) | 46.8 ± 1.1 | 37.0 ± 2.2 | 49.4 ± 1.8 | 44.4 ± 0.8 | -2.7 |
Table R5: Macro accuracy (%) of the ablation rows of Table 3 over three seeds as mean ± standard deviation, where the parameter-matched flat head is printed without its spread, and after 10 and 20 of the 40 outer iterations.
| Configuration | Macro acc. (40 it.) | After 10 it. | After 20 it. |
|---|---|---|---|
| PACO-PO (Ours, full) | 47.1 ± 0.8 | 44.9 | 45.5 |
| Flat 12-way head | 45.4 ± 0.4 (-1.7) | 43.0 | 43.8 |
| Flat 12-way head, parameter-matched | 45.6 (-1.5) | 42.1 | 43.4 |
| Without retrieval preview | 45.0 ± 1.5 | 41.9 | 43.6 |
| Learned depth only | 43.0 ± 2.1 | 39.9 | 42.5 |
| Learned effort only | 44.0 ± 1.0 | 41.7 | 43.3 |
| lambda frozen | 44.9 ± 0.3 | 42.1 | 43.2 |
Following your suggestion, we have added these results to Appendix A.9 as Tables 13 and 15 (Lines 1649–1658 and 1674–1684), summarized the seed spread in Section 5.3 (Lines 451–453), and updated the Limitations in Appendix A.0 (Lines 841–854).
Summarized W6: Positioning against Yue et al. (Weakness 6)
A6: We respectfully clarify that our setting differs from that of Yue et al. [3] in the per-query choice. Yue et al. allocate retrieved documents and generation iterations under a compute budget at the level of a task, whereas PACO-PO chooses for each query from a token-free retrieval preview under a budget that charges the generated reasoning tokens together with the retrieved context. The separation our paper describes is between per-query controllers of reasoning length and of retrieval depth.
Following your suggestion, we have limited the claimed separation to per-query controllers in the abstract (Lines 14–33), the caption of Figure 1 (Lines 46–50) and the first contribution (Lines 79–81), revised the related work on adaptive retrieval and reasoning effort (Lines 460–466), and added Yue et al. [3] to the related work of Appendix A.1 (Lines 884–901).
(Continued)
Summarized W7: Self-matches in the MATH reference corpus (Question 1)
A7: We respectfully clarify that with each training problem's own solution excluded from retrieval, PACO-PO keeps its MATH-500 accuracy of 51.1%. The training, probe and evaluation pools are disjoint, and the MATH-500 retrieval corpus holds the MATH training solutions. In the added run that removes each training problem's own solution from retrieval, Table R1 above shows 51.1% at a cost of 0.68 for PACO-PO, against 51.1% at 0.67 in Table 1, and 50.2% for the router.
Following your suggestion, we have added this result to Appendix A.9 as part of Table 17 (Lines 1703–1714), and added the self-match paragraph to the Limitations in Appendix A.0 (Lines 831–834).
Summarized W8: Several budgets and other reasoning models (Question 3)
A8: We respectfully clarify that a new budget needs a new training run under our training procedure, because the controller does not read B, and budget conditioning is the direction that removes this. Table R2 above gives HotpotQA at B = 0.5, 0.65 and 0.8 with the cascade and the router read near the cost of PACO-PO, and Table R1 above gives a recalibrated Qwen2.5-7B controller. The tables contain no model family with native effort control.
These results are in Appendix A.9 as Tables 16 and 17 (Lines 1690–1698 and 1703–1714).
References
[1] Zhihong Shao et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. 2024.
[2] Lihong Li et al. A contextual-bandit approach to personalized news article recommendation. WWW, 2010.
[3] Zhenrui Yue et al. Inference scaling for long-context retrieval augmented generation. ICLR, 2025.
Best Wishes,
The Authors of Paper 52793
Response to Reviewer a5Rx
Dear Reviewer a5Rx:
Thank you for recognizing the task value, the method design, and the method effectiveness. We have responded point by point to your concerns regarding the contribution, more datasets, baselines at matched cost, latency, statistical uncertainty, positioning, self-matches, and multiple budgets. In the marked copy appended to the revised PDF (pages 34–67), the revisions made in response to your comments are tagged with @Reviewer a5Rx. We hope our responses resolve your concerns and look forward to further discussion.
Summarized W1: Incremental methodology (Weakness 1)
A1: We respectfully clarify that the contribution is the problem and its formulation, while primal-dual relaxation and GRPO [1] are standard tools that PACO-PO runs on a one-step constrained problem. First, one expected token budget charges generated reasoning and retrieved context together, which couples effort and depth through cost. Second, a featurization is read from the retriever's scores and the index metadata before any passage enters the context. Third, the constraint is written relative to B, which lets one dimensionless price and one step size serve benchmarks with different token scales. Proposition 1 shows that the dual function is finite and convex without concavity of the inner problem, which a neural controller does not have.
Summarized W2: Three datasets, two over one index (Weakness 2, Question 2)
A2: Following your suggestion, we further evaluate PACO-PO and the difficulty router on 2WikiMultiHopQA, MuSiQue and PopQA, and on a mixed stream of the three benchmarks with one controller trained on the union of their training splits under one normalization and one budget. Table R1 shows that PACO-PO is more accurate than the router on the mixed stream at a lower cost, with 46.2% at 0.66 against 44.5% at 0.71. HotpotQA and NQ both retrieve from a Wikipedia index, which holds the retrieval stack fixed while the query type changes from multi-hop to single-hop, and the revised version contains no non-math reasoning set.
Table R1: Accuracy (%) and cost fraction beyond the three benchmarks, with PACO-PO and router accuracy as three-seed means and the recalibrated Qwen2.5-7B controller as a one-seed macro.
Following your suggestion, we have added these results to Appendix A.9 as Table 17 (Lines 1703–1714), and summarized them in Section 5.3 (Lines 454–457).
Summarized W3: Baselines at a single operating point (Weakness 3)
A3: Following your suggestion, we further read the cascade and the router at the cost of PACO-PO for three budgets on HotpotQA, each at its setting whose probe cost lies nearest that cost, beside the best mix of all 12 fixed cells. Table R2 shows that PACO-PO is above both baselines at each budget, with 47.6% against 45.8% for the router at B = 0.65 and 48.1% against 48.0% for the router and 47.7% for the cascade at B = 0.8, where the evaluation cost of each baseline is not printed and need not equal that of PACO-PO, and by our computation for this response each entry carries a binomial standard error of at most 1.58 points. On the probe split, the cascade threshold and the router mapping are tuned for the highest probe accuracy among the settings whose probe cost fraction is at most 0.85, and none of the three baselines reads B.
Table R2: HotpotQA accuracy (%) at three budget fractions, with the cascade and the router read at the setting whose probe cost lies nearest the cost of PACO-PO.
Following your suggestion, we have added these results to Appendix A.9 as Table 16 (Lines 1690–1698), and described the baseline implementations and how the cascade threshold and the router mapping are tuned in Appendix A.4 (Lines 1191–1205 and 1207–1216).