acceptodds
Under review as a conference paper at ICLR 2027

Confidence-Guided Query Refinement for Search-Augmented Reasoning

Abstract

Search-augmented reasoning enables language-model agents to acquire missing knowledge, but retrieval can fail when an agent generates a query that poorly expresses its current information need. We study this failure mode, which we term query unreliability, and show that token-level query confidence provides a pre-retrieval signal of retrieval quality and downstream answer correctness. Based on this observation, we introduce Confidence-Guided Query Refinement (QCQR), a post-hoc test-time method that executes confident queries unchanged while selectively rewriting, decomposing, or disambiguating low-confidence queries using an off-the-shelf reformulator. QCQR requires no additional training of the search agent or retriever. Across heterogeneous search agents and knowledge-intensive QA benchmarks, QCQR improves average answer accuracy by up to up to 6.18 percentage points, improves gold-document retrieval and downstream calibration, and requires a comparable number of external search calls. Unconditional and random refinement do not yield consistent gains, highlighting the importance of selectively intervening on uncertain queries. Our results establish query confidence as a practical signal for repairing unreliable search actions before retrieval.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agentic Search, Model Calibration, Uncertainty, Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_2hkSOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a method to rewrite, decompose, or disambiguate low-confidence queries to improve retrieval in search agents.

Critical strengths

The method is simple and training-free, which could help improve performance.

Critical weaknesses
  • The evaluation is limited. They use saturated benchmarks (back in 2018) to support their claims. In addition, they only use a subset of 200 examples and run the evaluation once, so the results are within noise. We do not know whether the improvements are real. The paper also does not state the backbone model used in the experiments, making their setup unclear.
  • Unclear how the mechanism helps improve retrieval. The paper does not give information about how frequently the agent rewrites its queries, and the effect on those questions.
  • The method can be replaced by training RL on the backbone, allowing it to make clearer queries.
Questions for authors
  • What model backbone did you use in the experiments?
  • How often the agent has to rewrite their queries?
  • Why did you use different seeds (42/40)?
6 comments

Part 1 of 2

W1/Q1. Evaluation scope, multi-seed results, and backbone specifications

We thank the reviewer for raising these concerns. We use the standard NQ, HotpotQA, 2Wiki, and MuSiQue benchmarks with the Wikipedia-2018 corpus to maintain a controlled comparison with established search agents works (Search-R1, ReSearch, R1-Searcher, HIPRAG). The 2018 date refers to the Wikipedia retrieval corpus, rather than the release date of all evaluation benchmarks. These experiments assess refinement under a controlled retrieval environment. The paper additionally evaluates on GAIA and WebWalkerQA using web search, providing evidence beyond the fixed Wikipedia index

To assess stability, we repeat QR (First) and QR (All) using seeds 40, 41, and 42 and report the mean ±\pm sample standard deviation:

MethodNQHotpotQA2WikiMuSiQueAverage
Search-R148.440.435.025.037.20
+QR (First)53.80±2.6553.80 \pm 2.6545.07±3.1445.07 \pm 3.1441.93±4.6241.93 \pm 4.6229.33±3.4029.33 \pm 3.4042.53±3.3742.53 \pm 3.37
+QR (All)54.00±2.50\mathbf{54.00 \pm 2.50}46.00±3.49\mathbf{46.00 \pm 3.49}43.73±6.13\mathbf{43.73 \pm 6.13}30.67±3.97\mathbf{30.67 \pm 3.97}43.60±3.97\mathbf{43.60 \pm 3.97}
R1-Searcher55.560.052.030.049.38
+QR (First)56.86±0.8056.86 \pm 0.8061.52±0.9061.52 \pm 0.9052.33±0.2952.33 \pm 0.2930.83±0.2930.83 \pm 0.2950.39±0.4750.39 \pm 0.47
+QR (All)58.43±1.11\mathbf{58.43 \pm 1.11}63.38±1.52\mathbf{63.38 \pm 1.52}52.67±0.76\mathbf{52.67 \pm 0.76}32.50±0.00\mathbf{32.50 \pm 0.00}51.74±0.84\mathbf{51.74 \pm 0.84}
HiPRAG61.550.058.524.448.60
+QR (First)65.00±1.8065.00 \pm 1.8057.00±5.2757.00 \pm 5.2766.33±4.5466.33 \pm 4.5430.84±4.7530.84 \pm 4.7554.79±3.7654.79 \pm 3.76
+QR (All)66.33±1.26\mathbf{66.33 \pm 1.26}58.17±4.25\mathbf{58.17 \pm 4.25}68.67±6.53\mathbf{68.67 \pm 6.53}32.58±4.85\mathbf{32.58 \pm 4.85}56.44±3.80\mathbf{56.44 \pm 3.80}
ReSearch53.061.953.037.551.35
+QR (First)58.75±3.8958.75 \pm 3.8966.37±1.1966.37 \pm 1.1959.00±4.3359.00 \pm 4.3343.33±2.0243.33 \pm 2.0256.32±3.2856.32 \pm 3.28
+QR (All)59.67±3.06\mathbf{59.67 \pm 3.06}70.82±0.62\mathbf{70.82 \pm 0.62}61.50±4.27\mathbf{61.50 \pm 4.27}46.67±3.25\mathbf{46.67 \pm 3.25}59.66±1.88\mathbf{59.66 \pm 1.88}

We also clarify the backbone configurations:

AgentBackboneParametersCheckpoint
Search-R1Qwen2.5-3B-Instruct3BSearch-R1 3B, GRPO v0.3
HiPRAGQwen2.5-3B-Instruct3BHiPRAG 3B, PPO
ReSearchQwen2.5-7B-Instruct7BReSearch 7B-Instruct
R1-SearcherQwen2.5-7B Base7BQwen-2.5-7B-base-RAG-RL

The Direct, CoT, RAG, and IRCoT baselines use Qwen2.5-3B-Instruct (3B parameters). We will add these specifications and the multi-seed results to the revised manuscript.

Part 2 of 2

W2/Q2. Query-refinement mechanism, intervention frequency, and effects on retrieval

Mechanism. QC$\rightarrow$QR intervenes before retrieval when the minimum query-token confidence falls below the agent-specific threshold. It generates rewrite, decompose, and disambiguate candidates, selects the candidate with the highest minimum-token confidence, and submits that query to the same retriever. The agent then continues reasoning using the retrieved evidence. Thus, the intervention changes the search query while keeping the retriever and search-agent weights fixed.

Refinement frequency. Appendix G.2, Table 12 reports the following intervention rates:

AgentSearch queries refined
Search-R131.30%
R1-Searcher11.23%
HiPRAG47.78%
ReSearch25.37%

These percentages refer to search queries rather than questions because a question may involve multiple search queries.

Effect on retrieval. The paper provides two complementary analyses. First, below-threshold queries have lower gold-document Hit@1 than above-threshold queries:

AgentBelow thresholdAbove threshold
Search-R140.4%48.7%
R1-Searcher34.7%49.4%
HiPRAG20.0%41.7%
ReSearch36.8%42.4%

These results support using confidence to identify queries associated with weaker retrieval.

Second, Table 3 compares retrieval before and after QC$\rightarrow$QR. Gold-document Hit@1 improves across all four agents and all three multi-hop datasets:

AgentHotpotQA Hit@12Wiki Hit@1MuSiQue Hit@1
Search-R162.80 →\rightarrow 70.2063.20 →\rightarrow 69.0049.00 →\rightarrow 58.00
R1-Searcher74.00 →\rightarrow 75.0076.50 →\rightarrow 78.5061.50 →\rightarrow 63.50
HiPRAG65.50 →\rightarrow 73.0063.50 →\rightarrow 69.5045.19 →\rightarrow 53.33
ReSearch79.38 →\rightarrow 83.5178.00 →\rightarrow 80.0059.50 →\rightarrow 62.00

Hit@3 also improves in every reported comparison, by 1.00–7.41 percentage points. Table 5 provides complementary evidence of reduced repetitive retrieval; for example, Search-R1's NovelRate@1 on NQ increases from 72.81% to 88.70%.

W3. Inference-time query repair versus additional reinforcement learning

Our motivation, however, is to improve already-trained search agents without requiring another computationally expensive training cycle for each backbone.

As discussed in the Introduction and Appendix D, training-based approaches such as Search-R1, ReSearch, R1-Searcher, and HiPRAG modify the agent's parameters. QC$\rightarrow$QR instead operates entirely at inference time: it uses the agent's existing token probabilities to identify uncertain search queries and an off-the-shelf reformulator to repair them before retrieval. Neither the search agent nor the retriever is retrained, and no additional reward-model training or RL rollouts are required.

This distinction is useful when deploying existing checkpoints or applying query refinement across different agents. We evaluate the same intervention framework on Search-R1, R1-Searcher, ReSearch, and HiPRAG, covering agents trained with both outcome-based and process-based rewards. The improvements on HiPRAG further suggest that inference-time refinement can provide benefits even after process-supervised RL training.

Q3. Consistent random seeds across experiments

We use the same three seeds 40, 41, and 42 for all experiments and report the mean ±\pm sample standard deviation across these runs.

We sincerely thank the reviewer again for the careful and constructive review. We hope that the clarifications and additional experiments address the concerns raised, and we would be happy to provide any further clarification. If the concerns have been satisfactorily addressed, we would be grateful if the reviewer could kindly reconsider the evaluation score.

I appreciate the authors' responses and clarification. They addressed my concern about the results' significant. However, it's still unclear:

  • Where the gains come from: in the Hit@1 improvement, how often does it turn incorrect answers into correct?
  • In addition, how you decide the threshold λ\lambda? Why do we need rewriting instead of making a model issue effective queries at first?
  • How often the refinement at example-level?
Reviewer_fUAKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies unreliable search queries and proposes QC→QR, which selectively rewrites, decomposes, or disambiguates queries using minimum token confidence. Without retraining the search agent or retriever, it improves average answer accuracy by 1.78–6.18 percentage points across four agents, with supporting retrieval-quality and cost analyses.

Critical strengths
  1. Repairing queries before execution is practically useful. Gains across four agents and additional open-web benchmarks support applicability (Tables 1 and 4).
  2. Intervention-budget-matched random refinement and unconditional refinement controls support selective intervention, rather than attributing all gains to extra generation (Table 2).
Critical weaknesses
  1. The methodological novelty is limited: minimum-token confidence and the RQ-RAG reformulation operations/checkpoint build on prior work (Sections 2–4). The contribution is primarily their integration into selective query repair. The weakest-link rationale is heuristic; Figure 3 links confidence to retrieval quality, but does not directly establish which queries benefit most from repair. Existing controls also leave the contribution of three-candidate confidence ranking unisolated.
  2. Similar search-call counts do not establish equal total cost. Refinement adds 0.76–2.63 generations per question. For Search-R1/HotpotQA, estimated runtime rises from 17 to 21.10 seconds (+24%) for a 3.2-point accuracy gain (Tables 1, 12–14). Runtime coverage is narrow, and comparison with alternative uses of the same total inference budget is missing.
  3. Evaluation clarity remains an issue: 65/200 HiPRAG–MuSiQue cases are excluded, with shared post-failure masks unclear (Appendix E.1); Table 3’s per-question retrieval metric conflicts with Section 5.2’s query-level description.
Questions for authors
  1. With the same candidate pool, how much does confidence ranking improve over random candidate selection? Does low query confidence predict larger repair gains after accounting for query length?
  2. Does QC→QR outperform additional agent sampling/search at matched total inference cost, including reformulator computation?
  3. Please clarify exclusion causes, common evaluation masks, and retrieval aggregation across turns, and provide paired uncertainty on the common set.
4 comments

Part 1 of 2

W1/Q1. Methodological contribution and confidence-based candidate selection

We agree that minimum-token confidence and the RQ-RAG refinement operations are inherited from prior work. Our contribution is their integration into a training-free, inference-time framework that uses an agent's query confidence to decide when to invoke query repair and then selects among multiple refinement operations. This enables existing search agents to improve without additional reinforcement learning or model training.

To examine the contribution of the refinement operation, we compare fixed rewrite, decompose, and disambiguate strategies with random action selection:

AgentDatasetRewriteDecomposeDisambiguateRandom+QR (All)
Search-R1NQ47.5047.0045.5047.0053.20
Search-R1HotpotQA39.5040.5037.5043.0043.60
Search-R12Wiki37.0034.0037.5036.5038.80
Search-R1MuSiQue17.5017.5013.5016.5027.80
ReSearchNQ52.5053.0050.5051.5059.00
ReSearchHotpotQA54.6449.4856.7054.6470.10
ReSearch2Wiki41.0041.0036.5040.5057.50
ReSearchMuSiQue35.0029.0033.0034.0043.50
R1-SearcherNQ47.6449.2148.6947.1257.60
R1-SearcherHotpotQA60.9060.9059.6260.2662.50
R1-Searcher2Wiki51.0051.5051.5051.0052.00
R1-SearcherMuSiQue36.5036.0035.5037.0032.50

Low-confidence queries benefit more from refinement, whereas refining high-confidence queries can degrade performance, as seen from refine all operations (Tab. 3).

W2/Q2. Total inference cost and comparison with additional agent sampling

We thank the reviewer for this suggestion. We compare QC$\rightarrow$QR (All) against additional agent sampling with five voting samples. Additional agent sampling generates five independent trajectories for each question and selects the final answer through majority voting across the five outputs.

AgentMethodNQHotpotQA2Wiki
Search-R1Additional sampling + voting (5)47.0044.0040.50
+QC$\rightarrow$QR (All)53.2043.6038.80
ReSearchAdditional sampling + voting (5)52.5059.0059.64
+QC$\rightarrow$QR (All)59.0070.1057.50
R1-SearcherAdditional sampling + voting (5)51.0053.5040.21
+QC$\rightarrow$QR (All)57.6062.5052.00
HiPRAGAdditional sampling + voting (5)55.0052.0056.00
+QC$\rightarrow$QR (All)65.0055.0066.50

Part 2 of 2

W3/Q3. Exclusion causes, shared evaluation masks, retrieval aggregation, and multi-seed uncertainty

We thank the reviewer for raising these concerns. For HiPRAG–MuSiQue, 65 of 200 questions were excluded because endpoint failures and timeouts prevented complete trajectory recording. Method-specific failures and invalid outputs were excluded rather than scored as incorrect. All compared methods use the same remaining 135 question IDs, ensuring a shared evaluation mask.

Retrieval is aggregated at the question level: Hit@3 equals one if any search turn retrieves a gold document among its top three results. Table 3 reports this question-level metric, whereas Section 5.2 presents query-level analysis. Both descriptions are correct but refer to different levels of analysis: Table 3 reports question-level retrieval performance, whereas Section 5.2 presents query-level retrieval analysis.

We repeat QR (First) and QR (All) using seeds 40, 41, and 42 and report the mean ±\pm sample standard deviation on the shared evaluation set.

MethodNQHotpotQA2WikiMuSiQueAverage
Search-R148.440.435.025.037.20
+QR (First)53.80±2.6553.80 \pm 2.6545.07±3.1445.07 \pm 3.1441.93±4.6241.93 \pm 4.6229.33±3.4029.33 \pm 3.4042.53±3.3742.53 \pm 3.37
+QR (All)54.00±2.50\mathbf{54.00 \pm 2.50}46.00±3.49\mathbf{46.00 \pm 3.49}43.73±6.13\mathbf{43.73 \pm 6.13}30.67±3.97\mathbf{30.67 \pm 3.97}43.60±3.97\mathbf{43.60 \pm 3.97}
R1-Searcher55.560.052.030.049.38
+QR (First)56.86±0.8056.86 \pm 0.8061.52±0.9061.52 \pm 0.9052.33±0.2952.33 \pm 0.2930.83±0.2930.83 \pm 0.2950.39±0.4750.39 \pm 0.47
+QR (All)58.43±1.11\mathbf{58.43 \pm 1.11}63.38±1.52\mathbf{63.38 \pm 1.52}52.67±0.76\mathbf{52.67 \pm 0.76}32.50±0.00\mathbf{32.50 \pm 0.00}51.74±0.84\mathbf{51.74 \pm 0.84}
HiPRAG61.550.058.524.448.60
+QR (First)65.00±1.8065.00 \pm 1.8057.00±5.2757.00 \pm 5.2766.33±4.5466.33 \pm 4.5430.84±4.7530.84 \pm 4.7554.79±3.7654.79 \pm 3.76
+QR (All)66.33±1.26\mathbf{66.33 \pm 1.26}58.17±4.25\mathbf{58.17 \pm 4.25}68.67±6.53\mathbf{68.67 \pm 6.53}32.58±4.85\mathbf{32.58 \pm 4.85}56.44±3.80\mathbf{56.44 \pm 3.80}
ReSearch53.061.953.037.551.35
+QR (First)58.75±3.8958.75 \pm 3.8966.37±1.1966.37 \pm 1.1959.00±4.3359.00 \pm 4.3343.33±2.0243.33 \pm 2.0256.32±3.2856.32 \pm 3.28
+QR (All)59.67±3.06\mathbf{59.67 \pm 3.06}70.82±0.62\mathbf{70.82 \pm 0.62}61.50±4.27\mathbf{61.50 \pm 4.27}46.67±3.25\mathbf{46.67 \pm 3.25}59.66±1.88\mathbf{59.66 \pm 1.88}

We sincerely thank the reviewer again for the careful and constructive review. We hope that the clarifications and additional experiments address the concerns raised, and we would be happy to provide any further clarification. If the concerns have been satisfactorily addressed, we would be grateful if the reviewer could kindly reconsider the evaluation score.

AuthorsOpenReview ↗

Follow-up on the Rebuttal

Dear Reviewer fUAK,

Thank you again for your review. We believe that our response addresses the concerns raised in your review. As the discussion period is approaching its end, we would appreciate it if you could take a look and let us know if any concerns remain or if any further clarification would be helpful.

Best regards,

The Authors

Dear Reviewer fUAK,

We believe that our response addresses the concerns raised in your review. As the discussion period is approaching its end, we would appreciate it if you could take a look and let us know if any concerns remain or if any further clarification would be helpful. If the concerns have been satisfactorily addressed, we would be grateful if the reviewer could kindly reconsider the evaluation score.

Best regards,

The Authors

Reviewer_xwPWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether an already-generated search query should be repaired before retrieval. QC→QR uses the minimum query-token probability to flag uncertain queries, generates rewrite, decomposition, and disambiguation candidates with an off-the-shelf reformulator, and executes its highest-confidence candidate. Neither the agent nor the retriever is retrained. Experiments cover four search agents on four QA benchmarks, with additional open-web tests.

Critical strengths

I like the clear separation between deciding to search and checking whether the query expresses the intended information need. The method is simple to add to existing agents, and Table 1 reports improvements across all four agents, including the process-supervised HiPRAG. The average gains range from 1.78 to 6.18 points. The unconditional, random-budget, and rejection-sampling controls are useful attempts to distinguish selective repair from simply adding a reformulator. The retrieval and calibration analyses provide supporting evidence beyond answer accuracy. The paper also reports reformulator generations and runtime rather than treating unchanged search-call counts as unchanged total cost.

Critical weaknesses

The comparisons need stronger uncertainty estimates and clearer handling of failed runs. Main runs use seed 42 while controls use seed 40. Appendix G.1 is helpful, but repeats only the two QR variants for Search-R1 and HiPRAG, not the original agents or controls. It therefore does not establish how stable the comparative gains are, especially the smaller improvements. Moreover, HiPRAG's MuSiQue evaluation excludes 65 of 200 questions; the paper does not clearly establish whether every compared method retains exactly the same question IDs. These issues make the size of the benefit harder to assess.

The confidence-specific design choices are not tested at the level where they operate. Appendix G.5 compares aggregations for predicting final-answer correctness from complete trajectories. That does not establish which aggregation best selects individual queries for repair, or whether the reformulator's minimum token probability reliably ranks its three candidates. The query-level retrieval correlation is encouraging, but the paper does not directly compare alternative gates or candidate selectors at matched refinement budgets.

Questions for authors

Could you report paired multi-seed comparisons of Original, QC→QR, and the key controls, with uncertainty intervals and a shared evaluation set, and clarify whether failures depend on the method? Could you also compare minimum confidence against alternative per-query aggregations at matched intervention rates, and compare confidence-based candidate selection against random selection or fixed refinement operations? These results would help establish which parts of this practical method are actually responsible for its gains.

3 comments

W1/Q1. Seed consistency and shared evaluation sets

We thank the reviewer for raising these concerns. To provide a clearer estimate of run-to-run variability, we extend the multi-seed (40,41,42) analysis to both QC$\rightarrow$QR variants for all four search agents and report the mean ±\pm sample standard deviation across the available independent runs

The repeated results show that QC$\rightarrow$QR (All) improves the average performance of Search-R1, R1-Searcher, HiPRAG, and ReSearch by 6.40, 2.36, 7.84, and 8.31 percentage points, respectively. The smaller gain for R1-Searcher should be interpreted more cautiously because it is closer to the observed run-to-run variation.

For HiPRAG on MuSiQue, 65 of the 200 questions were excluded because of endpoint failures associated with unusually long HiPRAG outputs. These cases were excluded rather than counted as incorrect. To ensure a fair comparison, we apply a shared evaluation mask: the same 135 retained question IDs are used for the original HiPRAG agent, QR (First), and QR (All). We will clarify the exclusion causes, retained sample sizes, shared question-ID masks, and number of successful runs in the revised manuscript.

MethodNQHotpotQA2WikiMuSiQueAverage
Search-R148.440.435.025.037.20
+QR (First)53.80±2.6553.80 \pm 2.6545.07±3.1445.07 \pm 3.1441.93±4.6241.93 \pm 4.6229.33±3.4029.33 \pm 3.4042.53±3.3742.53 \pm 3.37 (+5.33)
+QR (All)54.00±2.50\mathbf{54.00 \pm 2.50}46.00±3.49\mathbf{46.00 \pm 3.49}43.73±6.13\mathbf{43.73 \pm 6.13}30.67±3.97\mathbf{30.67 \pm 3.97}43.60±3.97\mathbf{43.60 \pm 3.97} (+6.40)
R1-Searcher55.560.052.030.049.38
+QR (First)56.86±0.8056.86 \pm 0.8061.52±0.9061.52 \pm 0.9052.33±0.2952.33 \pm 0.2930.83±0.2930.83 \pm 0.2950.39±0.4750.39 \pm 0.47 (+1.01)
+QR (All)58.43±1.11\mathbf{58.43 \pm 1.11}63.38±1.52\mathbf{63.38 \pm 1.52}52.67±0.76\mathbf{52.67 \pm 0.76}32.50±0.00\mathbf{32.50 \pm 0.00}51.74±0.84\mathbf{51.74 \pm 0.84} (+2.36)
HiPRAG61.550.058.524.448.60
+QR (First)65.00±1.8065.00 \pm 1.8057.00±5.2757.00 \pm 5.2766.33±4.5466.33 \pm 4.5430.84±4.7530.84 \pm 4.7554.79±3.7654.79 \pm 3.76 (+6.19)
+QR (All)66.33±1.26\mathbf{66.33 \pm 1.26}58.17±4.25\mathbf{58.17 \pm 4.25}68.67±6.53\mathbf{68.67 \pm 6.53}32.58±4.85\mathbf{32.58 \pm 4.85}56.44±3.80\mathbf{56.44 \pm 3.80} (+7.84)
ReSearch53.061.953.037.551.35
+QR (First)58.75±3.8958.75 \pm 3.8966.37±1.1966.37 \pm 1.1959.00±4.3359.00 \pm 4.3343.33±2.0243.33 \pm 2.0256.32±3.2856.32 \pm 3.28 (+4.97)
+QR (All)59.67±3.06\mathbf{59.67 \pm 3.06}70.82±0.62\mathbf{70.82 \pm 0.62}61.50±4.27\mathbf{61.50 \pm 4.27}46.67±3.25\mathbf{46.67 \pm 3.25}59.66±1.88\mathbf{59.66 \pm 1.88} (+8.31)

W2/Q2. Query-level confidence aggregation and candidate-selection ablations

We thank the reviewer for suggesting these ablations. We evaluate three fixed refinement operations—always rewriting, decomposing, or disambiguating—as well as random action selection.

AgentDatasetRewriteDecomposeDisambiguateRandom+QR (All)
Search-R1NQ47.5047.0045.5047.0053.20
Search-R1HotpotQA39.5040.5037.5043.0043.60
Search-R12Wiki37.0034.0037.5036.5038.80
Search-R1MuSiQue17.5017.5013.5016.5027.80
ReSearchNQ52.5053.0050.5051.5059.00
ReSearchHotpotQA54.6449.4856.7054.6470.10
ReSearch2Wiki41.0041.0036.5040.5057.50
ReSearchMuSiQue35.0029.0033.0034.0043.50
R1-SearcherNQ47.6449.2148.6947.1257.60
R1-SearcherHotpotQA60.9060.9059.6260.2662.50
R1-Searcher2Wiki51.0051.5051.5051.0052.00
R1-SearcherMuSiQue36.5036.0035.5037.0032.50

We also examine alternative token-level confidence aggregations. Across the four agents, the mean median-token confidence ranges from 0.94620.9462 to 0.99340.9934, the arithmetic-mean confidence ranges from 0.87700.8770 to 0.96330.9633, and the geometric-mean confidence ranges from 0.85370.8537 to 0.95640.9564. In contrast, minimum-token confidence ranges from 0.53750.5375 to 0.80520.8052 and exhibits substantially greater dispersion. Under the same numerical threshold, minimum-token confidence therefore identifies considerably more queries as low-confidence, whereas the other estimators smooth over isolated uncertain tokens.

We sincerely thank the reviewer again for the careful and constructive review. We hope that the clarifications and additional experiments address the concerns raised, and we would be happy to provide any further clarification. If the concerns have been satisfactorily addressed, we would be grateful if the reviewer could kindly reconsider the evaluation score.

AuthorsOpenReview ↗

Follow-up on the Rebuttal

Dear Reviewer xwPW,

Thank you again for your review. We believe that our response addresses the concerns raised in your review. As the discussion period is approaching its end, we would appreciate it if you could take a look and let us know if any concerns remain or if any further clarification would be helpful.

Best regards,

The Authors

AuthorsOpenReview ↗

Follow-up on the Rebuttal

Dear Reviewer xwPW,

We believe that our response addresses the concerns raised in your review. As the discussion period is approaching its end, we would appreciate it if you could take a look and let us know if any concerns remain or if any further clarification would be helpful. If the concerns have been satisfactorily addressed, we would be grateful if the reviewer could kindly reconsider the evaluation score.

Best regards,

The Authors

Reviewer_C5kJOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the problem that search agents sometimes issue unreliable retrieval queries, and proposes QC→QR, an inference-time method that does not require retraining the agent or the retriever. The method has two steps. First, it judges each query. Whenever the agent writes a query, the lowest token generation probability in the query is used as its confidence, and if the confidence is above a threshold, the query is sent to the retriever unchanged. Second, it refines low-confidence queries. When the confidence is below the threshold, the off-the-shelf RQ-RAG reformulator generates three candidates by rewriting, decomposing, and disambiguating the query, and the candidate with the highest reformulator confidence is used for retrieval. The authors evaluate the method with four search agents on NQ, HotpotQA, 2WikiMultiHopQA, and MuSiQue, and report average accuracy gains of 1.78 to 6.18 points with a similar number of search calls.

Critical strengths

1. Clear problem definition and clear writing

The paper singles out a specific failure mode. In this case, the agent correctly decides to search, but its query does not clearly express what it needs to find. Prior work mainly focuses on when and how often to search, and the paper clearly separates this problem from that line of work. The paper is well organized, and both the method and the experiments are easy to follow.

2. Plug-and-play with modest overhead

The method does not require retraining the agent or the retriever. It can be added directly to existing search agents and improves several different agents. It also barely changes the number of search calls and adds only modest extra runtime, so it is easy to use in practice.

Critical weaknesses

1. Limited novelty

The query confidence used in this paper is almost the same as in Search Wisely, since both take the minimum token probability over the search query. The main difference is that Search Wisely uses it during training, while this paper moves it to inference time. The refinement step also directly reuses the off-the-shelf RQ-RAG model and its three operations. In other words, every component of the method was proposed before, and the paper uses the same confidence as Search Wisely as a switch that decides when to call RQ-RAG to rewrite a query. The technical contribution at the method level is therefore limited, and the only new part is this particular use of the signal, namely deciding at inference time whether to rewrite a query.

2. Missing analysis of when refinement hurts

A low query confidence does not necessarily mean that the query is poorly written. The query may be perfectly clear while the model is simply unsure, and in that case a reformulator of limited capacity (Llama2-7B) may turn a good query into a worse one. However, the method triggers refinement whenever the agent is uncertain and then executes the candidate with the highest reformulator confidence without comparing it to the original query, so there is no guarantee that the refined query is better than the original. The authors also acknowledge in Appendix C that refinement can hurt a query that was already right, and refining all queries (Refine All) lowers accuracy for three of the four agents, but the paper does not analyze when and how often this happens.

Questions for authors
  1. Both the refinement decision (Eq. 4) and the candidate selection (Eq. 7) rely on the LLM's generation probabilities, but the paper does not explain how these probabilities are obtained. Section 4 states that the agents sample with temperature 0.7 and top-p 0.95, but it does not say whether the confidence is computed from the probabilities before or after these adjustments, nor which decoding settings the reformulator uses to generate candidates. In addition, there are many ways to estimate LLM confidence. For example, DRAGIN, discussed in the related work, combines entropy, attention, and stopword filtering, whereas this paper only compares four ways of aggregating the same token probabilities. Could the authors clarify the exact implementation and compare how different confidence estimation methods affect the results?

  2. The method rests on the premise that low-confidence queries retrieve poorly, but this relationship is only verified with one retriever (e5-base-v2) on three multi-hop datasets, and poor retrieval may also come from the retriever itself rather than from a badly written query. Do this relationship and the gains of the method still hold with a stronger retriever? In addition, the paper does not state the backbone model or parameter count of the four agents or of the Direct, CoT, RAG, and IRCoT baselines. Could the authors provide this information?

  3. Regarding whether the refined query is better than the original one, what fraction of refinements make retrieval worse? How does the method handle the case where all three candidates have low confidence? Does the method still help when the agent is much larger or more capable than the reformulator?

  4. The paper states that its evaluation follows ReSearch, but ReSearch uses gpt-4o-mini as the judge and also reports EM. This paper instead uses Qwen3-8B, reports only LLM-judge accuracy, and does not verify how accurate Qwen3-8B is as a judge, for example against human judgments. Why was Qwen3-8B chosen? Could the authors also report EM, Cover-EM, and F1 to show whether the gains hold across metrics?

6 comments

We thank the reviewers for their thoughtful comments and concerns.

Part 1 of 2

W1. Limited novelty: Contribution of training-free, confidence-triggered query repair

Our contribution is the formulation and evaluation of confidence-triggered query repair as a training-free inference-time intervention for search agents. Our method requires no retraining and can be applied to frozen agents with different architectures and search policies. RQ-RAG provides the repair operations, but our method determines when repair should be invoked and how a repair is integrated into an ongoing multi-turn search trajectory.

W2. Missing analysis of when refinement hurts: Frequency of harmful refinements

We thank the reviewer for highlighting this issue. To analyze when refinement hurts, we define a harmful refinement case as one in which the original query retrieves at least one gold document in the top three results, whereas none of the three refined queries retrieves any gold document in the top three. Such cases account for 2.00% of the evaluated cases for Search-R1, 4.38% for ReSearch, 2.68% for R1-Searcher, and 2.95% for HiPRAG

Q1. Confidence computation, decoding settings, and alternative confidence estimators

In our implementation, confidence is computed from the model's raw token distribution before applying temperature scaling or top-pp truncation. Specifically, after generating a search query, we evaluate each generated query token using the original model logits:

pt=softmax⁡(zt)yt,p_t = \operatorname{softmax}(z_t)_{y_t},

where ztz_t denotes the raw logits and yty_t is the generated token. Candidate generation uses stochastic decoding with temperature T=0.7T=0.7 and top-p=0.95p=0.95, whereas candidate confidence is computed from the reformulator's raw token probabilities.

When all three refinement candidates have low confidence, the method still selects the candidate with the highest confidence among them. This constitutes a relative ranking decision. We can extend the method to retain the original query or abstain from refinement when all candidate-confidence scores fall below a calibrated threshold.

Across the four agents, the mean median-token confidence ranges from 0.94620.9462 to 0.99340.9934, the arithmetic-mean confidence ranges from 0.87700.8770 to 0.96330.9633, and the geometric-mean confidence ranges from 0.85370.8537 to 0.95640.9564. In contrast, minimum-token confidence ranges from 0.53750.5375 ±\pm 0.200.20 and exhibits substantially greater dispersion. Thus, under the same numerical threshold, minimum-token confidence identifies considerably more queries as low-confidence, whereas the other estimators smooth over isolated uncertain tokens. However, overly aggressive refinement may also modify already effective queries and degrade performance, as observed with the Refine All baseline(Tab. 2)

Q2. Retriever dependence, stronger retrievers, and backbone specifications

We thank the reviewer for raising these important points. We use E5-base-v2 to remain consistent with the standard evaluation settings of established search agents, including Search-R1, ReSearch, R1-Searcher, and HiPRAG. We agree that retrieval failures may arise from limitations of the retriever rather than solely from poorly formulated queries. Future work can explore this direction by extending the analysis to larger retrievers.

Backbone specifications. We clarify the model configurations below:

MethodBackboneParametersCheckpoint
Search-R1Qwen2.5-3B-Instruct3BSearch-R1 3B, GRPO v0.3
HiPRAGQwen2.5-3B-Instruct3BHiPRAG 3B, PPO
ReSearchQwen2.5-7B-Instruct7BReSearch 7B-Instruct
R1-SearcherQwen2.5-7B Base7BQwen-2.5-7B-base-RAG-RL

The Direct, CoT, RAG, and IRCoT baselines all use Qwen2.5-3B-Instruct (3B parameters) as their backbone model. We will add these specifications to the revised manuscript.

Q3. Retrieval degradation, low-confidence candidates, and agent–reformulator capacity

To analyze when refinement hurts, we define a harmful refinement case as one in which the original query retrieves at least one gold document in the top three results, whereas none of the three refined queries retrieves any gold document in the top three. Such cases account for 2.00% of the evaluated cases for Search-R1, 4.38% for ReSearch, 2.68% for R1-Searcher, and 2.95% for HiPRAG. When all three refinement candidates have low confidence, the method selects the candidate with the highest confidence among the three branches. Thus, candidate selection is based on relative confidence, even when all candidates have low absolute confidence.

Part 2 of 2

Our method enables smaller search agents to achieve performance comparable to larger models at inference time, without additional training or the substantial computational cost of reinforcement learning.
Q4. Choice of Qwen3-8B as the evaluator and robustness across EM, Cover-EM, and F1

We thank the reviewer for highlighting this evaluation issue. We selected Qwen3-8B because it is an open-weight and reproducible evaluator that has recently been studied as an LLM judge [1,2]. In particular, SAGE evaluates a search-augmented Qwen3-8B judge against human majority votes and reports moderate-to-substantial agreement. We use a fixed evaluation prompt and identical decoding settings across all methods.

To determine whether the improvements persist without relying exclusively on an LLM judge, we additionally report Cover-EM (CEM), exact match (EM), and token-level F1 below. Refinement improves CEM in 14 of the 16 agent–dataset settings and ties in the remaining two. It improves EM in 15 of 16 settings and token-level F1 in 15 of 16 settings. Thus, the overall gains from QC$\rightarrow$QR remain consistent across semantic and lexical evaluation metrics.

AgentDatasetOriginal CEMAfter Refinement CEMOriginal EMAfter Refinement EMOriginal F1After Refinement F1
Search-R1NQ45.5052.5036.0036.5045.0449.08
Search-R1HotpotQA40.0042.5036.0038.5044.4449.27
Search-R12Wiki34.5046.0027.0040.5035.5846.72
Search-R1MuSiQue22.5022.5018.5017.0029.9232.67
ReSearchNQ49.0057.0039.5047.0048.0657.93
ReSearchHotpotQA52.5857.7345.3650.5256.3166.47
ReSearch2Wiki50.5051.0042.5043.0049.7051.21
ReSearchMuSiQue34.0040.5029.5033.0040.3146.19
R1-SearcherNQ50.2650.2640.8441.8852.5452.17
R1-SearcherHotpotQA15.3857.0515.3850.6421.7861.49
R1-Searcher2Wiki48.5052.0043.5045.0050.8853.67
R1-SearcherMuSiQue34.5036.0030.0031.0042.4743.20
HiPRAGNQ50.7859.380.784.698.6413.60
HiPRAGHotpotQA43.6448.649.0913.1818.9025.69
HiPRAG2Wiki60.2468.675.426.6317.7921.72
HiPRAGMuSiQue22.9033.593.827.6312.5519.18

Table: Original performance and performance after refinement across Cover-EM (CEM), EM, and token-level F1. All values are percentages. The higher value within each original–refinement pair is bolded; ties are bolded in both columns.

  1. Tu, Ni, and Bi (ACL 2026). How Long Reasoning Chains Influence LLMs’ Judgment of Answer Factuality.

  2. Badshah, Emami, and Sajjad (ACL 2026). SAGE: A Search-AuGmented Evaluation of Large Language Models on Free-Form QA.

We sincerely thank the reviewer again for the constructive feedback. We hope that the additional analyses address the concerns raised, and we would be happy to provide any further clarification. If the concerns have been satisfactorily addressed, we would be grateful if the reviewer could kindly reconsider the evaluation score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

We thank the reviewers for their thoughtful comments and concerns. We have revised the paper to provide additional evaluation results, quantify variability across runs, and clarify the experimental setup.

Summary of Revisions

The main updates are summarized below.

  • Multi-seed evaluation (Tables 1 and 2). We extended the QC→QR evaluation using seeds 40, 41, and 42, and updated the accuracy and token-level F1 results. QR variants are reported as mean ± sample standard deviation across the available runs to quantify variability across seeds.

  • Search-agent specifications (Table 10). We added the parameter sizes, backbone models, and official Hugging Face checkpoints for Search-R1, HiPRAG, ReSearch, and R1-Searcher.

  • Confidence distributions (Table 19). We added the mean and standard deviation of minimum confidence, geometric mean, median confidence, and arithmetic mean for each agent to characterize differences among aggregation methods.

  • Additional sampling baseline (Table 20). We added a comparison with additional sampling, where five trajectories are independently generated for each question and the final answer is selected by majority voting. QC→QR (All) achieves higher accuracy in most agent–dataset pairs in this comparison.

  • Additional evaluation metrics (Table 21). We added original and post-refinement results using Cover Exact Match (CEM), Exact Match (EM), and token-level F1 to examine whether improvements hold across different answer-matching metrics.

  • Refinement-action ablations (Table 22). We added variants that always select rewriting, decomposition, or disambiguation at each refinement step, together with random selection among the three actions. QC→QR (All) is included for reference, with the caption clarifying that the columns do not constitute a matched-protocol comparison.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.