Confidence-Guided Query Refinement for Search-Augmented Reasoning

Search-augmented reasoning enables language-model agents to acquire missing knowledge, but retrieval can fail when an agent generates a query that poorly expresses its current information need. We study this failure mode, which we term query unreliability, and show that token-level query confidence provides a pre-retrieval signal of retrieval quality and downstream answer correctness. Based on this observation, we introduce Confidence-Guided Query Refinement (QC$\to$QR), a post-hoc test-time method that executes confident queries unchanged while selectively rewriting, decomposing, or disambiguating low-confidence queries using an off-the-shelf reformulator. QC$\to$QR requires no additional training of the search agent or retriever. Across $4$ heterogeneous search agents and $4$ knowledge-intensive QA benchmarks, QC$\to$QR improves average answer accuracy by up to up to 6.18 percentage points, improves gold-document retrieval and downstream calibration, and requires a comparable number of external search calls. Unconditional and random refinement do not yield consistent gains, highlighting the importance of selectively intervening on uncertain queries. Our results establish query confidence as a practical signal for repairing unreliable search actions before retrieval.