SEQUIN: Learning Selective Query Intervention for Agentic RAG

Agentic retrieval-augmented generation (RAG) relies on intermediate search queries to acquire evidence during multi-step reasoning. Query optimization through backbone training requires parameter updates, while standalone rewriting may overlook the evolving reasoning–retrieval context. Moreover, rewriting already effective queries can degrade retrieval, motivating decisions about both how and when to intervene. We propose SEQUIN, a framework that learns how and when to rewrite intermediate queries while keeping the backbone frozen. Its trajectory-aware query rewriter is trained on beneficial rewrites identified through counterfactual rollouts. The intervention policy is initialized from counterfactual comparisons of intervention sequences and further trained through Intervention-Structured Policy Optimization (ISPO). Using final-answer rewards, ISPO combines forced-prefix exploration with episode-level and intervention-prefix advantage estimation. Across four multi-hop question-answering benchmarks, SEQUIN improves macro-average EM on Search-R1 by 2.5 percentage points and CEM on HiPRAG by 1.9 percentage points. On Search-R1, selective intervention reduces rewrite calls by approximately 30% on average across benchmarks relative to always rewriting, while improving macro-average EM. These results demonstrate that selective query intervention can improve frozen Agentic RAG systems without rewriting every intermediate query.