acceptodds
Under review as a conference paper at ICLR 2027

SEQUIN: Learning Selective Query Intervention for Agentic RAG

Abstract

Agentic retrieval-augmented generation (RAG) relies on intermediate search queries to acquire evidence during multi-step reasoning. Query optimization through backbone training requires parameter updates, while standalone rewriting may overlook the evolving reasoning–retrieval context. Moreover, rewriting already effective queries can degrade retrieval, motivating decisions about both how and when to intervene. We propose SEQUIN, a framework that learns how and when to rewrite intermediate queries while keeping the backbone frozen. Its trajectory-aware query rewriter is trained on beneficial rewrites identified through counterfactual rollouts. The intervention policy is initialized from counterfactual comparisons of intervention sequences and further trained through Intervention-Structured Policy Optimization (ISPO). Using final-answer rewards, ISPO combines forced-prefix exploration with episode-level and intervention-prefix advantage estimation. Across four multi-hop question-answering benchmarks, SEQUIN improves macro-average EM on Search-R1 by 2.5 percentage points and CEM on HiPRAG by 1.9 percentage points. On Search-R1, selective intervention reduces rewrite calls by approximately 30% on average across benchmarks relative to always rewriting, while improving macro-average EM. These results demonstrate that selective query intervention can improve frozen Agentic RAG systems without rewriting every intermediate query.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Retrieval Augmented Generation, Agentic RAG, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XcUxOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

SEQUIN improves intermediate search queries in agentic RAG while keeping the backbone frozen. A trajectory-aware rewriter produces alternative queries, and an intervention policy decides whether to keep or rewrite each query before retrieval. The rewriter is trained on beneficial rewrites identified through counterfactual rollouts. The policy is initialized from comparisons of intervention sequences and then optimized with ISPO. Across four multi-hop QA benchmarks, the paper reports gains for Search-R1 and HiPRAG, with additional results on Search-R1-7B. On Search-R1-3B, selective intervention also reduces rewrite calls relative to always rewriting.

Critical strengths
  1. The paper addresses a meaningful failure mode of query rewriting. Its experiments show that an untuned rewriter can hurt performance, whereas a trained rewriter and selective intervention improve final-answer accuracy. This supports the need to consider when an intermediate query should be left unchanged.
  2. The modular approach shows some portability across backbones. Positive results on both Search-R1 and HiPRAG suggest that the method is not tied to one agent implementation.
Critical weaknesses
  1. The practical motivation for a plug-and-play approach with a frozen backbone is not convincingly validated. The evaluated Search-R1 backbones are 3B and 7B, while inference adds a 3B query rewriter and a 0.5B intervention policy. Without results on a substantially larger or inaccessible backbone, or an end-to-end comparison with directly adapting a trainable backbone, it remains unclear when this more complex pipeline is preferable.
  2. The methodological novelty appears limited relative to prior work. Training a rewriter using feedback from a frozen reader was explored in [1]; learned query refinement in [2]; a plug-and-play retrieval policy around a frozen backbone in [3]; and selective use of rewriting in [4]. ISPO builds on the episode-level and step-level advantage framework of GiGPO [5], adapting it to intervention-action prefixes. The paper does not clearly establish a sufficiently distinct contribution beyond combining and adapting these existing ideas.

[1] Ma et al. “Query Rewriting in Retrieval-Augmented Large Language Models.” EMNLP, 2023. [2] Chan et al. “RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation.” COLM, 2024. [3] Fang et al. “SPARKLE: A Structured and Plug-and-play Agentic Retrieval Policy for Adaptive RAG Models.” ACL, 2026. [4] Shanian et al. “Better Together: Complementary Query Rewriting Under a Strong RAG Baseline.” EMNLP Industry Track, 2026. [5] Feng et al. “Group-in-Group Policy Optimization for LLM Agent Training.” NeurIPS, 2025.

Questions for authors
  1. For the trainable backbones used in the experiments, why train and deploy three separate language models rather than incorporate query rewriting and intervention into the backbone or a single auxiliary model? Could the authors compare their effectiveness and end-to-end costs, or validate the proposed motivation using a larger backbone that cannot be fine-tuned?
  2. Could the authors add a baseline without an intervention policy that retrieves with both the original and rewritten queries at every step, then deduplicates and merges the results? Please compare settings with the same final evidence budget as SEQUIN and with twice the evidence budget, and report answer quality and end-to-end costs.
2 comments
AuthorsOpenReview ↗

Response to Reviewer XcUx (Part 1/2)

Thank you for these comments and questions. We clarify SEQUIN’s methodological contributions relative to prior work and report additional dual-query retrieval comparisons below. All additional experiments use Search-R1-3B as the frozen backbone.

1. Methodological novelty and distinction from prior work.

We acknowledge the precedents identified by the reviewer. SEQUIN learns whether and how to modify intermediate queries already proposed by a frozen retrieval agent. Ma et al. [1] train a rewriter using frozen-reader feedback, including correctness-filtered supervision. SEQUIN evaluates rewrites at intermediate retrieval steps by preserving the trajectory prefix, substituting retrieved evidence, and continuing the agent to assess downstream outcomes. RQ-RAG [2] trains the model for query refinement and answer generation; SEQUIN keeps the existing agent’s parameters frozen. Its intervention policy chooses between retaining and rewriting an existing query, with retrieval performed in both cases. This differs from SPARKLE’s decisions about whether to retrieve, what query to generate, and which evidence to incorporate [3]. It also differs from the top-1 reranker-score threshold used to trigger query expansion and result merging in [4]: SEQUIN learns its intervention decisions from trajectory context and final-answer outcomes. ISPO explicitly builds on GiGPO’s episode- and step-level advantage framework [5], adapting it to keep/rewrite decisions through intervention-prefix grouping and combining it with forced-prefix exploration. Our contribution is a framework for selective rewriting of intermediate queries in Agentic RAG, with counterfactual supervision and policy optimization designed for this task.

[1] Ma et al. “Query Rewriting in Retrieval-Augmented Large Language Models.” EMNLP, 2023.

[2] Chan et al. “RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation.” COLM, 2024.

[3] Fang et al. “SPARKLE: A Structured and Plug-and-play Agentic Retrieval Policy for Adaptive RAG Models.” ACL, 2026.

[4] Shanian et al. “Better Together: Complementary Query Rewriting Under a Strong RAG Baseline.” EMNLP Industry Track, 2026.

[5] Feng et al. “Group-in-Group Policy Optimization for LLM Agent Training.” NeurIPS, 2025.

AuthorsOpenReview ↗

Response to Reviewer XcUx (Part 2/2)

2. Could the authors add a baseline without an intervention policy that retrieves with both the original and rewritten queries at every step, then deduplicates and merges the results? Please compare settings with the same final evidence budget as SEQUIN and with twice the evidence budget, and report answer quality and end-to-end costs.

We have added the dual-query retrieval baselines with matched and doubled evidence budgets, including their construction details, answer quality, and end-to-end runtime, to Appendix I and Table 8 of the revised manuscript.

(1) Baseline construction and answer quality.

We evaluated a baseline without an intervention policy that invokes the same trained query rewriter at every retrieval step and retrieves using both the original and rewritten queries. We interleave the two ranked lists, remove duplicate passages, and retain up to three unique passages (Merge-3) or six unique passages (Merge-6) per retrieval step. These settings match SEQUIN’s final evidence budget or double it, respectively. When fewer unique passages are available, we retain them without padding.

The following table reports EM percentages.

MethodHotpotQA2WikiMultiHopQABamboogleMuSiQueMacro-average
SEQUIN32.729.225.68.624.03
Merge-331.729.820.87.522.45
Merge-633.230.422.47.723.43

SEQUIN improves macro-average EM by 1.58 percentage points over Merge-3 and 0.60 percentage points over Merge-6, computed before rounding. Doubling the evidence budget improves the merging baseline, but its macro-average EM remains below SEQUIN. Merge-6 achieves higher EM on HotpotQA and 2WikiMultiHopQA, while SEQUIN performs better on Bamboogle and MuSiQue.

(2) End-to-end runtime, including the intervention policy’s cost.

We report total end-to-end evaluation time for each dataset in seconds, not single-question latency. Timing includes backbone inference, query rewriting, retrieval, and the applicable gating or merging operations. Model loading and warmup are excluded.

MethodHotpotQA2WikiMultiHopQABamboogleMuSiQue
SEQUIN1451.071601.57181.241598.41
Merge-31597.381750.86195.711840.58
Merge-61762.812056.97218.862112.46

Including SEQUIN’s gating cost, Merge-3 requires 8.0%–15.2% more evaluation time, and Merge-6 requires 20.8%–32.2% more time across the four benchmarks. In this setting, SEQUIN achieves higher macro-average EM and lower end-to-end runtime than both merging baselines while retaining a three-passage budget per retrieval step.

Reviewer_nK68OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SEQUIN, a framework that learns how and when to rewrite intermediate queries while keeping the Agentic RAG backbone frozen. A trajectory-aware query rewriter is trained on beneficial rewrites identified through counterfactual rollouts. An intervention policy is initialized through supervised learning and further optimized using Intervention-Structured Policy Optimization (ISPO), which combines forced-prefix exploration with episode-level and intervention-prefix advantages. Experiments cover four multi-hop QA benchmarks and multiple backbone configurations. On Search-R1-3B, SEQUIN improves macro-average EM from 21.5% to 24.0% while reducing rewrite calls by approximately 28.4% relative to always rewriting.

Critical strengths
  1. The paper addresses a practically relevant problem: improving intermediate queries without updating a deployed backbone. Selectively retaining effective queries is a reasonable way to avoid harmful interventions.

  2. Training supervision is grounded in downstream answer correctness rather than query quality alone. The modular design also preserves the backbone’s existing reasoning and retrieval interface.

  3. The no-rewrite, always-rewrite, and random-intervention baselines provide useful controlled comparisons. In particular, outperforming random intervention at a similar rewrite budget supports the value of learning where to intervene.

Critical weaknesses
  1. The contributions of ISPO’s specific mechanisms are not sufficiently isolated. Section 3.2 builds on GiGPO’s episode- and step-level advantage framework by introducing forced-prefix exploration and intervention-prefix grouping. In Table 2, SFT-only, ISPO-only, and the full method achieve macro-average EM scores of 22.3%, 23.6%, and 24.0%, respectively. These results support the benefits of reinforcement learning and supervised initialization, but without a comparison against standard policy optimization under the same rollout budget, they do not establish the benefits of ISPO’s particular design. The forced-exploration ablation in Section 5.2 only shows that the rewrite rate approaches 100% in Figure 3, without reporting the corresponding accuracy. The step-level advantage in Equation (6) is not separately ablated either. Moreover, grouping uses the retrieval-step index and preceding intervention actions, (t,a<t)(t, a_{<t}), even though grouped decisions may have different queries and reasoning contexts, as explicitly acknowledged by the authors. Further evidence is needed to establish whether this grouping provides useful local credit assignment beyond changing intervention frequency.

  2. Dependence on a single retrieval environment remains unresolved. Section 4.1 uses BM25 over Wikipedia 2018 throughout, returning three documents per query and allowing at most four retrieval steps per question. Both Search-R1 and HiPRAG originally use E5 dense retrieval. Consequently, the current experiments do not distinguish improvements that would hold in the backbones’ original retrieval environment from gains that primarily compensate for query–retriever mismatch after switching to BM25. The improvements across two backbones in Table 1 and Search-R1-7B in Appendix D provide encouraging evidence across model configurations, but all share the same retrieval setup. Repeating the no-rewrite, always-rewrite, and selective-rewrite comparisons with E5, while distinguishing retriever-specific retraining from direct transfer, would clarify the method’s applicability.

Questions for authors

Under the E5 retrieval environment originally used by the backbones, do the gains in Table 1 and the advantage of selective over always rewriting persist? Can the rewriter and intervention policy trained with BM25 transfer directly to E5, or is retraining necessary?

2 comments
AuthorsOpenReview ↗

Response to Reviewer nK68(Part 1/2)

Thank you for these comments. We provide additional ablation results for ISPO, clarify the intervention-prefix grouping, and evaluate SEQUIN under E5 retrieval, distinguishing direct rewriter transfer from intervention-policy retraining. All additional experiments reported below use Search-R1-3B as the frozen backbone.

W1. Contributions of ISPO’s mechanisms

(1) Matched-budget GRPO comparison isolates the step-level advantage.

We have added the matched-budget GRPO comparison to Section 5.2 and Table 2 of the revised manuscript. ISPO improves macro-average EM by 0.60 percentage points over the episode-only GRPO baseline, with gains on all four benchmarks. Both methods use the same SFT-initialized policy, fixed query rewriter, training questions, number of rollouts per question, and policy-update steps.

MethodHotpotQA2WikiMultiHopQABamboogleMuSiQueMacro-average
Original backbone30.628.420.86.321.53
GRPO (episode-only)32.429.024.08.323.43
ISPO32.729.225.68.624.03

All values are EM percentages.

This comparison directly addresses the requested step-level ablation. ISPO uses the combined advantage

Ai,t=Aiepisode+ωAi,tstep.A_{i,t}=A_i^{\mathrm{episode}}+\omega A_{i,t}^{\mathrm{step}}.

Setting ω=0\omega=0 yields the episode-only estimator used in our GRPO implementation:

Ai,t=Aiepisode=Ri−1G∑j=1GRj.A_{i,t}=A_i^{\mathrm{episode}}=R_i-\frac{1}{G}\sum_{j=1}^{G}R_j.

Every intervention decision within a rollout then receives the same episode-level advantage. The baseline retains the same forced-prefix exploration, forced-action loss weighting, and other optimization settings; only the step-level contribution is removed. Thus, this GRPO-style control isolates the contribution of the step-level signal within the same rollout and optimization procedure.

(2) Removing forced-prefix exploration produces always-rewrite behavior and lowers accuracy.

Removing forced-prefix exploration reduces macro-average EM from approximately 24.0% to 23.2%, matching Always Rewrite. This ablation retains supervised initialization, the same frozen backbone and trained query rewriter, and the remaining training settings.

Section 5.2 shows that the policy-sampled rewrite rate approaches 100%. We additionally evaluated the resulting checkpoint and confirmed that its answer accuracy matches Always Rewrite with the same trained rewriter. This supplements the rewrite-rate curve with the corresponding task accuracy: the loss of selective behavior is accompanied by lower answer accuracy. These results support the role of forced-prefix exploration in learning an effective selective intervention policy.

(3) Grouping aligns decision contexts, with benefits beyond rewrite frequency.

Grouping is performed within the same question. During ISPO training, the backbone and rewriter are frozen and use greedy decoding. With deterministic retrieval, identical intervention prefixes therefore produce identical pre-intervention contexts, including reasoning, queries, and retrieved evidence. Our earlier wording did not clearly distinguish this deterministic setting.

The ablation in (1) supports the usefulness of the resulting step-level signal, yielding a 0.60-point macro-average EM gain. Furthermore, Section 5.3 and Table 3 show that SEQUIN outperforms Random Intervention using the same rewriter: 24.0% versus 22.2% macro-average EM, with slightly fewer relative rewrite calls (71.6% versus 73.3%). This supports learning which queries to rewrite beyond adjusting overall rewrite frequency.

MethodMacro-average EM (%)Mean relative rewrite calls (%)
Random Intervention22.273.3
SEQUIN24.071.6
AuthorsOpenReview ↗

Response to Reviewer nK68(Part 2/2)

W2. Dependence on the retrieval environment

We have added the E5 retrieval results and clarified the rewriter-transfer and intervention-policy retraining settings in Section 5.5 and Table 4 of the revised manuscript. We additionally evaluated the original backbone (Search-R1-3B), Always Rewrite, and selective rewriting under E5. The query rewriter was directly transferred from BM25 without E5-specific fine-tuning, while the intervention policy was retrained under E5.

MethodHotpotQA2WikiMultiHopQABamboogleMuSiQueMacro-average
Original backbone31.430.832.010.326.13
Always Rewrite32.431.436.810.127.68
Selective Rewrite (ISPO)32.632.036.810.527.98

All values are EM percentages. Under E5, selective rewriting improves macro-average EM by 1.85 percentage points over the original backbone, with gains on all four benchmarks. It also exceeds Always Rewrite by 0.30 percentage points, performing better on three benchmarks and matching its performance on Bamboogle. These results show that SEQUIN’s gains persist under E5, including a smaller advantage over Always Rewrite. They also demonstrate that the BM25-trained rewriter remains effective without E5-specific fine-tuning when paired with an intervention policy retrained under E5. This experiment does not assess direct transfer of the intervention policy.

Reviewer_voVhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SEQUIN, which learns when and how to rewrite intermediate queries for a frozen agentic RAG backbone. A query rewriter is trained on teacher-generated rewrites selected through counterfactual rollouts. A separate intervention policy decides whether to keep each query or invoke the rewriter. The policy is initialized with supervised labels and then trained with ISPO, which combines forced-prefix exploration with episode-level and intervention-prefix advantages. The paper reports improvements on Search-R1 and HiPRAG. On Search-R1-3B, selective intervention improves macro-average EM from 23.2% to 24.0% over always rewriting with the same trained rewriter, while reducing rewrite calls by 28.4% on average across benchmarks.

Critical strengths
  1. I find selective query intervention an interesting idea. Rewriting every query can damage an already useful query, and using the reasoning-retrieval history to decide when to intervene is well motivated. Training from downstream answer outcomes also provides a relevant signal for this decision.

  2. The results show improvements across Search-R1-3B, Search-R1-7B, and HiPRAG. The comparison with always rewriting uses the same trained rewriter, and the random-intervention baseline has a similar rewrite budget. These controls provide useful evidence that learning when to rewrite has value beyond simply adding more rewrite calls.

Critical weaknesses
  1. The sources of the gains remain unclear. The trained always-rewrite baseline already improves EM from 21.5% to 23.2%, while SEQUIN reaches 24.0% (Table 2). The experiments do not isolate the benefit of counterfactual filtering from teacher-generated supervision, or ISPO from simpler RL objectives. Key data-generation details, including K and decoding settings, are missing. Without multi-seed results or uncertainty estimates, the reliability of the additional 0.8-point gain is also unclear.

  2. The two-model design needs stronger justification. Both modules receive the same inputs, but use separate 3B and 0.5B models. Missing controls include a shared model with separate gating and generation heads, or LoRA adapters on Search-R1 with its original weights frozen and adapters enabled only for gating and rewriting. These alternatives are not necessarily better, but their trade-offs should be assessed. Fewer rewrite calls alone do not establish lower end-to-end cost, since the gate runs at every retrieval step.

Questions for authors
  1. Training and attribution: What are K, the teacher and continuation decoding settings, and the filtering rate? Under matched data and rollout budgets, what evidence isolates the benefits of counterfactual filtering and ISPO from teacher supervision and simpler RL?

  2. LoRA baseline: Have you compared against LoRA adapters and a gating head on Search-R1 using the same supervision, with original weights frozen and adapters enabled only for gating and rewriting?

  3. Shared model: What accuracy or efficiency benefit does using separate models provide over a shared auxiliary transformer with gating and generation heads?

  4. Reliability and cost: Is the 0.8-point gain over always rewriting stable across seeds? What are the end-to-end latency and peak memory of the original agent, always rewriting, and SEQUIN, including the gate's cost?

2 comments
AuthorsOpenReview ↗

Response to Reviewer voVh(Part 1/2)

Thank you for these questions. We clarify the training settings and architectural choices below, and report additional comparisons examining the effects of counterfactual filtering and ISPO, stability across training seeds, and end-to-end computational costs. All additional experiments reported below use Search-R1-3B as the frozen backbone.

Q1. What are K, the teacher and continuation decoding settings, and the filtering rate? Under matched data and rollout budgets, what evidence isolates the benefits of counterfactual filtering and ISPO from teacher supervision and simpler RL?

We have added the candidate-generation, continuation-decoding, and filtering details to Appendix C.1, and the counterfactual-filtering and GRPO comparisons to Sections 5.1 and 5.2, respectively, with results in Table 2 of the revised manuscript.

1. Candidate Generation. We generate K = 4 candidate rewrites per eligible retrieval step using DeepSeek-V4-Flash with temperature 0. All candidates are generated in a single JSON response, conditioned on the question, preceding reasoning–retrieval context, and current query. After removing empty, duplicate, and unchanged queries, some steps contain fewer than four valid candidates.

2. Counterfactual Continuation. For each candidate, we preserve the original trajectory prefix and replace the retrieved evidence with the candidate's retrieval results. The frozen Search-R1-3B backbone uses greedy decoding (do_sample=False), with at most 256 new tokens per generation call, top-3 retrieval, and a maximum of four interaction turns (including those already consumed by the prefix). If needed, a final generation call is allowed without further retrieval.

3. Filtering Rate. From 209,628 generated candidate rewrites, counterfactual filtering produces 19,937 training examples, corresponding to a yield of 9.51% relative to the candidate pool.

4. Contribution of Counterfactual Filtering. We compare unfiltered teacher supervision, which selects one teacher-generated rewrite per retrieval step without counterfactual evaluation, against supervision constructed through counterfactual filtering.

Rewriter Training DataHotpotQA2WikiBamboogleMuSiQueAvg.
No Rewrite30.628.420.86.321.53
Unfiltered Teacher Supervision31.128.620.07.021.68
Counterfactually Filtered Supervision32.128.724.07.823.15

All values are EM percentages. Counterfactual filtering improves macro-average EM by 1.48 percentage points over unfiltered teacher supervision. However, the training-set sizes differ, so this comparison does not fully isolate the filtering effect under a matched data budget.

5. Contribution of ISPO. We compare ISPO with GRPO using the same SFT-initialized policy, fixed rewriter, training questions, rollouts per question, and policy-update steps.

Policy OptimizationHotpotQA2WikiBamboogleMuSiQueAvg.
GRPO32.429.024.08.323.43
ISPO32.729.225.68.624.03

Under matched training budgets, ISPO improves macro-average EM by 0.60 percentage points over GRPO, with improvements across all four benchmarks.

AuthorsOpenReview ↗

Response to Reviewer voVh(Part 2/2)

Q2. Have you compared against LoRA adapters and a gating head on Search-R1 using the same supervision, with original weights frozen and adapters enabled only for gating and rewriting?

We have not evaluated this baseline. SEQUIN targets settings where backbone parameters are inaccessible or adaptation is not permitted, while intermediate retrieval queries and returned evidence remain accessible. Backbone-attached LoRA adapters and a gating head require access to the model’s internal architecture. Although Search-R1 itself is trainable, we use it as a reproducible backbone and keep its parameters frozen to study settings where backbone adaptation is unavailable or not permitted.

Q3. What accuracy or efficiency benefit does using separate models provide over a shared auxiliary transformer with gating and generation heads?

SEQUIN addresses two complementary tasks: improving intermediate queries and deciding when to intervene. We therefore design two modules with distinct roles: a query rewriter and an intervention policy. This separation allows us to optimize intervention decisions while keeping the trained rewriter fixed, with a smaller 0.5B policy running at each retrieval step and the 3B rewriter invoked only when selected. The ablations in Table 2 support the effectiveness of these components: training the rewriter improves macro-average EM from 20.1% to 23.2%, while the full framework achieves 24.0%. Table 3 further shows a 28.4% reduction in rewrite calls compared with Always Rewrite. These results support the effectiveness of the current framework. Future work will explore a shared auxiliary Transformer with gating and generation heads to assess the relative accuracy–efficiency trade-offs of the two architectures.

Q4. Is the 0.8-point gain over always rewriting stable across seeds? What are the end-to-end latency and peak memory of the original agent, always rewriting, and SEQUIN, including the gate's cost?

We have added the three-seed stability results to Appendix H (Table 7) and the end-to-end runtime and peak GPU memory measurements to Appendix G (Table 6) of the revised manuscript. The cost measurements include SEQUIN’s gating overhead.

1. Stability Across Seeds. We repeated intervention-policy training with three seeds (41, 42, and 43), keeping the backbone, trained query rewriter, training data, and evaluation settings fixed. Always Rewrite uses the same fixed rewriter and serves as a shared reference. This experiment therefore measures variability in intervention-policy training.

MethodMacro-average EM (%)
Always Rewrite (fixed reference)23.15
SEQUIN (seed 41)23.75
SEQUIN (seed 42)24.03
SEQUIN (seed 43)23.85
SEQUIN (mean ± std)23.88 ± 0.14

We report the mean and sample standard deviation of macro-average EM across the three seeds, computed before rounding. SEQUIN outperforms Always Rewrite in all three runs, with gains ranging from 0.60 to 0.88 percentage points and an average gain of 0.73 percentage points. These results support the consistency of the improvement across the evaluated policy-training seeds.

2. End-to-End Runtime. We measure total evaluation time (seconds), including backbone inference, retrieval, rewriting, and gating where applicable. Model loading and warmup are excluded. These measurements represent dataset-level runtimes, not per-question latencies.

MethodHotpotQA2WikiBamboogleMuSiQue
Original Agent1289.671356.50158.361405.92
Always Rewrite1538.491711.28195.611752.84
SEQUIN1451.071601.57181.241598.41

Including gating overhead, SEQUIN reduces runtime by 5.7%–8.8% compared with Always Rewrite, while remaining 12.5%–18.1% slower than the original agent.

3. Peak GPU Memory. We report peak GPU memory allocated by PyTorch during inference, including the weights of all models loaded in the evaluation process and SEQUIN’s intervention policy. Memory used by the separately hosted retriever is excluded. All values are in GiB.

MethodHotpotQA2WikiBamboogleMuSiQue
Original agent18.6224.6717.7021.69
Always Rewrite24.5526.5323.9628.66
SEQUIN25.4928.0324.8928.07

SEQUIN incurs additional memory overhead compared with the original agent and does not consistently reduce peak memory relative to Always Rewrite. Its measured efficiency benefit lies in reduced runtime relative to Always Rewrite, with gating overhead included.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their careful assessments and constructive feedback. Their comments have helped us clarify SEQUIN’s methodological contributions, strengthen the experimental evidence, and more precisely characterize its deployment assumptions and accuracy–efficiency trade-offs. We have revised the manuscript in response to the reviewers’ comments, with revisions marked in blue.

We summarize the main additions and clarifications below:

  1. Additional evidence on counterfactual supervision and policy optimization (Reviewers voVh and nK68). We added candidate-generation, decoding, and filtering details, together with comparisons against unfiltered teacher supervision and an episode-only GRPO-style control. Counterfactual filtering improves macro-average EM by 1.48 percentage points over unfiltered supervision. Under matched rollout and optimization settings, adding ISPO’s step-level advantage improves macro-average EM by 0.60 percentage points, with gains on all four benchmarks. We also report that removing forced-prefix exploration leads to nearly always-rewrite behavior and reduces macro-average EM from approximately 24.0% to 23.2% (Sections 5.1–5.2; Table 2; Appendix C.1).

  2. Stability across intervention-policy training seeds (Reviewer voVh). We repeated policy training with three seeds (41, 42, and 43) while keeping the backbone, trained rewriter, data, and evaluation settings fixed. SEQUIN outperforms the shared Always Rewrite reference in all three runs, with gains of 0.60–0.88 percentage points and an average gain of 0.73 percentage points. We report the mean and sample standard deviation and clarify that this experiment measures variability in intervention-policy training (Appendix H; Table 7).

  3. End-to-end computational costs, including gating overhead (Reviewer voVh). We added dataset-level evaluation runtimes and peak GPU memory measurements for the original agent, Always Rewrite, and SEQUIN, with the measurement scope and memory accounting specified. Including gating overhead, SEQUIN reduces runtime by 5.7%–8.8% relative to Always Rewrite while achieving higher macro-average EM. Compared with the original agent, SEQUIN remains 12.5%–18.1% slower, making the additional computational cost explicit (Appendix G; Table 6).

  4. Evaluation under dense retrieval (Reviewer nK68). We added experiments with E5 retrieval, directly transferring the BM25-trained rewriter without further fine-tuning and retraining the intervention policy under E5. SEQUIN improves macro-average EM by 1.85 percentage points over the original backbone and 0.30 percentage points over Always Rewrite, showing that the gains persist under dense retrieval (Section 5.5; Table 4).

  5. Dual-query retrieval baselines with matched and doubled evidence budgets (Reviewer XcUx). We added baselines that retrieve with both the original and rewritten queries, deduplicate and merge the results, and retain up to three passages (Merge-3) or six passages (Merge-6) per step. SEQUIN achieves higher macro-average EM by 1.58 and 0.60 percentage points, respectively. Including SEQUIN’s gating cost, Merge-3 requires 8.0%–15.2% more evaluation time and Merge-6 requires 20.8%–32.2% more time. We also report the benchmark-level differences (Appendix I; Table 8).

  6. Clearer methodological positioning and architectural rationale (Reviewers voVh, nK68, and XcUx). We clarify that SEQUIN learns whether and how to rewrite intermediate queries proposed by a frozen agent, with retrieval performed under either action. We distinguish this setting from prior query-rewriting and retrieval-policy approaches, explicitly acknowledge ISPO’s foundation in GiGPO, and explain intervention-prefix grouping under deterministic rollouts. We also clarify the intended setting where backbone adaptation is unavailable or not permitted. We appreciate the reviewers’ suggestions regarding backbone-attached adapters and a shared auxiliary Transformer, and plan to explore these alternatives in future work to assess their accuracy–efficiency trade-offs.

Detailed point-by-point responses are provided under each review. We hope this summary helps the reviewers and Area Chair navigate the additional evidence and revisions. Thank you for your time and consideration, and we welcome any further questions or requests for clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.