acceptodds
Under review as a conference paper at ICLR 2027

SearchSkill : Teaching LLMs to Use Search Tools with Evolving Skill Banks

Abstract

Teaching language models to use search tools is not only a question of whether they search, but also of whether they issue good queries. This is especially important in open-domain question answering, where broad or copied queries often waste retrieval budget and derail later reasoning. We propose SearchSkill, a framework that makes query planning explicit through reusable search skills. At each step, the model first selects a skill, then generates a search or answer action conditioned on the selected skill card. The skill inventory itself is not fixed: SearchSkill maintains an evolving SkillBank, expands or refines it from recurrent failure patterns, and reconstructs affected trajectories before supervised training. The resulting two-stage SFT recipe aligns training with the inference-time protocol of skill selection followed by skill-grounded execution. Across open-source and closed-source models, SearchSkill improves exact match on knowledge-intensive QA benchmarks and yields better retrieval behavior, including fewer copied first queries, more atomic hop-focused queries, and more correct answers within a small search budget. These results suggest that explicit skill-conditioned query planning is a lightweight alternative to treating search as an undifferentiated action.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
tool use, agent skills, multi-hop question answering

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_WSbxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SEARCHSKILL tackles a real annoyance: search agents tend to just dump the whole multi-hop question into the retriever. The fix is a select–read–act loop, before each step the model picks a skill from a SkillBank, reads its card, then issues a query or answers. The bank isn't hand-written; it grows in four curriculum stages (B0→B4, six → ~20 cards), driven by failure analysis, with skills from successful rollouts "protected" from later revision.

Critical strengths
  1. The core idea is clean and actually testable. Making query planning explicit — pick a skill, read the card, then act — turns a vague "search better" goal into a token-level, trainable interface. The factorization in Eq. 1 is simple, and the two-stage SFT genuinely aligns training with the inference protocol (Stage II removes the skill hint, so the model must really route). That's a nicer story than bolting on another RL reward.

  2. The SkillBank evolution is a real mechanism, not window dressing. Staged growth from direct lookup → bridge → comparison → long-hop, with protection of validated skills and revision driven by failure traces, is principled. Table 4 shows staging beats one-shot by ~5 points. And the bank is portable: frozen B4 helps three closed-source models (Fig. 5) and transfers to WebWalker with zero retraining.

  3. Diagnostics go well beyond EM. Copy rate, atomic-hop rate, searches per question, Correct@≤3 — these actually test the motivating claim instead of just reporting accuracy. The paired SFT-vs-RL analysis (Fig. 6) is honest, and they openly admit GRPO doesn't help accuracy. Case studies are readable and convincing. Also: 1.2h of LoRA SFT vs ~10h of GRPO is a genuinely cheap recipe.

Critical weaknesses
  1. The headline number rides on a proprietary teacher. The 42.34 comes from GPT-5.4-generated skills and GPT-5.4 trajectories. The self-generated variant gets 38.60 — i.e., basically ties Search-R1's 38.68. So "self-directed skill discovery elicits latent capability" is oversold. Table 3's distillation control doesn't rescue it either: the SkillBank itself is GPT-5.4-authored, so it's distillation in a different costume, and the control differs in more than "skill conditioning."

  2. Baseline fairness and statistical rigor are shaky. Baselines look copied from original papers rather than rerun under this E5/top-3/budget-5 setup, and some are implausible (s3 at 15.00 avg). Evaluation is 16 validation records, greedy decoding, single run, no seeds or significance tests, EM only. Worse, the bank is built from HotpotQA/2Wiki/MuSiQue — the same datasets where gains concentrate — while MuSiQue stays stuck at ~15 EM.

Questions for authors

see weakness

5 comments
AuthorsOpenReview ↗

Response to Reviewer WSbx (Part 1/3)

We sincerely thank you for the thoughtful, candid and encouraging review. We greatly appreciate your recognition of the clean select–read–act interface, the SkillBank evolution as a real mechanism and the diagnostics beyond EM. Your questions on teacher dependence and evaluation rigor guided us to separate skills from distillation and to re-test all baselines more rigorously, which strengthens both our claims and their empirical support.

W1: The headline number rides on a proprietary teacher. The 42.34 comes from GPT-5.4-generated skills and GPT-5.4 trajectories. The self-generated variant gets 38.60 — i.e., basically ties Search-R1's 38.68. So "self-directed skill discovery elicits latent capability" is oversold. Table 3's distillation control doesn't rescue it either: the SkillBank itself is GPT-5.4-authored, so it's distillation in a different costume, and the control differs in more than "skill conditioning."

1) Separating skills from teacher distillation

The paper's Table 3 shows that our skill workflow elicits extra capability. This comes from two sources: the model learns to select and apply skills in its reasoning, and the cards themselves carry content.

First, we exclude GPT-5.4's knowledge, that is, whether the gain is plain distillation from GPT-5.4 without our workflow. The paper's Table 3 gives this control on 7B-Instruct, in EM:

HotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQAAvg.
SearchSkill, GPT-5.4 skills and trajectories43.0044.5015.5038.4041.6061.0451.4042.21
GPT-5.4 trajectories without skills38.0032.5018.0033.6038.6458.2046.6037.93

Distilling GPT-5.4's knowledge on these questions is not enough without our workflow. With the same teacher and budget, skill-structured training still outperforms direct trajectory distillation.

Second, we restrict the test to skill conditioning: we reuse our whole pipeline and only shuffle the content of the skill cards. EM of 7B-Instruct, seed 42, on the enlarged benchmarks:

HotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQAAvg.
SearchSkill41.242.218.943.239.461.951.142.6
Shuffled cards39.741.217.939.236.660.251.040.8

Without correct card content, EM drops on every dataset, by 1.8 on average (McNemar p = 0.003, 95% CI [+0.4, +2.9]). The card content is thus essential for reasoning, which further shows the role of skills. We will add this part to the main text with an explanation.

2) Self-generated skills against large-scale RL

Our workflow is not designed to rely on a particular model for its gains. We use GPT-5.4 mainly for practicality and efficiency, since an API makes the bank easy to build. The training cost of the self-generated variant against Search-R1, 7B-Instruct on the paper's benchmarks:

MethodTrainingHardwareTraining timeMacro EM
Search-R1GRPO8 H100about 50–100 h, estimated38.68
SearchSkill, self-generated skillstwo-stage SFT4 H1000.92 h, measured38.60

Our original aim is to elicit the model's capability in a more efficient way, and we do achieve it. With 4 H100 GPUs and under one hour of SFT, the 7B model writing its own skills matches the capability of large-scale RL, which shows that self-directed skill discovery does elicit the latent capability of the 7B model.

AuthorsOpenReview ↗

Response to Reviewer WSbx (Part 2/3)

W2: Baseline fairness and statistical rigor are shaky. Baselines look copied from original papers rather than rerun under this E5/top-3/budget-5 setup, and some are implausible (s3 at 15.00 avg). Evaluation is 16 validation records, greedy decoding, single run, no seeds or significance tests, EM only. Worse, the bank is built from HotpotQA/2Wiki/MuSiQue — the same datasets where gains concentrate — while MuSiQue stays stuck at ~15 EM.

1) Baselines are re-run, and s3 is scored by strict EM

We thank the reviewer for the careful reading. As stated in the Evaluation paragraph of Sec. 4.1, the main table re-ran every baseline with the shared E5 retriever, top-3 passages, five-search budget and greedy decoding. To address the concern further, we enlarged the datasets to the 1,000-question scale; Bamboogle officially has only 125 test questions. The EM results are as follows:

MethodHotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQAAvg.
ZeroSearch30.427.87.222.436.058.947.332.9
Search-o133.031.814.336.831.458.944.635.8
Search-R139.733.216.738.434.553.646.137.5
StepSearch36.940.221.844.033.752.045.539.2
Smart-Searcher38.940.618.743.235.958.947.340.5
SearchSkill-Base, seed 4241.942.216.940.038.461.051.741.7
SearchSkill-Base, seed 7353140.642.719.738.438.861.351.341.8
SearchSkill-Base, seed 7553141.943.117.736.839.661.652.942.0
SearchSkill-Base, mean41.542.718.138.438.961.352.041.8 ± 0.1
SearchSkill-Instruct, seed 4241.242.218.943.239.461.951.142.6
SearchSkill-Instruct, seed 7553142.441.518.243.238.362.551.842.6
SearchSkill-Instruct, seed 290342.240.118.940.839.362.349.741.9
SearchSkill-Instruct, mean41.941.318.742.439.062.250.942.3 ± 0.4

Paired significance tests against every baseline:

ZeroSearchSearch-o1Search-R1StepSearchSmart-Searcher
McNemar p, largest over all six seeds< 2e-35< 3e-26< 1e-15< 3e-10< 5e-4
Instruct mean gain, paired bootstrap 95% CI+9.5 [+7.9, +10.9]+6.5 [+5.1, +8.0]+4.9 [+3.6, +6.3]+3.2 [+1.7, +4.6]+1.8 [+0.4, +3.2]

Our method leads every baseline, and the advantage passes significance tests.

s3 itself has strong search ability. Its 15.00 comes from strict EM, whereas its authors use span containment, so we scored our re-run of s3 in our harness under both criteria:

s3HotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQA
Strict EM, as in the main table20.014.43.510.46.536.317.3
Span containment, its own criterion48.951.720.535.258.472.663.6

Most mainstream methods report EM, so for fairness we keep EM as the common standard, while we acknowledge that s3 also has strong search ability.

AuthorsOpenReview ↗

Response to Reviewer WSbx (Part 3/3)

2) Evaluation set, seeds, metrics and decoding

Actually, the 16 records are the SFT validation split used to monitor training, not the evaluation set. The paper evaluates 3,725 questions, and the enlarged sets above contain 5,799.

The two tables above already give multi-run greedy results. Greedy decoding has no run-to-run variance: we re-ran every baseline three times and show one row because the runs are identical. To address the concern further, we also report sampling decoding against Search-R1 and F1 for every baseline. Sampling uses temperature 0.7 with three runs on the paper's benchmarks, in EM:

HotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQAAvg.
Search-R1, run 133.027.011.529.636.052.844.933.5
Search-R1, run 234.026.510.532.033.353.044.333.4
Search-R1, run 333.525.511.532.833.753.744.933.7
Search-R1, mean33.526.311.231.534.353.244.733.5
SearchSkill, seed 42, run 139.543.013.530.436.459.551.239.1
SearchSkill, seed 42, run 240.042.513.535.237.160.350.639.9
SearchSkill, seed 42, run 338.543.516.040.837.359.951.141.0
SearchSkill, seed 42, mean39.343.014.335.536.959.951.040.0

F1 on the enlarged benchmarks for every baseline:

MethodHotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQAAvg.
ZeroSearch40.934.915.034.546.267.556.142.2
Search-o144.440.722.645.242.468.453.245.3
Search-R149.136.622.646.241.259.351.043.7
StepSearch47.847.432.654.142.360.352.848.2
Smart-Searcher49.948.128.257.246.567.455.750.4
SearchSkill-Instruct, seed 4252.548.727.353.348.369.857.851.1
SearchSkill-Instruct, seed 7553153.448.826.953.548.370.258.251.3
SearchSkill-Instruct, seed 290352.947.327.851.748.869.856.650.7
SearchSkill-Instruct, mean52.948.327.352.848.569.957.551.0 ± 0.3

Under sampling SearchSkill leads Search-R1 by 6.5 EM, and under F1 it leads every baseline, so our advantage holds under these changes. The gains are not concentrated on the three evolution datasets. On the enlarged MuSiQue every seed reaches 18 EM or more, so it is not stuck at 15.

We are deeply grateful for your thorough and constructive review, which helped us separate the role of skills from the teacher and strengthen the statistical evidence. We hope these results resolve your concerns and would be glad to discuss any questions.

Reviewer_ytwuOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SearchSkill makes query planning explicit. Before each search or answer action, the policy selects skills from an evolving SkillBank and conditions on the skill cards. The bank is grown over a dataset curriculum from rollout feedback and then used to build trajectories for two-stage SFT. With self-generated skills, a 7B Qwen model matches Search-R1 (38.60 vs. 38.68 macro EM) using SFT only. With GPT-5.4-generated skills it reaches 42.34, and the frozen bank also helps closed-source models.

Critical strengths
  1. Strong results at low cost. SFT on self-generated skills is competitive with RL-trained search agents, and the paper reports query-level gains: fewer copied first queries, more atomic hop queries, and more correct answers within 3 searches (Fig. 4).
  2. Informative ablations. The GPT-5.4 trajectory control without skills (37.93 vs. 42.21) shows the gain is not only teacher distillation. The no-card, seed-bank and staged vs. one-shot ablations support the design.
Critical weaknesses
  1. Limited statistical support. Each subset within the multi-hop datasets contains only 125–200 questions, and experiments were conducted as single runs; furthermore, the margins between many of the reported results are as small as 1–3 EM (Exact Match) points. Additionally, it remains unclear whether the baseline models were re-run under the same evaluation framework; Appendix H presents several instances of malformed Search-R1 outputs.
  2. Need for further clarification on data sources. Appendix D.3 incorporates "failure-replay samples from early evaluations," and Table 10 includes 1,445 such records. It is essential to confirm that there is no overlap between this data and the question sets used for evaluation.
Questions for authors
  1. Were the baselines re-run under your retriever and 5-search budget, and what is their malformed-output rate? Can you add CIs, or multiple SFT seeds?
  2. What data were the "earlier evaluations" behind the failure-replay and replay records run on? Is any evaluation question included?
5 comments
AuthorsOpenReview ↗

Response to Reviewer ytwu (Part 1/3)

We sincerely thank you for the thoughtful and encouraging review, and for recognising the strong results at low cost, the query-level gains and the informative ablations. Your constructive comments on statistical support and data provenance guided us to enlarge the benchmarks, re-train with three seeds, test a second harness and audit the training data, which further strengthen the soundness of our claims.

W1: Statistical support is limited. The multi-hop sets have 125–200 questions each and a single run, while many margins are 1–3 EM points. It is also unclear whether the baselines were re-run in the same harness; App. H shows several malformed Search-R1 outputs.

Q1a: Were the baselines re-run under your retriever and 5-search budget? Can you add CIs, or multiple SFT seeds?

1) Larger evaluation sets, three seeds and significance tests

In the paper, the 200-question multi-hop test sets are the same as those released by Tool-Star [1], and Bamboogle officially has only 125 test questions. As suggested, we enlarged every other set to the 1,000-question scale, re-ran every baseline three times, and re-trained SearchSkill with three seeds (E5 top-3, at most five searches, greedy decoding), as set in the main text. The EM results are as follows:

MethodHotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQAAvg.
ZeroSearch30.427.87.222.436.058.947.332.9
Search-o133.031.814.336.831.458.944.635.8
Search-R139.733.216.738.434.553.646.137.5
StepSearch36.940.221.844.033.752.045.539.2
Smart-Searcher38.940.618.743.235.958.947.340.5
SearchSkill-Base, seed 4241.942.216.940.038.461.051.741.7
SearchSkill-Base, seed 7353140.642.719.738.438.861.351.341.8
SearchSkill-Base, seed 7553141.943.117.736.839.661.652.942.0
SearchSkill-Base, mean41.542.718.138.438.961.352.041.8 ± 0.1
SearchSkill-Instruct, seed 4241.242.218.943.239.461.951.142.6
SearchSkill-Instruct, seed 7553142.441.518.243.238.362.551.842.6
SearchSkill-Instruct, seed 290342.240.118.940.839.362.349.741.9
SearchSkill-Instruct, mean41.941.318.742.439.062.250.942.3 ± 0.4

Under greedy decoding the three runs of each baseline give identical results, so one row is shown. SearchSkill, trained three times with different seeds, stays above every baseline on average.

Paired significance tests against every baseline:

ZeroSearchSearch-o1Search-R1StepSearchSmart-Searcher
McNemar p, largest over all six seeds< 2e-35< 3e-26< 1e-15< 3e-10< 5e-4
Instruct mean gain, paired bootstrap 95% CI+9.5 [+7.9, +10.9]+6.5 [+5.1, +8.0]+4.9 [+3.6, +6.3]+3.2 [+1.7, +4.6]+1.8 [+0.4, +3.2]

Our method consistently outperforms these baselines, and every p-value is below 0.001.

2) A second harness with a different retriever

As stated in the main text, all methods share the same retriever, corpus, top-3 passages, five-search budget, decoding and scoring script. Whether the harness matters is an interesting angle, so we built a second one: BM25 with the index released by Search-R1 over the same wiki-18 corpus, top-3 passages, five searches, greedy decoding and the same scoring. Macro EM under the two harnesses:

HarnessZeroSearchSearch-o1Search-R1StepSearchSmart-SearcherSearchSkill
E5, as in the paper33.235.137.237.840.341.9
BM2528.731.131.333.934.437.7

Under the new harness our lead over the strongest baseline grows from 1.7 to 3.4 EM. We thank the reviewer for this angle, which further shows that our model is stable, and we will add it to the paper.

[1] Tool-Star: Empowering Multi-Tool Collaborative Web Agent via Reinforcement Learning, SIGIR'26. Tool-Star is an RL framework for multi-tool reasoning agents, and its released evaluation data provides the multi-hop test sets: https://github.com/RUC-NLPIR/Tool-Star/tree/main/evaluation/data

AuthorsOpenReview ↗

Response to Reviewer ytwu (Part 2/3)

Q1b: What is their malformed-output rate?

Malformed outputs

Rates of invalid actions that end the episode, in the same harness, on the 3,725 questions of the paper's benchmarks:

Search-R1StepSearchSearch-o1ZeroSearchSmart-SearcherSearchSkill
6.0%0.7%0.2%0.1%0.1%0.16%

Actually, our setting is fair: every model produces a few malformed outputs, ours included. The malformed outputs in App. H are real behaviour of the released Search-R1 checkpoint, not a reproduction error. Below, Search-R1-Instruct (GRPO, 7B) as reported by its authors is compared with our run, in EM:

HotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQAAvg.
Reported by its authors38.634.616.240.042.962.342.739.6
Our run, as in the paper38.535.514.038.434.553.646.137.2
Our run, forced answer41.039.515.038.436.056.648.339.3

To address the concern, we examined why Search-R1 has the highest rate. It was trained to favour searching, and even under the harness it often uses up its five searches without giving an answer, as the example at the end of this part shows. Once we force an answer, EM rises to 39.3, close to the authors' 39.6. In the main text we keep the original setting without forced answers, since it applies equally to SearchSkill and every baseline and is therefore fair to all.

SearchSkill, at 41.9–42.6, still leads by 2.7–3.4 EM, so the comparison in the main table is fair. We will add this analysis to the appendix, and we thank the reviewer for spotting it.

Example: Where did the husband of Muza Niyazova die? Search-R1 finds Saparmurat Niyazov, his home village Gypjak and its distance to Ashgabat in five searches, then stops without an answer; when forced, it answers Ashgabat, which is correct.

AuthorsOpenReview ↗

Response to Reviewer ytwu (Part 3/3)

W2: Data provenance needs clarification. App. D.3 mixes in "failure-replay examples from earlier evaluations", and Table 10 includes 1,445 replay records. It should be confirmed that none of these overlap with the evaluation questions.

Q2: What data were the "earlier evaluations" behind the failure-replay and replay records run on? Is any evaluation question included?

1) Sources of the replay records

The "earlier evaluations" are dev samples scored during bank evolution, not the evaluation sets. The training pool is huge, so random sampling rarely shows the model hard cases; we therefore drew 15 questions at random from the bank-evolution dev data so that training sees these typical errors.

Their failures drive skill revision, and replaying a random sample of them, 15 questions and 43 rows, shows the policy how the revised skills repair those errors. The 1,445 replay records of the paper's Table 10 come from these sources:

RowsSourceSplitEvaluation questions
1,231multi-hop trajectoriestrainnone
171single-hop trajectoriestrainnone
43, from 15 questionsbank-evolution failuresdev samples0 by ID, 1 by normalised text

Dev and evaluation data do carry an overlap risk. A check against every evaluation question finds no ID match and one HotpotQA question by normalised text: row hotpotqa_failure_dev_262 with its two replays, What NIFL Premier Intermediate League team did Sean Connor play for?

We checked the output of every seed of our model, and none answers this question correctly, so it has no effect on the results.

2) Removal and injection controls

To exclude any potential effect, we re-trained the model with identical settings after removing only this question's three rows, and again after removing all 43 failure-replay rows. We further injected 1, 15 or 50 evaluation questions into training on purpose, to measure what overlap does. EM on the paper's benchmarks:

Training dataHotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQAAvg.
With replay, as in the paper40.544.513.043.239.461.951.141.94
Without the overlapping question40.548.015.036.839.062.252.041.93
Without 43 failure-replay rows42.043.017.540.037.762.750.841.96
1 evaluation question injected42.043.517.537.638.963.650.441.93
15 evaluation questions injected40.042.517.536.838.761.950.941.19
50 evaluation questions injected44.041.018.537.637.562.250.641.63
EM on the 50 injected questions, clean → injected38.0 → 52.0

Overall, all EM results fluctuate within the confidence interval. This gives two conclusions. First, moderate overlap has no effect on the benchmark as a whole: the model may answer the overlapping questions, but the changed training data also shifts its answers elsewhere. Second, although the model does better on the injected questions themselves, from 38.0 to 52.0 EM, this gain does not raise the final EM.

This suggestion is especially valuable. We will add an anti-overlap check that removes evaluation questions from the sampled dev data, and the appendix will describe these sources in more detail.

We sincerely thank you again for your helpful suggestions, which have made our evaluation more rigorous and our data provenance clearer. We hope our responses address your concerns and look forward to any further feedback.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Response to Reviewer 94k6 (Part 1/2)

We sincerely thank you for the careful and constructive review. We greatly appreciate your recognition of the matched distillation control, the card ablations and the frozen-bank transfer, and your note that our safeguards deserve credit. Your question on the final training/evaluation boundary guided us to audit every replay record and run removal and injection controls, which makes the held-out interpretation of our results explicit.

W1: The final training/evaluation boundary after replay is added is not sufficiently specified. Appendix D.1 describes initial evaluation-ID overlap removal, B.1 describes normalized-question filtering for the Qwen-generated setting, and D.2 explicitly uses the processed D.1 training pool for bank evolution. These safeguards deserve credit. However, D.3 subsequently adds failure-replay examples from earlier evaluations to the trajectory manifest without identifying those splits or reporting a final overlap check. Table 10 separately lists 1,445 replay records in the 3,017-record SFT pool; it does not say how many derive from earlier-evaluation failures. These are record counts, not distinct evaluation questions. This is not evidence of established contamination, but it leaves the held-out interpretation of the central GPT-teacher training results unresolved. The fixed-bank transfer evidence is not invalidated by this concern.

Q1: Which splits supplied the earlier-evaluation failures added in D.3, and were they distinct from the final evaluation questions? Please report final-manifest overlap counts by ID and normalized question after all replay additions. If overlap is found, what do the unaffected results show? The initial D.1 filtering and D.2 bank provenance are already documented; the question concerns the later additions.

1) Sources and overlap counts

This question is crucial, and we thank the reviewer for the careful questions. The training pool is huge, so random sampling rarely shows the model hard cases; we therefore drew 15 questions at random from the bank-evolution dev data so that training sees these errors.

Their failures drive skill revision, and replaying a random sample of them, 15 questions and 43 rows, shows the policy how the revised skills repair those errors. The 1,445 replay records of the paper's Table 10 come from these sources:

RowsSourceSplitEvaluation questions
1,231multi-hop trajectoriestrainnone
171single-hop trajectoriestrainnone
43, from 15 questionsbank-evolution failuresdev samples0 by ID, 1 by normalised text

Although the 15 questions come from dev data, they do carry a risk of overlapping with the evaluation sets. A check against every evaluation question finds no ID match and one HotpotQA question by normalised text: row hotpotqa_failure_dev_262 with its two replays, What NIFL Premier Intermediate League team did Sean Connor play for?

We checked the output of every seed of our model, and none answers it correctly, so it affects neither the results nor the fairness of the comparison.

2) Removal and injection controls

To exclude the effect of overlap, we re-trained the model with identical settings after removing only this question's three rows, and again after removing all 43 failure-replay rows. We also injected 1, 15 or 50 evaluation questions into training on purpose. EM on the paper's benchmarks:

Training dataHotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQAAvg.
With replay, as in the paper40.544.513.043.239.461.951.141.94
Without the overlapping question40.548.015.036.839.062.252.041.93
Without 43 failure-replay rows42.043.017.540.037.762.750.841.96
1 evaluation question injected42.043.517.537.638.963.650.441.93
15 evaluation questions injected40.042.517.536.838.761.950.941.19
50 evaluation questions injected44.041.018.537.637.562.250.641.63
EM on the 50 injected questions, clean → injected38.0 → 52.0

The average EM of every setting fluctuates within the normal range. First, moderate overlap has no effect on the benchmark as a whole: the model may answer the overlapping questions, but the changed training data also shifts its answers elsewhere. Second, although the model does better on the injected questions themselves, from 38.0 to 52.0 EM, this gain does not raise the final EM.

We thank the reviewer for this suggestion. We will add an anti-overlap check to our pipeline and describe this effect in detail in the appendix.

AuthorsOpenReview ↗

Response to Reviewer 94k6 (Part 2/2)

Q2: What subset, checkpoint, and evaluation scope explain Figure 3's full-bank HotpotQA/2Wiki values of 59.5/50.4, compared with 43.0/44.5 in Figure 2 and the main 7B-Instruct results? Is H.4's eight-query Search-R1 example from a separate diagnostic run rather than the stated five-search setting? These are local interpretation questions; the case study alone does not establish unequal main-table budgets.

1) Figure 3's values 59.5 and 50.4

Figure 3 asks whether an activated skill category is actually needed for the answer, not only selected. Its subset is the questions on which Comparison/Join is activated, 37 of 200 on HotpotQA and 119 of 200 on 2Wiki. The checkpoint and runs are the same 7B-Instruct ones as the main table, and the scope is the paper's 725 multi-hop questions. Comparison/Join in Figure 3 and in the main table:

DatasetActivated questionsCorrectEM on activated subset, Fig. 3Correct on whole benchEM on all 200, main table
HotpotQA372259.58643.0
2Wiki1196050.48944.5

Figure 3 and the main table are different statistical views. The activated questions answered correctly with these skills are included in the correct answers counted by the main table.

2) The eight-query example in H.4

Actually, as stated in the Evaluation paragraph of Sec. 4.1, the main-table setting is fair. The eight-query example does come from a separate diagnostic run of our early Search-R1 reproduction. We further show below that a cap of five or eight searches changes EM little, with the same trend for SearchSkill and Search-R1. EM on the paper's benchmarks with a cap of five or eight searches:

Method, capHotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQAAvg.
Search-R1, 538.535.514.038.434.553.646.137.2
Search-R1, 840.539.015.038.435.755.947.938.9
SearchSkill, seed 42, 540.544.513.043.239.461.951.141.9
SearchSkill, seed 42, 842.542.016.543.239.261.550.942.3

Different search budgets have little effect on the results. We thank the reviewer for this insightful question, and we will add the cases under both the five- and eight-search caps to the appendix.

Thank you very much for the careful reading and the precise questions, which helped us clarify the training/evaluation boundary and the interpretation of our analyses. We hope our answers resolve your concerns and welcome any further discussion.

AuthorsOpenReview ↗

Follow-up to Reviewer 94k6

Dear Reviewer 94k6,

Thank you again for your careful and constructive review. As the discussion period is drawing to a close, we understand that this is a busy time and you may not yet have had the chance to return to our response. To save your time, please allow us to summarize it briefly, together with one additional check.

(1) Final training/evaluation boundary. As you requested, we checked the entire final SFT manifest, after every replay addition, against all 3,725 evaluation questions:

SourceRecordsOverlap by IDOverlap by normalized text
Coverage supplement1,57200
Train-split trajectories1,40200
Failure replay, 15 dev questions4301 question, 3 rows
Total3,01701 question

No seed of our model answers this question correctly, and the controls confirm it has no effect:

SettingMacro EM
As in the paper41.94
Overlapping question excluded from evaluation41.97
Re-trained without the overlapping question41.93
Re-trained without all 43 failure-replay rows41.96
1 / 15 / 50 evaluation questions injected on purpose41.93 / 41.19 / 41.63

(2) Figure 3 and H.4. Figure 3 uses the same checkpoint and runs as the main table, computed only on questions where Comparison/Join is activated (37/200 on HotpotQA, 119/200 on 2Wiki). The eight-query case in H.4 comes from a separate diagnostic run, and a five- or eight-search cap changes the results little.

We would greatly value your thoughts on whether these additions resolve your concerns, and we would be glad to discuss or clarify anything further in the remaining time. Thank you again for the comments that have made our paper stronger.

Best regards,

The Authors

AuthorsOpenReview ↗

Follow-up Response to Reviewer 94k6

Dear Reviewer 94k6,

Before the rebuttal period ends, we would like to add one result on the held-out interpretation raised in your W1.

Held-out interpretation of the training results (W1). You noted that the replay additions leave the held-out reading of the GPT-teacher training results open. We therefore built a result that is held out by construction and does not use the GPT-teacher pool at all. The 3B model trained on its self-generated skills (filtered as in B.1) was further trained with reinforcement learning, where we extend the reward of the paper's GRPO study with process terms matched to the skill framework, namely skill-action consistency and completion of the skill loop. Its training questions come from the training splits with all evaluation questions removed. EM on the paper's benchmarks:

3B modelHotpotQA2WikiMuSiQueBamboogleNQTriviaQAPopQAAvg.
Search-R1-Instruct (GRPO)29.031.010.528.033.656.043.933.14
SearchSkill, GPT-5.4 skills35.541.516.027.237.554.346.636.94
SearchSkill, self-generated skills + skill-framework RL36.543.013.025.640.360.148.438.13

Obtained without the pool in question, this model exceeds the GPT-teacher model by 1.2 EM, so the central conclusion does not rest on the replay additions. As rebuttal time is limited, we present 3B now; the 7B counterpart and the details will be included when we revise our paper.

We would greatly appreciate the chance to discuss this, or any remaining detail, with you before the discussion closes.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.