acceptodds
Under review as a conference paper at ICLR 2027

HybridResearcher: Bridging the Structured-Unstructured Boundary for Deep Research Agents

Abstract

While LLM agents are increasingly capable of gathering and synthesizing evidence from a single homogeneous source, such as structured tables or unstructured data, real-world analytical questions frequently span heterogeneous sources. For example, answering one enterprise question may require both a result computed over an internal database and evidence found on the web. Recent evaluations on benchmarks like HybridDeepResearch show that even frontier agents struggle when evidence must cross this boundary. While these benchmarks have exposed the limitation, methodologies to explicitly teach cross-source reasoning remain absent. To address this, we introduce HybridResearcher, a 4B agent trained to bridge evidence across the structured and unstructured boundary. To build it, we synthesize cross-source bridge trajectories by starting with validated single-source tasks—such as existing text-to-SQL or web search queries—and fabricating the missing source around a bridge entity drawn from the seed's own output. We retain a sample only when strict construction gates confirm that neither side alone suffices, ensuring the data requires true cross-source interdependence. Finally, we train on this curated dataset using supervised fine-tuning followed by reinforcement learning. Evaluated on the existing HybridDeepResearch benchmark, HybridResearcher improves over our base model, Qwen3.5-4B, by 30.0 points pass@8 and surpasses the far larger Qwen3.5-397B-A17B by 8.3 points, while also improving over its base model on established, held-out single-source benchmarks (BrowseComp and BrowseComp-Plus), demonstrating that this complex cross-source reasoning can be taught directly to small models without triggering catastrophic forgetting.

Primary area:
foundation or frontier models, including LLMs
Keywords:
deep research, hybrid deep research, question answering, deep research agent, agentic model training

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_4dKjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies deep-research agents that must carry evidence between a SQL database and web search within one reasoning chain. It builds hybrid training items by extending validated single-source seeds in two directions. Text-to-SQL tasks receive a web leg made of coined facts, with fabricated gold documents and hard negatives. Hard web-search questions receive a synthetic SQLite database that produces or consumes the bridge entity. Gates are meant to ensure that neither source alone suffices. Qwen3.5-4B is fine-tuned on these items plus existing HyDRA and text-to-SQL data, and then trained with RL using a five-vote LLM judge. On the 120-question hard subset of HybridDeepResearch, run in the authors' local corpus and harness, pass@8 rises from 30.83 to 60.83, above Qwen3.5-397B (52.50) and below Claude Sonnet 4.6 (65.83). Single-source performance on BrowseComp, BrowseComp-Plus and Spider 2.0-lite also improves.

Critical strengths
  1. Extending validated single-source seeds across the boundary, with coined facts and anti-shortcut gates, is a practical way to obtain hard, checkable hybrid data.
  2. The gains are large for a 4B model and do not hurt single-source ability: BrowseComp and BrowseComp-Plus improve by 8–12 pass@8 points (Table 2).
  3. The paper is clearly written, the construction is documented in enough detail to reproduce, and the per-rollout failure analysis (App. I) is more informative than what such papers usually provide.
Critical weaknesses
  1. All hybrid results come from a single 120-question benchmark run in the authors' own local corpus and harness. That is the same environment, including ~2.4M fabricated documents, in which the 4B model was trained, while the larger baselines see it zero-shot.
  2. There is no variance. With single runs, the "beats Qwen3.5-397B" result (27.19 vs. 26.77 avg@8), the RL gain over SFT (+1.7 pass@8) and the RL ablations (1.6–2.0 points) are hard to separate from noise.
  3. It is unclear how much of the gain comes from the proposed construction. Existing HyDRA plus raw text-to-SQL data already lifts pass@8 from 30.83 to 47.50, leaving about 12 points for the new data.
  4. The anti-shortcut gates, the paper's core idea, are never ablated against ungated hybrid synthesis, and they are not uniform. For search-originated items, nothing checks that the web side is unsolvable without the bridge, and the App. D.3 example looks answerable from the shark-sequel clue alone.
  5. Possible overlap between the HDR test set and the training sources (HyDRA, BIRD/Spider seed databases) is not discussed.
Questions for authors
  1. What are the HDR hard-subset results with live web search, for your model and for the baselines?
  2. At fixed data volume, how does SFT on hybrid items built without the gates compare to the full recipe?
  3. How was overlap between the HDR hard subset and HyDRA or the BIRD/Spider seed databases ruled out, and how much do results vary across training seeds?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 4dKj (Part 1)

Thank you for your review and for appreciating that "gains are large", our methods has "enough detail to reproduce" and our analysis "is more informative than what such papers usually provide". Please refer to our response to your comments below:

W1 + Q1. Evaluation with Live Web Search

We evaluated on the HybridDeepResearch hard subset with live web search, replacing the local retrieval service with the same live search and page-fetching tools used for BrowseComp (Appendix H.1). The live web contains none of our fabricated documents, so every model faces this environment zero-shot.

ModelAvg@8 (%)Run Std (%)95% CI of Avg@8 (%)Pass@8 (%)95% CI of Pass@8 (Bootstrap) (%)
Qwen3.5-4B (Base)10.832.52[8.73, 12.94]38.33[30.00, 47.50]
HybridResearcher-4B (+ SFT)22.813.42[19.95, 25.67]55.83[46.67, 64.17]
HybridResearcher-4B (+ RL)25.422.92[22.97, 27.86]59.17[50.00, 67.50]

As shown in the table above, with live web search, SFT and RL improve over the base model by 17.50 and 20.84 points in Pass@8 and 11.98 and 14.59 points in Avg@8 (p<0.001p < 0.001). These results show that the gains are not an artifact of the local corpus or the fabricated documents and transfer to the open web.

W2: Variance and Statistical Significance

Thank you for this point. We evaluated our models across 8 independent runs on the 120-question HybridDeepResearch benchmark:

ModelAvg@8 (%)Run Std (%)95% CI of Avg@8 (%)Pass@8 (%)95% CI of Pass@8 (Bootstrap) (%)
Qwen3.5-4B (Base)7.712.17[5.89, 9.52]30.83[22.50, 39.17]
HybridResearcher-4B (+ SFT)23.752.52[21.64, 25.86]59.17[50.00, 68.33]
HybridResearcher-4B (+ RL)27.192.81[24.83, 29.54]60.83[51.67, 69.17]

As shown in the table above, both SFT and RL outperform the base model by more than 16.0 points in Avg@8 (p<0.001p < 0.001) and more than 28.3 points in Pass@8. RL further improves over SFT by 3.44 points in Avg@8 (Welch's tt-test, p=0.02p = 0.02). With these error bars, HybridResearcher-4B remains comparable to much larger frontier models such as Claude Sonnet 4.6 (65.83 Pass@8). We will include these results in the final version.

W3. Contribution of the Proposed Data

To our data’s contribution, we compare against HyDRA-based mixtures at matched training-set sizes. For each comparison, we sample the same number of rows from our SFT mixture while keeping the final recipe's per-source proportions. All other settings follow the final recipe.

SFT Training DataRowsOverall Pass@8 (%)S2SQL Pass@8 (%)SQL2S Pass@8 (%)Parallel Pass@8 (%)Overall Avg@8 (%)
Base Qwen3.5-4B-30.8322.5020.0050.007.71
HyDRA only (10k)10k45.8347.5025.0065.0013.33
(i) Ours (no HyDRA), matched10k53.3355.0027.5077.5021.04
HyDRA + Raw SQL (Table 13, 36k)36k47.5045.0022.5075.0013.96
(ii) Ours (Synthesized Only), matched36k56.6760.0027.5082.5021.35
Full SFT (Ours + HyDRA + Raw SQL)101k59.1765.0035.0077.5023.75

At the same size, our synthesized data alone outperforms HyDRA + raw SQL by 9.17 Pass@8 and 7.39 Avg@8, and it is better on all three task types. HyDRA + raw SQL barely improves SQL2S (20.00 →\rightarrow 22.50), the direction where the web search must start from a SQL result, while our data targets this bridging with multi-hop constraints and hard negatives. When combined with existing data, our data adds 20.0 / 12.5 Pass@8 points on S2SQL / SQL2S and raises Avg@8 from 13.96 to 23.75 (+70% relative). In addition, all RL data is constructed by our pipeline, so the RL gains also come from our data. We will include this experiment in the final version.

AuthorsOpenReview ↗

Response to Reviewer 4dKj (Part 2)

W4 + Q2. Cross-Source Necessity and the Anti-Shortcut Gates

The gates are applied during construction, and rejected items, along with their documents and trajectories, were not retained. Rebuilding an ungated set at matched volume would require rerunning the full synthesis and teacher rollouts, which is not feasible within the rebuttal period. We instead compare trajectories with and without the gates. Without the gates, 79.5% of accepted teacher trajectories never execute a substantive SQL query (Section 3.1). With the gates, nearly all trajectories do, and most carry an entity returned by SQL into a later web search:

SubsetExecute non-schema SQL (%)Use an SQL-returned entity in a later search (%)
SQL-originated SQL2S99.9983.90
SQL-originated S2SQL100.0080.30
Search-originated SQL2S100.0070.33
Search-originated S2SQL100.0068.30

Since single-source trajectories do not improve hybrid performance at a matched budget (Section 3.1), an ungated set would be unlikely to teach the cross-source step. For S2SQL items, the answer is the result table of a database that exists only within the item, so it cannot be obtained without SQL, and the hidden anchor must be resolved on the web before the correct query can be written. For SQL-originated SQL2S items, the web facts are coined and pass a no-document probe and a mask test, so they cannot be found without the bridge returned by SQL, while the answer itself lies in the web documents. For search-originated SQL2S items, the answer also lies on the web, and the gate guarantees that the bridge can only be obtained through SQL.

W5 + Q3 : No Overlap Between Training Data and the Evaluation Set

When constructing our training data, we ensured that there is zero overlap between it and HybridDeepResearch (HDR). We also ensured that the training data and HDR use disjoint databases. HDR is built on LiveSQLBench, a benchmark designed to be contamination-free, whose databases do not appear in BIRD, Spider, or any other training source. The databases behind HyDRA come from BIRD and Spider, and our SQL-originated seeds come from the sources in Table 7, none of which include LiveSQLBench. Each search-originated item ships with its own newly synthesized database.
Repeating the full SFT and RL pipeline across seeds was beyond our compute budget. However, several results indicate that the gains are not due to training noise. The gain over the base model (+19.48 Avg@8) is about seven times the run-to-run standard deviation. The SFT ablation (Table 13) and the matched-size comparison (response to W3) each come from separately trained models, and adding our data improves performance in every case. The gains also hold under live web retrieval (response to W1), and three single-source benchmarks (Table 2).

Thanks for the rebuttal, which addressed most of the concerns. I will raise my score. Please ensure to include these results and discussions into the revised paper.

Reviewer_P439OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper goes after a gap nobody had really trained for: questions where a result from a SQL database has to be carried into a web search, or vice versa. Their key move is not building data from scratch — they take already-validated single-source seeds (a text-to-SQL task, or a hard search question), then fabricate the missing source around a "bridge entity" pulled from the seed's own output, so difficulty is inherited for free.

Critical strengths
  1. The diagnosis is sharp and it's measured, not asserted. They actually ran the obvious baselines — finetune on SQL-only, search-only, and the union — and showed none of them help (31.67 / 15.83 / 29.17 vs. base 30.83). And they quantify the shortcut problem at 79.5%. That reframes the whole paper from "we made more data" to "here's why the obvious thing fails, and here's the fix." Most hybrid-data papers skip this step entirely.

  2. The data construction is the real contribution. Seeding from validated tasks means you don't have to re-establish difficulty. The anti-shortcut gates are deterministic wherever possible — membership gate, SQL-AST execution check, singleton intersection, mask test — rather than "ask a judge if it's hard." Coined pseudo-word facts plus typed hard negatives that break exactly one constraint is a genuinely clever touch: it makes parametric recall useless by construction.

  3. Results hold up where it counts, and the analysis explains them. Beating your own 397B flagship with a 4B is a nice headline, but Table 2 is more convincing: BrowseComp, BrowseComp-Plus and Spider 2.0-lite all improve after hybrid training, so no catastrophic forgetting. And §6 actually opens the box — SFT teaches reading over blind searching, RL sharpens queries and cuts repeats — instead of just reporting deltas.

Critical weaknesses
  1. Everything rests on one 120-question benchmark. That's small, and it's the same benchmark that motivated the whole design, so the synthesis recipe is arguably shaped to fit it. Ordering a 4B at 60.8 against Claude's 65.8 on 120 items needs error bars or a second hybrid benchmark before I'd trust it. Also, training uses a frozen 21M-document local corpus while BrowseComp evaluates against the live web — that transfer is assumed, not tested.

  2. RL's gains are uneven and partly negative, and the paper glosses over it. SQL2S pass@8 drops 35.0 → 30.0 after RL, and Spider 2.0-lite drops 65.19 → 62.96. The headline only quotes avg@8. With 8 samples per question, no seed variance, and no significance tests, I can't tell which deltas are real. Worse, the judge (Qwen3.8-27B) is same-family as the base model being improved — that deserves an ablation.

Questions for authors

see weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer P439 (Part 1)

Thank you for your review and for highlighting that our data construction is “a genuinely clever touch” and our diagnosis is “sharp and measured”. Please refer to our response to your comments below:

W1. (a). Independent Training Data

None of our training data is derived from HybridDeepResearch. Our constructed hybrid tasks are seeded from public text-to-SQL datasets (Appendix B, C) and public deep-research questions sets (Appendix D.1, Table 10), and all fabricated databases, documents, and questions are generated from these seeds. The design choice in Section 3 target failure modes we measured on our own training data, rather than properties of the HybridDeepResearch benchmark.

W1. (b). Evaluation Scale and Uncertainty

We evaluate on the official balanced hard subset of HybridDeepResearch (120 questions). Following your suggestion, we report variation across the 8 independent runs and 95% confidence intervals for both metrics:

ModelAvg@8 (%)Run Std (%)95% CI of Avg@8 (%)Pass@8 (%)95% CI of Pass@8 (Bootstrap) (%)
Qwen3.5-4B (Base)7.712.17[5.89, 9.52]30.83[22.50, 39.17]
HybridResearcher-4B (+ SFT)23.752.52[21.64, 25.86]59.17[50.00, 68.33]
HybridResearcher-4B (+ RL)27.192.81[24.83, 29.54]60.83[51.67, 69.17]

As shown in the table above, both SFT and RL outperform the base model by more than 16.0 points in Avg@8 (p<0.001p < 0.001) and more than 28.3 points in Pass@8. RL further improves over SFT by 3.44 points in Avg@8 (Welch's tt-test, p=0.02p = 0.02). With these error bars, HybridResearcher-4B remains comparable to much larger frontier models such as Claude Sonnet 4.6 (65.83 Pass@8). We will revise Section 5.1 to state this comparison precisely and add the confidence intervals to Table 1 in the final version.

W1. (c). Evaluating HybridDeepResearch Against the Live Web

Thank you for this point. We already evaluated BrowseComp on the live web (Table 2), where HybridResearcher-4B improves over the base model by 8.21 Pass@8 and 9.33 Avg@8. To test the transfer directly on hybrid tasks, we further evaluated our models on the hard-set HybridDeepResearch benchmark over 8 independent runs using live web search (replacing the local retrieval corpus with live web search and page-visiting tools while keeping the judge unchanged):

ModelAvg@8 (%)Run Std (%)95% CI of Avg@8 (%)Pass@8 (%)95% CI of Pass@8 (Bootstrap) (%)
Qwen3.5-4B (Base)10.832.52[8.73, 12.94]38.33[30.00, 47.50]
HybridResearcher-4B (+ SFT)22.813.42[19.95, 25.67]55.83[46.67, 64.17]
HybridResearcher-4B (+ RL)25.422.92[22.97, 27.86]59.17[50.00, 67.50]

As shown in the table above, SFT and RL improve over the base model by 11.98 and 14.59 points in Avg@8 (p<0.001p < 0.001) and by 17.50 and 20.84 points in Pass@8, and RL achieves the best performance on both metrics (25.42% Avg@8 and 59.17% Pass@8). These results show that the cross-source reasoning learned with the local corpus transfers to open-web retrieval. We will add this experiment in the final version.

AuthorsOpenReview ↗

Response to Reviewer P439 (Part 2)

W2. (a). Post-RL Changes

As observed in prior work [1-3], RL sharpens the policy toward high-reward trajectories, which improves per-sample accuracy while reducing sample diversity and therefore Pass@k. Our results follow this pattern. On SQL2S, Avg@8 rises from 8.75 to 12.81 (+46% relative), while Pass@8 changes by 2 of 40 questions. This gain comes from more reliable use of retrieved evidence: when the gold answer appears in a search result, RL answers correctly in 27.2% of rollouts, compared with 18.5% for SFT (Table 3, Section 6.1). Over the full hard subset, RL improves Avg@8 over SFT by 3.44 points, which is statistically significant (p=0.02p = 0.02, see our response to W1).

Our RL data consists entirely of long-horizon hybrid tasks that require multiple searches, page visits, and SQL queries. Spider 2.0-lite is a SQL-only task with the database schema given in the prompt. Our models use about 20 tool calls per rollout on it, all SQL queries, compared with about 55 calls on HybridDeepResearch spread across SQL, web search, page visits, and knowledge lookups. RL therefore does not target this setting, and Avg@8 on Spider 2.0-lite stays nearly unchanged after RL (41.39 →\rightarrow 40.83). The Pass@8 difference corresponds to 3 of 135 questions. After RL, the model remains well above the base model on both metrics (+3.70 Pass@8 / +10.18 Avg@8).

[1] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models, arXiv 2024.
[2] Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?, NeurIPS 2025.
[3] The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models, arXiv 2025.

W2. (b). Judge Robustness Across Model Families

Thank you for this suggestion. We re-scored all 2,880 trajectories (3 models ×\times 8 runs ×\times 120 questions) using three independent judge families (GLM-5.3, GPT-5.5, and Claude-Sonnet-5.5) using the same judge prompt and 5-vote majority as our original Qwen3.8-27B judge:

Evaluated ModelQwen3.8-27B (Our Judge in Paper) Pass@8 / Avg@8 (%)GLM-5.3 Pass@8 / Avg@8 (%)GPT-5.5 Pass@8 / Avg@8 (%)Claude-Sonnet-5.5 Pass@8 / Avg@8 (%)
Qwen3.5-4B (Base)30.83 / 7.7130.83 / 8.3330.83 / 7.9230.83 / 7.71
HybridResearcher-4B (+ SFT)59.17 / 23.7557.50 / 23.3356.67 / 22.7157.50 / 22.60
HybridResearcher-4B (+ RL)60.83 / 27.1960.83 / 27.4060.00 / 26.0461.67 / 26.67

All four judges give the same ranking (RL > SFT > Base). Across judges, Avg@8 varies by at most 1.36 points, within the run-to-run standard deviation (2.17 to 2.81), and Pass@8 by at most 2.50 points (3 of 120 questions). The Qwen judge also does not favor our models: the highest Pass@8 for our RL model comes from Claude Sonnet 5.5. Since Qwen3.8-27B is also the RL reward judge, the consistent RL gains under all three non-Qwen judges show that these gains are not an artifact of the reward judge. We will add this analysis in the final version.

AuthorsOpenReview ↗

Follow-up reminder for Reviewer P439

Dear Reviewer P439, we appreciate the time and effort you've put into reviewing our submission. As the discussion period ends on Nov 18, we want to follow up and see if our response has addressed your comments. In particular, we have (1) reported run-to-run variance and 95% confidence intervals on the official balanced hard subset, (2) evaluated HybridDeepResearch with live web search, where our models keep large gains over the base model, (3) explained that RL mainly improves per-rollout consistency (Avg@8), which is in line with prior work, and (4) re-scored all trajectories with three judges from other model families, which give the same ranking and confirm that the RL gains are not an artifact of the Qwen reward judge. We welcome any further discussion on your comments and are happy to clarify additional points before the discussion period ends.

Comments on the paper

5 comments
AuthorsOpenReview ↗

Response to Reviewer 9Mbb (Part 1)

Thank you for your review and for highlighting that our paper has “a practically relevant target”, “meaningful empirical gains”, and “useful experimental coverage”. Please refer to our response to your comments below:

W1 + Q1. Contribution of Our Data Under Matched Budgets (SFT data)

Part of this comparison is already in Section 3.1, where SQL-only, Search-only, and Union SFT controls are trained with a matched budget of 10k examples; we will state this budget explicitly in the final version. We extend it with HyDRA only and with the final mixture without HyDRA, both at 10k examples:

SFT Dataset (10k Matched Budget)Overall Pass@8 (%)S2SQL Pass@8 (%)SQL2S Pass@8 (%)Parallel Pass@8 (%)Overall Avg@8 (%)
Base Qwen3.5-4B (No SFT)30.8322.5020.0050.007.71
SQL-only31.6725.007.5062.506.98
Search-only15.830.0010.0037.503.85
Union (SQL-only + Searc-only)29.170.0017.5070.009.90
HyDRA-only45.8347.5025.0065.0013.33
Ours (final mixture w/o HyDRA)53.3355.0027.5077.5021.04

At the same budget, single-source data does not improve over the base model, and the union drops S2SQL to 0.00, so single-source trajectories do not teach the bridge between sources. The final mixture without HyDRA outperforms HyDRA only by 7.50 Pass@8 and 7.71 Avg@8 and is better on all three task types. These results show that the gains come from our cross-source data rather than training volume.

W1 + Q1. RL Data Curation and RL vs. Continued SFT

The curation comparison in Section 6.2 evaluates our curation procedure as a whole against no curation, using the same training recipe and the same number of RL steps, so the training compute is matched. Its effect is visible at the level of each rollout group: 71% of groups in the curated set provide reward contrast for policy updates, compared with 38% in the uncurated pool. This ratio does not depend on dataset size, so it directly reflects the difficulty filter.

Regarding continued SFT, RL learns from both successful and failed rollouts, while continued SFT on successful trajectories learns from successes only. Our hard gating (Section 6.4) relies on this difference. It assigns zero reward to degenerate behaviors such as repeated tool calls, which reduces repeated calls from 49% to 11% and rollouts without a valid answer from 10.7% to 0.0%. Continued SFT on successful trajectories provides no direct signal against such behaviors.

W2 + Q2. Cross-Source Necessity and the Anti-Shortcut Gates

The gates are applied during construction, and rejected items, along with their documents and trajectories, were not retained. Rebuilding an ungated set at matched volume would require rerunning the full synthesis and teacher rollouts, which is not feasible within the rebuttal period. We instead compare trajectories with and without the gates. Without the gates, 79.5% of accepted teacher trajectories never execute a substantive SQL query (Section 3.1). With the gates, nearly all trajectories do, and most carry an entity returned by SQL into a later web search:

SubsetExecute non-schema SQL (%)Use an SQL-returned entity in a later search (%)
SQL-originated SQL2S99.9983.90
SQL-originated S2SQL100.0080.30
Search-originated SQL2S100.0070.33
Search-originated S2SQL100.0068.30

Since single-source trajectories do not improve hybrid performance at a matched budget (Section 3.1), an ungated set would be unlikely to teach the cross-source step. For S2SQL items, the answer is the result table of a database that exists only within the item, so it cannot be obtained without SQL, and the hidden anchor must be resolved on the web before the correct query can be written. For SQL-originated SQL2S items, the web facts are coined and pass a no-document probe and a mask test, so they cannot be found without the bridge returned by SQL, while the answer itself lies in the web documents. For search-originated SQL2S items, the answer also lies on the web, and the gate guarantees that the bridge can only be obtained through SQL.

AuthorsOpenReview ↗

Response to Reviewer 9Mbb (Part 2)

W3. Submission Rates and Cross-Source Reasoning

Thank you for this suggestion. We measured the share of rollouts that submit an answer for the single-source SFT controls in Section 3.1:

ModelSubmission Rate (%)Overall Pass@8 (%)Overall Avg@8 (%)
Base Qwen3.5-4B49.230.837.71
Search-only SFT13.015.833.85
SQL-only SFT44.231.676.98
Union (SQL + Search) SFT32.429.179.90
Hybrid SFT (Ours)96.859.1723.75
Hybrid RL (Ours)100.060.8327.19

The single-source controls are trained on agent trajectories that end with a submitted answer, yet they submit less often than the base model on hybrid tasks. Submitting an answer here requires completing the cross-source chain, for example resolving an entity on the web before writing the SQL query in S2SQL, so the submission rate itself reflects cross-source ability rather than protocol adherence alone. Missing answers also do not explain the gap in Pass@8. The base model submits a valid answer in at least one rollout for 98 of 120 questions, but solves only 37 of them. Even if it solved all 22 remaining questions, its Pass@8 would be at most 49.17%, below both our SFT (59.17%) and RL (60.83%) models. The base model's missing answers mainly come from degenerate generation, where a single reasoning turn repeats until it exhausts the 32K-token limit (43.2% of its rollouts, Appendix I.1).

W4. Uncertainty Estimates of Key Comparisons

We evaluated 8 independent runs on the 120-question HybridDeepResearch benchmark:

ModelAvg@8 (%)Run Std (%)95% CI of Avg@8 (%)Pass@8 (%)95% CI of Pass@8 (Bootstrap) (%)
Qwen3.5-4B (Base)7.712.17[5.89, 9.52]30.83[22.50, 39.17]
HybridResearcher-4B (+ SFT)23.752.52[21.64, 25.86]59.17[50.00, 68.33]
HybridResearcher-4B (+ RL)27.192.81[24.83, 29.54]60.83[51.67, 69.17]

As shown in the table above, both SFT and RL outperform the base model by more than 16.0 points in Avg@8 (p<0.001p < 0.001) and more than 28.3 points in Pass@8. RL further improves over SFT by 3.44 points in Avg@8 (Welch's tt-test, p=0.02p = 0.02). With these error bars, HybridResearcher-4B remains comparable to much larger frontier models such as Qwen3.5-397B and Claude Sonnet 4.6. We will include these results in the final version.

W5. Judge Accuracy and Robustness

Thank you for this suggestion. Judges from different model families are unlikely to share the same systematic errors. We re-scored all 2,880 trajectories (3 models ×\times 8 runs ×\times 120 questions) using three independent judge families (GLM-5.3, GPT-5.5, and Claude-Sonnet-5.5) using the same judge prompt and 5-vote majority as our original Qwen3.8-27B judge:

Evaluated ModelQwen3.8-27B (Our Judge in Paper) Pass@8 / Avg@8 (%)GLM-5.3 Pass@8 / Avg@8 (%)GPT-5.5 Pass@8 / Avg@8 (%)Claude-Sonnet-5.5 Pass@8 / Avg@8 (%)
Qwen3.5-4B (Base)30.83 / 7.7130.83 / 8.3330.83 / 7.9230.83 / 7.71
HybridResearcher-4B (+ SFT)59.17 / 23.7557.50 / 23.3356.67 / 22.7157.50 / 22.60
HybridResearcher-4B (+ RL)60.83 / 27.1960.83 / 27.4060.00 / 26.0461.67 / 26.67

All four judges give the same ranking (RL > SFT > Base). Across judges, Avg@8 varies by at most 1.36 points, within the run-to-run standard deviation (2.17 to 2.81), and Pass@8 by at most 2.50 points (3 of 120 questions). The Qwen judge also does not favor our models: the highest Pass@8 for our RL model comes from Claude Sonnet 5.5. Since Qwen3.8-27B is also the RL reward judge, the consistent RL gains under all three non-Qwen judges show that these gains are not an artifact of the reward judge. We will add this analysis in the final version.

AuthorsOpenReview ↗

Response to Reviewer 9Mbb (Part 3)

Q3. Post-RL Changes

As observed in prior work [1-3], RL sharpens the policy toward high-reward trajectories, which improves per-sample accuracy while reducing sample diversity and therefore Pass@k. Our results follow this pattern. On SQL2S, Avg@8 rises from 8.75 to 12.81 (+46% relative), while Pass@8 changes by 2 of 40 questions. This gain comes from more reliable use of retrieved evidence: when the gold answer appears in a search result, RL answers correctly in 27.2% of rollouts, compared with 18.5% for SFT (Table 3, Section 6.1). Over the full hard subset, RL improves Avg@8 over SFT by 3.44 points, which is statistically significant (p=0.02p = 0.02, see our response to W4).

Our RL data consists entirely of long-horizon hybrid tasks that require multiple searches, page visits, and SQL queries. Spider 2.0-lite is a SQL-only task with the database schema given in the prompt. Our models use about 20 tool calls per rollout on it, all SQL queries, compared with about 55 calls on HybridDeepResearch spread across SQL, web search, page visits, and knowledge lookups. RL therefore does not target this setting, and Avg@8 on Spider 2.0-lite stays nearly unchanged after RL (41.39 →\rightarrow 40.83). The Pass@8 difference corresponds to 3 of 135 questions. After RL, the model remains well above the base model on both metrics (+3.70 Pass@8 / +10.18 Avg@8).

[1] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models, arXiv 2024.
[2] Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?, NeurIPS 2025.
[3] The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models, arXiv 2025.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.