acceptodds
Under review as a conference paper at ICLR 2027

U2: Can Models Learn to Ask the Unseen and the Unsolvable?

Abstract

As models become increasingly capable, recent benchmarks have introduced challenging questions that require models to perform complex reasoning and utilize external tools, such as web search or coding sandboxes. However, these benchmarks are costly to construct and typically demand significant effort from human experts for question design and annotation. Consequently, these datasets are difficult to scale up and update over time, posing a challenge as models risk training on similar, static questions. To address this problem, we propose training a question generator that takes academic materials, such as research papers, as input and outputs complex questions requiring a comprehensive understanding of the source text. Specifically, we employ reinforcement learning (RL) and design several rewards to incentivize the model to generate increasingly difficult questions. The most critical of these is the “difficulty reward,” which utilizes another powerful model to answer the generated questions during rollout. This solver model yields an answer distribution, where higher uncertainty indicates a more challenging question. Following training, we employ the generator to construct the Unseen and nsolvable mark (), composed of three 500-question tracks: (1) open-ended questions requiring comprehensive, citation-grounded reports; (2) fact-heavy non-STEM QA requiring extensive searching and reasoning; and (3) calculation-heavy STEM QA requiring source-grounded derivation and computation. Across these fixed evaluation sets, frontier models equipped with external tools retain substantial room for improvement. We further introduce , a fully self-evolving setting in which the question generator, difficulty model, and target solver are all derived from the same base model. The generator produces questions that the base model itself finds difficult, and the validated questions are then used to train a fresh solver initialized from the same base model. The resulting GPT-OSS-20B model improves over the base by 4.8, 3.3, and 3.6 points on GAIA, FRAMES, and HLE.

Primary area:
datasets and benchmarks
Keywords:
Deep Research, Data Synthesis, Reinforcement Learning, Self Improvement

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_XVUyOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes U2, a framework for learning to generate challenging, source-grounded questions from academic literature. A GPT-OSS-20B generator is optimized using GRPO with format-dependent rewards and a factual-consistency gate supplied by an SFT-trained verifier. For short-form questions, difficulty is measured through sampled-answer entropy from a tool-enabled solver that is not given the source paper. Multi-agent procedures filter questions and construct reference answers.

The framework produces U2-Bench, which has three 500-question tracks: research reports, fact-heavy non-STEM QA, and calculation-heavy STEM QA. Validation includes model and tool-access comparisons, generator checkpoint evaluations, human assessment of 82 question–answer pairs, and a Fact500 ambiguity audit with a controlled repair intervention. A same-base U2-Train experiment reports improvements of 4.8, 3.3, and 3.6 percentage points on GAIA, FRAMES, and HLE, respectively.

Critical strengths
  1. The paper directly investigates whether ambiguity can inflate the difficulty reward. Section 6 audits all 500 Fact500 questions without revealing the intended source or reference answer. For the 20 identified defects, the authors compare three versions: the original question, a minimal repair that restores the missing constraint, and a placebo paraphrase. Repairs reduce normalized entropy by 0.139 relative to placebo, and the reported 95% confidence interval excludes zero. This supports the interpretation that omitted constraints increase entropy within the identified defective items. It directly addresses a central failure mode of the proposed reward.

  2. The paper demonstrates the feasibility and practical utility of building literature-grounded evaluations automatically. The framework produces three distinct 500-question evaluation tracks and demonstrates their use for both model assessment and downstream training. The matched source-access analysis on Fact500 is particularly useful. Accuracy increases from 55.6% without tools to 75.8% with web search and code, and to 86.2% when the intended source is supplied (Appendix C.2). These results support the utility of the generated questions for evaluating evidence acquisition and source-grounded problem solving. U2-Train also provides preliminary evidence of training utility through improvements on three external benchmarks.

Critical weaknesses
  1. Major — Existing validation does not sufficiently establish the robustness of benchmark scores or the capability interpretations drawn from them.

The paper includes meaningful validation: human evaluation, error-taxonomy analysis, a complete Fact500 ambiguity audit, and independent Python calculations. Nevertheless, important gaps remain in connecting these checks to the reliability of the reported scores.

Section 3.2 reports that 84.1% of the 82 human-reviewed question–reference pairs were fully correct. Appendix A.5 distinguishes uniqueness defects, scoring issues, granularity disagreements, and reference-answer errors, so the remaining 15.9% should not be read as a reference-error or ambiguity rate. However, several things remain unclear: the sample’s track composition and representativeness, the inter-annotator agreement, and whether the identified corrections were incorporated into the evaluated snapshot.

Scoring sensitivity is substantial. GPT-5.5 receives 75.8% under the LLM judge but 42.2% under rule-based scoring (Section 5.2). This discrepancy alone does not show that the judge is wrong, since accepting semantically equivalent answers could explain part of it. However, independent expert adjudication of actual solver outputs is needed to determine whether the difference reflects valid flexibility or erroneous acceptance.

For Calc500, Appendix D.1 reports independent Python solutions, but not their coverage, agreement rate, or how disagreements were handled. Appendix E explicitly acknowledges the absence of an independent validity audit. Consequently, attributing Calc500 score differences specifically to assumption selection and multi-step derivation remains insufficiently supported.

These gaps materially limit conclusions about benchmark difficulty and model capability. Independent experts should separately validate question uniqueness, reference correctness, and judge decisions on stratified samples from both QA tracks. The authors should also report whether correcting the identified Fact500 defects and reference errors changes the principal results.

  1. Major — The experiments demonstrate overall training benefits but do not isolate the additional contribution of entropy-based difficulty selection.

Appendix C.1 compares the generator before and after training under a fixed source pool and screening policy. With 500 candidates at each checkpoint, usable yield increases from 3.6% at step 0 to 28.4% at step 200. This supports the effectiveness of the combined training procedure within the stated protocol. Figure 5 also provides a meaningful comparison between difficulty oracles.

However, as Appendix C.1 acknowledges, these experiments do not separate the difficulty reward from the formatting, grounding, and other rewards. They establish improved generation under the combined objective, but not the specific added value of entropy-based optimization.

Similarly, U2-Train’s improvements over the base model show positive transfer after additional training. They do not establish that entropy-selected questions outperform equally sized, otherwise comparable training data. The gains could come from additional optimization, source content, answer verification, or dataset composition. Appendix E acknowledges the missing matched training-data controls and the lack of training-seed variability, but these remain consequential evidence gaps.

The decisive controls are:

RL without the difficulty reward. Solver training on randomly selected versus entropy-selected questions, with dataset size, verification, and training budget matched.

Comparing these variants on held-out sources and held-out solvers would test whether the differences generalize beyond the selection protocol. Multiple training seeds would establish whether the downstream gains are reproducible. A compute-matched generate-and-filter baseline would also clarify whether RL offers an advantage over simply generating more candidates.

  1. Minor issues and reporting corrections (not decisive for my recommendation):

Section 5.1 contains an arithmetic inconsistency. On 500 questions, an increase from 55.6% to 75.8% corresponds to 278 versus 379 correct answers, a net increase of 101. The reported 135 recoveries and 16 regressions instead imply a net increase of 119. These cannot describe the same paired evaluation and require clarification. The representative examples do not substantiate the claimed complexity. Table 7 effectively supplies the requested answer category within the question. Table 8 supplies the formula and all required quantities, leaving only substitution and unit conversion. These examples do not show that the entire benchmark is easy, but they are weak illustrations of extensive searching or source-grounded derivation. Some reproducibility details are insufficiently specified. The paper provides the optimization objectives, oracle sample count, entropy threshold, and annotation committee configuration. However, the following need further specification:

  • the actual reward coefficients;
  • the entropy-sampling temperature and canonicalization details;
  • the verifier’s training size and held-out performance;
  • the U2-Train dataset size, solver reward, and training configuration.
Questions for authors

Clarification requests:

  1. Human validation and corrections: How were the 82 reviewed pairs sampled, and how are they distributed across tracks? What was the inter-annotator agreement? How were the errors and disagreements identified in Appendix A.5 reflected in the final references and the reported scores?
  2. Fact500 audit and evaluated snapshot: Were the 20 defects identified in Section 6 removed or repaired before the reported evaluations? If not, how do the principal results change after correction? Was an independently reviewed sample used to estimate defects that the audit may have missed?
  3. Scoring and numerical verification: What are the evaluation judge’s exact model ID and complete prompt? Are expert judgments available for solver outputs on which the scoring methods disagree? For the independent Python solutions in Appendix D.1, what proportion of Calc500 was covered, what was the agreement rate, and how were disagreements resolved?

Clarification and requested controls (could materially affect my assessment):

U2-Train protocol and controls: What are the training-set size and track proportions, the answer-generation and verification models, and the GSPO reward? Which external benchmark splits, sample sizes, and evaluation budgets were used, and how was overlap between training and evaluation data checked? Are results available for removing the difficulty reward, or for training on matched randomly selected data?

6 comments

Thank you for recognizing the source-blind audit, controlled repair experiment, source-access analysis, and training application. We clarify the evidence supporting benchmark quality and distinguish the contributions established by the existing experiments.

Weakness 1 + Questions 1–3: Reliability of the benchmark, reference answers, and evaluation.

Response. Our quality-control procedure combines source-grounded generation, reference-answer adjudication, professional human review, and source-blind validity analysis. These stages address different aspects of benchmark quality.

During generation, an SFT-trained factual verifier checks each reasoning paragraph against its source context. Reference construction uses independent answer proposals, cross-critique, and rotated adjudication, with tolerance-aware numerical agreement for calculation-heavy questions.

The professional review of 82 question–reference pairs reports 84.1% fully correct references, 98.8% question relevance, and 96.3% full source support. Answer uniqueness is assessed separately, with 89.0% judged to admit a single clear answer. As explained in Appendix A.5, the remaining non-fully-correct cases include distinct categories such as reference incompleteness, granularity disagreements, and uniqueness defects. They should not be combined into a single ambiguity rate. We will clarify the sampling frame and annotation protocol; the original study does not report inter-annotator agreement.

The complete Fact500 source-blind audit independently tests question specification by searching for materially different supported answers satisfying every explicit constraint. The controlled intervention then compares original questions, minimal repairs restoring a missing qualifier, and placebo paraphrases.

Rebuttal Table 4: Controlled changes in normalized entropy, reported in Section 6.

Paired comparisonMean changeBootstrap 95% CI
Minimal repair − original−0.181[−0.330, −0.041]
Placebo − original−0.042[−0.231, +0.147]
Minimal repair − placebo−0.139[−0.264, −0.037]

The repair effect persists against the placebo control, supporting the explicit-constraint check used by the reject-or-rewrite procedure. The reported Fact500 scores describe the fixed evaluation set; this intervention measures entropy changes rather than post-repair leaderboard accuracy.

For response scoring, semantic evaluation follows the approach used in SimpleQA [1], BrowseComp [2], HLE [3], and FRAMES [4]. Open-ended evaluation adopts DeepEval [5], whose original paper provides human-alignment experiments. We will distinguish this published methodological evidence from U2-specific evaluator calibration.

For calculation-heavy questions, Appendix D.1 describes the reference-construction procedure and sandboxed Python checks. We will keep these checks distinct from the complete Fact500 validity audit and clarify their coverage and disagreement-handling protocol separately from model accuracy.


Weakness 2: Contribution of difficulty selection and dependence on GPT-5.5.

Response. The paper provides evidence at both the generation and downstream-training stages.

Under a fixed source pool and screening policy, Appendix C.1 evaluates 500 candidates at each generator checkpoint:

Rebuttal Table 5: Generator improvement under the combined training objective.

Training stepUsable-question yieldWell-posednessMean normalized entropy
03.6%44%0.71
20028.4%83%0.93

The simultaneous improvements in usable yield, well-posedness, and entropy support the effectiveness of the combined generation objective within the stated screening protocol. The difficulty-model ablation additionally shows that oracle choice changes the challenge level of the selected questions.

U2-Train demonstrates external training utility: the same-base GPT-OSS-20B solver improves from 59.3 to 64.1 on GAIA, 76.2 to 79.5 on FRAMES, and 16.6 to 20.2 on HLE. We attribute these gains to the complete pipeline and distinguish them from the marginal contribution of entropy selection, which requires matched component-level controls.

Regarding oracle dependence, U2-Bench evaluates the selected questions across GPT, Gemini, and Claude families. U2-Train further studies a separate setting in which the generator, difficulty model, and target solver are distinct instances initialized from the same GPT-OSS-20B base model. Thus, its difficulty signal targets the base model’s own capability gaps rather than relying on GPT-5.5 as the difficulty oracle.


[1] Measuring Short-Form Factuality in Large Language Models.

[2] BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents.

[3] Humanity’s Last Exam.

[4] Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation.

[5] LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wild.

Weakness 3: Inconsistent recovery and regression counts in Section 5.1.

Response. Thank you for identifying this reporting inconsistency. The reported accuracies correspond to 278 correct answers without tools and 379 with tools, giving a net increase of 101 questions, or 20.2 percentage points. The submitted recovery/regression pair of 135/16 does not reproduce these totals.

We will replace the inconsistent breakdown with the supported aggregate statement:

Tool access increases accuracy from 55.6% to 75.8%, corresponding to 101 additional correct answers on the 500-question set.

The aggregate accuracy result remains unchanged. Any paired recovery/regression breakdown must use the same item-level labels and reproduce these margins.


Weakness 4 + Question 4: Implementation details and verifier training.

Response. The three learned components have distinct training procedures: the generator uses GRPO, the verifier uses SFT, and the solver uses GSPO.

Generator. Section 2.2 defines the task-specific reward components and normalized answer entropy. Appendix A.2 gives the GRPO objective, including the clipped policy-ratio surrogate, group-relative advantage, KL penalty, and numerical stabilization. All formats share a factual gate: a failed paragraph-level source-consistency check sets the trajectory reward to zero.

Open-ended generation combines format, lexical self-containment, paper selection, and coverage. Short-form generation combines format, self-containment, section selection, and difficulty; calculation-heavy generation additionally uses a quantitative-structure reward.

Verifier. Section 2.3 and Appendix A.2 specify supervised fine-tuning on MAS-adjudicated paragraph labels using next-token cross-entropy under a fixed binary output format. The verifier is frozen during generator optimization. The label-construction procedure includes independent assessment, source-span extraction, cross-examination, and rotated adjudication.

Solver. Section 7 initializes a fresh GPT-OSS-20B solver from the same base model as the generator and difficulty model, and trains it with GSPO on the generated, validated questions. The reported transfer results concern this complete same-base pipeline.

We will consolidate the remaining numerical hyperparameters, reward coefficients, data partitions, sampling and canonicalization settings, and external evaluation configurations in the implementation description, while keeping the three components separate.


Additional concern: The appendix examples do not adequately illustrate the technical complexity.

Response. We will supplement the appendix with two technical examples from our broader generated calculation-question pool, with explicit provenance, identifiers, and solution sketches.

Example 1: Cutoff-dependent integration, item 597151fddde298da.

For ϵ>0\epsilon>0, determine the Λ→∞\Lambda\to\infty limit of the following integral over an ellipsoidal domain:

I11=i∫l12/(1+ϵ)+l22+l32+l42≤Λ2l2−4l12(l2+1)3 d4l.I_{11}=i\int_{l_1^2/(1+\epsilon)+l_2^2+l_3^2+l_4^2\leq\Lambda^2}\frac{l^2-4l_1^2}{(l^2+1)^3}\,\mathrm{d}^4l.

The solver must account for the ellipsoidal cutoff and its finite boundary contribution. Applying spherical angular averaging before treating the cutoff would incorrectly give zero. The result is

I11=iπ28+4ϵ−ϵ2−81+ϵ2ϵ2.I_{11}=i\pi^2\frac{8+4\epsilon-\epsilon^2-8\sqrt{1+\epsilon}}{2\epsilon^2}.

Neither this expression nor its derivation is supplied in the question. The task requires reasoning about cutoff geometry, boundary contributions, and the limiting procedure.

Example 2: Convergence of a matrix-integral observable, item a570a95f6559e793.

Determine whether the second moment ⟨Tr⁡XD2⟩\langle\operatorname{Tr}X_D^2\rangle exists for the six-dimensional bosonic SU(2)SU(2) Yang–Mills matrix integral.

The partition function is finite, but the second moment diverges logarithmically along commuting-matrix directions. The key reasoning step is to distinguish normalization of the measure from convergence of the observable.

Together, these examples illustrate reasoning about regulator-dependent finite contributions and observable convergence, complementing the aggregate evaluations with concrete technical problems and solution explanations.

Thank you for the thoughtful response and for correcting the Section 5.1 numbers. I appreciate the clarifications. However, some of my main concerns would benefit from more concrete evidence. Any information on the following would be very helpful:

Corrected results. Since the reported Fact500 scores use the original set, it would be helpful to see whether the main results change after removing or repairing the 20 identified defects.

Judge validity. Section 5.2 states that the judge and prompt are reported explicitly, but I could not find the short-form judge model or prompt in the paper. Could you share them? Given the gap between LLM-judge and rule-based scores (75.8 vs. 42.2), a small manual check of cases where the two methods disagree would also help readers interpret the scores.

Calc500 checks. Could you share the coverage and agreement rate of the independent Python solutions (Appendix D.1)?

Entropy selection. If results on random vs. entropy selection, or without the difficulty reward, are available, I would be glad to see them. Otherwise, slightly scoping the related claims may be sufficient.

Details. If possible, please include the U2-Train data size, how overlap with the evaluation benchmarks was checked, and key values not given in the paper (reward coefficients, oracle sampling temperature, and verifier training size and accuracy).

I understand time is limited. Partial answers are welcome.

Reviewer_WjjhOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper trains a question generator with reinforcement learning to produce challenging, source-grounded questions from academic literature. It uses a frozen verifier for factual consistency and answer entropy from a tool-enabled solver to reward difficulty in short-term QA. The resulting U2-Bench contains three 500-question tracks: open-ended reports, non-STEM factual QA, and STEM calculation QA. Evaluations show remaining headroom for frontier models, supported by human annotation checks and a benchmark validity audit. The paper also introduces U2-Train, where the generator, difficulty model, and solver share the same base initialization, yielding improvements on GAIA, FRAMES, and HLE.

Critical strengths
  • This study introduces a practical question-generation framework, combining RL-based generation, solver answer entropy for difficulty, and a factual-consistency gate. U2-Bench covers three task families and remains challenging with tools: the best reported accuracies on Fact500 and Calc500 are 76.2% and 77.4%.
  • The source-blind audit covers all 500 Fact500 questions and identifies 20 defects. A controlled repair experiment shows that omitted constraints can inflate entropy. Oracle-choice and evaluation-metric ablations also clarify how these choices affect benchmark scores.
  • U2-Train demonstrates a training application using generator, difficulty-model, and solver instances from the same base initialization. The resulting solver improves by 4.8, 3.3, and 3.6 points on GAIA, FRAMES, and HLE, respectively.
Critical weaknesses
  • Evaluation depends heavily on model judges. Figure 6 shows substantially different short-term scores across evaluation rules, but it doesn't establish which rule best matches human judgments. The open-ended results also lack calibration against human ratings or alternative judges. This leaves uncertainty about absolute score interpretation and the reliability of close model comparisons.
  • U2-Train doesn't isolate the benefit of difficulty-based selection. Table 5 compares the trained solver only with the base model. Without matched training data and compute controls, the gains could reflect additional training rather than entropy-based selection. Comparisons with random selection or generation without the difficulty reward would clarify the contribution. The single training configuration also leaves variability unquantified.
  • The stability of entropy-based selection isn't established. Fact500 uses eight sampled answers and a normalized-entropy threshold of 0.7, but the paper doesn't assess selection stability across repeated sampling, decoding settings, or answer canonicalization choices. The oracle-choice ablation demonstrates dependence on the difficulty model, while GPT-5.5 also serves as an evaluated system. Although acknowledged, these dependencies limit confidence that the selected difficulty is reproducible across configurations.
  • In the 82-item human evaluation, only ~84% of reference answers are fully correct, 11% are partially correct, and 4.9% are incorrect (Table 3). The complete Fact500 audit is useful, but it targets omitted constraints rather than comprehensively checking reference correctness. Calc500 has no independent validity audit. These gaps leave uncertainty about how much reported error reflects model failures versus dataset defects.
Questions for authors

Please address the weaknesses above. There are several questions about the weaknesses to help clarify the evidence:

  • About the U2-Train controls, were any comparisons conducted with the same training budget and data volume but without entropy-based selection? These would help distinguish the benefit of the proposal selection method from additional training.
  • How were the 82 human-reviewed items sampled across tracks, and were identified reference errors corrected before evaluation?
  • Regarding selection stability, have you measured whether repeated oracle sampling selects similar questions under the entropy threshold? Any existing checks across sampling settings or canonicalization choices would help assess the reliability of the difficulty signal.
  • Is there any human agreement or cross-judge validation for the short-form evaluator or open-ended scores? This would help interpret the metric differences in Figure 6 and the reliability of model comparisons.
3 comments

Thank you for recognizing the construction framework, validity analysis, and external transfer results. We clarify the rationale for semantic evaluation, the contribution supported by U2-Train, and the scope of the existing human review.

Weakness 1 + Question 4: Why use an LLM judge rather than rule-based scoring or exact match?

Response. Our evaluation target is whether a response gives the correct answer, including semantically equivalent expressions. Free-form responses can use different wording, entity aliases, or equivalent numerical representations. Reference-based semantic judging recognizes these valid variations.

This approach follows closely related benchmarks:

  • SimpleQA [1], Section 2.4, uses a prompted ChatGPT classifier to compare predictions with reference answers and assign correct, incorrect, or not-attempted labels.
  • BrowseComp [2], Section 2.3, explicitly uses an AI grader to assess semantic equivalence between predicted and reference answers.
  • Humanity’s Last Exam [3], Section 4.1, uses GPT-4o to assess correctness while accounting for equivalent formats such as fractions and decimals.
  • FRAMES [4], Section 3.1, uses an LLM autorater for free-form answers and reports 96% agreement accuracy and Cohen’s kappa of 0.889 against human annotations.

Our paper reports all three scoring approaches:

Rebuttal Table 1: GPT-5.5 accuracy under different evaluation rules on fact-heavy QA.

Evaluation ruleAccuracy (%)
LLM judge75.8
Rule-based scoring42.2
Exact match20.4

These comparisons make scoring sensitivity visible. We motivate the primary metric by semantic correctness and the task format, while retaining deterministic metrics as complementary results. We will make the judge prompt and accepted equivalence criteria explicit.

Related long-form evaluations also use LLM judges, including DeepResearch Bench [5] for report quality and citation support, and MT-Bench [6] for reference-guided or preference-based response assessment.


Weakness 2 + Question 1: Does U2-Train improve performance because of difficulty selection or simply additional training?

Response. The reported experiment establishes the external training utility of the complete U2-Train pipeline. A fresh GPT-OSS-20B solver trained on the generated, validated questions improves on all three external benchmarks.

Rebuttal Table 2: External transfer from training GPT-OSS-20B on U2-generated questions.

ModelGAIAFRAMESHLE
GPT-OSS-20B base59.376.216.6
+ U2-generated-data training64.179.520.2
Absolute gain (percentage points)+4.8+3.3+3.6

The generator, difficulty model, and fresh solver are separate instances initialized from the same base model. Thus, the experiment demonstrates that questions generated and selected around the base model’s capability gaps provide useful training data, with gains extending beyond the generated question distribution.

A matched random-validated control would isolate the marginal contribution of entropy selection. The reported base-versus-trained comparison supports the transfer benefit of the complete pipeline. We will state this attribution explicitly and distinguish the pipeline-level result from component-level claims.


Weakness 3 + Question 3: Reproducibility of difficulty selection.

Response. The short-form pipeline uses a tool-enabled GPT-5.5 difficulty oracle, K=8 sampled answers, post-canonicalization answer classes, and entropy-based filtering. The oracle receives the generated question without the intended source paper being directly injected; general web search and code execution remain available.

This evaluates the ability to locate and reason over evidence without privileged source context. We will place the oracle identifier, sampling settings, canonicalization rules, entropy normalization, threshold, and access policy together in the revised implementation description.

The existing oracle-choice ablation demonstrates that the difficulty model changes the challenge level of the selected questions. We will distinguish this result from sensitivity to sampling and canonicalization choices.

Weakness 4 + Question 2: Scope of the 82-pair human review and treatment of identified errors.

Response. The 82-pair study is a source-aware professional review of fact-heavy question–reference-answer pairs from the construction pipeline. Annotators assess correctness, relevance, and support against the source papers. It complements the source-blind audit, which separately examines whether the question’s explicit constraints determine the target.

Rebuttal Table 3: Existing professional review of 82 fact-heavy question–reference pairs.

DimensionRatingPercentage
Reference correctnessFully correct84.1%
Reference correctnessPartially correct11.0%
Reference correctnessIncorrect4.9%
Question relevanceRelevant98.8%
Source supportFully supported96.3%
Answer uniquenessSingle clear answer89.0%

The results distinguish reference completeness from question ambiguity: an incomplete reference can receive a partially correct label even when the question has a unique answer. Consequently, the combined 15.9% not rated fully correct is not an ambiguity rate.

The reported model scores concern the original fixed evaluation set. The source-blind audit and reject-or-rewrite procedure are additional validity checks. We will distinguish the original evaluation from release-time corrections and clarify that the 82-pair review concerns fact-heavy QA rather than cross-track judge calibration.


References

[1] Measuring Short-Form Factuality in Large Language Models.

[2] BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents.

[3] Humanity’s Last Exam.

[4] Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation.

[5] DeepResearch Bench: A Comprehensive Benchmark for Deep Research Agents.

[6] Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.

Thank you for your detailed response. Most of my concerns have been addressed, raising the score accordingly.

Reviewer_Lp1uOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an RL-based framework for training a question generator that produces source-grounded, high-difficulty evaluation items from academic literature, and uses it to build U2-Bench: three 500-question tracks spanning open-ended reports, fact-heavy non-STEM QA, and calculation-heavy STEM QA. A key ingredient is a difficulty reward computed as the normalized entropy of answer distributions sampled from a strong, tool-enabled solver, coupled with a frozen verifier that gates paragraph-level factual consistency. The paper demonstrates substantial headroom for frontier models on U2-Bench and introduces U2-Train, a fully self-evolving variant where the generator, difficulty model, and solver are all derived from the same base model; the resulting solver exhibits measurable gains on external benchmarks (GAIA, FRAMES, HLE).

Critical strengths
  1. The formulation of question generation as RL with a composite reward that includes a normalized-entropy difficulty signal from a tool-enabled solver is an elegant and practical mechanism for scalable difficulty selection, and the framework is explicitly designed so that benchmark versions can be regenerated as solvers improve.
  2. The paragraph-level factual gate, trained via SFT on multi-agent adjudicated labels, effectively ensures that generated questions remain grounded in the source context rather than achieving apparent difficulty through ambiguity or hallucination. Crucially, this is complemented by an independent source-blind validity audit (with a controlled minimal-repair intervention) that separates genuine difficulty from underspecification, and by a lexical self-containment reward that suppresses prompt-local references.
  3. The U²-Train setting derives the question generator, difficulty model, and solver as separate instances from a shared base-model initialization, using the base model's own answer-entropy to select questions that expose its capability gaps. This is a compelling demonstration of self-curriculum data generation, and the resulting solver's gains on external benchmarks (GAIA, FRAMES, HLE) provide evidence of transfer beyond the generated question distribution.
Critical weaknesses
  1. The central difficulty signal is tied to a proprietary oracle (GPT-5.5) that is also an evaluated model on the same benchmark, raising concerns about selector–evaluator entanglement and potential overfitting to oracle idiosyncrasies; while acknowledged and partially ablated, the dependence remains a core vulnerability.
  2. Entropy at small K (K=8) can be noisy and is sensitive to canonicalization of answers and sampling temperature; the paper lacks a stability analysis across K, decoding settings, and canonicalization strategies.
Questions for authors
  1. How stable is the entropy-based difficulty score with respect to K (number of samples), decoding temperature, and answer canonicalization heuristics?
  2. For the open-ended track, have you conducted any human evaluation on a stratified subset to calibrate DeepEval’s absolute levels and pairwise ordering? If so, how do human preferences correlate with the three reported metrics?
1 comment

Thank you for recognizing the entropy-based difficulty signal, paragraph-level factual verification, source-blind validity analysis, and same-base U2-Train setting.

Weakness 1 + Question 1: Stability of entropy-based difficulty under different sampling and normalization settings.

Response. We use entropy as an operational measure of a specified solver’s uncertainty under a fixed access policy. The short-form pipeline samples K=8 answers per question, canonicalizes them into answer classes, and computes entropy from the empirical class probabilities. This aggregates evidence across multiple responses rather than assigning difficulty from a single answer.

The difficulty-model ablation in the paper demonstrates that oracle choice meaningfully controls the challenge level. Selection using the weaker oracle produces questions on which several evaluated systems approach 80% no-tool accuracy, whereas selection using GPT-5.5 produces a more challenging set with scores around 50%. This supports the intended role of the difficulty model in adapting question generation to model capability.

We will consolidate the oracle identifier, decoding configuration, canonicalization procedure, entropy normalization, and selection threshold in the revised implementation description. We will distinguish the existing oracle-choice ablation from sensitivity to K, temperature, and answer normalization, and scope the corresponding claims explicitly.


Question 2: Human support for the open-ended DeepEval evaluation.

Response. Our open-ended evaluation adopts DeepEval from LiveResearchBench [1], whose original paper explicitly evaluates alignment with human experts in Appendix B. DeepEval uses dimension-specific protocols: checklist-based assessment for presentation and organization, and issue-based additive scoring for factual/logical consistency and citation association.

The authors report that identifying individual issues and aggregating penalties achieves higher human agreement than directly assigning an overall score. Using a GPT-5/Gemini-2.5-Pro judge ensemble, they obtain 82.0% human agreement on 128 identified inconsistencies and 85.9% agreement on 128 identified citation-association issues.

These published human-alignment results motivate our adoption of DeepEval. We will explicitly cite this methodological evidence and clarify the evaluation protocols in the revision. The reported agreement figures belong to the original DeepEval study and its judge ensemble.


References

[1] LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wild. Appendix B.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.