- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether incorporating additional answer comparison pairs and refining scoring dimensions can provide more effective supervision for preference optimization in scientific question-answering (QA). Drawing on existing literature, the authors utilized a large language model (LLM) to construct a QA dataset for the inverse design of titanium alloys. The dataset comprises 4,000 questions, each featuring four candidate answers, an overall score, and task-specific sub-scores. Using the Qwen3.5-9B model, the authors compared "binary DPO"—constructed from pairs with the largest score gaps—against "dense DPO," which utilizes six pairs per question. Training pairs were selected based on criteria including rubric-based sub-scores, scalar score differences, and random sampling. Automated evaluation across a standardized set of 413 questions revealed that both "clean-start" binary DPO and dense DPO achieved a score of approximately 92.6. Certain task-specific sub-scores showed high correlation, and the Jaccard similarity between selection sets derived from rubric-based differences versus scalar score differences was 0.943 (at the 50th percentile). Across three experimental runs, no consistent advantage for rubric-based selection was established. Furthermore, an analysis of training data volume indicated that reducing the total amount of processed data could still yield performance comparable to that of filtered training sets. The primary contribution of this work lies in its empirical diagnosis of issues related to scoring redundancy, the composition of preference pairs, and the confounding effects of training data volume.
Critical strengths
1.The research addresses a problem of practical value. By analyzing supervision density, scoring structures, and filtering strategies—all based on the same question-answering resource—the authors provide a useful reference for constructing preference data in scientific domains.
- The experimental analysis is rigorous, and the reporting of results is transparent. The authors examine potential confounding factors, such as the evaluation process and the volume of training data, while clearly delineating the evidentiary boundaries regarding the use of multiple random seeds and exploratory findings.
Critical weaknesses
1.There is a lack of independent verification for the data and evaluations. Although the authors traced the origins of the literature and conducted manual spot checks, they did not provide a comprehensive report on the verification results for the preference rankings. Furthermore, there is no independent assessment of the new responses generated by the models; the primary conclusions remain heavily reliant on automated scoring systems.
2.The experimental controls are not sufficiently matched. The control conditions simultaneously alter question coverage, training volume, and update configurations. Moreover, the number of repetitions for key comparisons is limited, making it difficult to disentangle the effects of the supervisory structure from those of the training configuration.
Questions for authors
1.What are the sample size for manual verification, the backgrounds of the reviewers, and the accuracy of their preference rankings? Is it possible to provide independent evaluations of the newly generated model responses?
2.Do the results across multiple random seeds—while maintaining consistent initialization, a fixed batch size, matched question coverage, and an identical training budget—still support the primary observations?
3.What is the scope of accessibility for the data, as well as the configurations used for generation and scoring? For restricted datasets, can an auditable subset be provided to allow for independent verification of the preference labels and key findings?
2 comments
Thanks for the detailed response. The human validation numbers (800 instances, Pearson 0.822, QWK 0.796) are welcome, though the rebuttal itself notes the subset was built to contrast high and low answers, so they do not speak to ranking accuracy across the full quality range. The matched-budget runs help, but two seeds and a 900 versus 889 update gap leave them, in the authors' own words, short of statistical equivalence. Independent evaluation of newly generated answers is still missing. On data, no auditable subset is offered pending project approval, which limits external verification of the preference labels. These core concerns survive the rebuttal.
Human Validation, Experimental Controls, and Reproducibility
We have revised Section 4.2 and added Appendix G to make the available evidence and its limitations more explicit.
1. Human verification: sample size, evaluator background, and agreement
The underlying TitaniumQA resource underwent a construction-stage human validation study.
We sampled 100 questions from each of four reasoning types, selecting the highest- and lowest-scored original candidate answers for each question. This produced 400 questions and 800 answer-level instances.
Three evaluators with materials-science backgrounds independently rated the anonymized answers on a 0–100 scale. The mean of their ratings served as the human reference.
Appendix G.5 now reports the results:
These statistics compare the construction-stage fused LLM scores with mean human ratings. They provide evidence of human–LLM scoring agreement on the original candidate answers.
We also wish to clarify their scope. Because the subset intentionally contrasts high- and low-scored answers, these statistics should not be generalized to the full range of answer quality. They are not inter-rater agreement estimates or a separately measured pairwise preference-ranking accuracy.
Independent expert evaluation of newly generated model outputs remains unavailable. Such evaluation would require substantial additional specialist effort across many open-ended scientific responses. Our downstream experiments instead employ a frozen rubric-based evaluator to maintain a consistent scoring scale, while acknowledging that this does not remove systematic evaluator bias.
2. Matched-budget experiments and multiple seeds
To address the concern about changing question coverage and optimization budgets, we added a supplementary source-grouped pair-density experiment (Section 5.5, Table 6, Appendix G.1–G.2).
The one-pair random and six-pair conditions share 2,394 training questions, the same SFT initialization, effective batch size 16, and nominal optimization hyperparameters.
They process 14,364 and 14,223 pair exposures over 900 and 889 updates, respectively.
Across two training seeds, one-pair random obtains 93.21/93.02, compared with 92.48/92.83 for six-pair training.
The seed-42 comparison also replicates in direction on a second, previously unused 786-question held-out split (93.30 versus 92.64).
These are approximately matched controls rather than a fully factorial experiment. The two-seed results provide an additional robustness check but are insufficient to establish method-level statistical equivalence.
3. Source-overlap sensitivity and auditability
We additionally examined the effect of source-grouped versus question-level splitting (Appendix G.3).
Under the question-level split, 95.1% of test questions have a primary reference source represented in training; under source grouping, the corresponding overlap is zero.
Using matched training-question counts and task proportions, the corrected DPO scores were 90.39 for source grouping and 90.14 for question-level splitting.
We observed no clear overall inflation associated with reference-source reuse in this single-seed comparison. However, the two conditions also differ in test-question composition and source coverage, so this result does not prove the absence of all forms of leakage.
The new source-grouped train/test/held-out sets contain 186/87/54 primary reference sources, respectively, with no overlap between their primary reference identifiers. We explicitly acknowledge that this does not rule out overlap in auxiliary retrieved evidence or the evaluator's training annotations.
4. Data availability and reproducibility
The complete structured dataset remains subject to project-specific release restrictions.
To support reproducibility within those constraints, the revised manuscript documents the data schema, source-grouped splitting protocol, pair-construction rules, optimization configurations, evaluation procedure, and aggregate audit statistics. The Reproducibility Statement also describes the anonymized supplementary code package containing releasable training, evaluation, analysis, and configuration files.
Releasing actual data instances, including a sampled auditable subset, would require the applicable project approval. We therefore distinguish available implementation and audit materials from restricted underlying records rather than promising unrestricted data access.
We hope the added human-validation statistics, controlled pair-density experiments, source-overlap analysis, and expanded reproducibility documentation address the reviewer's concerns while making the remaining limitations transparent.