- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper compares hard-only, soft-only, and mixed-constraint RLVR training, arguing that reward reliability matters more than constraint diversity. Human-annotated verifier audits and noise-injection experiments motivate H1S, which retains hard constraints while limiting soft constraints to one per instruction. Experiments report improved instruction following on five benchmarks, largely preserved general capabilities, and lower training latency.
Critical strengths
-
H1S is simple and practically useful. Table 3 reports average scores of 62.81 and 68.65 for the 7B and 32B models, exceeding the corresponding mixed-training results. The measured reduction from 19.63 to 8.23 minutes per step at 32B is also substantial.
-
The human-annotated verifier audit and comparisons across model families usefully highlight reward reliability as a confounder in instruction-following research. The additional judge-consistency analysis strengthens the evidence that the reported soft-constraint gains are not specific to one evaluation judge.
Critical weaknesses
-
The central causal claim is not established. Section 4.2 varies training-data volume and per-instruction constraint count when studying “diversity,” so these factors are not isolated. Its noise calibration is also inconsistent with the stated intervention: overriding rewards to 1 with probability gives negative-response recall . Matching recall therefore requires , rather than the recall difference used in the paper. Matching recall alone also does not match reward precision or systematic judge errors. These experiments demonstrate sensitivity to reward corruption, but do not establish the claimed primacy of precision over diversity.
-
The robustness analysis contains consequential reporting errors. In Table 8, the five VerlIF-Pro Hard-only scores average to 61.38, not the reported 64.98, reducing its advantage over the reported Mix average from 4.04 to approximately 0.44 points. Section F.1.1 also claims a significant Hard-only lead over Mix on every evaluated benchmark, although the VerInstruct-Pro* rows show Mix ahead on CFBench and FollowBench. A significant difference cannot establish superiority in the opposite direction. These inconsistencies weaken confidence in the quantitative support for the main conclusion.
Overall, the conclusions of this paper (Precision Bounds Diversity) seem limited to the specific setting and are difficult to generalize. It is widely acknowledged that data diversity is crucial.
Questions for authors
After correcting the noise calibration, does the central conclusion hold when training-set size and per-instruction constraint count are controlled and constraint-type diversity is varied separately?
5 comments
Follow-Up on Responses
Dear Reviewer,
Thank you again for your review.
With the discussion period closing soon, we would like to respectfully follow up and inquire if you have the opportunity to review our responses to your comments.
We sincerely look forward to your feedback.
Thank you for your reply. I will maintain my score (Weak rejection) and stand by my view: Overall, the conclusions of this paper (Precision Bounds Diversity) seem limited to the specific setting and are difficult to generalize. It is widely acknowledged that data diversity is crucial.
We thank the reviewer for the rigorous review. We address the two weaknesses below.
W1-1: Confounded factors in the Section 4.2 diversity experiment
We agree this weakness. To address this, we fix the training-data volume and vary only the number of constraints retained per example. Starting from the Mix training set, we keep 0, 1/4, 2/4, 3/4, or 4/4 of each example's constraints, and train all five variants under the identical recipe.
Test performance does not increase monotonically with the number of constraints. The 1/4 variant is the strongest on IFEval (0.823) and ties the 2/4 variant on CFBench (0.53); the full-constraint (4/4) variant is no better than these low-constraint variants. Because the number of training examples is held fixed across all five runs, the comparison isolates constraint count from data scale.
This is consistent with the original conclusion of Section 4.2: once a minimal level of constraint diversity is reached, adding more constraints yields no consistent gain.
W1-2: Noise calibration
We agree that the previous noise level was miscalibrated and thank the reviewer for the correction. Under the stated forced-positive intervention, the negative-response recall of the corrupted reward is r(p) = (1 − p)·r_code. Matching the judge's negative-response recall of 30.6% therefore requires
We reran the matched conditions at these levels and updated Figure 4 (https://ibb.co/XrHmHCDY). We observe:
Relative to the original calibration, the gaps change only marginally: 0.5–0.8 → 0.8–0.9 points on IFEval, and 4–7 → 4–7.8 points on CFBench. The qualitative conclusion is preserved: noise closes about 70% of the in-distribution gap (2.5–3.0 points at p = 0 → 0.8–0.9 points at the matched level) and erases the Hard-only OOD advantage on CFBench. This is not change the original conclusion.
W2: Reporting errors in Table 8 and Appendix F.1.1
We thank the reviewer for catching this error and apologize for the oversight. The five per-benchmark VerIF-Pro Hard-only scores in Table 8 average 61.38, not 64.98; We will correct the Table 8 average accordingly.
We will also revise the claim in Appendix F.1.1 to: "Hard-only training outperforms Soft-only training on hard-constraint test sets and performs comparably on soft-constraint test sets; overall, it performs comparably to Mix training." The revised claim is consistent with Table 1 (lines 178–181): "Hard-only training outperforms Soft-only on average and matches the Mix variants, including on soft IF benchmarks."
The corrected result supports the two conclusions of the main paper: (i) training with higher-precision rewards generalizes better than training with lower-precision rewards; and (ii) a high-precision reward with limited diversity, e.g., Hard-only, can match the generalization of larger and more diverse data, e.g., Mix training.
Narrowing this claim does not weaken the main paper's conclusions.