- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper analyzes task selection in a single-round proposer–solver pipeline for code generation. It evaluates learnability- and IFD-based filtering, finding no statistically distinguishable aggregate improvement over the unrefined pool in the main experiments, and investigates proposer diversity.
Critical strengths
- Separating selection from proposer updates addresses whether filtering adds value beyond task generation. The multi-model comparisons are informative, and the discussion appropriately acknowledges statistical-power limits and confounding in the diversity analysis.
Critical weaknesses
- Lemma 1 does not hold for the effective support defined in §3. For a two-logit softmax with rewards , an on-policy expected-reward gradient step of size moves above , to approximately . Finite sampling also does not ensure . The containment chain in Proposition 2 is therefore unjustified: selection stays within a realized pool, but this does not establish the claimed restriction to the base proposer’s effective support.
- The IFD result concerns a different selection protocol from the cited method. Li et al. (NAACL 2024, §2.2) exclude scores above ; this paper selects the highest scores and reports selected means of , and in Table 2. This may be a reasonable RL adaptation, but its negative result does not establish the performance of the cited filtered procedure. The distinction matters for interpreting one of the two tested refinement operators.
Questions for authors
- What sampling or update assumptions would make Lemma 1 valid, and what nontrivial support conclusion remains under them?
- Was excluded before selection, using the same score reported in Table 2? If omitted, does the result persist with that filter under a comparable training budget?
We sincerely thank you for your careful examination and review.
Q1 / W1: Effective support and finite sampling.
Response. We agree that Lemma 1 does not hold for the stated effective-support definition. In your example, the update increases the success probability from 0.099 to approximately 0.100603, crossing the threshold 0.1. A task with probability 0<p≤ϵ can also appear in N independent draws with probability 1−(1−p)N>0.
Selection guarantees F(Dz)⊆Dz. Containment in a fixed set Sz additionally requires sampling to remain restricted to Sz, for example through a fixed hard mask; ordinary on-policy RLVR does not provide this guarantee. We will correct the support statement. Pool containment constrains the tasks refinement can select, but does not bound solver generalization.
Q2 / W2: IFD threshold and rerun.
Response. We agree that the reported arm differs from the original IFD procedure. Li et al. exclude scores above 1 before ranking [1], whereas our selected means of 1.21, 1.24, and 1.64 are incompatible with that exclusion under the reported score definition. We will label this as an IFD ranking adaptation to RL and clarify the implementation.
The current results concern only the tested IFD ranking adaptation. Whether they persist with thresholded selection remains open. Adding the threshold alone would still not reproduce the original instruction-tuning pipeline, which also uses a pre-experienced scoring model.
Reference
[1] From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning