- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies whether real-user queries discarded by preference-data filtering can still provide useful material for new training tasks. RESOURCE trains a source-grounded instruction generator with self-training and response-aware GRPO, searches over candidate instructions using response rewards, and constructs fresh preference pairs for downstream DPO. Source-matched experiments compare raw and recycled supervision across data scales, test mixtures with retained data, and examine instruction transfer to other target models.
Critical strengths
-
The problem is practical. A discarded query can still contain useful code, context, or task material, so recycling is a reasonable complement to filtering. The paper also clearly distinguishes constructing a new task from recovering the user's original intention.
-
The source-matched comparisons are a strong part of the design. Raw and recycled conditions use the same sources, training-set size, and preference-construction procedure. Table 1 shows consistent RESOURCE improvements across three benchmarks and all four tested data scales, making the main empirical result fairly convincing.
-
The study goes beyond the raw-versus-recycled comparison. The mixture results and cross-model instruction transfer are useful, and Table 4 separates contributions from recycler training and instruction search. Together they support the method as a usable pipeline rather than just a few successful examples.
Critical weaknesses
-
The comparisons match the final training data, but not the total construction cost. RESOURCE uses up to eight instruction candidates with four response probes each, followed by sixteen fresh responses, and also requires SFT and GRPO for the recycler. RAW only needs the final preference-construction stage. The reported gains are useful, but the practical value would be clearer with total generation/training costs and a stronger use of the same construction budget for the baseline.
-
The necessity of the discarded source content is not isolated. The task can change substantially, and the validity checks mainly establish eligibility rather than semantic grounding or usefulness. There is no clear source-removed or source-shuffled control showing how much of the gain comes from meaningful source reuse rather than general synthetic-task generation and reward-based search. This limits the specific conclusion about recycling, even if the resulting data are useful.
-
The evaluation leaves some uncertainty. The three benchmarks share a local Llama-3.1-70B judge, and the main construction pipeline also uses Llama models. This does not invalidate the comparisons, but independent judging would strengthen them. In addition, Tables 1–4 report standard errors without making repeated downstream-training variability clear enough. Evaluation uncertainty and training-seed uncertainty should be distinguished.
Questions for authors
-
What are the measured construction and recycler-training costs, and over how much recycled data are these costs amortized? Please use existing measurements if available.
-
Do you have an existing source-removed or source-shuffled control, or a systematic grounding audit? If not, can the claim be narrowed to the demonstrated utility of the generated data rather than the necessity of source reuse?
-
Do the reported standard errors include independent downstream DPO runs, or only evaluation variation? Please also clarify whether any comparisons have already been checked with an independent judge.
Response to Reviewer W9YY
We appreciate the reviewer’s thoughtful comments and feedback on our paper!
We agree that recycling incurs additional construction cost. Our question is whether each discarded example can somehow become useful by investing in one-time recycler training and subsequent generation. We do not claim recycling is more compute-efficient than using these examples directly.
Here are the measured costs for the one-time training and construction, in NVIDIA A40 GPU-hours.
The same trained recycler is reused across our experiments without retraining. Its one-time training cost is therefore amortized over 8K sources, corresponding to approximately 18 GPU-hours per 1K sources. As such, recycling trades additional construction compute for better downstream performance from the same discarded sources (e.g., 46.20 to 54.86 in AlpacaEval2). We will make this trade-off explicit in the revision to help assess its practical value. Thank you!
Following your suggestion, we additionally evaluated general synthetic data generation, including source-free synthesis and self-instruct-style generation using discarded queries as seeds. All methods use the same amount of 2K data, the same response generation configuration and preference construction.
The results above clearly show the gap between using discarded queries as seeds/sources and source-free synthesis. This highlights the usefulness of discarded source content for recycling.
Thank you for the suggestions. We additionally evaluated with an independent GPT-4o judge (gpt-4o-2024-11-20) using the same 2K-trained models. ReSource outperforms Raw across all three benchmarks under this judge as well. In addition, even with GPT-4.1 (gpt-4.1-2025-04-14), the same trend holds although the absolute scales differ across judges: AlpacaEval2 LC increases from 22.35 for Raw to 25.19 for ReSource.
The SE in the paper reflects evaluation uncertainty. To further verify, we additionally ran five downstream training runs per setting using multiple seeds. The table below shows AlpacaEval2 LC under the original local judge, with mean ± SE across five runs. We will explicitly distinguish these two types of uncertainty in the revision.
Thank you so much again for your time and valuable suggestions! We hope these additional results and clarifications address your concerns, and we would appreciate it if you could reconsider your assessment based on our response.