- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This is a paper with a clear line of reasoning and attractive results achieved on a low budget; however, the experimental protocol and its positioning in terms of novelty are not yet sufficient to support strong conclusions.
I believe that ReDiF is indeed effective —‘single GPU, LoRA, very few training samples, and 5–8-step students’.However,it cannot yet be demonstrated that it is universally superior to existing distillation methods, nor can it be conclusively proven that its advantages stem from RL rather than experimentation budget, early stopping, parameterisation, ect.
Critical strengths
1.The paper's strongest idea is simple. ReDiF does not match the teacher at every denoising step. It optimizes only the final output.
2.The paper studies a practical setting. Training uses one GPU, a few hours, small batches, and few conditioning signals. The goal is to produce a usable student as quickly as possible. This setting matters for researchers with limited compute.
3.The paper is generally clear. The experiments are also reasonably thorough.
Critical weaknesses
1.The implementation details are inconsistent. Sec. 4.1, Alg. 1, and App. H say that teacher references are generated during training. They also say that teacher outputs are not stored. Tab. 1 and Apps. C and P say something different. They describe a teacher-sample pool that is generated once, saved to disk, and reused by all methods.
2.The baseline comparison has a narrow scope. It mainly measures adaptation speed under a low training budget. It does not measure general distillation performance. The results show that ReDiF learns quickly under the authors' protocol. They do not show that ReDiF is always more efficient or better than existing methods.
3.The paper also claims that PPO is better than direct reward backpropagation. However, the two settings differ in more than the optimizer. Several factors change at the same time. The experiment therefore does not isolate the effect of RL.
Questions for authors
1.Please clarify how teacher references are generated. Are they generated online for every rollout, or precomputed and stored before training? And also report the exact number of teacher forward passes.
2.How was the training budget selected? Was it chosen using ReDiF's held-out FID? If so, could this stopping point favor ReDiF over slower-converging baselines?
3.The PPO and direct-backpropagation settings differ in more than the optimizer. Can the authors provide a comparison with the same sampler, trajectory length, and compute budget?
We warmly thank the reviewer for the careful and detailed review, and in particular for finding the inconsistency in how we described the teacher samples. Fixing it has made the paper clearer.
W1 and Q1. How teacher samples are generated, and the number of teacher forward passes. The reviewer is right that the text described two procedures. The method itself does not need stored teacher samples; they can be generated during training. In our experiments we generated them once and cached them, because every baseline had to be trained from scratch with the same teacher, and the same cache is shared by all methods. The teacher sampler is deterministic, so caching does not change the results, and a run with no cache at all gives the same FID within one standard deviation. We would kindly ask the reviewer to see the Data-free training and the cached teacher samples section of our general comment "Data-free and ReDiF vs. direct backprop clarifications", which gives the exact teacher forward-pass counts and the results without a cache. We rewrote Sections 3.3 and 4.1, Algorithm 1 and Appendices C, E and H of the revised paper so that they describe one procedure.
W2 and Q2. How the training budget was chosen. We set the budget from the compute we target: a few GPU-hours on a single A100. Held-out FID was not used for this choice. Within that target, the number of epochs was set slightly past the point where ReDiF's training reward converges, so that it represents the converged regime rather than one particular point: the reward converges around epoch 90 on ImageNet-64 and around epoch 14 on COCO, and we trained to 100 and 20 epochs. The same number of epochs was then used for every method, and every reported number comes from the checkpoint with the highest training reward within that budget.
The reviewer is right that this could favour ReDiF over slower baselines, and we appreciate this point. This is why we trained all methods on ImageNet-64 for twice the budget (200 epochs): ReDiF has a lower FID than every baseline at every evaluated epoch, and no baseline reaches it in this range (Appendix D.1 of the revised paper). Two baselines first get worse (FID above 35 around epochs 80–100) and then improve, so at our budget they are still recovering. We agree that our results show strength in the low-training regime and not general superiority; this is the claim we make (we would kindly ask the reviewer to see the Scope section of our general comment "Novelty and scope of the work").
W3 and Q3. Same sampler, trajectory length and compute budget. We would be very grateful if the reviewer could read the ReDiF vs. direct backpropagation section of our general comment "Data-free and ReDiF vs. direct backprop clarifications", where we answer this question in detail with new experiments. Direct backpropagation and ReDiF share the stochastic sampler (η = 1), the number of steps, the reward, the initialization, the teacher cache, the batch and the epoch budget. On EDM direct backpropagation goes through all 8 steps, and truncating it changes its FID by about 0.1. On SD1.5 direct backpropagation must be truncated to the last step because the full chain does not fit on one GPU, while ReDiF trains on the full chain without truncation; this memory advantage is one of the main strengths of our method.