- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes LateSyn and studies how to optimize a synthetic data generator using actual training improvement from the data as the reward. The authors note that this reward comes only after learner training and evaluation, and becomes outdated under asynchronous execution. LateSyn first uses a critic to predict future data utility and update the generator immediately, then applies residual correction after true results return. The authors theoretically show that delay affects only the residual, and report better results than the baselines on three agent environments: Search-QA, ALFWorld, and WebShop.
Critical strengths
The problem the paper focuses on has practical relevance, in that the reward becomes stale under asynchronous execution when the learner's actual training improvement is used as the reward, and the proposed predict-then-correct method is also simple and easy to integrate into existing workflows.
Critical weaknesses
The paper's core empirical claim lacks supporting evidence. The authors attribute the gains to reducing the signal exposed to delay, but on Search-QA the delayed residual ratio is about 0.99, indicating that the mechanism has almost no effect, yet the improvement there is the largest. In Figure 4, λ is zero before step 10, so LateSyn and the λ=0 control should follow identical code paths and their curves should coincide, but the curves have already separated from step 2, and the gap cannot be attributed to the method. Synchronous LU that directly uses the true learning outcome as the reward performs worse than the proxy baseline on two of the three environments, and the paper's starting point is also not supported by its own data. In addition, the experiments use only one 4B model and two seeds, so the main conclusions are difficult to establish.
Questions for authors
- What is the essential difference in problem setting or method between this paper and learning-to-teach, teacher–student curriculum, and data valuation, where rewarding the data generator with the learner's actual progress has already been studied?
- Does the paper's premise that actual learning performance is more suitable as a reward than proxy signals hold, even though synchronous LU is worse than the proxy baseline on two environments and the correlation between LU and held-out improvement is only about 0.25?
- Why is such a large degree of asynchrony needed in practical settings, given that one-step delay performs better than the synchronous case in Figure 3, the high-delay problem appears only at M=8, and the paper reports no benefit in time or throughput?