- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies an interesting problem in coding-agent reinforcement learning. Existing environment construction pipelines may filter tasks according to rollout outcomes from a screening model. In CODEMIDAS, tasks with all-pass or all-fail screening results are removed. The authors argue that this filtering rule may be too strong. Based on this observation, the paper proposes ASSAY. It recovers 2,628 filtered environments, pretrains a critic using outcome records collected during environment construction, and uses single-rollout asynchronous actor–critic optimization.
Critical strengths
The main idea is simple but important. The paper clearly separates environment quality from optimizer-dependent training utility. I think this distinction is useful for future RL data construction. The analytical motivation is also clear. The paper shows that a small screening group can easily be unanimous even when a larger future training group is not degenerate.
Critical weaknesses
1: The effect of recovered RL data and recovered-task critic supervision is not fully separated This is my main concern. When the filtered tasks are recovered, they can affect ASSAY in at least two ways. First, they are added into the RL task pool. Second, their construction-time records can also provide additional data for verifier-grounded value pretraining.
2: The quality of the recovered environments is still uncertain The recovered tasks have passed the earlier quality checks, but these checks are not perfect. One concerning result is the online leakage-guard trigger rate. The retained tasks have a trigger rate of 0.9%, while the screening-all-pass recovered tasks have a much higher rate of 6.8%. This suggests that some recovered tasks may have more exploit or verifier problems. The online guard is useful, but the paper also acknowledges that it cannot detect all verifier weaknesses.
Questions for authors
Please see above.
Needless to say I will keep my score