- Rating
- 3
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper focuses on the post-training, and it points out that SFT may suffer from the in-distribution overfitting, and RL may suffer from the sparse rewards. Therefore, it propose the MupCoder, which includes a token-level RL framework that provides dense process supervision without requiring auxiliary reward models. This approach equips the model with multi-perspective hints, such as reference demonstrations and test cases, and trains LLMs via the on-policy self-distillation. It shows the empirical improvements on diverse code generation benchmarks than other baselines.
Critical strengths
- The paper is well-written and easy to follow.
- The empirical experiments are conducted over models in different sizes and evaluated on diverse benchmarks, which is convincing.
- The ablations are also sufficient.
Critical weaknesses
- One concern might be the novelty, since I think this approach looks like the combination of existing methods and pipelines. For example, the hint information is similar to the privileged information in value model, which has been studied before. and the reward design is just reasonable.
- I am also wondering if it is possible to put all three hints into a whole prompt? Will it be better or not?
Questions for authors
Please refer to weakness.