Exploring Pass-Rate Reward in Reinforcement Learning for Code Generation

Reinforcement learning (RL) from unit-test feedback has become a standard post-training recipe for improving large language models (LLMs) on code generation. However, the *pass-all-tests* binary reward can be sparse, yielding no learning signal on challenging problems where none of the sampled solutions passes all tests. A common remedy is to use the *test-case pass rate* as a surrogate reward. In this work, we study pass-rate rewards in RL for code generation and report a consistent pattern across base models and algorithms: despite alleviating reward sparsity, pass-rate rewards do not reliably improve final performance over binary rewards in controlled experiments. To understand this discrepancy, we analyze reward density and the resulting gradient directions. We find denser rewards but weak local transfer to verified full-pass references when groups contain only partial-pass responses. Sample-level probes reveal positive and negative reference-transfer effects within the same group, while semantic analysis shows that higher pass rates can favor algorithmic errors over near-correct solutions. Together, these findings identify a mismatch between partial test-case success and guidance toward full solutions, motivating rewards that align intermediate feedback with full correctness.