The Catfish Effect: Heterogeneous Competition in Group-Relative Learning

Learning systems do not evolve in isolation: their progress is shaped by the populations against which they compete. Yet most reinforcement learning methods focus on improving the learner itself, while largely treating the composition of its competitive population as fixed. This raises a broader question: can introducing a heterogeneous competitor reshape learning without providing demonstrations or direct supervision? We investigate this question through Catfish GRPO, a group-relative reinforcement learning framework that introduces a frozen, moderately stronger model as a competitor rather than a teacher. The Catfish contributes only a scalar task outcome to reshape relative learning signals and guide rollout allocation; its generations, tokens, log probabilities, and parameters never enter the student loss. In our main comparison, Catfish GRPO exceeds the strongest evaluated baseline by up to 2.5 percentage points in-domain and 3.2 percentage points out-of-domain. Beyond performance, we study how heterogeneous competitors alter learning dynamics. Catfish encourages greater behavioral diversity early in training, increasing the chance that the student discovers correct on-policy trajectories on difficult problems and expanding the learning opportunities available in later stages. The quality of the competitor also plays a critical role: weaker or poorly matched competitors can substantially diminish the benefit, while stronger competitors are not necessarily better. Together, these findings position Catfish GRPO not only as an RL method, but also as a controlled testbed for studying how population composition and heterogeneous competition shape learning and co-evolution.