INFUSER: Influence-Guided Self-Evolution Improves Reasoning

Self-evolution offers a scalable path to stronger reasoning: a pretrained language model improves itself with only minimal external supervision. Yet existing methods either depend on extensively curated or teacher-generated training data, or, when the generator runs unsupervised, reward it by a difficulty heuristic that need not improve the solver. We introduce **INFUSER**, an iterative co-training framework with two co-evolving roles: a *Generator* that drafts questions and reference golden answers from a pool of unstructured, automatically collected documents, and a *Solver* that improves by training on them. The solver is trained with standard correctness rewards against the generator-provided answers, while the generator is rewarded by an *optimizer-aware influence score* that measures whether each proposed question would actually improve the solver on the target distribution. Because this continuous, noisy influence score is poorly served by standard GRPO, we propose **DuGRPO**, a dual-normalized variant of GRPO, for generator training. Together, these turn the document pool into an *adaptive curriculum* that favors questions useful to the current solver, not just hard ones. On Qwen3-8B-Base, INFUSER outperforms strong self-evolution baselines with over 20% relative improvement on Olympiad and SuperGPQA benchmarks, and an 8B INFUSER co-evolving generator outperforms a frozen 32B thinking generator on math and coding. Ablations confirm each design choice is necessary. Code is available at