Selecting What to Transmit: Cross-Generational In-Context Reinforcement Learning

In-context reinforcement learning promises that a trained network can continue to improve on a new instance by reading its own attempts, without updating its parameters. Prior work seeks better inference performance while maintaining efficiency, typically by accumulating more information in the context window. This is a trade-off: more information in the context window raises the computational cost of inference. Inspired by the intergenerational accumulation of knowledge in human societies, we move from within-generation to cross-generational inference. To this end, we propose CGIRL, which partitions inference into successive generations. Within each generation, the policy reads a fixed short window. Across generations, MAP-Elites, an evolutionary algorithm, selects the transmitted experience from candidates archived by their return dynamics, rather than transmitting the full history or a single incumbent. On the travelling salesperson problem and a sequence-recall task, experiments show that CGIRL achieves state-of-the-art performance and efficiency.