Preselection-Free Knowledge-Grounded Reinforcement Learning

Existing agents solve sequential decision-making tasks by accessing knowledge, such as expert-designed data, or navigating the environments through trial and error. Research has explored knowledge as supervision or reinforcement learning for interactive improvement, but typically studied independently. Recent work introduces a unified paradigm called Knowledge-Grounded Reinforcement Learning (KGRL), which studies how RL can efficiently leverage external knowledge and how integrating these two complementary signals can enhance learning. However, prior KGRL methods require an expert-preselected knowledge set for each task, which is hard to obtain in real-world applications. Without it, performance can degrade significantly due to misleading information. We propose a trade-and-curate mechanism for KGRL that trades stochastic information among knowledge policies and curates knowledge-policy fusion, making the agent resilient to unhelpful information in an unselected knowledge set. Our experiments in KGRL MiniGrid environments show that the trade-and-curate mechanism achieves up to 89.6% performance improvement without knowledge preselection.