Fork of Thought: Eliciting Self-Triggered Rethinking in LLM Agents via Branch-Sampled Reinforcement Learning

LLM agents can benefit from rethinking during multi-step interaction, but effective rethinking requires learning both when to intervene and how to provide useful guidance. Jointly training these behaviors is challenging: task-level rewards provide indirect credit for individual interventions, and advice can become less effective as the acting policy evolves. We introduce Fork of Thought, a reinforcement learning framework for learning self-triggered rethinking within a single LLM. At selected decision points, the framework branches a shared interaction prefix into alternative advice-conditioned continuations and a no-intervention control. These branches provide comparative feedback for learning when to invoke rethinking and how to generate useful advice, while task-level rewards train the agent’s overall behavior. We further adopt a two-stage training procedure that first focuses on rethinking under a constrained acting policy, then jointly optimizes acting and rethinking. The framework aims to align rethinking with the evolving agent policy and make it an adaptive component of sequential decision-making.