Selective Shortcut Value Learning for Long-Horizon Offline Goal-Conditioned RL

One source of difficulty in long-horizon offline goal-conditioned reinforcement learning (GCRL) is that sparse rewards must propagate through many Bellman updates before primitive actions become well separated in value. We introduce Selective Shortcut Value Learning (SSVL), which shortens this propagation directly in primitive action-value learning using stopped trajectory segments and a single-discount shortcut continuation. A frozen shortcut-aligned reference selectively compresses segment rewards, while an auxiliary Shortcut Value Consistency (SVC) loss helps the separately learned IQL state value preserve shortcut relations; both are used only during training, and deployment remains a single primitive goal-conditioned policy. We analyze how shorter propagation strengthens progress-state separation and separately characterize the logged-continuation conditions under which shortcut regression targets preserve primitive-action preference. Across diverse state-based and visual OGBench tasks, SSVL is competitive with strong offline GCRL methods, including hierarchical approaches.