Planning-Aware Continual Traversability Learning for Off-Road Navigation

Off-road traversability estimation is particularly challenging due to diverse terrain and continuous changes in visual appearance and physical properties. To address this problem, several recent works formulate traversability estimation as a continual learning problem. These methods manage replay memory and prioritize update samples using perception-level criteria, including feature diversity and traversable-terrain feature changes. They have improved traversability prediction on newly encountered terrain while reducing forgetting of previously learned terrain. However, their perception-level criteria focus on visual changes rather than downstream planning and may prioritize samples with little influence on path selection. Moreover, these works regard untraversed terrain as non-traversable regardless of its actual traversability. This can unnecessarily constrain safe routes and degrade navigation performance in complex off-road environments. In this paper, we propose PACT, a planning-aware continual traversability framework that selects planning-relevant samples to adapt the model and counterfactually identifies obstacles and traversable terrain in untraversed regions. PACT comprises two novel modules: Planning-Aware Acquisition (PAA) and Counterfactual Asymmetric Replay (CAR). PAA prioritizes samples whose candidate paths exhibit high uncertainty and low traversability. This directly links sample selection to downstream planning rather than to perception-level criteria alone. CAR identifies obstacles and traversable terrain in untraversed regions by counterfactually evaluating how each candidate region affects planning. For asymmetric replay, identified obstacles are combined with current observations to create new training samples, while safe terrain is used for contrastive learning to maintain consistent representations across observations. On the GrandTour continual benchmark, PACT achieves the highest average planning success and lowest average prediction error with fewer samples than the baselines.