acceptodds
Under review as a conference paper at ICLR 2027

Planning-Aware Continual Traversability Learning for Off-Road Navigation

Abstract

Off-road traversability estimation is particularly challenging due to diverse terrain and continuous changes in visual appearance and physical properties. To address this problem, several recent works formulate traversability estimation as a continual learning problem. These methods manage replay memory and prioritize update samples using perception-level criteria, including feature diversity and traversable-terrain feature changes. They have improved traversability prediction on newly encountered terrain while reducing forgetting of previously learned terrain. However, their perception-level criteria focus on visual changes rather than downstream planning and may prioritize samples with little influence on path selection. Moreover, these works regard untraversed terrain as non-traversable regardless of its actual traversability. This can unnecessarily constrain safe routes and degrade navigation performance in complex off-road environments. In this paper, we propose PACT, a planning-aware continual traversability framework that selects planning-relevant samples to adapt the model and counterfactually identifies obstacles and traversable terrain in untraversed regions. PACT comprises two novel modules: Planning-Aware Acquisition (PAA) and Counterfactual Asymmetric Replay (CAR). PAA prioritizes samples whose candidate paths exhibit high uncertainty and low traversability. This directly links sample selection to downstream planning rather than to perception-level criteria alone. CAR identifies obstacles and traversable terrain in untraversed regions by counterfactually evaluating how each candidate region affects planning. For asymmetric replay, identified obstacles are combined with current observations to create new training samples, while safe terrain is used for contrastive learning to maintain consistent representations across observations. On the GrandTour continual benchmark, PACT achieves the highest average planning success and lowest average prediction error with fewer samples than the baselines.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Continual Learning, Traversability Estimation, Off-Road Navigation, Motion Planning, Experience Replay

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HMCiOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors develop a method for continually adapting a vision-based traversability estimation model for offroad environments, where their novelty lies in the use of a planner to help select observations to be used for self-supervised traversability learning. The "PAA" module produces a set of candidate trajectories using MPPI, and this set is retained for traversability model training only when the trajectories have a large spread and a (i.e. "high uncertainty") and low traversability. The "CAR" module detects obstacles from image segmentation masks by replacing each mask with a freespace label and checking whether re-running MPPI produces drastically different candidate trajectories after this change. Upon collecting a repository of obstacles, CAR augments the training set by pasting segmented obstacles into other training images. The authors report performance improvements across the board.

Critical strengths

The counterfactual reasoning framework for detecting obstacles is a good justification for using planning to inform traversability data acquisition Composing obstacles with separate images to augment the training dataset is an intriguing approach The proposed method does improve sample efficiency compared to some prior works

Critical weaknesses

The claim that "low traversability and high uncertainty" is a good heuristic for sample selection needs to be further justified The related works section does not cover all relevant prior works and should be expanded The work does not compare against state-of-the-art methods like SALON, which demonstrate extreme sample efficiency with limited pretraining The flow of the paper is difficult to follow

Lastly, the contribution is jeopardized by insufficient discussion of/comparison with prior works. For example, this work does not compare against the 2025 work "SALON" (https://ieeexplore.ieee.org/abstract/document/11128268), which demonstrates adaptation in seconds to novel terrain with limited pretraining. Hence, although the proposed work demonstrates an improvement in sample efficiency compared to the reported baselines, it's unclear where it stands in the current state of the art. The related works section also omits several works in the field (e.g. https://ieeexplore.ieee.org/abstract/document/10160856, https://proceedings.mlr.press/v270/triest25a.html).

Questions for authors

Can you further explain the intuition behind the "low traversability and high uncertainty" heuristic? How does the projection of traversability features from the image space to the BEV space account 3D information loss (e.g. driving under the leaves of a tree)? How much data is used for the pretrained traversability model, and how does that compare to SALON?

3 comments

Thank you for taking the time to review our work and for providing thoughtful suggestions. We are pleased that you found counterfactual reasoning to provide a useful rationale for planning-guided traversability data acquisition, found the obstacle-composition augmentation interesting, and recognized the sample-efficiency gains over the reported baselines. Below, we address your concerns and suggestions. The main revisions are highlighted in blue in the revised paper.


[W1 & Q1] Intuition and justification for the “low traversability and high uncertainty” heuristic: Thank you for suggesting a clearer justification of our sample-selection criteria. We hypothesize that observations with low traversability and high uncertainty include regions that are not trained enough and obstacles useful for mitigating forgetting. Specifically, these observations mainly correspond to cases in which a broad traversable region has not been sufficiently learned or there are obstacles that largely affect candidate trajectories. The untrained region in the first case is essential for adapting to new environments, and the obstacles in the second case can be used to generate useful samples for mitigating forgetting by combining with observations. Table 1 in the main paper demonstrates that leveraging our planning-oriented criteria enables PACT to outperform baselines, supporting our hypothesis. Note that our criteria directly link sample selection to downstream planning, unlike previous works that select samples using perception-oriented criteria, such as feature diversity.

In addition, we also conducted additional experiments to evaluate whether using both criteria is more effective than using either criterion alone. Using both criteria achieved the highest planning success and the lowest RMSE and forgetting with the fewest selected observations. These results suggest that low traversability and high uncertainty are complementary criteria to select useful samples for continual traversability learning. The table below summarizes the results.

Selection criterionTrigger-selected observations ↓Planning success (%) ↑RMSE ↓Forgetting ↓
Low traversability only33293.810.11410.0063
High uncertainty only16486.190.10990.0071
Low traversability and high uncertainty12294.760.10810.0050

We added a new section (Appendix D.4 in the revised paper) to describe the experimental settings and results discussed in this response. Table D4 reports prediction, forgetting, and planning performance, while Table D5 reports observation usage and computation. Further details of the sample-selection criteria are provided in Section 3.2. We hope these explanations and additional results address your concern about the sample-selection heuristic.


[W2] Expansion of the related work section: Thank you for pointing out the missing references. We have expanded Section 2 and the reference list to include HDIF, Velociraptor, and SALON, together with GSAT, TravSUITE, ALTER, and Online Prototype Adaptation. In the revised paper, we provide detailed discussions of these methods and their differences from PACT in Sections 2.1–2.2. Below, we describe the differences between PACT and SALON, the most closely related of these methods.

SALON and PACT both use stored experience to support adaptation and mitigate forgetting. SALON manages its buffer based on visual-feature classes and speeds to preserve diverse experience. PACT selects observations using planning-oriented criteria and separates obstacle and safe-terrain information to preserve important knowledge for navigation. The key difference is that SALON manages stored experience based on perception-oriented criteria and driving speed, whereas PACT guides observation selection and knowledge preservation through downstream planning.

[W3] Comparison with state-of-the-art methods like SALON: Following your suggestion, we added SALON and COTRATE as baselines for comparison with recent methods.

We first conducted additional experiments on GrandTour. PACT achieved higher planning success and safe–hazard consistency, and lower forgetting than SALON and COTRATE. Consistency measures the percentage of 800 visually similar safe–hazard pairs for which both regions are correctly classified.

SALON balances stored experience based on visual-feature classes and speeds, which do not explicitly prioritize important obstacle information for planning. This may lead to prediction errors near obstacle boundaries and contribute to SALON’s collision rate of 27.14%. COTRATE preserves diverse visual features through feature selection and replay, but we found that it was less effective at distinguishing visually similar safe terrain and hazards. This is reflected in its safe–hazard consistency of 67.75%. Unlike these two baselines, PACT explicitly identifies obstacle regions through counterfactual diagnosis and reinforces obstacle recognition through composition with different backgrounds, while separately preserving safe-terrain representations. These results suggest that preserving both types of knowledge through asymmetric replay contributes to more reliable safe–hazard discrimination and higher planning success. The following table summarizes the results.

MetricSALONCOTRATEPACT
RMSE ↓0.10540.14850.1081
AUROC ↑0.98940.98300.9950
Safe IoU ↑0.96720.84970.9696
Balanced accuracy ↑0.96770.90970.9695
Forgetting ↓0.00650.00650.0050
Safe–hazard consistency (%) ↑90.0067.7595.38
Planning success (%) ↑70.9566.6794.76

In the revised paper, we added these results to Tables 1–3, which report overall continual-learning performance, safe–hazard discrimination, and offline global planning performance, respectively. Figures 3–4 also include the added baselines’ predictions and planned paths.

To evaluate generalization beyond GrandTour, we conducted additional experiments on the WVN dataset. Across these evaluations, PACT showed strong overall performance and achieved the highest mean planning success among all evaluated methods. In particular, on WVN, PACT achieved a planning success rate of 52.2% for 10–15 m queries, exceeding SALON and COTRATE by 25.0 and 15.0 percentage points, respectively. These results suggest that PACT’s advantages are not confined to a particular training seed, mission order, or the GrandTour dataset. The following table summarizes the WVN results.

MetricSALONCOTRATEPACT
RMSE ↓0.19580.16130.1351
AUROC ↑0.98050.96870.9802
Safe IoU ↑0.88150.92180.9700
Balanced accuracy ↑0.92130.93970.9309
Forgetting ↓0.08540.01580.0031
Planning success, 10–15 m (%) ↑27.237.252.2
Planning success, 3–6 m (%) ↑67.890.092.6

In the revised paper, we expanded Appendix C.1 to describe the added baseline implementations and Appendices D.1–D.3 to present comparisons across training seeds and mission orders, along with mission-level learning dynamics. We also added a new section (Appendix F) to describe the experimental settings and results on the WVN dataset discussed in this response.


[W4] Clarity of the paper’s flow: After reviewing the paper, we found that the explanations in Sections 3.1 and 4.4 could be easier to follow and revised them accordingly. In the final paragraph of Section 3.1, we clarified PAA’s role in observation selection and CAR’s separate decisions about adaptation and obstacle storage for replay. In the first paragraph of Section 4.4, we added an explanation distinguishing MPPI’s role in observation selection from A*’s role in offline planning evaluation. We also specified that the same A* planner evaluates cost maps derived from each model’s traversability predictions.

If any explanations or parts of the paper’s flow remain unclear, please let us know so that we can revise them accordingly.

[Q2] 3D information loss in the projection from image space to BEV space: Thank you for highlighting this issue. When projecting image-space traversability predictions into BEV, we apply height filtering to reduce the mixing of overhead observations with ground predictions. We explain this process in more detail below.

We exclude prediction points more than 1.5 m above the robot base-frame origin before XY aggregation. Specifically, we use depth and camera calibration to associate image predictions with 3D points, apply the height filter, and then aggregate the retained predictions into XY cells. This removes high overhead points, such as foliage, before their traversability scores are combined with ground predictions in the same cell. We chose the upper cutoff considering the robot’s approximate height, using the base frame to express height relative to the robot. Depth-based projection of image-space traversability predictions into BEV is also used in V-STRONG, while height-based point filtering is used in SALON’s ANYmal mapping implementation.

We added the explanation of height filtering discussed in this response to Appendix C.3 in the revised paper, including the rationale for the cutoff and the subsequent ground/elevated fusion. Table C3 lists the corresponding settings.


[Q3] Data used for the pretrained traversability model and comparison with SALON: Thank you for this insightful question about pretraining data requirements. We agree that clarifying the data used before adaptation is important for a fair comparison.

Our initial traversability model was trained on 5,885 Mission A frames, with 785 separate validation frames. PACT and all baselines, including SALON, start from the same pretrained model. We use this common initialization, together with shared inputs and supervision, to compare observation-selection, adaptation, and knowledge-retention strategies from the same starting point.

We could not directly compare the total data used before adaptation with the original SALON because its data requirements for visual clustering, roughness calibration, and GP initialization have not yet been confirmed. SALON uses a pretrained visual backbone and visual clusters constructed from previously collected images. We have emailed the SALON authors to clarify these quantities and will incorporate the verified information into the manuscript upon receiving their response.

We added details of the initialization data and the purpose of the shared initialization to Appendix B.1 in the revised paper.


Thank you again for your suggestions. We have expanded the related-work discussion, added the PAA ablations and baseline comparisons, and clarified the BEV projection, common initialization, and presentation. We welcome further discussion of any remaining concerns. If our response has addressed your primary concerns, we would greatly appreciate it if you could update your score to reflect that. We are also willing to discuss with you if you have any further concerns.

Reviewer_zHuHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents PACT, a framework for continual learning of traversability in offroad terrains. PACT serves as a perception model for robots navigating unstructured scenarios and utilizes self-supervised and continual learning to adapt to unseen environments on the fly. During traversing, a pre-trained base model predicts traversability maps and candidate trajectories. A mechanism called PAA (Planning-Aware Acquisition) then selectively samples observations to be used for online training of the model. Selection is based on predicted trajectory divergence (high) and traversability (low), with the intuition that mainly difficult and uncertain scenarios contribute effectively to the models training progress. After selection, a mechanism called CAR (Counterfactual Asymmetric Replay) extracts obstacles counterfactually (an obstacle is considered an obstacle if it sufficiently affects planning results when removed) and train the model in two main ways: It mixes extracted obstacles with traversable backgrounds and directly supervises traversability prediction on mixed obstacles; and it contrastively aligns predicted embeddings for safe regions to safe teacher regions while pushing them away from obstacle embeddings. The authors conduct experiments on a self-constructed benchmark on the GrandTour dataset and compare the method to other continual learning methods. The results show that PACT learns safer perception with fewer samples, ultimately resulting in higher global planning success.

Critical strengths

The paper introduces a new, well-performing method for online adaption of traversability perception for unstructured offroad environments. The paper is mostly written in an understandable manner and comes with a substantial appendix diving deeper into the method. It convincingly conveys that considering planning for selecting samples for perception is important. In addition, the paper introduces a benchmarking methodology for continual learning methods on the public GrandTour dataset which will aid the community in conducting further research.

Critical weaknesses

The paper compares its method only on the self-constructed benchmark on the GrandTour dataset, which limits direct comparability to prior publications. An evaluation on the older WVN dataset would add credibility and make it easier for researchers to judge approaches based on comparisons done by multiple parties.

Questions for authors

3.2: "Candidate trajectories can spread widely across broad safe regions, so high uncertainty alone does not necessarily indicate a need for further learning." -> As a reader, this sentence (also mentioned in other forms earlier in the paper) raises questions as to why this should be the case. Only later in the paper, when it is clarified that trajectory divergence is used as a measure of uncertainty, does it become clear why the authors come to this conclusion. This should be clarified from the start; or instead, "trajectory divergence" should be used directly in the explanations instead of "uncertainty".

3.3: The term "verified obstacle intervention" is introduced without explanation. This can be confusing to readers.

Things to improve the paper that did not impact the score:

  • Terms used in 3.1 (e.g. ItI_t, YtY_t, etc.) should be denoted in Figure 1.
1 comment

Thank you for your thoughtful review and helpful suggestions. We appreciate your positive assessment of planning-informed sample selection, the detailed appendix, and the GrandTour continual-learning benchmark. We address your concerns and suggestions below. The main revisions are highlighted in blue in the revised paper.


[W1] Evaluation on the WVN dataset: We appreciate your suggestion to extend the evaluation beyond GrandTour. Following your suggestion, we conducted additional experiments on the WVN dataset, comparing PACT with seven baselines across four environments. We also added SALON and COTRATE for comparison with recent methods.

PACT achieved the best results on all metrics reported in the table. In particular, PACT achieved 15.0 percentage points higher planning success than COTRATE, the strongest planning baseline, for 10–15 m queries and 2.6 percentage points higher success for 3–6 m queries. These results indicate that PACT’s advantages in prediction accuracy, knowledge retention, and planning performance are not specific to GrandTour. The following table summarizes the results.

MethodRMSE ↓Forgetting ↓Safe IoU ↑Planning success: 10–15 m (%) ↑Planning success: 3–6 m (%) ↑
CM0.16270.04450.934533.375.2
Reservoir0.15840.04010.923330.672.2
WVN0.16580.04000.91596.171.9
LwF0.22500.06750.73671.747.0
IMOST0.18540.06190.891420.665.2
SALON0.19580.08540.881527.267.8
COTRATE0.16130.01580.921837.290.0
PACT0.13510.00310.970052.292.6

We added a new section (Appendix F in the revised paper) to describe the WVN experimental settings and results discussed in this response. Tables F1–F2 report prediction, forgetting, and planning performance, while Figures F1–F3 show retention and planning examples. We hope these additional experiments address your concern about evaluation beyond GrandTour.


[Q1] Clarifying the meaning of “uncertainty”: Thank you for pointing out that the meaning of “uncertainty” becomes clear only later in the paper. We now use “trajectory divergence” from its first mention and explain that it describes the spatial spread of candidate trajectories. This makes the meaning clear from the outset.

In the revised paper, we replaced “uncertainty” with “trajectory divergence” when referring to this quantity in the abstract, Introduction, Figure 1 caption, and Sections 3.1–3.2. We also revised the definition preceding Eq. (1).


[Q2] Meaning of “verified obstacle intervention”: Thank you for asking us to clarify this term. In “verified obstacle intervention,” “intervention” refers to replacing an obstacle region’s traversability values with the designated safe value in Eq. (3), and “verified” means that the region satisfies CAR’s geometric and planning-oriented criteria.

Even with this definition, the term may make CAR’s procedure difficult to follow. In the revised paper, we therefore replaced it with a direct description of the traversability replacement and its role in CAR’s adaptation and obstacle replay decisions in Section 3.3.

We also clarified the obstacle-verification criteria in Appendix B.3 and the continual update procedure in Algorithm B1 of Appendix B.6.


[N1] Denoting the terms used in Section 3.1 in Figure 1: Figure 1 now directly labels the observation x_t = (I_t, D_t, q_t), traversability prediction T_t, supervision Y_t, and validity mask Ω_t using the notation in Section 3.1. We also replaced “uncertainty” in the diagram with “Traj. Divergence.” These labels make the correspondence between the framework and the method definitions explicit.


Thank you again for helping us strengthen the evaluation and improve the clarity of the paper. If our response has addressed your primary concerns, we would greatly appreciate it if you could update your score to reflect that. We are also willing to discuss with you if you have any further concerns.

Reviewer_T8PkOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents PACT, a planning-aware continual traversability learning framework for off-road navigation that addresses the limitations of perception-oriented sample selection and conservative untraversed terrain labeling. PACT introduces Planning-Aware Acquisition (PAA), which selects update samples whose candidate trajectories exhibit high uncertainty and low traversability, directly coupling model adaptation to downstream planning needs. It also incorporates Counterfactual Asymmetric Replay (CAR), which counterfactually evaluates untraversed candidate regions using SAM2 masks to isolate planning-relevant obstacles from safe terrain, subsequently storing obstacles for composition and safe regions for contrastive learning. Validated on the 8-mission GrandTour continual benchmark, PACT achieves superior prediction accuracy (lowest average RMSE of 0.1081 and lowest average forgetting of 0.0048) and higher downstream planning success (94.76% vs. 71.90% for the best baseline Reservoir) while utilizing 81.0% fewer adaptation frames.

Critical strengths
  • As detailed in Section 3.2 and Figure 1, the Planning-Aware Acquisition module establishes a direct coupling between upstream traversability estimation and downstream planning by triggering updates only when candidate trajectories exhibit both high uncertainty and low traversability.
  • Counterfactual Asymmetric Replay leverages SAM2-based counterfactual evaluation to reliably isolate true obstacles from safe terrain, effectively resolving conservative mislabeling in untraversed areas with over 95% consistency.
  • In Section 4.2, the empirical evaluation across eight GrandTour missions confirms that PACT achieves a 94.76% offline global planning success rate in Table 3 while reducing adaptation training frames by 81.0% compared to strong baselines in Table 1.
  • The paper demonstrates commendable evaluation rigor through controlled component ablations in Table 4 as well as multi-seed and task-order robustness analyses in Table D3.
Critical weaknesses
  • As acknowledged in Section 5 and measured in Appendix E.2 and E.3, executing the complex control flow—particularly SAM2 mask extraction taking three to four seconds per frame—introduces substantial processing delays. These severe runtime overheads make real-time online adaptation practically unfeasible during active physical navigation.
  • The Planning-Aware Acquisition module relies on fixed heuristic thresholds for trajectory divergence and mean traversability, as specified in Table B2 and Section 5. Because these hand-tuned parameters are tied to specific motion lattices and vehicle speeds, the framework lacks dynamic self-calibration guarantees and may require tedious retuning whenever deployed under different robot dynamics, operating speeds, or planning horizons.
Questions for authors

Since realistic off-road deployment lacks explicit task-transition signals and ground-truth validation sets, how does PACT perform when evaluated as a single, continuously updated model checkpoint across the streaming sequence without per-mission validation monitoring, and does its performance degrade significantly without oracle checkpoint selection?

3 comments

We appreciate your careful review and constructive feedback. We are pleased that you recognized the connection between perception adaptation and planning needs, the effectiveness of counterfactual obstacle diagnosis, and the experimental analysis through component ablations, multiple seeds, and task-order evaluations. The main revisions are highlighted in blue in the revised paper.


[W1] Runtime overheads and the feasibility of real-time online adaptation during navigation: Thank you for your constructive feedback. In PACT, mask generation is the main bottleneck of computational overhead. While we use SAM2 to generate masks in our experiments, PACT is compatible with any segmentation model. Fortunately, recent studies have accelerated SAM segmentation models while preserving segmentation quality. For example, EfficientViT-SAM accelerates SAM while maintaining accuracy on its reported benchmarks, while EfficientTAM achieves faster video segmentation with performance comparable to SAM2. We believe that combining PACT with such efficient models facilitates real-time online adaptation, reducing significant runtime overheads.

In addition, we claim that lowering the prompt-grid density can substantially reduce runtime overhead without replacing segmentation models. To validate our claim, we conducted experiments with the SAM2 prompt grid reduced from 48×48 to 16×16. The results are summarized in the table below. Reducing the prompt grid substantially shortened mask-generation time while maintaining comparable prediction performance. Although planning success decreased, it remained higher than that of all baselines in the original submission. We expect that combining a more efficient segmentation model with a lower prompt-grid density will further reduce computational overhead and help enable real-time online adaptation.

Metric48×4816×16
Mean SAM2 mask-generation time (s) ↓3.6950.471
RMSE ↓0.10810.1091
Safe IoU ↑0.96960.9735
Planning success (%) ↑94.7685.71
Collision rate (%) ↓5.2413.81

We added a new section (Appendix E.5 in the revised paper) to describe the experimental settings and results discussed in this response. Table E4 reports runtime measurements, while Table E5 reports prediction, forgetting, and planning performance after adaptation. We hope these additional experiments help address your concerns about PACT’s computational overhead.


[W2] Fixed heuristic thresholds and retuning for different robot dynamics, operating speeds, or planning horizons: Thank you for raising this concern. To facilitate reconfiguration under different operating conditions, PACT provides explicit criteria for selecting the trajectory-divergence and traversability thresholds, together with a systematic procedure for generating candidate trajectories. These criteria and procedures provide a reproducible starting point for retuning. We explain these criteria and the trajectory-generation procedure in more detail below.

The trajectory-divergence and traversability thresholds are defined with reference to the spatial planning horizon and the supervision scale, respectively. We set the divergence threshold to 40% of the planning horizon, giving 3.2 m for the 8-m horizon used in our experiments. This selects observations with substantial separation between elite trajectories and provides a reference for adjusting the threshold when the horizon changes. The traversability threshold is set to 0.70, the midpoint of the 0.6–0.8 supervision range assigned to SAM-expanded traversable regions, to include observations with intermediate traversability scores when trajectory divergence is also high.

Candidate trajectories can be regenerated systematically for the specified robot motion conditions. Given the robot’s kinematic model, motion state, and motion limits, the procedure generates feasible trajectories and selects representative trajectories to initialize MPPI. Individual paths therefore do not need to be designed manually. Together with the threshold-selection criteria, this procedure makes the configuration explicit and reproducible when adapting PACT to different operating conditions.

We added the explanations discussed in this response to Appendices B.2–B.3 in the revised paper. Appendix B.2 describes candidate-trajectory generation, while Appendix B.3 explains the rationale for the two thresholds. Table B2 lists the numerical settings.

[Q1-1] A single, continuously updated model checkpoint without explicit task-transition signals or per-mission validation monitoring: We are glad to discuss PACT’s performance under these conditions. We conducted an additional experiment in which a single model was continuously updated without explicit task-transition signals, per-mission validation monitoring, or oracle checkpoint selection, and evaluated the final checkpoint. We also included SALON and COTRATE for comparison with recent methods.

PACT achieved the best overall performance among the eight methods. Its lowest forgetting and highest planning success support the effectiveness of its memory management and asymmetric replay in preserving knowledge and finding collision-free paths under these conditions. The following table summarizes the results.

MethodPlanning success (%) ↑RMSE ↓AUROC ↑Safe IoU ↑Balanced Accuracy ↑Forgetting ↓
CM55.240.12960.99400.89850.93910.0172
Reservoir71.900.16000.98180.95020.90530.0347
WVN72.860.15770.98960.93990.92540.0341
LwF28.570.21660.99280.48110.73970.0644
IMOST67.140.13330.99260.92010.95080.0324
SALON72.860.15840.99160.84430.91570.0526
COTRATE52.860.22960.96320.72490.84790.0676
PACT76.670.12120.98750.95780.94890.0004

In the revised paper, we added Appendix D.6 to describe the experimental settings and results of continuous adaptation, with the comparison across methods reported in Table D7.


[Q1-2] Performance without oracle checkpoint selection: To examine PACT’s performance without oracle checkpoint selection, we conducted an additional experiment using the last checkpoint after fixed-epoch training. We kept the mission boundaries unchanged and removed validation monitoring and early stopping.

PACT achieved the best results on all six reported metrics, with planning success only 0.95 percentage points below the submitted result. These results indicate that PACT maintains high planning performance without relying on oracle checkpoint selection. The following table summarizes the results.

MethodPlanning success (%) ↑RMSE ↓AUROC ↑Safe IoU ↑Balanced Accuracy ↑Forgetting ↓
CM67.620.14690.94930.94350.93860.0307
Reservoir31.900.26280.97410.70680.84230.1662
WVN29.520.22450.97970.75590.86860.1127
LwF0.000.28090.98360.09940.54860.0838
IMOST15.710.34840.96170.49600.73980.2571
SALON12.860.31490.96530.57830.77620.2053
COTRATE0.000.47250.58080.08650.54180.3169
PACT93.810.11660.99210.96890.95740.0145

In the revised paper, we added Appendix D.5 to describe the experimental settings and results without oracle checkpoint selection, with the comparison across methods reported in Table D6. We hope these additional experiments address your concerns about reliance on task-transition signals and validation-based checkpoint selection.


Thank you again for encouraging these evaluations. The revised appendix now reports the runtime and adaptation results from these additional experiments. If our response has addressed your primary concerns, we would greatly appreciate it if you could update your score to reflect that. We are also willing to discuss with you if you have any further concerns.

Reviewer_T8PkOpenReview ↗

Follow-up on Rebuttal

Thanks for the thorough rebuttal and extra experiments, especially the streaming and non-oracle checkpoint evaluations that really addressed my main concern. I would like to maintain my current score and stay with a positive recommendation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their careful evaluations and constructive feedback. Based on your suggestions, we have clarified several aspects of the paper and strengthened its empirical support through additional experiments and analyses. We summarize the main revisions below.

Paper Revision Summary

The main revisions are highlighted in blue in the revised paper.

  • We extended our evaluation beyond the GrandTour benchmark by adding experiments on the WVN dataset across four environments and three training seeds, with detailed experimental settings and full results provided in Appendix F. (zHuH)

  • We added SALON and COTRATE to Tables 1–3 and Figures 3–4 to expand the comparison with recent methods, with implementation details in Appendix C.1. We also added comparisons across training seeds and mission orders in Appendices D.1–D.2 to examine robustness to these variations. (HMCi)

  • We expanded Appendix D.4 to explain the intuition behind the “low traversability and high uncertainty” sample-selection criteria. We also added ablation experiments to examine the benefit of using both criteria, with results in Tables D4–D5. (HMCi)

  • We added experiments to reflect the absence of explicit task-transition signals and ground-truth validation sets in realistic off-road deployment. In Appendix D.6, we evaluated a single, continuously updated model checkpoint without explicit task-transition signals or per-mission validation monitoring. In Appendix D.5, we examined whether performance degrades without oracle checkpoint selection. (T8Pk)

  • We added experiments with reduced SAM2 prompt density in Appendix E.5 to examine its effects on runtime overhead and adaptation performance. We also discussed the use of efficient segmentation models to further reduce runtime overhead and support real-time online adaptation. (T8Pk)

  • We revised the Related Work section to include the suggested studies and clarify the similarities and differences between these methods and PACT. (HMCi)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.