- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors develop a method for continually adapting a vision-based traversability estimation model for offroad environments, where their novelty lies in the use of a planner to help select observations to be used for self-supervised traversability learning. The "PAA" module produces a set of candidate trajectories using MPPI, and this set is retained for traversability model training only when the trajectories have a large spread and a (i.e. "high uncertainty") and low traversability. The "CAR" module detects obstacles from image segmentation masks by replacing each mask with a freespace label and checking whether re-running MPPI produces drastically different candidate trajectories after this change. Upon collecting a repository of obstacles, CAR augments the training set by pasting segmented obstacles into other training images. The authors report performance improvements across the board.
Critical strengths
The counterfactual reasoning framework for detecting obstacles is a good justification for using planning to inform traversability data acquisition Composing obstacles with separate images to augment the training dataset is an intriguing approach The proposed method does improve sample efficiency compared to some prior works
Critical weaknesses
The claim that "low traversability and high uncertainty" is a good heuristic for sample selection needs to be further justified The related works section does not cover all relevant prior works and should be expanded The work does not compare against state-of-the-art methods like SALON, which demonstrate extreme sample efficiency with limited pretraining The flow of the paper is difficult to follow
Lastly, the contribution is jeopardized by insufficient discussion of/comparison with prior works. For example, this work does not compare against the 2025 work "SALON" (https://ieeexplore.ieee.org/abstract/document/11128268), which demonstrates adaptation in seconds to novel terrain with limited pretraining. Hence, although the proposed work demonstrates an improvement in sample efficiency compared to the reported baselines, it's unclear where it stands in the current state of the art. The related works section also omits several works in the field (e.g. https://ieeexplore.ieee.org/abstract/document/10160856, https://proceedings.mlr.press/v270/triest25a.html).
Questions for authors
Can you further explain the intuition behind the "low traversability and high uncertainty" heuristic? How does the projection of traversability features from the image space to the BEV space account 3D information loss (e.g. driving under the leaves of a tree)? How much data is used for the pretrained traversability model, and how does that compare to SALON?
3 comments
[W3] Comparison with state-of-the-art methods like SALON: Following your suggestion, we added SALON and COTRATE as baselines for comparison with recent methods.
We first conducted additional experiments on GrandTour. PACT achieved higher planning success and safe–hazard consistency, and lower forgetting than SALON and COTRATE. Consistency measures the percentage of 800 visually similar safe–hazard pairs for which both regions are correctly classified.
SALON balances stored experience based on visual-feature classes and speeds, which do not explicitly prioritize important obstacle information for planning. This may lead to prediction errors near obstacle boundaries and contribute to SALON’s collision rate of 27.14%. COTRATE preserves diverse visual features through feature selection and replay, but we found that it was less effective at distinguishing visually similar safe terrain and hazards. This is reflected in its safe–hazard consistency of 67.75%. Unlike these two baselines, PACT explicitly identifies obstacle regions through counterfactual diagnosis and reinforces obstacle recognition through composition with different backgrounds, while separately preserving safe-terrain representations. These results suggest that preserving both types of knowledge through asymmetric replay contributes to more reliable safe–hazard discrimination and higher planning success. The following table summarizes the results.
| Metric | SALON | COTRATE | PACT |
|---|---|---|---|
| RMSE ↓ | 0.1054 | 0.1485 | 0.1081 |
| AUROC ↑ | 0.9894 | 0.9830 | 0.9950 |
| Safe IoU ↑ | 0.9672 | 0.8497 | 0.9696 |
| Balanced accuracy ↑ | 0.9677 | 0.9097 | 0.9695 |
| Forgetting ↓ | 0.0065 | 0.0065 | 0.0050 |
| Safe–hazard consistency (%) ↑ | 90.00 | 67.75 | 95.38 |
| Planning success (%) ↑ | 70.95 | 66.67 | 94.76 |
In the revised paper, we added these results to Tables 1–3, which report overall continual-learning performance, safe–hazard discrimination, and offline global planning performance, respectively. Figures 3–4 also include the added baselines’ predictions and planned paths.
To evaluate generalization beyond GrandTour, we conducted additional experiments on the WVN dataset. Across these evaluations, PACT showed strong overall performance and achieved the highest mean planning success among all evaluated methods. In particular, on WVN, PACT achieved a planning success rate of 52.2% for 10–15 m queries, exceeding SALON and COTRATE by 25.0 and 15.0 percentage points, respectively. These results suggest that PACT’s advantages are not confined to a particular training seed, mission order, or the GrandTour dataset. The following table summarizes the WVN results.
| Metric | SALON | COTRATE | PACT |
|---|---|---|---|
| RMSE ↓ | 0.1958 | 0.1613 | 0.1351 |
| AUROC ↑ | 0.9805 | 0.9687 | 0.9802 |
| Safe IoU ↑ | 0.8815 | 0.9218 | 0.9700 |
| Balanced accuracy ↑ | 0.9213 | 0.9397 | 0.9309 |
| Forgetting ↓ | 0.0854 | 0.0158 | 0.0031 |
| Planning success, 10–15 m (%) ↑ | 27.2 | 37.2 | 52.2 |
| Planning success, 3–6 m (%) ↑ | 67.8 | 90.0 | 92.6 |
In the revised paper, we expanded Appendix C.1 to describe the added baseline implementations and Appendices D.1–D.3 to present comparisons across training seeds and mission orders, along with mission-level learning dynamics. We also added a new section (Appendix F) to describe the experimental settings and results on the WVN dataset discussed in this response.
[W4] Clarity of the paper’s flow: After reviewing the paper, we found that the explanations in Sections 3.1 and 4.4 could be easier to follow and revised them accordingly. In the final paragraph of Section 3.1, we clarified PAA’s role in observation selection and CAR’s separate decisions about adaptation and obstacle storage for replay. In the first paragraph of Section 4.4, we added an explanation distinguishing MPPI’s role in observation selection from A*’s role in offline planning evaluation. We also specified that the same A* planner evaluates cost maps derived from each model’s traversability predictions.
If any explanations or parts of the paper’s flow remain unclear, please let us know so that we can revise them accordingly.
[Q2] 3D information loss in the projection from image space to BEV space: Thank you for highlighting this issue. When projecting image-space traversability predictions into BEV, we apply height filtering to reduce the mixing of overhead observations with ground predictions. We explain this process in more detail below.
We exclude prediction points more than 1.5 m above the robot base-frame origin before XY aggregation. Specifically, we use depth and camera calibration to associate image predictions with 3D points, apply the height filter, and then aggregate the retained predictions into XY cells. This removes high overhead points, such as foliage, before their traversability scores are combined with ground predictions in the same cell. We chose the upper cutoff considering the robot’s approximate height, using the base frame to express height relative to the robot. Depth-based projection of image-space traversability predictions into BEV is also used in V-STRONG, while height-based point filtering is used in SALON’s ANYmal mapping implementation.
We added the explanation of height filtering discussed in this response to Appendix C.3 in the revised paper, including the rationale for the cutoff and the subsequent ground/elevated fusion. Table C3 lists the corresponding settings.
[Q3] Data used for the pretrained traversability model and comparison with SALON: Thank you for this insightful question about pretraining data requirements. We agree that clarifying the data used before adaptation is important for a fair comparison.
Our initial traversability model was trained on 5,885 Mission A frames, with 785 separate validation frames. PACT and all baselines, including SALON, start from the same pretrained model. We use this common initialization, together with shared inputs and supervision, to compare observation-selection, adaptation, and knowledge-retention strategies from the same starting point.
We could not directly compare the total data used before adaptation with the original SALON because its data requirements for visual clustering, roughness calibration, and GP initialization have not yet been confirmed. SALON uses a pretrained visual backbone and visual clusters constructed from previously collected images. We have emailed the SALON authors to clarify these quantities and will incorporate the verified information into the manuscript upon receiving their response.
We added details of the initialization data and the purpose of the shared initialization to Appendix B.1 in the revised paper.
Thank you again for your suggestions. We have expanded the related-work discussion, added the PAA ablations and baseline comparisons, and clarified the BEV projection, common initialization, and presentation. We welcome further discussion of any remaining concerns. If our response has addressed your primary concerns, we would greatly appreciate it if you could update your score to reflect that. We are also willing to discuss with you if you have any further concerns.
Thank you for taking the time to review our work and for providing thoughtful suggestions. We are pleased that you found counterfactual reasoning to provide a useful rationale for planning-guided traversability data acquisition, found the obstacle-composition augmentation interesting, and recognized the sample-efficiency gains over the reported baselines. Below, we address your concerns and suggestions. The main revisions are highlighted in blue in the revised paper.
[W1 & Q1] Intuition and justification for the “low traversability and high uncertainty” heuristic: Thank you for suggesting a clearer justification of our sample-selection criteria. We hypothesize that observations with low traversability and high uncertainty include regions that are not trained enough and obstacles useful for mitigating forgetting. Specifically, these observations mainly correspond to cases in which a broad traversable region has not been sufficiently learned or there are obstacles that largely affect candidate trajectories. The untrained region in the first case is essential for adapting to new environments, and the obstacles in the second case can be used to generate useful samples for mitigating forgetting by combining with observations. Table 1 in the main paper demonstrates that leveraging our planning-oriented criteria enables PACT to outperform baselines, supporting our hypothesis. Note that our criteria directly link sample selection to downstream planning, unlike previous works that select samples using perception-oriented criteria, such as feature diversity.
In addition, we also conducted additional experiments to evaluate whether using both criteria is more effective than using either criterion alone. Using both criteria achieved the highest planning success and the lowest RMSE and forgetting with the fewest selected observations. These results suggest that low traversability and high uncertainty are complementary criteria to select useful samples for continual traversability learning. The table below summarizes the results.
We added a new section (Appendix D.4 in the revised paper) to describe the experimental settings and results discussed in this response. Table D4 reports prediction, forgetting, and planning performance, while Table D5 reports observation usage and computation. Further details of the sample-selection criteria are provided in Section 3.2. We hope these explanations and additional results address your concern about the sample-selection heuristic.
[W2] Expansion of the related work section: Thank you for pointing out the missing references. We have expanded Section 2 and the reference list to include HDIF, Velociraptor, and SALON, together with GSAT, TravSUITE, ALTER, and Online Prototype Adaptation. In the revised paper, we provide detailed discussions of these methods and their differences from PACT in Sections 2.1–2.2. Below, we describe the differences between PACT and SALON, the most closely related of these methods.
SALON and PACT both use stored experience to support adaptation and mitigate forgetting. SALON manages its buffer based on visual-feature classes and speeds to preserve diverse experience. PACT selects observations using planning-oriented criteria and separates obstacle and safe-terrain information to preserve important knowledge for navigation. The key difference is that SALON manages stored experience based on perception-oriented criteria and driving speed, whereas PACT guides observation selection and knowledge preservation through downstream planning.