acceptodds
Under review as a conference paper at ICLR 2027

Hierarchical Perception– Decision Test-Time Adaptation for UAV Vision-Language Navigation

Abstract

Unmanned aerial vehicle vision–language navigation (UAV-VLN) has attracted increasing attention, requiring an embodied agent to follow natural-language instructions by sequentially reasoning over visual observations in open 3D environments and executing navigation actions. Despite recent progress, more drastic viewpoint and scene variations in unseen environments can induce substantial train–test distribution shifts, manifested as cross-modal misalignment and feature drift at the perception stage, as well as biased action prediction at the decision stage. To address these issues, we propose Hierarchical Perception– Decision Test-Time Adaptation for UAV-VLN. At the perception stage, we propose a Memory-Augmented Cross-modal Perception Adaptation (MACPA) module. By constructing an environment semantic memory bank and introducing adaptive tokens, MACPA generates adaptive semantic guidance prompts to mitigate semantic drift. At the decision stage, we propose an Entropy-Gated Historical Decision Calibration (EHDC) module, which aggregates high-confidence historical action distributions based on the similarity between the current state and cached features, alleviating decision drift and enhancing the consistency and stability of predictions. Extensive experiments on the UAV-VLN benchmarks, OpenFly and OpenUAV, demonstrate that our method achieves better navigation performance and more stable decision behavior than state-of-the-art approaches in unseen environments.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Unmanned aerial vehicle vision–language navigation, test-time adaptation, perception–decision adaptation.

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_yWn1OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes HPD, a gradient-free test-time adaptation framework for UAV vision-language navigation. HPD combines MACPA, which uses an online semantic memory and adaptive tokens to improve perception under distribution shift, and EHDC, which calibrates uncertain action predictions using retrieved historical decisions. Experiments on OpenFly and OpenUAV report improved navigation performance over baseline and competing TTA methods.

Critical strengths
  1. This paper proposes an effective hierarchical test-time adaptation framework for UAV-VLN. HPD jointly adapts perception and decision making under distribution shift, leading to consistent improvements on OpenFly and OpenUAV.

  2. HPD keeps the backbone frozen and avoids test-time backpropagation. On the 1B model, it adds only 68 MB of GPU memory and 21 ms per step. This modest overhead makes its navigation gains practically appealing.

Critical weaknesses
  1. The novelty of MACPA needs to be better clarified. IDEA [1] has already explored storing historical adaptation knowledge in an external memory for online VLN adaptation and reusing such knowledge based on the relevance between the current and historical environments. The core paradigm of MACPA appears highly similar to that of IDEA. The authors should therefore clarify the methodological novelty of MACPA over IDEA and provide a performance comparison.

  2. The effectiveness of EHDC requires further justification. EHDC retrieves similar states from the historical cache using visual features. However, in VLN, actions depend not only on the current visual observation but also on the instruction and navigation context. Thus visual similarity does not necessarily imply decision similarity, and visually similar states may require entirely different optimal actions.

  3. The effectiveness of entropy gating also requires justification. When the current prediction approaches maximum entropy, the fusion increasingly favors the historical prediction. However, high uncertainty in the current model does not guarantee that the retrieved historical information is reliable.

[1] Turning Adaptation into Assets: Cross-Domain Bridging for Online Vision-Language Navigation, ICML 2026

Questions for authors
  1. How does MACPA differ from IDEA, and how does it compare under the same evaluation protocol?

  2. Why does EHDC use visual features alone for historical retrieval?

  3. How does entropy gating avoid over-relying on unreliable historical predictions when the current model is highly uncertain?

1 comment

W1&Q1. We thank the reviewer for the valuable suggestion. We acknowledge that IDEA has explored the use of external memory to store and reuse historical adaptation knowledge. The main differences between the two methods lie in the memory contents and their utilization mechanisms. IDEA employs source-domain feature statistics as distribution alignment constraints, optimizes Soft Visual Prompts through gradient-based updates, and combines Fisher guidance with convex hull projection to facilitate the cross-domain transfer of historical adaptation parameters. In contrast, MACPA keeps model parameters frozen and does not perform gradient-based parameter optimization. Instead, it uses historical visual states as Keys and instruction-conditioned semantics as Values, enabling the online reuse of historical semantic information and enhancement of current perceptual representations through similarity-based retrieval and EMA updates. Thus, IDEA primarily focuses on parameter-level test-time optimization and cross-domain transfer, whereas MACPA emphasizes feature-level historical semantic retrieval and online enhancement. It is worth noting that IDEA incorporates both source-domain feature statistics for additional distribution alignment and gradient-based parameter optimization, whereas MACPA does not involve model parameter updates during testing. Accordingly, the two methods explore different ways of leveraging historical information under distinct adaptation settings. We sincerely thank the reviewer again for the insightful comments and constructive suggestions.

W2&Q2. We thank the reviewer for this valuable suggestion. EHDC uses visual features for historical retrieval primarily because visual observations directly reflect the current environmental layout, the relative positions of landmarks, and local motion states. During continuous navigation, these visual cues exhibit a degree of temporal continuity. For example, when a UAV continues moving forward, consecutive observations typically share similar visual scenes and environmental structures. Therefore, historical forward actions associated with similar visual states can provide useful references for current decision-making. In contrast, when the UAV performs a turning action, changes in the viewing direction and the relative positions of objects in the scene generally lead to corresponding changes in visual features, providing cues for distinguishing different local motion states. Accordingly, visual state similarity is used to retrieve historical actions primarily to exploit environmental continuity and provide historical experience for local action selection, rather than to directly infer the overall navigation intent. Meanwhile, MACPA complements task-relevant information through instruction-conditioned historical semantic retrieval, allowing the two modules to provide complementary support at the perception and decision levels. We also acknowledge that visual similarity alone cannot fully guarantee action compatibility across different navigation instructions, which remains a limitation of the current design. Future work will further explore instruction- and navigation-progress-conditioned memory retrieval mechanisms that incorporate task-semantic compatibility constraints while preserving visual state matching, thereby improving the reliability of cross-instruction historical action reuse. We sincerely thank the reviewer again for the insightful comments and constructive suggestions.

W3&Q3. We thank the reviewer for raising this important concern. We agree that high uncertainty in the current prediction does not necessarily imply that historical decision priors are reliable. Within the HPD framework, we address this issue from two aspects. First, EHDC does not indiscriminately store all historical predictions; instead, it applies confidence-based filtering to retain only high-confidence action priors, thereby improving the reliability of historical decision information. Second, MACPA provides complementary support for the semantic consistency of historical decisions at the perception level. Specifically, the action distributions stored in the EHDC cache are generated by the policy network based on cross-modal navigation-state representations. By enhancing current perceptual representations through instruction-conditioned historical semantic retrieval, MACPA helps improve the task-semantic consistency of these cross-modal representations, thereby indirectly increasing the relevance of the historical action information accumulated and reused by EHDC to the current navigation task. Therefore, the utilization of historical decisions is influenced not only by entropy gating but also by confidence-based action filtering and upstream semantic representations. We sincerely thank the reviewer again for the insightful comments and constructive suggestions.

Reviewer_u494OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes HPD, a hierarchical perception–decision test-time adaptation framework for UAV vision–language navigation in unseen environments, without updating backbone parameters. HPD consists of two modules: MACPA, which retrieves instruction-conditioned semantic memory using visual features and generates adaptive semantic prompts, and EHDC, which maintains low-entropy action prototypes to calibrate uncertain predictions. Experiments on OpenFly and OpenUAV demonstrate navigation improvements, supported by ablation studies, efficiency analysis, hyperparameter sensitivity, and evaluation-order analysis.

Critical strengths

S1. Practical motivation and efficiency. The paper addresses efficient test-time adaptation without backpropagation. Table 3 reports only 21 ms additional inference latency and 68 MB memory overhead, demonstrating a favorable performance–efficiency trade-off. S2. Consistent improvements under the same backbone. Table 3 shows that HPD improves SR from 23.14% to 28.85% and SPL from 21.46% to 25.13% using the same 1B backbone. Table 4 further shows that MACPA and EHDC each provide improvements, while their combination performs better.

Critical weaknesses

W1. Inconsistent experimental statistics and ablation settings. In Table 2, the relationship between Full, Easy, and Hard is unclear. If Easy and Hard partition Full, their weighted averages should be consistent across NE, SR, OSR, and SPL. However, the implied subset weights differ substantially beyond rounding errors. If these results come from independent evaluations with different adaptation histories, the protocol should be clarified. Moreover, Table 5 reports baseline performance (SR 23.14%, SPL 21.46%) when N = 0 or K = 0, whereas EHDC alone achieves SR 26.11% and SPL 24.65% in Table 4. It is unclear whether EHDC remains enabled in Table 5. These inconsistencies directly affect experimental verifiability and reproducibility. W2. Incomplete algorithmic definitions and implementation details. Equation (6) uses the historical prior p̃ₜ, but its computation is not fully defined. Equation (5) describes cache contents and replacement rules without specifying similarity normalization, softmax temperature, handling of empty action categories, or cache read/write order. For MACPA, the initialization of L₀, the source of cross-modal attention parameters, and the value of μ in Eq. (4) are unspecified. Furthermore, the 1B checkpoint, MLP action-head training, and possible source-domain pretraining of additional modules are insufficiently documented. Freezing parameters at test time does not resolve these reproducibility concerns. W3. Unverified reliability of cross-instruction decision reuse. EHDC retrieves historical action prototypes using pooled visual features, without explicitly considering the current instruction or navigation progress. Similar observations may require different actions under different instructions or at different navigation stages. Moreover, when prediction entropy approaches its maximum, αₜ approaches 1 in Eq. (6), potentially allowing historical priors to override instruction-conditioned predictions without verifying task compatibility. Although the failure analysis acknowledges mismatched historical priors, targeted quantitative evidence is missing.

Questions for authors

Q1. Experimental consistency: Are Full/Easy/Hard in Table 2 statistics from a single evaluation or independent runs? When N = 0 or K = 0 in Table 5, is EHDC enabled? Please clarify the configurations relative to Table 4. Q2. Algorithmic reproducibility: Please provide the complete computation of p̃ₜ, cache read/write procedures, and empty-cache handling. Also clarify MACPA initialization, attention parameter sources, and offline training of additional components. Q3. Evaluation protocol: How were hyperparameters selected, and which validation data were used? Are OpenFly baseline results quoted or re-evaluated on the same public subset? Please provide OpenUAV results using the identical backbone checkpoint without HPD.

1 comment

W1&Q1. We sincerely thank the reviewer for pointing out these issues. These results were indeed obtained from independent evaluations on different subsets. In the revised version, we have updated the complete results, which remain comparable to those reported in the current version. We have also carefully checked and corrected the corresponding experimental settings. We sincerely thank the reviewer again for the careful review and valuable suggestions.

W2&Q2. We thank the reviewer for the valuable comments. To address the insufficient descriptions of the algorithmic definitions and implementation details, we have supplemented the revised manuscript with the relevant information. Specifically, we have clarified the computation of the historical prior �0��0� in EHDC, including cosine similarity calculation using L2-normalized inner products, Top-1 retrieval, empty-slot handling, and cache read/write order. We also specify that no additional softmax operation or temperature parameter is required. Furthermore, we have provided the MACPA implementation details, including the initialization of �1��1� memory using 1,000 samples from the training set, the fine-tuning of cross-modal attention weights on the training data, and the momentum coefficient �2��2�. We have also clarified the language backbone configuration (Meta Llama-3.2-1B, with a hidden dimension of 2,048), as well as the input representation, ten-class discrete action output, and cross-entropy training objective of the MLP action classification head. These additions provide a more complete description of the algorithm and model configuration, improving the reproducibility of our approach. We sincerely thank the reviewer again for the constructive suggestions.

W3. We thank the reviewer for this valuable comment. We acknowledge that EHDC currently retrieves historical action prototypes based on pooled visual features, without explicitly modeling the current navigation instruction or task progress, which represents a limitation of the current design.

Our motivation is that visual observations continuously change during navigation and provide rich environmental state information. Retrieving historical information associated with similar visual states can therefore provide supplementary decision references for current action prediction. Meanwhile, MACPA in HPD complements task-relevant information through instruction-conditioned historical semantic retrieval, providing semantic priors for current perception. Thus, EHDC primarily leverages visual state similarity to retrieve historical decision references, whereas MACPA enhances task-relevant semantics in current perceptual representations, allowing the two modules to provide complementary information at different levels.

Furthermore, we acknowledge that high prediction entropy does not necessarily guarantee the reliability of historical decision priors, and entropy gating alone may introduce mismatched historical information. To mitigate this risk, EHDC not only uses entropy gating to regulate the use of historical information but also incorporates an entropy-based selection mechanism during memory updates, prioritizing historical decisions with lower prediction entropy to improve the reliability of cached action information. Future work will further explore instruction- and progress-aware historical information retrieval and reliability assessment to improve the accuracy of historical decision calibration.

Q3. We thank the reviewer for raising this point. Hyperparameters were selected based on validation performance using a subset of 500 Seen trajectories (35.9%) and subsequently kept fixed for all evaluations. We also conducted ablation studies on the Unseen split to examine the generalizability of the selected configuration. We apologize for the lack of clarity regarding the OpenFly baselines. The reported baseline results were directly quoted from the original paper, and we additionally provide our own reproduced results for comparison.

SplitNESROSRSPL
Seen9433.855.730.3
Unseen185.421.049.818.1
Reviewer_7CyzOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses train–test distribution shift in Unmanned Aerial Vehicle Vision-Language Navigation (UAV-VLN). The authors decompose the navigation pipeline into a perception stage and a decision stage, and propose HPD, a gradient-free test-time adaptation framework that adapts both stages without updating backbone parameters:

  1. MACPA (perception): an online semantic memory bank (keys = visual-state prototypes, values = instruction-conditioned semantic embeddings, capacity K ≤ 1000) plus N = 5 adaptive tokens that retrieve the top-5 nearest entries, softly aggregate them into environment-adaptive semantic prompts, and are themselves updated by an EMA (λ = 0.6). Memory values are refined online by momentum updates.
  2. EHDC (decision): a class-wise key-value cache storing one high-confidence action-distribution prototype per action class; when prediction entropy exceeds a threshold (γ = 0.3), the current action distribution is convexly fused with the cached historical prior, weighted by an uncertainty-dependent coefficient αtα_t.
Critical strengths
  1. Well-motivated problem at a genuine gap. Perception-level semantic drift and decision-level instability under viewpoint/scene variation are real failure modes in UAV-VLN, and to my knowledge this is the first TTA framework designed specifically for UAV-VLN; existing TTA-for-VLN work targets indoor/ground settings. The two-stage decomposition is a natural and useful framing of the problem.
  2. Strong efficiency story with evidence. Unlike gradient-based TTA competitors, HPD adds only ~68 MB GPU memory and ~21 ms per-step latency, while outperforming LTPO by 1.68 SR / 0.73 SPL at 221 ms lower latency (Table 3). This matters for onboard UAV deployment and is one of the most convincing parts of the paper.
  3. Thorough ablation and sensitivity coverage. Module-level ablation (Table 4), cross-TTA comparison with per-method implementation details (Table 3, appendix §6), sweeps over NN, KK, λ\lambda, and γ\gamma, and a 5-seed test-ordering robustness study (SR varies ≤ 0.5 pp) together constitute an unusually complete hyperparameter/robustness package for this area. I verified all reported gain claims in Tables 1-4 for internal consistency and found no numeric discrepancies.
Critical weaknesses
  1. The novelty is compositional and the delta is not argued. MACPA closely combines known mechanisms, online key-value memory retrieval (in the line of TDA/AdaNPC-style caching) with EMA-updated adaptive prompt tokens (test-time prompt tuning) , and EHDC closely combines a TDA-style per-class high-confidence cache with SAR-style entropy filtering. The claimed contribution is the hierarchical coupling of perception- and decision-level adaptation, yet the paper provides no analysis of why the two stages must be coupled or what behavior emerges from the coupling beyond the sum of parts; the ablation (Table 4) shows strictly additive gains (+2.4 SR from MACPA, +3.0 from EHDC, +5.7 combined), consistent with a pure composition hypothesis. A direct comparison against "apply an existing parameter-free TTA method (e.g., TDA-style caching) to the same OpenFly-Agent" is missing, so the marginal value of the proposed design over its own components' predecessors is unestablished.
  2. Evaluation coverage is inconsistent across benchmarks. All TTA baselines are compared only on OpenFly (Table 3). The OpenUAV table (Table 2) contains only non-TTA methods (Random/Fixed/CMA/OpenUAV/NavFoM), so the central claim of being a better test-time adaptation method is never tested on the second benchmark. Additionally, the 7B results in Table 1 have no implementation description anywhere (how prompts are injected into the frozen LLaMA-7B, whether the action head is retrained), making them unauditable as reported.
  3. No statistical significance for the main claims. All main tables report single-run numbers. The only multi-seed experiment (Fig. 5) reports SR 28.65 ± 0.22, which puts the headline gains over NavFoM (+1.89 SR seen, +3.90 unseen) near or within seed noise. The relationship between the Fig. 5 mean (28.65) and the Table 4 headline (28.85) is also unexplained.
  4. The TTA setting is blurred by training-split memory initialization. MACPA's memory is initialized with K = 1000 samples drawn by trajectory-balanced sampling from the training split (stated only in appendix §6, not the main text). This is not the standard source-free TTA setting and gives HPD an implicit information advantage over pure test-time methods (SAR, FSTTA) in Table 3, undermining the "same backbone, same protocol" fairness claim. The main text must state this, and the fairness implications need analysis or a source-free variant.
  5. Limited reproducibility. No code release statement, no random seeds, and no description of how the 1B OpenFly-Agent was trained (data, steps, losses). The provenance and license of the LLaMA-1B replacement backbone are unstated. Several reported numbers (e.g., the 7B configuration) cannot be reproduced from the paper as written.
  6. The OSR regression on OpenFly is explained post-hoc without evidence. On OpenFly Test-Seen (7B), OSR drops 64.3 → 57.5 with HPD; the text attributes this to "a more conservative and goal-directed stopping strategy," but no stopping-behavior analysis (stop accuracy, progress rate, distance-at-stop distribution) is provided to support this causal claim.
Questions for authors
  1. Coupling vs. composition. What evidence can you provide that the perception–decision coupling in HPD produces behavior beyond an additive combination of its parts? Concretely: how does HPD compare against applying TDA (or an equivalent per-class key-value cache with pseudo-label refinement) directly to the OpenFly-Agent, both in final metrics and in failure-mode profile? If the gains are additive, why is a two-stage framework necessary rather than either module alone?
  2. 7B protocol. For the 7B OpenFly-Agent + HPD results in Table 1: how are the MACPA prompt tokens injected into the frozen LLaMA-7B (cross-attention layers, input embeddings, adapter insertion?), and is any component (action head, projection layers) retrained for the 7B configuration? Please specify the full protocol.
  3. Memory initialization and fairness. Since the memory bank is initialized from 1,000 training-split samples, do the compared TTA baselines in Table 3 receive equivalent access to training data? Would HPD still outperform TDA/SAR/FSTTA under a strict source-free protocol (cold-start memory, no training-split initialization)? Please report a source-free variant.
  4. Statistical robustness of the OpenUAV gains. Can you provide multi-seed results (≥ 3 runs) with confidence intervals for Table 2, particularly for the gains over NavFoM (+1.89 SR seen, +3.90 unseen)? Are these differences statistically significant?
  5. TTA baselines on OpenUAV. Why are SAR/FSTTA/TDA not compared on OpenUAV? Are there obstacles (e.g., action-space mismatch, simulator differences), and if not, can you add this comparison? Without it, the TTA claim is supported on only one of the two benchmarks.
2 comments

W1&Q1. We thank the reviewer for the valuable comments. Although online memory retrieval, EMA updates, and entropy filtering are established techniques, we tailor them to the characteristics of UAV navigation by designing a functionally complementary two-stage adaptation mechanism. Specifically, MACPA retrieves historical information based on visual states and leverages instruction-conditioned semantic information to enhance current perceptual representations, whereas EHDC uses high-confidence historical actions and entropy gating to adaptively calibrate action distributions at the decision level. Regarding the additive effects of the two stages, MACPA enhances current perception through historical semantic retrieval, although redundant or semantically mismatched information may affect action prediction. EHDC provides complementary historical action references for decision calibration, helping mitigate the influence of unreliable information on action selection. Conversely, EHDC alone cannot compensate for missing historical semantic information. Thus, the two stages complement each other through semantic representation enhancement and action decision calibration. Furthermore, as shown in Table 3, compared with TDA, HPD improves SR and SPL by 2.29 and 2.42 percentage points, respectively, while reducing NE by approximately 5 m. These results further demonstrate the advantages of the proposed two-stage framework over directly applying existing cache-based TTA methods. We sincerely thank the reviewer again for the constructive comments.

W2&Q2&Q5. We thank the reviewer for this valuable suggestion. We acknowledge that TTA comparisons across different benchmarks remain incomplete. Since different TTA methods require specific model adaptations and evaluation protocol adjustments, while repeated evaluations across multiple random seeds incur considerable computational overhead, we will further explore TTA comparisons on OpenUAV. Regarding the 7B configuration, it follows the same adaptation procedure as the 1B model. Before testing, both the MLP action prediction head and the cross-attention head are trained, while the LLaMA-7B backbone remains frozen. During testing, all three components remain frozen. The semantic prompts generated by MACPA are incorporated as input embeddings alongside navigation instructions, historical information, and visual observations. These implementation details have been clarified in the revised manuscript to improve reproducibility.

W3&Q4. We thank the reviewer for this valuable suggestion. Our main results were originally obtained from multiple runs with a fixed random seed. To more comprehensively account for the effect of random trajectory ordering on the proposed method, we instead report the mean results across five random seeds for all experiments and have updated the revised manuscript accordingly. On OpenUAV, the SR decreases by 0.95 percentage points under this evaluation protocol, while the overall performance remains comparable. We sincerely thank the reviewer again for the careful review and constructive suggestions.

W4&Q3. We thank the reviewer for the valuable comment. We acknowledge that the description of this experimental setting in the original manuscript was not sufficiently precise. Accordingly, we have removed the inappropriate statements regarding the comparative experiments in the revised manuscript and clarified that the HPD semantic memory is initialized with 1,000 trajectory-balanced samples from the training set and subsequently updated online during testing. This initialization is intended to provide an initial semantic prior at the beginning of test-time adaptation, thereby supporting memory retrieval and decision calibration. We sincerely thank the reviewer again for this constructive suggestion.

W5. We thank the reviewer for the valuable comments. We plan to release the code along with the corresponding training and evaluation configurations. For both the 1B and 7B configurations, the action prediction head is trained before testing, while all other components remain frozen. During testing, both the language backbone and the action prediction head remain frozen. The semantic representations obtained through MACPA retrieval and aggregation are fed into the model as input embeddings. HPD performs test-time adaptation through online memory updates, token-state updates, and action distribution calibration, without any gradient-based optimization. These details have been added to the revised manuscript. We sincerely thank the reviewer again for the constructive suggestions.

W6. We thank the reviewer for the valuable suggestion. The action-distribution analysis in the supplementary material further supports this interpretation. HPD reduces the proportion of long-distance forward actions (Forward 9) from 84.2% to 72.1%, while increasing short-distance forward movements and left/right turns, indicating a greater preference for local positional and directional adjustments and less over-traversal near the target. This behavior is consistent with the results in Table 1. Although OSR decreases from 64.3% to 57.5%, SR increases from 34.3% to 38.6%, narrowing the gap between OSR and SR from 30.0 to 18.9 percentage points. Meanwhile, SPL increases from 24.9% to 36.2%. These results suggest that HPD improves task completion primarily by reducing over-traversal and promoting more effective local action adjustments. We sincerely thank the reviewer again for the constructive comments.

Reviewer_6eWgOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Hierarchical Perception–Decision Test-Time Adaptation (HPD) for UAV vision–language navigation in unseen environments. The framework consists of Memory-Augmented Cross-modal Perception Adaptation (MACPA), which retrieves historical semantic information to mitigate perception drift, and Entropy-Gated Historical Decision Calibration (EHDC), which uses cached action distributions to refine uncertain predictions. Both modules operate without test-time gradient updates. Experiments on OpenFly and OpenUAV demonstrate improved navigation performance, with relatively low inference overhead.

Critical strengths
  • The paper is well-written and easy to follow, with a clear motivation and a well-organized presentation of the proposed perception and decision adaptation modules.
  • The proposed EHDC module provides a meaningful approach to stabilizing action predictions by adaptively incorporating historical decisions based on prediction uncertainty.
  • The proposed method achieves consistent improvements on OpenFly and OpenUAV, outperforming existing TTA baselines with relatively low inference overhead. The component-wise ablations and supplementary analyses further support its effectiveness.
Critical weaknesses
  • Although MACPA and EHDC effectively leverage historical information, their advantages over simpler history modeling strategies remain unclear. Comparisons with conventional approaches, such as reusing historical visual tokens, temporal feature aggregation, or recurrent history modeling, would help clarify whether the additional memory retrieval and decision calibration mechanisms provide meaningful benefits over simpler alternatives.
  • Although MACPA aims to mitigate semantic drift by retrieving relevant historical information, the paper provides limited analysis of the retrieval process itself. Visualizing the retrieved memory entries and their similarity or attention weights, particularly in successful and failed navigation cases, would help clarify whether MACPA effectively retrieves instruction-relevant semantic information.
  • Several in-text citations appear without proper parentheses or consistent formatting, making parts of the paper difficult to read. The citation formatting should be reviewed and corrected for better readability.
Questions for authors

Please refer to the weaknesses section.

1 comment

W1. We thank the reviewer for this valuable suggestion. We agree that direct comparisons with simpler history modeling strategies would help further clarify the contributions of the proposed mechanisms. The existing ablation studies provide indirect evidence of the effectiveness of MACPA and EHDC. On the OpenFly Test-Unseen split, each module independently improves navigation performance, while their combination yields further gains, demonstrating their complementary contributions. Moreover, HPD achieves these performance improvements with only modest additional computational overhead. Overall, the existing results provide indirect evidence of the effectiveness of the proposed historical memory retrieval and decision calibration mechanisms in terms of performance gains, module complementarity, and computational efficiency.

W2. We thank the reviewer for this valuable suggestion. We acknowledge that the original manuscript provides limited visualization of MACPA's retrieval behavior. MACPA stores historical semantic features aggregated in an instruction-conditioned manner and continuously updates them during online navigation. These features serve as implicit semantic priors to facilitate the updating of learnable prompts and enhance current perceptual representations. It is important to note that these memory features are latent semantic representations rather than explicitly aligned with specific visual object regions or category labels, and their primary role is to provide implicit semantic guidance for current perception. Consequently, the retrieval weights cannot be directly mapped to specific object regions or category responses in the images, and simply visualizing the retrieved memory entries and their weights may not adequately reveal their actual semantic contributions. We sincerely thank the reviewer again for this valuable suggestion.W

W3. We thank the reviewer for pointing out the citation formatting issues. We have carefully checked and revised all in-text citations, standardized the formatting of parenthetical citations, added missing spaces before citations, and replaced manually typed citation numbers with standard LaTeX citation commands to ensure consistency. We also adjusted parentheses and punctuation around certain citations to improve readability.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.