acceptodds
Withdrawn from ICLR 2027

PrivacyNav: Controlling Visual Representation Disclosure in Vision-Language Navigation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-Language Navigation (VLN) systems rely on semantically rich visual representations for scene understanding and action generation. However, in modular, cross-device, or remote inference, these representations are often transmitted beyond the trusted perception boundary, where an honest-but-curious recipient could recover policy-defined sensitive information. To address this issue, we propose PrivacyNav, a state-adaptive visual disclosure framework that controls which visual evidence is disclosed and how much is disclosed at each navigation state, while keeping the underlying VLN policy frozen. PrivacyNav comprises two modules. Privacy-Aware Evidence Estimator predicts token-level disclosure safety together with state-level disclosure risk, yielding localized and contextual signals. Adaptive Visual Disclosure Controller ranks visual tokens by jointly considering disclosure safety and host-native navigation evidence, and maps state-level risk to a state-specific token retention ratio. We conduct extensive experiments across four heterogeneous VLN architectures, complemented by module ablations, operating-point analyses, and real-world deployment. Compared with disclosing unmodified representations, PrivacyNav reduces macro sensitive-concept recovery by 3.23–10.38 percentage points, with navigation success rate losses of 0.81–5.11 percentage points. These results demonstrate consistent reductions in sensitive semantic recoverability under the evaluated protocols, with host-dependent disclosure–utility trade-offs rather than a formal privacy guarantee.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language Navigation, Embodied AI, Vision-Language Models, Visual Disclosure Control, Semantic Recovery

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2DpsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PrivacyNav investigates the issue of intermediate visual representations leaking sensitive semantics in modular vision-language navigation (VLN) systems. The proposed method first predicts token-level sensitivity and state-level risk using a weakly supervised estimator. It then ranks tokens by combining disclosure safety with navigation-relevant signals from the host model, dynamically adjusting the token retention ratio based on risk while keeping the navigation policy frozen throughout the process. Experiments cover four VLN architectures, 1,839 R2R val-unseen navigation episodes, COCO image caption recovery, 973 independently annotated navigation states, and real-robot deployments.

Critical strengths

Controlling information disclosure before visual representations cross trust boundaries holds practical value for modular embodied systems. The authors integrated this approach into four heterogeneous, frozen VLN models and conducted real-robot deployments, providing meaningful evidence that the method adapts well across different architectures.

In Table 2, with similar average navigation token retention ratios, removing disclosure-aware selection causes the macro-average recovery rate of sensitive concepts to rise from 40.06% to 50.50%. Furthermore, the authors used non-overlapping scene splits and manually verified the independently constructed in-domain evaluation labels. The paper explicitly distinguishes empirical drops in recovery rates from formal privacy guarantees, enhancing the credibility of the reported results.

Critical weaknesses

Section 3.1 excludes attacks that infer information from token counts and selection patterns; however, Appendix B explicitly states that the deployed StreamVLN data packets include the selected tokens alongside their indices. Because the retention ratio directly depends on sensitivity risk, this observable information could itself expose sensitive scene states. Additionally, the fixed image caption recovery model and room query protocol cannot distinguish between "a genuine reduction in sensitive information" versus "the visual representation shift merely causing a specific recovery model to fail." Although the authors explicitly acknowledge these limitations, these evaluation gaps substantially weaken the evidence that the method can defend against the posited curious receiver.

The paper has not fully demonstrated a superior trade-off between information disclosure and navigation performance. Table 1 compares only single configurations for each baseline, whereas performance curves are reported solely for PrivacyNav's parameters. For example, Gaussian achieves a navigation success rate of only 7.29% on Uni-NaVid—far below PrivacyNav's 47.96%—making it an inadequate control for comparing privacy at similar navigation performance levels. A similar token retention ratio also fails to prove an advantage under equal navigation performance. For the state-adaptive mechanism, Table 2 shows that while navigation success rate improves, the sensitive information recovery rate also slightly increases. Existing results do not sufficiently establish its advantage over reasonably tuned simple baselines.

Questions for authors
  1. If a sensitive concept detector is trained on post-defense representations and granted access to the actual transmitted token count and indices, does the drop in sensitive information recovery rate persist?

  2. When matched to navigation success rates similar to PrivacyNav, what are the sensitive information recovery rates of tuned VisionZip, Gaussian, and a naive approach that selects tokens solely based on sensitivity?

Reviewer_op6gOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper addresses representation level leakage in split or remote inference VLN pipelines, where intermediate visual tokens cross a trust boundary and can be inverted into captions that name sensitive content. The proposed method, PrivacyNav, bolts two modules onto a frozen VLN host. A Privacy Aware Evidence Estimator predicts a per token disclosure safety score and a per state risk score, trained with weak supervision from a frozen VLM room classifier and an off the shelf object detector. An Adaptive Visual Disclosure Controller then maps state risk to a token retention count and ranks tokens by a weighted blend of disclosure safety and the host’s own attention, keeping the highest scoring tokens. Experiments cover four frozen VLN hosts, with navigation measured on R2R val-unseen and leakage measured by a frozen caption inversion attacker on COCO, plus a smaller in domain probe over navigation states and a physical robot deployment. The method reduces recovery of sensitive concepts by a few percentage points at a modest cost in navigation success rate.

Critical strengths

The negative result on efficiency pruning is valuable and cleanly demonstrated. At a matched token budget, the token compression baseline leaves sensitive concept recovery essentially unchanged.

The comparison against additive noise is informative in a non obvious direction. Gaussian perturbation degrades general caption quality more than PrivacyNav does while still leaking more sensitive content. The implication, that coarse high energy semantics survive perturbation while fine descriptive detail does not, is a genuine insight and a good argument for targeted selection over undirected noise.

Critical weaknesses
  1. The connection to navigation is tenuous, and the paper is better positioned as general trustworthy/robust AI work. Nothing in the core mechanism is navigation specific. A learned per token safety head, host attention as a utility proxy, rank fusion, and top scoring selection form a generic recipe for split inference defense. The cited papers are from the general privacy preserving ML literature, and the method would transfer unchanged to any pipeline that transmits visual tokens.

Considering the evaluation, the privacy numbers come from COCO, which is not navigation data at all. Only the utility axis is embodied. The one genuinely in domain privacy experiment uses a weaker query protocol, runs on a much smaller set of observations, and supports within host comparisons only, as the authors themselves note. So the privacy claim is a generic vision claim using a navigation label. The work is better served by repositioning as a general split inference defense and evaluate across several domains, such as VQA, retrieval, captioning, and action recognition, to show the approach generalizes, with VLN as one instance.

  1. The protection is partial. After defense, the attacker still names a correct sensitive concept for the large majority of frames that contain one, and recovers a substantial fraction of sensitive concepts overall. This is attenuation, not protection, and the writing does not sufficiently convey that a user’s practical exposure changes little.

  2. No adaptive attacker. The authors disclose this, but it undercuts the central claim. The information remains in the retained tokens, and the mean fill scheme used on one host leaks the mask pattern, which marks exactly the positions the estimator judged risky. A decoder finetuned on PrivacyNav outputs would have both the retained evidence and that spatial hint. Defenses in this area have repeatedly fallen to adaptive attacks, so this is the experiment that matters most, which is missing.

  3. The second module does not do what the introduction says it does. The second challenge is framed as a privacy problem. The ablation shows that state adaptive retention improves navigation success while slightly increasing sensitive recovery. It is a utility recovery mechanism, not a privacy mechanism. The framing should match the finding.

  4. Operating points are not matched within a row. On at least one host the navigation retention and the COCO retention differ substantially, so the utility and privacy figures in the same table row come from different settings.

  5. Thin real world evidence. The caption audit uses a handful of observation sources with correlated repeats. The long horizon navigation result rests on a small number of episodes and is reported without uncertainty.

  6. Lastly, the paper exposition is poor, with unnecessary jargon and obtuse phrasing. The AI disclosure states that AI was used to "assist with scientific writing" but it would seem the AI did a poor job and ended up obfuscating the message altogether.

Questions for authors

The authors are encouraged to address the concerns raised, particularly about repositioning the core central claims of the work as it's relationshipn to robot navigation is teneous at best.

Reviewer_3aMvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents PrivacyNav, a framework designed to reduce sensitive information leakage in Vision-Language Navigation (VLN). It estimates the sensitivity of visual tokens and adjusts which tokens are shared based on their importance for navigation and the privacy risk of the current scene. Experiments on four VLN models show that PrivacyNav can reduce sensitive information recovery while maintaining reasonable navigation performance.

Critical strengths
  • The paper addresses an important privacy issue in VLN systems. The proposed approach is practical and can be applied to existing navigation models without retraining them.
  • The experiments cover four different VLN models and include ablation studies and real-world testing, which help demonstrate the effectiveness and applicability of the proposed framework.
Critical weaknesses
  • The privacy evaluation mainly relies on fixed recovery models. It is unclear whether the proposed approach would remain effective against stronger attackers who are aware of the defense and can adapt their recovery methods.
  • The method causes some loss in navigation performance, and the advantage of adaptive token retention over simpler fixed-retention strategies is not fully demonstrated. Moreover, privacy and navigation are mainly evaluated on different datasets (COCO and R2R), making it difficult to directly assess the privacy-utility trade-off.
Questions for authors
  • How well would PrivacyNav perform against attackers trained specifically to recover information from its modified visual representations?
  • Can the authors compare PrivacyNav with other privacy-preserving approaches under similar navigation performance or token retention ratios?
  • Can the authors provide more evidence showing that adaptive token retention consistently performs better than fixed token retention?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.