- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
PrivacyNav investigates the issue of intermediate visual representations leaking sensitive semantics in modular vision-language navigation (VLN) systems. The proposed method first predicts token-level sensitivity and state-level risk using a weakly supervised estimator. It then ranks tokens by combining disclosure safety with navigation-relevant signals from the host model, dynamically adjusting the token retention ratio based on risk while keeping the navigation policy frozen throughout the process. Experiments cover four VLN architectures, 1,839 R2R val-unseen navigation episodes, COCO image caption recovery, 973 independently annotated navigation states, and real-robot deployments.
Critical strengths
Controlling information disclosure before visual representations cross trust boundaries holds practical value for modular embodied systems. The authors integrated this approach into four heterogeneous, frozen VLN models and conducted real-robot deployments, providing meaningful evidence that the method adapts well across different architectures.
In Table 2, with similar average navigation token retention ratios, removing disclosure-aware selection causes the macro-average recovery rate of sensitive concepts to rise from 40.06% to 50.50%. Furthermore, the authors used non-overlapping scene splits and manually verified the independently constructed in-domain evaluation labels. The paper explicitly distinguishes empirical drops in recovery rates from formal privacy guarantees, enhancing the credibility of the reported results.
Critical weaknesses
Section 3.1 excludes attacks that infer information from token counts and selection patterns; however, Appendix B explicitly states that the deployed StreamVLN data packets include the selected tokens alongside their indices. Because the retention ratio directly depends on sensitivity risk, this observable information could itself expose sensitive scene states. Additionally, the fixed image caption recovery model and room query protocol cannot distinguish between "a genuine reduction in sensitive information" versus "the visual representation shift merely causing a specific recovery model to fail." Although the authors explicitly acknowledge these limitations, these evaluation gaps substantially weaken the evidence that the method can defend against the posited curious receiver.
The paper has not fully demonstrated a superior trade-off between information disclosure and navigation performance. Table 1 compares only single configurations for each baseline, whereas performance curves are reported solely for PrivacyNav's parameters. For example, Gaussian achieves a navigation success rate of only 7.29% on Uni-NaVid—far below PrivacyNav's 47.96%—making it an inadequate control for comparing privacy at similar navigation performance levels. A similar token retention ratio also fails to prove an advantage under equal navigation performance. For the state-adaptive mechanism, Table 2 shows that while navigation success rate improves, the sensitive information recovery rate also slightly increases. Existing results do not sufficiently establish its advantage over reasonably tuned simple baselines.
Questions for authors
-
If a sensitive concept detector is trained on post-defense representations and granted access to the actual transmitted token count and indices, does the drop in sensitive information recovery rate persist?
-
When matched to navigation success rates similar to PrivacyNav, what are the sensitive information recovery rates of tuned VisionZip, Gaussian, and a naive approach that selects tokens solely based on sensitivity?