- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies window-level DP fine-tuning for VLA models, assuming visual observations, instructions, and robot states are public. The authors propose DP-VLA_MC, which combines tighter gradient clipping with noise-aware second-moment correction (adapted from DP-AdamBC) and a variance floor to adjust AdamW updates under flow-matching objectives. They also introduce Action-Ref, a shadow-model-calibrated membership inference metric.
Critical strengths
- Clean ablations on optimizer dynamics under gradient noise
- good writing
Critical weaknesses
-
The paper only achieves non-trivial task utility at (with single-step ). In DP theory, yields an upper bound factor of , providing zero formal privacy protection. Evaluating a DP method exclusively in a non-private regime () invalidates its core contribution as a privacy-preserving framework
-
Defining privacy at the single-window level while treating visual observations/states as public and randomly assigning ~50% of windows within the same trajectory as members is unrealistic.
-
Even on the non-private (NP) model, the proposed Action-Ref attack achieves a pooled AUROC of only (and on Object and Goal suites), which is barely above random guessing ().
-
Limited methodological novelty: DP-VLA-MC is a direct application of DP-AdamBC (Tang et al., 2024) with a variance floor, and Action-Ref is a standard difficulty-calibrated shadow attack (Watson et al., 2022).
Questions for authors
-
Why did you choose full-parameter fine-tuning (3.6B parameters) and ?
-
Since each test episode contains both member and non-member windows, how much does intra-episode interpolation degrade your MIA AUROC?
-
What is the Action-Ref AUROC if membership is split at the episode level rather than the window level?