- Rating
- 4
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The authors consider a setting where an attacker sees batches of data being fed into a sequential learning algorithm and perturbs the data so as to drive the learned parameters toward the attacker's target. The attacker has full knowledge of the non-adaptive attacker is able to duplicate and remove points. After formalizing the scenario as an MDP, the authors present an efficient attack and analyze it both empirically and theoretically.
Critical strengths
- The paper is very clearly written and easy to follow (although it would benefit from having a notation table).
- The threat model is restricted enough to be mathematically friendly to work with while also being realistic enough to be applicable.
- The theoretical analysis is both elegant and useful.
Overall it's a nice paper that performs a standard-methods attack for an interesting and novel setting. While the method alone is probably not efficient or powerful enough to immediately applicable in real-world settings, the manuscript lays the groundwork for future work (including theoretical and empirical analysis, as well as addressing the engineering challenges of deploying the solution and further settings such as an adaptive defender).
Critical weaknesses
- There is a bit of a bait-and-switch. The first line of the abstract motivates the topic using LLMs, but then experiments are all on linear regression and logistic regression (on as high as 30-dimensional data).
- The experiments are lacking. Logistic and linear regression are the simplest of learners, and only extremely simple datasets, none of which are from canonically sequential domains are evaluated.
- The method seems slow. The authors highlight the computational difficulty of solving the combinatorial optimization problem and describe how they address this challenge in section 3.4. However, given that even MNIST was PCA'd down to d=30, I'm guessing the method is too slow for even moderately sized data.
Questions for authors
- Am I right in that it's too slow to run on MNIST in its native dimension?
- What's the limitation holding you back from running on e.g., MLPs or other (non-linear) learners?
- In the third bullet point of the listed main contributions, you say "up to function-approximation and surrogate errors." What does that mean?
3 comments
I thank the authors for their response. I maintain my (high) assessment of the work.
Thank you!
We appreciate your encouraging evaluation of this exciting work!
We appreciate the reviewer’s thoughtful suggestions. We have adjusted our manuscript and experiments to show that our approach is efficient and powerful enough for higher-dimensional datasets and complex learning models of the victim.
(W1) Misleading abstract opening
We thank the reviewer for pointing this out. We agree that the abstract’s opening can cause a misleading impression that our method is evaluating an enormous-scale learner like an LLM. We have removed the confusing opening and clarified the scope of our study in the revised abstract.
(W3, Q1) Runtime on native-dimension MNIST
Our use of 30-dimensional PCA features is consistent with the experimental setup of Zhang, Zhu, and Lessard (2020) and Margiotta et al. (2023). To validate the runtime in a higher dimension, we ran the experiments on the full-dimensional MNIST and included the result in Section 5 with the full-dimensional experiment. The new result has shown that the FEVI incurs the lowest distance to target and the validation cost, while using the least amount of training time.
(W2, Q2) Extension to MLP
We also realize the value of providing more validation on the MLP-based victim models. The same concern was also raised by the other two reviewers. Therefore, we conducted the following experiments on the MLP victim model. We implemented a small MLP victim with a single 8-node hidden layer and a tanh activation layer for a regression task on a noisy sinusoidal wave. The result shown in the revised manuscript highlights how our proposed approach can handle a more complex learning model that the victim deploys. Compared with the model-free RL baselines, the FEVI achieves the lowest cumulative test cost by a slight margin. It consistently achieves the lowest parameter distance to the target. Our approach also attains the lowest validation cost throughout all the training episodes.
(Q3) Ambiguous wording in the main contribution
Thank you for catching this slip in our draft. When we said “up to”, we meant “derived from”. Our finite-sample bound on the optimality gap is derived from the function-approximation and surrogate errors, which results in O(n−0.5) decay rate. We realized the confusion in the wording and, thus, we have updated the third bullet point in the main contribution to the following
...derived from the function-approximation and surrogate errors. Our analysis also reveals how the problem specification affects the upper bound of the surrogate approximation error.