- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies label-preserving data modifications that induce a trigger response under a designated training order while remaining inactive under random orders. It proposes a trajectory-dependent construction and evaluates the effect across image, audio, and graph tasks.
Critical strengths
The order-dependent poisoning objective is interesting and highlights a training-process risk that dataset checksums alone cannot capture.
The experiments cover several modalities, and the appendices openly report unsuccessful settings and accuracy costs. These disclosures help clarify the phenomenon’s scope.
Critical weaknesses
First, the central approximation is insufficiently justified. Appendix A.1 drops $ (v^\top H_j y(t))$ on the grounds that it cancels across orders. However, the transverse trajectory depends on the order, so this cancellation does not generally follow. The omitted term could contribute at the same order as the curvature effect being analyzed.
Next, the normal behavior under random orders is not fully established. Table 1 reports random-order trigger sensitivity but omits its clean accuracy. Near-zero sensitivity alone does not establish equivalence to clean-data training. Moreover, the 64-epoch run has a residual ratio of 6.20 and random-order sensitivity of 0.057, so it does not demonstrate persistence together with the single-pass stealth properties.
Then, I find that Appendix B reports an unsuccessful public ResNet-20 setting before moving to custom CNNs. Accuracy costs are also substantial in some settings: on Speech Commands, the designated run is 19.9 points worse than clean-data fine-tuning.
Overall, I lean toward weak rejection. The conditional poisoning effect is interesting, but the theoretical explanation and the claim of normal random-order behavior need stronger support
Questions for authors
What assumptions justify cancelling across orders? Can you bound its contribution or measure it against the retained curvature term along the actual trajectories?
Could you report random-order clean accuracy alongside matched clean-data runs, with uncertainty estimates? Is there a multi-epoch setting that simultaneously preserves task performance, low random-order sensitivity, and a residual ratio below one?
Does the effect hold on a standard pretrained architecture under fine-tuning settings that preserve clean accuracy? For the unsuccessful ResNet-20 case, is the proposed relaxation-length explanation supported by measurements?
Which split was used to select the working points from the 273 CIFAR-10 and 223 SVHN configurations? Were the selected configurations then evaluated on untouched data and fresh random orders?
6 comments
W2, part1 and Q2, part1: behavior under random orders
Thank you for pointing out this issue. Table 1 gave the base accuracy and two accuracy drops and left the random-order run out. The revised Table 1 (p. 5) reports the base accuracy and the three accuracies after fine-tuning, so the comparison you ask for can be read off directly.
W2 part2 and Q2, part2: overall: task performance, low random-order sensitivity, and a residual ratio
Yes, we frequently encounter runs that meet the criteria. For example, on SVHN, With the designated order repeated in each of four passes and Adam at a learning rate of , modifying batches, the three quantities hold together: the random-order sensitivity is , the designated sensitivity is , and the residual ratio is .
The -epoch CIFAR-10 run you cite is reported for a different purpose, which we should have separated more clearly: it shows that the accuracy cost of the designated run falls as the number of passes grows, from points at one pass to points at . You are right that it does not demonstrate the single-pass stealth properties at the same time. Its residual ratio is because the construction of those runs does not bound the total gradient at , and its random-order sensitivity is because the modified data are seen times under any order. This row is about cost and not about stealth, and we point to the SVHN row above for a multi-epoch setting that meets all three conditions.
W1 and Q1: orthogonal term
The transverse term. You are right on both points. The transverse part y(t) of the displacement is determined by the order, so v⊤Hjy(t) does not cancel between two orders. We have corrected the discussion in Appendix A.1.
We perform measurements along the actual trajectories as asked. The specific settings and results are as follows.
Setting.
From w0 we run the designated order and four random orders. At every step we take the batch presented at that step and compute one Hessian-vector product hj=Hjv, where Hj is the Hessian of that batch's loss at w0; the trajectory quantities Z(t) and y(t) are read from the actual weights w(t). Summing the exact one-step update along each trajectory gives
Z(m)=P+C+T+S,
where P=−η∑tbσ(t) is the push, C=−η∑tZ(t)v⊤Hσ(t)v is the retained curvature term, T=−η∑ty(t)⊤hσ(t) is the dropped transverse term, and S is the remainder of second and higher order. P depends only on the set of batches and is the same for all orders.
C and T are the two halves of the same first-order term v⊤Hσ(t)u(t), split only because the recursion tracks Z; together they pull Z back by 3.021 in the designated order and by 3.242 to 3.327 in the four random orders.
Changes to the paper. All changes are marked in blue in the revised PDF.