- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a method for certifying the robustness of classifiers for structured event-based data, such as those obtained from event cameras, against group deletion. Specifically, the authors assume that the input tensor is divided into groups and use a Randomized Smoothing framework extended to group deletion to derive a worst-case certified radius against group deletions. Also, by using the fact that deleting inactive groups containing no events does not change the input, the authors extend the worst-case certificate to probabilistic guarantees against random loss, bursty loss, and hybrid threats combining adversarial deletion and random loss. The authors also show that the proposed method gives larger certified radii compared with existing methods on five benchmarks.
Critical strengths
- The paper proposes an RS-based certification framework that accounts for group-level deletion in structured data, such as data obtained from event cameras. The framework also provides guarantees against random and bursty losses that may occur in practical systems.
Critical weaknesses
- Several extensions of RS that account for data structures, such as graphs, point clouds, image patches, and sequences, have already been proposed. However, the novelty of the proposed method relative to these existing approaches is not sufficiently clear. Experimental comparisons with related structured RS methods are also absent.
- The accuracy–robustness trade-off controlled by p_keep is not sufficiently investigated. Although Figure 1 shows how certified accuracy varies with p_keep, the relationship between smoothed classification accuracy and certified radius is not directly evaluated.
Questions for authors
- What is the specific theoretical novelty of the proposed method compared with existing structured RS frameworks, such as Hierarchical Randomized Smoothing (Scholten et al., 2023)? In particular, could the worst-case and physical-loss certificates presented in this paper be derived by applying existing frameworks to SST groups? If not, please clarify what prevents such a derivation. Would it also be possible to include theoretical or experimental comparisons with these closely related methods?
- Could the authors directly examine the relationship between smoothed classification accuracy and average certified radius (ACR) as p_keep varies across the datasets? In particular, it would be helpful to clarify where the current choice of p_keep, selected by maximizing ACR, lies in the accuracy–robustness trade-off.
5 comments
Thank you for your response and the additional results addressing my previous concerns. The newly provided results of accuracy–robustness trade-off resolve most of my concerns about the point. However, as I still have concerns about the novelty of the proposed method meets the high standards of ICLR, I will maintain my original score for now.
Clarification Requested on Remaining Novelty Concerns
I understand your decision. However, could you please clarify your remaining concerns regarding novelty? You mentioned the "high standards of ICLR." Could you specify which novelty criteria our work fails to meet?
As explained in our rebuttal, your previous novelty concerns have been directly addressed, and we believe the novelty of our contributions is technically justified. Novelty should be assessed based on concrete technical arguments and evidence, rather than a subjective perception of what constitutes a "high standard." We would appreciate specific technical reasons for maintaining this concern.
Novelty vs. structured RS, and the full accuracy–ACR trade-off
Thank you for the clear summary. Both questions are answered below, including the full trade-off data you asked for.
Weakness 1 and Question 1: How is EventCert new compared with structured RS such as HRS? Can existing frameworks derive its certificates?
The worst-case certificate, yes. The paper does not claim it as new (l. 59–68). For Bernoulli group deletion, the likelihood ratio between the smoothed DS(X) and X lies in 0,qk−1 with qk=(1−pkeep)k. The Neyman–Pearson lemma then gives exactly Eq. 3, so no structured RS framework using this noise on SST groups can certify more. HRS certifies ℓp-bounded changes to at most r entities. Deleting a group is an ℓ2 change of 2.8–15.6 (Table 22), far above the ≈0.65 that ℓ2 smoothing certifies. That is why the ℓ2 route certifies 0 groups (Table 3).
The physical-loss certificates, no. Every structured RS method certifies a fixed set Ak(X). EventCert certifies a probability over a random loss process: PrS∼Π[g(DS(X))=A^]≤PrS∼Π[H>kwc(X)],H=∣S∩N(X)∣. This needs three ingredients that those frameworks do not have:
These certificates show that independent-loss guarantees overstate safety. On N-MNIST at 5% loss, certified accuracy is 85.6% under independent loss but only 14.5% under bursts (Table 5).
Table 3 already compares three RS baselines (randomized ablation on our groups, RS-Del-style deletion, Gaussian) with the same models and budgets. EventCert certifies 3.8–5.6× more. At pkeep=0.1, ablation's ceiling is 6 groups and ours is 45.
Weakness 2 and Question 2: Smoothed accuracy vs. ACR across pkeep.
From our existing logs (seeds 0–2, certified at every pkeep). An asterisk marks the selected value, which for SHD is 0.2 or 0.3 depending on the seed.
We will add this table to Appendix D.
Summary
We hope this resolves both questions, and we respectfully ask you to reconsider your score.
AI disclosure: The language has been polished and improved by AI. All analysis and computational work were done by the authors.