- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper explores refusal filter stacks in safety classifiers. The core observation is that redundancy makes individual ablations uninformative about component filter contributions. Several approaches are considered for improved stack selection, including iterative removal using greedy backward elimination.
Critical strengths
- The paper focuses on a practical problem affecting real deployments.
- The proposed methods improve performance of the stack over the baseline by a considerable amount.
Critical weaknesses
- The conceptual novelty appears limited. The central observation that redundant veto components can have zero leave-one-out effect despite contributing in other configurations is a familiar consequence of redundancy, as the authors acknowledge. The empirical characterization is useful, but I am not convinced that the findings on this deliberately untuned stack establish a sufficiently general or consequential contribution for ICLR.
- The abstract is overloaded with numerical results, notation, and qualifications, which obscure the central question and takeaway. Clarity would improve substantially if it prioritized the problem, the experimental approach, and the main finding, leaving most detailed statistics and secondary results to the main text.
Questions for authors
See weaknesses