- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
SafeLens proposes a "fast-and-slow" cascading framework for video content moderation: S1 uses a lightweight probe + frame-level captioning for initial screening, and S2 uses a Chain-of-Thought VLM for in-depth reasoning on low-confidence samples. On the data side, the authors apply TracIn-based influence filtering to the SafeWatch public subset (roughly 6.5% of the original 2M), retaining only 48K samples (2.4% of the full set) and augmenting them with CoT traces. The backbone is Qwen3-VL-2B; CoT generation uses Qwen3.5-27B; captioning uses Florence-2-Large. On the SafeWatch-GenAI test set, the system reports 76.7% Avg ACC and 75.3% Macro F1, with inference latency of approximately 1.76s.
Critical strengths
- Clearly motivated problem: The tension between "uniform large-model inference is too expensive" and "simple classifiers are insufficiently accurate" is a genuine pain point in real deployment. A fast-and-slow cascade has direct engineering value.
- Data filtering angle is meaningful: Using influence functions to curate a high-quality 2.4% subset from noisy data, with an ablation in Appendix K showing filtered vs. unfiltered (73.9→76.7%), supports the "quality over quantity" argument.
- Efficiency-accuracy trade-off analysis is fairly complete: Threshold τ sweep (Figure 4), nine backbone combinations (Figure 5, Tables 9/10), runtime breakdown (Table 11), and FLOPs comparison (Figure 6) map out a deployable parameter space.
- Code and data release: An anonymous repository is available, and the filtered dataset is also released, which is favorable for reproducibility.
Critical weaknesses
Methodological credibility of influence filtering
-
Circularity risk: Influence analysis requires a backbone model to compute gradients (Eq. 1). The authors first fine-tune Qwen3-VL-2B on the noisy full training set to obtain Mθ_sw, then use this noise-contaminated model to determine which samples are "influential." If the backbone itself is biased by mislabeled examples, negatively influential samples may precisely be the correct examples that disagree with the erroneous labels. The paper does not validate this.
-
Validation set noise: The reference point for influence is SafeWatch-Real (644 samples). If this validation set itself has label noise (the authors themselves show noise examples from the SafeWatch training set in Appendix L, but do not characterize validation set quality), then the selection criterion of "positive average influence within class" rests on a noisy reference.
-
Threshold sensitivity: The two conditions in Algorithm 1 Line 4 (positive within-class influence, non-negative aggregate influence) are hard thresholds. The paper does not show how many samples were removed at each step, how the class distribution of the retained set shifts, or how final performance varies with threshold choices. From 130K public subset to 48K, the retention rate is roughly 37%, but is this number reproducible?
-
TracIn approximation limitations: Using gradient inner products of the last layer as TracIn is a very rough approximation. The authors acknowledge in Limitations that Hessian-based methods might be better, but provide no comparison quantifying the error introduced by this approximation.
Questions for authors
see as the above
4 comments
Reviewer v8tK - 2/2
W-3: Threshold sensitivity: The two conditions in Algorithm 1 Line 4 (positive within-class influence, non-negative aggregate influence) are hard thresholds. The paper does not show how many samples were removed at each step, how the class distribution of the retained set shifts, or how final performance varies with threshold choices. From 130K public subset to 48K, the retention rate is roughly 37%, but is this number reproducible?
WR-3: Thank you for the observation. In influence analysis, using a zero threshold to distinguish beneficial from detrimental samples is a standard practice in prior work [5, 6]. Therefore, we believe that varying the threshold is not necessary for our analysis. We clarify below.
- There is no tunable threshold. Both criteria use the sign of the influence score. In prior work, positive influence marks a beneficial sample and negative influence marks a detrimental sample [5,6]. Zero is therefore the natural decision boundary, and varying it would require an arbitrary hyperparameter with no principled meaning.
- Class-wise sample reduction. We provide the percentage of each class removed and each class's share of the unfiltered data:
| Class | % of Class Removed | Share of Unfiltered Data |
|---|---|---|
| Sexual | 67.8% | 0.35 |
| Abuse | 44.5% | 0.03 |
| Violence | 79.7% | 0.13 |
| Misinfo | 34.4% | 0.10 |
| Illegal | 26.8% | 0.02 |
| Extreme | 57.7% | 0.05 |
| Safe | 57.2% | 0.30 |
-
The filtered set is a deterministic function of the training data, the validation set, and the backbone. Code, filtered subset, datasets, and models are all publicly available, so the retention rate is reproducible.
-
We have added this table to the revised Appendix B.
W-4: TracIn approximation limitations: Using gradient inner products of the last layer as TracIn is a very rough approximation. The authors acknowledge in Limitations that Hessian-based methods might be better, but provide no comparison quantifying the error introduced by this approximation.
WR-4: Thank you for the observation. For large datasets, TracIn is a scalable approach to influence analysis that has been adopted in prior work and these studies demonstrate its effectiveness in their respective analyses [5, 6]. Our results provide further empirical support: the filtered data consistently outperforms the unfiltered data. We clarify below.
- Last-layer, Hessian-free TracIn is a scalable choice made in existing work [5, 6]. Hessian-based estimation for a 2B VLM over about 100K+ 644 training-validation pairs is impractical at this scale. This is the reason we use a Hessian-free method.
- The relevant measure is the quality of the selected data. Filtered data achieves +2.8 Avg. Acc. over unfiltered data and +3.7 Avg. Acc. over random selection of equal size and class distribution.
- These results show the approximation is sufficient to select high-quality data, and SafeLens outperforms all baselines with this setting.
- The influence method is a design choice that practitioners can adapt to their resources, and a Hessian-based method can be plugged into our pipeline without changes to the rest of SafeLens.
References:
- [1] Humane, P., et. al., Influence Functions for Efficient Data Selection in Reasoning. NeurIPS 2025 Workshop on Efficient Reasoning.
- [2] Kwon, Y., et. al., DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Models. ICLR 2024.
- [3] Yang, et. al., Dataset Pruning: Reducing Training Data by Examining Generalization Influence. ICLR 2023
- [4] Chen et al., SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanations, ICLR 2025; OpenReview.
- [5] Pruthi et al., Estimating Training Data Influence by Tracing Gradient Descent, NeurIPS 2020.
- [6] Vitel et. al., First is Not Really Better Than Last: Evaluating Layer Choice and Aggregation Strategies in Language Model Data Influence Estimation, ICLR 2026.
Dear Reviewer v8tK,
Thank you once again for your insightful review. With the deadline approaching in just one day, we would be happy to answer any remaining questions.
If our rebuttal has addressed your concerns, we would also kindly request for a reconsideration of the score.
Kind Regards,
Authors.
Dear Reviewer v8tK,
We would like to thank you once again for taking the time to review our work and share your valuable feedback. We are happy to answer any additional questions you may have.
As the rebuttal period is ending in a few hours, if our responses have addressed your main concerns, we would be grateful if you would reconsider your assessment.
Kind Regards,
Authors.
Reviewer v8tK - 1/2
Dear Reviewer v8tK,
Thank you for your time and effort in reviewing our paper. We appreciate your observations and provide clarifications for each point below.
W-1: Circularity risk: Influence analysis requires a backbone model to compute gradients (Eq. 1). The authors first fine-tune Qwen3-VL-2B on the noisy full training set to obtain Mθ_sw, then use this noise-contaminated model to determine which samples are "influential." If the backbone itself is biased by mislabeled examples, negatively influential samples may precisely be the correct examples that disagree with the erroneous labels. The paper does not validate this.
WR-1: Thank you for this observation. We provide empirical validation of this concern below.
W-2: Validation set noise: The reference point for influence is SafeWatch-Real (644 samples). If this validation set itself has label noise (the authors themselves show noise examples from the SafeWatch training set in Appendix L, but do not characterize validation set quality), then the selection criterion of "positive average influence within class" rests on a noisy reference.
WR-2: Thank you for raising this point. We characterize the validation set quality below.