- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Δ-Audits, an adaptive performance monitoring framework designed to detect harmful model updates after an initial white-box audit. The framework combines an initial white-box inspection phase with a subsequent query-limited black-box monitoring phase, allowing auditors to assess whether deployed models have deviated from their previously audited versions. The authors introduce Twister, a bi-level optimization algorithm that constructs an optimized monitoring query distribution by anticipating a rational model provider's utility-maximizing behavior under monitoring constraints. By strategically assigning query probabilities to audit samples, the proposed method aims to detect harmful model updates efficiently while reducing the incentives for providers to manipulate audited models. Experiments show that Twister achieves higher detection power with fewer queries than uniform and performance-based monitoring, while substantially reducing manipulation gains under the considered adversarial setting.
Critical strengths
- The paper addresses the problem of maintaining audit validity after model deployment, which is an important research direction for ensuring the continuous trustworthiness of ML systems.
- The proposed Twister algorithm introduces a bi-level optimization formulation that explicitly considers the conflicting objectives of auditors and model providers.
- The paper is generally well-written.
Critical weaknesses
- The problem of continuously monitoring deployed models through black-box queries has been explored in previous studies, such as [1]. This work has already investigated statistical approaches for detecting behavioral changes or monitoring model fairness. Although the proposed Twister algorithm introduces bi-level optimization to account for an adaptive provider's utility-maximizing behavior, the paper does not sufficiently establish its advantages over existing monitoring approaches. The experimental evaluation primarily compares against uniform sampling and a standard performance-based test, without incorporating stronger monitoring or active query-selection baselines. The authors should provide a clearer discussion of the methodological differences from prior work and conduct more comprehensive comparisons to substantiate the claimed contributions.
- The experiments are restricted to two benchmark datasets, Folktables and CelebA, using a linear model and a simple CNN, respectively. These relatively simple architectures may not adequately represent the complexity of modern deployed ML systems. In particular, the proposed Twister algorithm relies on bi-level optimization to simulate the provider's response, raising concerns about its computational scalability to larger and more complex models. Although the paper reports computational costs for the evaluated models, these results do not establish how the method scales with model size or dataset size. The authors should consider evaluations on more modern architectures and provide additional scalability analysis to demonstrate the generalizability and practical applicability of the proposed framework.
- Although Twister demonstrates improved detection efficiency for harmful model updates, the results in Figure 2 indicate that the proposed method may also produce relatively high false alarm rates for beneficial updates. For example, in the Lending scenario, the average rejection probability reaches approximately 0.26 for updates that improve auditor utility by 0.13, with some model pairs exhibiting rejection probabilities above 0.6. This is concerning because the proposed framework triggers a new full white-box inspection whenever an alarm is raised, potentially introducing substantial unnecessary auditing costs. Moreover, although the framework is intended for periodic monitoring, the experiments primarily evaluate individual monitoring runs without investigating the cumulative false alarm probability or auditing overhead across repeated monitoring rounds. The authors should provide a more systematic analysis of the trade-off between detecting harmful updates and avoiding false alarms, particularly under long-term deployment scenarios.
- The manuscript contains several unfinished placeholders. For example, Appendix A (Extended Related Work) contains the placeholder text "Common techniques include blah bla blaw." . Additionally, Section 4 contains an unresolved citation placeholder, "CITE[celeba]", in the description of the CelebA dataset. The authors should carefully revise the manuscript and resolve these issues.
[1] Richter, Leo, et al. "An auditing test to detect behavioral shift in language models." International Conference on Learning Representations. Vol. 2025. 2025.
Questions for authors
See Weaknesses above.
3 comments
Thank you for the response.
While some concerns have been addressed, the scalability and long-term monitoring overhead remain insufficiently evaluated. Additionally, the presence of unfinished placeholders in the original submission raises concerns about the manuscript's readiness. Therefore, I will maintain my original score.
Dear Reviewer z97c,
Thank you for your fast response and for acknowledging some of your concerns were addressed. On the scalability concern you had, as we mention in the common response, we added a moderation scenario and demonstrated that our method can be applied to LLM scale. Is there a specific model/scenario you would like to see in the paper?
Best regards, Paper authors
Dear Reviewer z97c, our common answers address your concerns about the number of scenarios and computational scale, the corrected Lending false-alarm results, and the unfinished placeholders.
Relation to change detection literature Our contribution is inspection-time query-distribution design that anticipates a utility-maximizing provider's response. [RHMK24] (which we also cite) is close to our work in that it aims at detecting changes that affect a score and not just any model change. However, motivated by a regulation scenario, we add an additional assumption: the knowledge of the provider training objective to inform which points to select.
Relation with active testing literature Similar to us, active testing aims at selecting the most "informative" queries to estimate or test some metric. However, active testing setups either assume full access to the model, which we don't assume after the audit or only black-box queries. Moreover, [GLMT+24] showed that in adversarial settings, no active testing strategy can perform better than uniformly random queries, impossibility that we bypass by adding a monitoring phase after the query selection.
Continuous monitoring We choose to expose our method (optimize the test distribution) on a simple binomial test to ease exposition.
Twistercan straightforwardly be extended to the testing-by-betting framework of [RHMK24].References
[GLMT+24] Augustin Godinot, Erwan Le Merrer, Gilles Trédan, Camilla Penzo, and François Taïani. Under manipulations, are some AI models harder to audit? In 2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 644–664, April 2024.
[RHMK24] Leo Richter, Xuanli He, Pasquale Minervini, and Matt Kusner. An auditing test to detect behavioral shift in language models. In The Thirteenth International Conference on Learning Representations, October 2024.