- Rating
- 2
- Confidence
- 1
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies post-hoc auditing of multi-group fairness for black-box ML models. It proposes “bias probes” based on cross-group relational queries and further considers model-extraction resistance and auditing against a fairness-aware adversarial model owner.
Critical strengths
-
The topic is relevant, and the paper provides a fairly comprehensive theoretical treatment rather than only an empirical auditing method.
-
The paper attempts to connect efficient fairness estimation, interpretability and model confidentiality within one framework, and evaluates the method on several standard fairness datasets and classifiers.
Critical weaknesses
-
The core idea is to replace ordinary model queries with Cross-Group Queries, where the auditor submits one example from each protected group and receives only relational comparisons rather than the underlying predictions. While technically interesting, I am not fully convinced that this represents a sufficiently substantial advance over the existing active auditing/active learning literature for ICLR. The paper needs to have a much clearer explanation of what fundamentally becomes possible with bias probes that cannot be achieved by adapting existing active fairness auditing approaches.
-
The main experiments use COMPAS, German Credit and Student Performance, with linear models, random forests and MLPs. For claims concerning practical black-box auditing, model confidentiality and adversarial model owners, stronger evaluation on larger and more realistic modern ML systems would make the contribution more convincing.
Questions for authors
-
How would Cross-Group Queries be implemented in a realistic auditing scenario where the auditor has access only to a conventional prediction API?
-
Could the authors clarify more sharply the technical novelty relative to existing active fairness auditing and comparison-query/disagreement-based active learning?
-
How well does the approach scale to modern high-dimensional models, multiple or intersecting protected attributes, and larger numbers of groups?
2 comments
Follow up on our response
Dear Reviewer,
Thank you again for your constructive feedback on our work. We submitted our responses a few days ago and updated the manuscript. We wanted to kindly follow up, as the discussion period is coming to an end. We would very much appreciate it if you could revisit your evaluation in light of our responses, should they have addressed your concerns. If you have any further questions or concerns, please do not hesitate to let us know. We look forward to hearing your thoughts.
Kind regards,
Authors,
We appreciate the reviewer’s time and effort in carefully reading our paper and for providing constructive feedback. We respond to each point below and describe the clarifications and corrections made to the manuscript.
Implementation of Cross-Group Queries. Cross-Group Queries (CGQ) can be implemented through a middleware layer around a standard prediction API. A conventional API returns individual predictions, whereas our approach requires comparisons across protected groups. This functionality can be provided by an additional inference wrapper connected to the model owner's prediction API. Given a coupled set of samples, the wrapper queries the underlying model for each sample, computes the corresponding cross-group comparison, and returns only the comparison result to the auditor. Individual predictions remain internal to the wrapper and are not disclosed. The auditor can therefore interact with this wrapper through a dedicated CGQ endpoint, without access to the model under audit or its prediction API.
Technical novelty relative to existing literatture. Our work provides a general framework for actively auditing multi-group fairness with an arbitrary number of protected groups, while accounting for an adaptive adversarial model owner. The framework provides guarantees on audit accuracy and produces interpretable reports, even when the model owner attempts to conceal unfair behavior.A key distinction between our approach and existing fairness-auditing methods concerns the object of the auditor's learning task. While existing approaches typically assess discriminatory behavior through statistical parity between two groups, our formulation generalizes the problem to an arbitrary number of groups. Moreover, existing methods broadly follow two strategies: direct estimation of the fairness measure or model reconstruction followed by plug-in estimation. The former estimates group-dependent quantities from sampled queries, while the latter learns an approximation of the model to evaluate its fairness properties. A summary of these approaches are in Appendix B. Our approach is different: instead of estimating the fairness measure directly or reconstructing the entire model, the auditor learns a probe that captures the model's discriminative behavior across protected groups. At each round, the auditor requests a comparison across groups on a selected coupled sample. These responses progressively refine the probe space until the auditor identifies a probe that witnesses a significant group discrepancy. This approach has two main advantages. First, the learned probes provide interpretable audit report of the detected group discrepancies. Second, learning the relevant relative behavior can be substantially easier than reconstructing the underlying model, as established by our information-theoretic separation result. Overall, our framework provides a distinct approach to active multi-group fairness auditing and establishes a foundation for studying robustness against adaptive adversarial model owners. We have updated the contribution section clarifying this in the revised draft (line 107).
Scalability to high-dimensional models and groups sizes. The scalability of our framework depends primarily on the complexity of the probe class, rather than on the input dimension or the number of protected groups alone. First, input dimension does not directly enter the definition of multi-group fairness which is defined in terms of protected groups (the unique feature relevant to estimate multi-group fairness). However, our auditor also identifies regions of the input space that exhibit substantial unfairness, providing interpretable audit reports. The complexity of this search is captured by the statistical complexity of the probe class, through quantities such as the multi-star number and the Daniely-Shwartz dimension. Second, our framework does not require protected groups to be disjoint. It therefore also covers intersecting protected groups. The dependence on the number of groups k is appears in the multi-star number and the Daniely-Shwartz dimension. We provide a more detailed discussion in Appendix E.2 (paragraph beginning at line 1879). Finally, scalability to modern high-dimensional models depends on the complexity of their induced probe classes. These classes may be substantially more complex than those arising from the restricted linear models used in our illustrative separation result. Consequently, our theoretical guarantees remain applicable, but the resulting sample complexity depends on the structure of the induced probe class.