- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper is an application of double machine learning (DML) [Chernozhukov et al., 2024] to AI Assurance. AI Assurance is the process of auditing a AI to test its compliance with a set of policies. More precisely, the article deals with content moderation with simulations using Jigsaw dataset.
The paper contains interesting data transform to augment the Jigsaw dataset with synthetic binary topic variable, to test in particular how DML can be used to detect content moderation manipulation under varying levels of confounding selection rate ratio skew.
Among other results, the authors show that even using a part of the available features, DML can improve the precision of the estimation.
They also claim that DML can be applied for other wilder scenarios (no doubt about that).
Overall Assessment
This paper presents an elegant idea on an important problem, and I would be glad to raise my score towards acceptance if my concerns are addressed. Reproducibility (W2) is the decisive point for me: without the code and further experimental details, I would have to lower my recommendation. The related work (W1) could be improved, but I do not consider it a deal-breaker.
Critical strengths
S1. Clarity and presentation. I would like to thank the authors for the quality and clarity of their writing. Reading this paper is a real pleasure. The introduction sets up the problem very effectively, and the overall flow is excellent: whenever a question came to mind, the following paragraph answered it.
S2. Timeliness and originality. The research is timely. While surrogate models have been used for AI assurance before, this is the first time I have seen the problem approached through the lens of Double Machine Learning (DML). The idea is simple, clever, and appears to work well, which makes it a valuable contribution to the field.
Critical weaknesses
W1. Related work and choice of baselines. The related work section would benefit from a broader coverage of the (black-box) auditing literature, in particular recent work such as:
[1] Robust ML Auditing Using Prior Knowledge (ICML 2025). This paper proposes a general black-box auditing framework and highlights the importance of prior knowledge. It would be valuable to discuss how the role and accuracy of such prior knowledge relate to the subsets of feature dimensions (25%, 50%, 75%, 100%) considered in the submitted paper. I am not asking for an experimental comparison, but rather for a discussion that positions this work within the existing literature.
[2] Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation (AIES 2018). This is one of the earlier works using a surrogate model to audit a model. Although its setting differs, it should be acknowledged in Section 2 (again, as a discussion rather than an experimental comparison).
[3] The DSA Transparency Database: Auditing Self-Reported Moderation Actions by Social Media (Proc. ACM HCI, 2025). This work seems particularly relevant and should be discussed (see Q2).
In addition, the baselines are not described, and the paper does not explain why they are relevant for the comparison (see Q4). Finally, I would encourage the authors to connect the detection of manipulation under confounding skew to the literature on fairwashing, which addresses a closely related concern.
W2. Reproducibility. My main concern is the reproducibility of the work. I acknowledge the reproducibility statement at the end of the paper, but in its current form it is insufficient. Since DML is an existing method, a central contribution of the paper lies in its application to AI assurance and in the accompanying experimental evaluation. In that context, sharing the code is essential for the community to verify and build upon the results. Moreover, the data augmentation procedure described in Sections 4.1 and 4.2 should be explained and illustrated in more detail, for example with pseudo-code, a worked example, the intuition behind each step, key takeaways, and plots of the resulting data distributions. Such material is typically provided in the appendix alongside the code.
Questions for authors
Clarification requests
-
Q1. Is the DML coefficient reported in Section 4 the average treatment effect discussed in Section 3?
-
Q2. What are the links and implications of your work with respect to [3]? For instance, could DML be used to verify the information reported in the DSA Transparency Database? Conversely, could the information in that database be used to improve the DML-based approach?
-
Q3. Could you clarify the role of Kapusta’s framework in your approach?
-
Q4. Could you explain how the baselines were chosen? A short description of each baseline and the reasons for selecting them would help. This also relates to the reproducibility of the comparison (W2).
-
Q5. Could the data transformations described in l. 212–215 and l. 258–261 be used in other contexts beyond this paper? Do you consider them a contribution of the paper? Is there a more standard approach in the literature? Please provide more details, including at least one illustrative example and the code.
Suggestions for improvement
-
If space allows, please elaborate on the statement in l.159–161. This point is very important, as it reflects a major limitation shared by much of the black-box auditing literature.
-
Related to Q4 and Q5: please provide additional information to improve reproducibility, including examples illustrating the data transformations. It would also be very helpful to report the raw values of the metrics before thresholding (rather than only summaries such as “4/5 fails”), and to release the code.
-
Please consider adding a short conclusion or a list of key takeaways at the end of the paper.
Minor remark
Please use a consistent format for percentages and proportions throughout the paper (e.g., “1” vs. “100%”, “94.7” vs. “0.947”).
2 comments
I thank the authors for engaging the discussion. As the discussion is ending really soon, I'm not able to read again the paper in the mean time or dig for additional answers.
I'll take everything into account during the internal discussion to give my final score.
Updated paper for better reproducibility and attached code for data generation
Thank you very much for your suggestions and comments. We have updated the paper in response, focusing specifically on reproducibility and related work. We also attached code for the data generation process. Please see details below.
W2. Reproducibility.
*We updated the paper to lay out the data generation process in Appendix A, lines 756--881. We include (A.1) an algorithmic description, (A.2) a pedagogical example of a toy dataset that walks through each step, along with visuals to help support understanding, and (A.3) a brief derivation of the calculation involved to achieve the toxicity rate ratio. We also attached the code for a python implementation as supplemental material.
*We updated the paper with descriptions of the baselines including our motivation for including them. See Section 4.2 lines 252—264.
*We attached code for data generation, but releasing the code repository is difficult since it's treated as highly restricted in our organization. We are working on a way to get it released. In the mean time we updated the paper with your reproducibility suggestions. As an anecdote, much of our code and findings have been reproduced internally on private data. We are confident in the reproducibility of our results and will try to get the code released to make it easier for others to reproduce and build upon it. *We added Appendix C to describe the content moderation classifiers and hyperparams more clearly at the request of another reviewer.
W1. Related work.
Thanks for these suggestions on related work. We agree with all them. We updated the related work (Section 2) to describe the relation to black box auditing and designed the discussion in part around your questions about how ideas from specific papers might interact with our work (Lines 99-116). We also updated the fairness paragraph to include fairwashing (Lines 130-136).
Clarification requests:
Q1. Correct. We updated the paper changing the first few occurrences of coefficient to "DML ATE estimate" and use "DML estimate" in all the proceeding ones for consistency with rest of paper.
Q2. This is a great question. We updated the related work to include a discussion about our work and DSA. We believe that DSA can help us form hypotheses, understand the data better in order to come up with more comprehensive nuisance features, and also help do root cause analysis. See Lines 107--116 in Section 2 for more detail.
Q3. More specifically their idea of an assurance claim aligns with ours as a way to turn a nebulous AI assurance problem into a falsifiable scientific hypothesis for which we can gather evidence. However, our view of AI assurance is not quite as broad as theirs, as we consider mitigation a separate but related problem.
Q4. Yes --- we updated the paper with a new section describing and motivating the baselines (See Section 4.2 lines 252—264).
Q5. We updated the paper with an appendix that includes the description of the data generating process and an illustrative example. We hadn't thought about it as a main contribution of the paper, but perhaps an ancillary one to help answer our research questions. But perhaps it could be used in other contexts. Any time you want to generate a binary variable that is not completely disconnected from the text, drawing a random hyperplanes seems like a reasonable thing to do, and certainly better than flipping a coin. There's possibly an interesting connection to simhash here, in that enough random topics might eventually uncover the cosine similarity between embeddings of content pairs (i.e, number of topics in common means higher cosine similarity). This gives some weight to the idea that our topics are semantically grounded in the text even if they are not directly interpretable. As for skew, we reassign points closest to the hyper plane to achieve skew but over/under sampling is another thing that is done for example, in the distribution-shift literature to rebalance data.
Suggestions for improvement:
"If space allows..." There's a lot to say here but based on real world experience we often have a strong prior about how a black box ML model should work for a particular task and what features it ought to use. We can also use whatever resources, if any, we have available (e.g., people, documentation, policy documents) to peer a little deeper into the box. From here, we can test how faithful our nuisance models are to the black box model on the data and iterate as needed.
"Related to Q4 and Q5...." We updated the paper to address both Q4 and Q5.
We updated the paper with a conclusion.
Regarding percent/proportions we made several consistency edits and will continue to go through the paper.