acceptodds
Withdrawn from ICLR 2027

A Resource-Based Framework for AI Auditing: Access, Data, and Compute

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

AI audits have garnered signficant attention in recent months, as the ability to test and monitor AI models plays a crucial factor in safe development and deployment. However, AI audit success hinges on an auditor's capabilities; in particular, what information and resources they have at their disposal. Ongoing debates have surrounded the question: *How much model access is needed to conduct an AI audit?* Auditors seek more model access, while companies resist it for reasons related to confidential business information (trade secrets) and privacy. This tug-of-war remains unresolved. Further, this debate misses a related question: *How much data and compute do auditors need?* In this work, we systematically investigate both questions. We provide a framework that varies each of these three dimensions—model access, data, and compute—and measures their joint effect on audit success. We apply this framework to three tasks. We find that (1) some model access is important, but there are strong diminishing returns; (2) data and compute matter more for traditional fairness audits, but model access matters more for manipulation and lie detection; and (3) these three resources can be fungible, meaning that model access is not the only salient factor. These results suggest that audit governance should jointly characterize (access, data, compute) requirements in relation to audit objectives. Further, our framework serves as a template that can be applied to different audit tasks and inform the degree of auditor authority.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI, audits, framework, access

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_tBryOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper fits in the field of auditing remote ML or LLM models.

The authors propose an "access/data/compute" framework, where the interaction of all three means is observed in the success metrics of an audit. Three audit types are considered: demographic parity assessment, manipulation detection, and lie detection for LLMs.

This paper is empirical only (no theory).

The concluding claims made are about the need to consider all three means at all times when a novel audit task instanciation arrives.

Critical strengths
  • The overall topic of understanding the conditions under which audits succeed or not is of interest. This paper instantiates such a question with a particular (yet imperfect, in my opinion) focus
Critical weaknesses
  • Artificial framework: Related work on audits places data (here d) as a primary citizen. The current paper adds access and compute as a clear value proposition. Yet I claim that, in the current paper at least, these two additions are artificial. First, the "compute" aspect is in my opinion irrelevant: auditors today are state institutions or tasked private companies with considerable means to perform their audits, so studying the "compute" aspect by solely considering that an auditor can have access to even hundreds of thousands of parameters only for computing is not realistic. The paper claims to be practical, so I think here it is not: this is far from what is occurring even today in practice. Second, the "access" mean is also artificial: I am obviously ok with l=0 and l=1 access. But when l>=2, this means that the model provider will open "part" of her model, which is not realistic. As mentioned by the authors, they keep their model as a trade secret, so this is an all-or-nothing openness that is witnessed in practice. Institutions might force full access to a model (by law, for instance), but not to "several layers". This means that I am fine with the study of the "data" dimension, but we are then back to what most of the related work already does; and because the paper claims practical impact, artificial core assumptions break the whole realism of the proposed framework.

  • There is no theoretical lead-in in this paper. So we are left with a pure numerical assessment of some chosen cases. The three case studies are interesting, and the bibliography is relevant; yet, the reader is then directly forced into an overly technical/practical choice of a totally arbitrary parameter set to instantiate them (Table 1). Then tiny paragraphs very briefly describe some observations. So what about the generalization of these experiments to other parameters, audited models, or tasks? I am left with the impression that, besides the actual numbers presented, I cannot say anything about the stability of these results to any change to any arbitrary metric/parameter/task/target model presented in the paper, and then no knowledge gains beyond a point in the landscape of potential audits.

  • See below for a questionable mapping of related work's use of hypothesis classes (generally on the auditee side) to a mere number of parameters in the auditor's model (auditor side then). In general, I find the related work part to be good, but the choices made to represent their leads in the current framework are far below and totally arbitrary. For instance, for the lie detection task, page 5: "We teacher-force each transcript through its target to extract output probabilities and hidden states over the reply; reply text at ℓ = 0 is encoded as a random projection of GPT-2 token counts (k = 128), the log probs at ℓ = 1 are represented by an 8-dimensional summary, and hidden layer activations at ℓ ≥ 2 are projected to 256 dimensions." This is so complex and arbitrary that I find it impossible to link it to the precise knowledge and conclusions from related work.

details:

  • Some claims such as (Section 6): "Second, access to hidden states does not seem to help in out-of-distribution performance, plausibly because the extra access lets probes specialize on the deception they were trained on" are misleading, as they are talking about what the auditor does with its arbitrary choice of "compute" mean, not about the hardness of the task itself. Overfitting is a problem local to the auditor, not an inherent impossibility to address the audit task. The distinction between both is unclear in the paper.

  • References are misformatted: no mention of the conference in most of them (AI statement claims search for biblio, so I guess this must be fixed next time)

Questions for authors
  • Do you have practical examples of companies giving to an auditor access to layers 2, 3, ..., k, but not the whole model?

  • How do you go from (page 3) from hypothesis classes being central to audits, to flattening all to a "compute" that only considers the size of an auditor's model? I do not see such a straight parallel; hypothesis classes are much more abstract and powerful for what they allow in reasoning about bounds and possibilities of audits in general.

Reviewer_RudtOpenReview ↗
Rating
1
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that an AI audit's success depends not only on how much model access the auditor has, but also on data (the number of queries) and compute (the capacity of the auditor's probe). It proposes a reusable framework that sweeps all three resources together and traces a performance surface, showing which resources can substitute for which.

Critical strengths

It adds a useful new angle to a policy debate that matters. The debate over auditor access has mostly treated access as the only variable, set against firms' concerns about trade secrets and privacy.

Critical weaknesses
  1. The experimental settings are too small to support the governance conclusions, and the compute axis does not mean the same thing across them. The two feed-forward case studies audit tiny MLPs trained on synthetic 5-dimensional data.

  2. Several headline findings follow largely from how the tasks are built, and fungibility is never quantified. The DP gap is defined entirely by the model's outputs over the input distribution. It is therefore unsurprising that internal access helps little and more queries help a lot.

Questions for authors
  1. How does the probe compare with a simple plug-in estimator for the DP gap?

  2. Can you quantify fungibility?

  3. Do the access-level rankings survive other featurizations?

  4. Do the compute conclusions hold when the probe is not much larger than the target?

Reviewer_JWQ3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper asks how much an auditor needs of three resources (model access, query data, and “compute”) to audit a model well, rather than asking only about access. It sweeps a grid over (access level, number of queries n, probe parameter count p), fits a probe at each cell, and reads off performance surfaces and level sets. It applies this to three tasks: estimating a classifier’s demographic parity gap (1,650 small synthetic MLPs), detecting input substitution “fairwashing” (1,500 small MLPs), and lie detection on four open-weight LLMs using Liars’ Bench.

Critical strengths
  • Reframing the access debate to include data and auditor capacity is a reasonable and policy-relevant question, and the three tasks were chosen to have genuinely different structure (output-level estimand, consistency check, latent property).
  • The seed protocol separates exogenous variation (target models, splits, query draws) from probe optimization noise, which is more careful than typical probing papers.
Critical weaknesses
  • Per C.1, each transcript is teacher-forced “including the stored system prompt,” and reply-token states attend to it. The paper says the auditor does not see the system prompt, but the activations encode it. Since nearly all lies come from “deceive” prompts and nearly all honest replies from neutral prompts, the probe may be detecting the instruction, not the lie. This alone could explain the jump to ~0.99 at the first hidden layer and the poor transfer to settings with different prompt structure. Needed controls: report AUROC restricted to deceive-prompt transcripts (lie vs. ignored-instruction honest), and rerun extraction with the system prompt removed.
  • Level 0 for lie detection is a random projection of GPT-2 bag-of-token counts into 128 dims. The in-distribution task is “does the reply call a known true/false fact the wrong way,” which a capable LLM judge (as used in Liars’ Bench) could likely solve with zero labeled data. The headline “access to a hidden layer is crucial” is really “access beats a bag-of-words probe.” Same issue for DP: the obvious estimator is the plug-in group difference of ŷ over the n queries, which is unbiased here because labels and queries come from the same p_test. Without it, we cannot tell whether the “compute” axis is just the probe learning to compute a mean.
  • My understanding is that p is probe parameter count (500 to 200K). The real auditor costs are queries and forward passes through a 70B model, plus labeled data. Policy conclusions about compute fungibility do not follow. The logistic regression reference in Table 17 matches the best MLP almost everywhere, which suggests the compute axis is close to flat for lie detection and much of the reported “compute helps modestly” is noise or tuning.
Questions for authors

Covered in the weakness section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.