acceptodds
Desk rejected at ICLR 2027

Controlling the Probability of Unknown Unknowns in Pre-Deployment Testing of AI Systems

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

In pre-deployment testing of AI systems such as AI agents, a developer can often deal with *seen* failure modes — leaking API keys, deleting important databases, or installing a malicious package — by implementing fixes or making contingency plans for when the same issue might occur again. What is more challenging are the *unknown unknowns:* the *unseen* failure modes that haven't surfaced during testing. The decision to stop testing and deploy therefore faces an inherent tradeoff: continue testing to observe more behaviors, at the cost of time and resources, or stop testing and deploy, at the risk of failure modes remaining undiscovered. How then does the developer decide when they have tested enough? To answer this, we develop an algorithm that certifies when the *probability of an unseen failure mode* is sufficiently small. The algorithm constructs an anytime-valid upper confidence sequence on this quantity, so the developer can repeatedly test a system, inspect the bound, and decide to stop when it falls below a chosen threshold without invalidating the statistical guarantee. Our problem can be interpreted as a sequential variant of the missing-mass estimation problem which gave rise to the classic Good-Turing estimator (Good, 1953). On SWE-Chat, a dataset of real-world coding agent traces (Baumann et al., 2026), we implement our procedure and show that it achieves tighter upper bounds than a simple multiple-testing corrected Good-Turing baseline, and even dominates a fixed-horizon (non-multiple-testing-corrected) Good-Turing upper confidence bound, which is not anytime valid. Additionally, to better match realistic evaluation pipelines, we extend our method to handle sampling from a different task distribution and imperfect automated failure labeling.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI evaluation, AI safety, sequential inference, missing mass estimation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_S7kMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is focused on "when to stop testing an AI system" by estimating the probability of upper bound of meeting an unseen failure. It proposes a mixture of restarted betting processes and extends the framework to known distribution shift and imperfect failure labeling. Experiments use synthetic data and repeated sampling from an annotated SWE-Chat dataset.

Critical strengths
  • The topic is interesting, and the setting is close to the abstract workflow of how people testing systems.
  • The theory part seems to be sound.
Critical weaknesses
  • The paper uses much narrower margins than the official ICLR template. This allows more content within the page limit.
  • I think the setting is over-simplified. The most critical part in real-world testing is how to ensure the test-coverage. The whole paper is based on an assumption that the more you test, the more coverage you have. However, in the real-world, this would rarely happen. A simple example is that you tried multiple tests with integer input for an system, but the bug only occurs when inputs are floats. This problem is classic in software testing scope (see "STADS: Software Testing as Species Discovery" TOSEM, 2018), which also discussed the "when-to-stop-testing" topic. In agent system testing scenario, since the input space can be very large, I don't think such complexity can be easily ignored.
  • The paper's scope is unclear to me. It looks like a pure theory paper where only formulations in the methods without any real-world application example of "agent system". But the title claimed a relationship to the "agent system" scenario to me. If it really related to "agent system testing", I think at least STADS should be discussed or compared in the paper.
  • From my understanding, the quality of the experiment highly depends on the definition of "unseen fail". The labels seems all come from LLM judge. I think it matters since if we classify all failures into the same type, them the bound drops to zero right after finding the first failure.
Questions for authors
  • Is this a pure theory paper?
  • Can you use the correct paper format?
  • Can you compare or at least discuss the difference between this paper and STADS?
1 comment

On the margins of the paper. Thank you for catching the margins issue. (A spurious package import had changed the margins.) Our updated manuscript has the correct margins. We moved some of the background on SAVI, discussion of intuition for warm-starts and choices of the hyper parameter lambda, and discussion of the technical challenges of our results, to the appendix. We also moved some of the empirical setup details to the appendix.

On the connection to software testing. We disagree with the claim that our “assumption that the more you test, the more coverage you have”. Instead, the analogous statement in our setting is “the more you observe real-world usage of a system, the more coverage with respect to that distribution you have”. We’ve added a discussion of STADs and relation to software testing to the paper; their results (and those in the software testing literature) are not anytime valid and can therefore not be used for optimal stopping while preserving statistical validity. Our setting is different from classic software testing because with AI systems the goal is not test-coverage (this may be impossible with output spaces as large as AI systems!). Instead, the goal is a guarantee with respect to a distribution of usage for a population of beta testers or historical usage data.

On the real-world application. The paper’s primary contributions are the method and associated theory guarantees. The empirics are in support of establishing the finite-sample data efficiency of our method compared to other sensible approaches. We instantiate the method on real AI-agent outputs, but the goal of the empirics is to validate the method in circumstances in which a “ground-truth” missing mass can be observed. In real applications, no such ground-truth missing mass is available. Thus, we try to use a setting that is as close to a real operational setting as possible without losing our ability to measure ground truth and compare to other approaches.

On the importance of the unseen and fail functions. We agree (and discuss in the paper, see the first sentence of limitations in the discussion section) that these definitions are very important in practice and depend on the deployment context. Our contribution solves what we believe to be an important and interesting technical question of how to stop testing once the definitions of unseen and fail have been fixed and determined to be relevant to the deployment decision at hand. Future work examining how to choose these definitions would be valuable but is inherently context-dependent and should not be expected to be resolved in our work (and definitely not at the level of abstraction at which we are able to solve the optimal/optional stopping question), which proposes and establishes guarantees once these definitions have been satisfactorily determined. Our method (just like any statistical estimator) will not yield useful results if the quantity being measured (e.g., a trivial fail function that never marks anything as a failure) is very different from the real-world phenomenon it is intended to capture.

Reviewer_ayX2OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks when testing can stop with a statistical guarantee that the probability of encountering a previously unseen failure mode is sufficiently small. It formulates this as sequential missing-mass estimation and constructs an anytime-valid upper confidence sequence, allowing developers to inspect results and stop adaptively without invalidating coverage. The method mixes betting processes started at different times, using the monotonic decrease of unseen-failure probability to establish validity. It also develops extensions for known distribution shift and imperfect failure labeling. Experiments resample SWE-Chat sessions and examine alternative taxonomies, online novelty rules, and sessions containing multiple failure modes. The main experiment reports 97.3% simultaneous coverage at a nominal 95% level and tighter bounds than the evaluated Good–Turing baselines.

Critical strengths
  1. A principled solution to an important sequential evaluation problem. Algorithm 1 and Proposition 3.2 address the practical distinction between estimating unseen-failure probability at a predetermined sample size and deciding when to stop after inspecting results. The proof exploits monotonicity of the missing mass to construct a valid process despite the changing inferential target. This is a substantive statistical contribution with applications beyond AI evaluation.

  2. The experiments test several representations of failure novelty. Beyond the main fixed taxonomy, the paper evaluates an alternative taxonomy, online embedding-based novelty, and set-valued failure observations. These checks support the claim that the procedure is not restricted to a single categorical clustering. The explicit disclosure that the imperfect-label experiment achieves no ground-truth labeling savings is also valuable.

Critical weaknesses
  1. The imperfect-label stopping-time guarantee is not established by the provided proof. In the proof of Theorem A.3, the conditional-mean calculation identifies the expectation of the importance-weighted (Zi)(Z_i) with the true unseen-failure probability, although (Zi)(Z_i) uses the potentially incorrect label (h~i)(\tilde h_i). Lemma A.5 bounds labeling disagreement; it does not establish this equality, and the subsequent positive-drift argument needs to account for the discrepancy. This concern specifically affects the extension’s stopping-time guarantee; the authors should supply a corrected argument, additional assumptions, or a revised error allocation.

  2. The deployment interpretation depends on unvalidated operational labels. The SWE-Chat experiment resamples a finite empirical population whose failure labels are gated by dataset scores and then revised by an LLM; 255 of 993 initially failed test sessions are relabeled successful, and novelty categories are also automatically constructed. This permits a legitimate statistical evaluation against those constructed labels, but does not independently establish that the labels capture substantive failure modes or shared mitigation requirements. An independent audit of failure and novelty judgments would strengthen the connection between the statistical certificate and the deployment-safety motivation.

Questions for authors
  1. How does the positive-drift argument in Theorem A.3 accommodate disagreement between (h~i)(\tilde h_i) and fail? Does the stated stopping-time bound require a stronger margin or a different labeling-error allocation?

  2. Algorithm 2 restricts the betting parameter to ([0,(B+ζ)−1])([0,(B+\zeta)^{-1}]), whereas Appendix C.2 describes the known-shift experiment as using a grid in ((0,1)). Which range was implemented, and was the grid rescaled?

  3. Was there independent human validation of the revised failure labels and same-mitigation novelty judgments beyond the 40-session development set? Please summarize any existing validation.

1 comment

Reviewers R9Bv and ayX2 identified the same set of strengths and weaknesses. We respond to their shared comments/questions in the response to reviewer R9Bv. Reviewer ayX2, please feel free to respond either in this thread or in the other one.

Reviewer_R9BvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper formulates pre-deployment failure discovery as sequential inference on the remaining probability of encountering an unseen failure. Under a fixed system and a monotone novelty rule, it constructs an upper confidence sequence by mixing betting processes across starting times and betting parameters. This yields an anytime-valid stopping criterion in the basic setting with exact failure labels. The paper also proposes extensions for adaptive task sampling with known importance weights and imperfect failure labels obtained through selective auditing. Experiments resample a finite population of SWE-Chat sessions, using LLM-generated failure descriptions and taxonomies, and compare the proposed bounds with pointwise and Bonferroni-corrected Good–Turing bounds.

Critical strengths
  1. A useful statistical formulation of failure discovery. The paper correctly treats the remaining unseen-failure probability as a random, history-dependent quantity. Exploiting its monotonicity to support valid adaptive stopping is a principled contribution to evaluation methodology.

  2. Experiments examine several relevant variations and disclose limitations. The evaluation includes repeated trials, simultaneous coverage, alternative taxonomies, online novelty rules, and sessions containing multiple failure modes. The authors also explicitly report that the imperfect-labeling experiment achieves no reduction in ground-truth labeling. This transparency helps distinguish demonstrated results from prospective benefits.

Critical weaknesses

The stopping-time proof does not establish the claimed imperfect-labeling extension.

In the proof of Theorem A.3, particularly page 19, lines 1000–1008, the argument effectively uses [ \mathbb{E}!\left[ \frac{P(X_i)}{Q_i(X_i)}Z_i ,\middle|,\mathcal F_{i-1} \right] =R_{i-1}^{\mathrm{unseen}}. ] However, Algorithm 2 defines (Z_i) using the selectively audited label (\widetilde h_i), which can differ from the true failure indicator. Importance weighting corrects the sampling distribution; it does not remove labeling bias.

Writing the left-hand side as (\widetilde r_{i-1}), the PAC labeling guarantee provides, on its validity event, an error bound of the form [ \left|\widetilde r_{i-1} -R_{i-1}^{\mathrm{unseen}}\right|\leq\zeta, ] rather than the equality used in the proof. This matters because the betting factor at the stopping threshold is [ 1+\lambda\left(\varepsilon-\zeta -\frac{P(X_i)}{Q_i(X_i)}Z_i\right). ] The condition (R_{i-1}^{\mathrm{unseen}}<\gamma<\varepsilon-\zeta) therefore does not, by itself, establish the positive drift required by the proof.

For example, with (P=Q), (\varepsilon=0.10), and (\zeta=0.05), a true unseen-failure probability of (0.02) and an additional unseen false-positive probability of (0.04) are compatible with the stated labeling-error bound. The proxy indicator then has mean (0.06), making the expected betting factor (1-0.01\lambda), despite the true risk being below an admissible (\gamma=0.03). A separate argument controlling proxy-induced novelty is needed to establish the claimed delay bound relative to the true-risk crossing time.

Appendix B.3 does not empirically resolve this issue: every session before stopping receives a ground-truth label, so the experiment never exercises the proposed use of erroneous proxy labels. Consequently, both the generalized stopping guarantee and its practical labeling-cost benefit remain insufficiently established.

The comparisons and validation do not establish a convincing deployment-relevant advantage.

The empirical baselines are two versions of the same Good–Turing bound with a data-independent confidence width. Their looseness does not establish an advantage over relevant variance-adaptive sequential methods.

A concrete alternative follows from the paper’s own monotonicity assumption. In the exact-label setting, define [ Y_i=\mathrm{fail}(A_i), \mathrm{unseen}(A_i,H_{i-1}). ] Then [ \mathbb E[Y_i\mid\mathcal F_{i-1}] =R_{i-1}^{\mathrm{unseen}}, \qquad R_t^{\mathrm{unseen}} \leq \frac1t\sum_{i=1}^{t} \mathbb E[Y_i\mid\mathcal F_{i-1}]. ] Existing empirical-Bernstein confidence sequences, such as Howard et al. (2021, Theorem 4), cover this average conditional mean without requiring a common mean across observations. By the displayed inequality, their upper bounds also bound the current unseen-failure probability. This is a direct derivation of a relevant comparator from existing theory.

Such a comparator may have worse detection delay than the restart mixture, but that is precisely the distinction that should be established. Comparisons with it, and a reported ablation against the non-restarted betting process discussed in Section 3.1, would clarify the incremental contribution. The current results mainly establish superiority over conservative fixed-width baselines.

Furthermore, the application study establishes coverage for a stipulated empirical distribution and labeling pipeline. The reference population contains 4,961 previously collected sessions; dataset-labeled successes are treated as nonfailures, and the LLM judge relabels 255 of the 993 remaining dataset-labeled failures as successes. The “true” missing mass is subsequently computed using those labels and the constructed taxonomy. This is a legitimate simulation target, but it supplies no independent validation that failures were correctly identified or that grouped incidents share an effective mitigation.

This distinction is consequential for the deployment framing. Observing a failure category removes it from the unseen-risk target even if its recurrence remains dangerous. The experiments do not validate mitigation effectiveness or transfer the certificate to a system modified during testing. The practical interpretation therefore needs substantially stronger evidence or narrower claims.

Questions for authors
  1. Theoretical clarification: How does the proof of Theorem A.3 account for the difference between the conditional mean of the proxy-based (Z_i) and the true unseen-failure probability? Can the stated delay bound be established while explicitly retaining false-positive and false-negative contributions?

  2. Implementation clarification: Algorithm 2 restricts (\lambda) to ([0,(B+\zeta)^{-1}]), whereas Appendix C.2 states that the known-shift experiment uses a grid in ((0,1)), with (B\approx1.5). Was that grid rescaled? Without rescaling or another restriction, betting factors can become negative, invalidating the nonnegative-supermartingale argument.

  3. Comparative evidence: How does the method compare with an empirical-Bernstein confidence sequence for the average conditional novelty probability, and with the non-restarted betting process? These comparisons would materially clarify the benefit attributable to the restart construction.

  4. Validation of the application: Is there independent human adjudication of the failure labels and category merges, including an audit of sessions initially labeled successful? What evidence supports the interpretation that membership in the same category implies coverage by an existing mitigation?

3 comments
AuthorsOpenReview ↗

Main response

Reviewers R9Bv and ayX2 identified the same set of strengths and weaknesses. We respond to their shared comments and questions here.

  1. On the error in the imperfect labeling extension. Thank you both for identifying this issue. The original proof relied on a conditional-expectation equality that holds for the algorithm presented in the body of the paper, but not necessarily for the imperfect labeling extension. Fortunately, this equality is stronger than necessary to establish our theoretical guarantees. In our updated manuscript, we have revised the proofs of simultaneous coverage and stopping time guarantees to state weaker sufficient conditions on the conditional mean. We then show that these conditions hold for the imperfect-labeling extension under a simple additional assumption. Intuitively, we require that true successes cannot be counted as discoveries of new failure modes, even if the imperfect labeler mistakenly flags them as failures (false positives). This directly rules out the counterexample raised by reviewer R9Bv. In practice, this is a natural condition that can be satisfied operationally by verifying flagged failures before counting them as unseen. We provide full details of the changes required in the proofs at the end of this rebuttal for those who closely followed the proof.

  2. On the realisticness of the empirics for real-world pre-release testing pipelines. We agree that we do not independently validate whether the failures identified share the same mitigations. This is a deployment-specific question! For example, a company deploying a customer service agent may have different mitigations available to them, depending on the company; similarly, a coding agent developer may care about different mitigations depending on the context where the agent will be deployed. Whether a particular mitigation adequately addresses failures in a system depends on the effects of the failures, and since we are using publicly available data and are not actually deploying an AI system, we cannot begin to resolve these questions. We instead take a simple, reproducible approach using LLM judges as a proof of concept, with no independent human adjudication. The reviewers also note “LLM judge relabels 255 of the 993 remaining dataset-labeled failures as successes”, but the original dataset-labeled failures are also from LLM judges, and we decided to re-run the annotation because the previous annotations came from an older, less capable model. We note that no reviewer disputes the statistical validity of our empirics (just the relevance to deployment applications), and we think statistical validity and apples-to-apples comparisons to other approaches is the crux of the value of our empirics.

  3. On the betting parameters in Appendix 3.2. Yes, we ensured non-negativity of the supermartingale in our empirical validation of the extension to known distribution shift. The statement that the lambda grid is (0,1) was a typo, and our implementation did correctly rescale the grid. Thank you for catching this.

Response to comments specific to reviewer R9Bv.

  1. We thank the reviewer for suggesting an additional baseline. We have added the empirical-Bernstein confidence sequences from Howard et al. (2021, Theorem 4). On the SWE-chat dataset, our method achieves a tighter bound than the empirical-Bernstein confidence sequence. Please see figure 1 in the updated manuscript. One intuition for why our method should typically work better than an approach like an empirical Bernstein running means approach is that empirical Bernstein approaches cannot stop quickly if the missing mass remains above epsilon for a long period of time. In particular, if the missing mass stayed above kεk\varepsilon for T rounds, the process would take at least kTkT rounds before the mean missing mass dropped below epsilon. Our approach does not have this issue because of the mixture over re-start processes.

  2. We also report the requested ablation against the non-restarted betting process. Please see figure 10 in the updated manuscript. On the SWE-chat dataset, we find that the restart-mixture results in modestly tighter bounds than the non-restarted betting process. We remark that the restart-mixture is important for theoretical optimality, but the empirical benefit on this particular dataset is modest.

==================================================================

Appendix: Additional details on the the correction to the proof of the imperfect labeling extension:

  1. For the more general stopping time guarantee, let (Vt)t≥1(V_t)_{t\ge 1} be a sequence of unseen failure statistics. We require the following conditional mean condition:

E[Vt∣Ft−1]≤Rt−1unseenE[V_t | F_{t-1}] \le R_{t-1}^{{unseen}} a.s. for all t≥1t \geq 1

This is a simple upper bound condition, which is weaker than the equality assumed in the original proof. The stopping time argument goes through under this condition as follows:

a. To establish positive log-wealth drift, we want E[log⁡(1+λ(ϵ−ζ−Vt)∣Ft−1]>0{E}[\log(1+\lambda(\epsilon - \zeta - V_t) | F_{t-1}] > 0.

b. Under the conditional mean bound, E[λ(ϵ−ζ−Vt)∣Ft−1]≥λ(ϵ−ζ−Rt−1unseen){E}[\lambda(\epsilon - \zeta - V_t) | F_{t-1}] \ge \lambda(\epsilon - \zeta - R_{t-1}^{{unseen}}).

c. After Tγ,Rt−1unseen<γT_{\gamma}, R_{t-1}^{\mathrm{unseen}} < \gamma and γ<ϵ−ζ\gamma < \epsilon - \zeta, so E[λ(ϵ−ζ−Vt)∣Ft−1]≥λ(ϵ−ζ−γ)>0\mathbb{E}[\lambda(\epsilon - \zeta - V_t) | {F}_{t-1}] \ge \lambda(\epsilon - \zeta - \gamma) > 0.

d. The log is locally linear so we can combine this with the above facts to establish positive drift: E[log⁡(1+λ(ϵ−ζ−Vt)∣Ft−1]E[\log(1+\lambda(\epsilon - \zeta - V_t) | F_{t-1}] is approximately equal to E[λ(ϵ−ζ−Vt)∣Ft−1]{E}[\lambda(\epsilon - \zeta - V_t) | F_{t-1}]. More rigorously, since VtV_t is bounded (from bounded likelihood ratio), we can apply a Taylor approximation bound: $ E[\log(1+\lambda(\varepsilon-\zeta-V_t))| F_{t-1}]] \geq \lambda E[\varepsilon-\zeta-V_t|\mathcal F_{t-1}]-C\lambda^2$ and show that for sufficiently small λ\lambda, the strictly positive first-order term guarantees positive log-wealth drift.

  1. Next, we show that the PAC labeling algorithm satisfies this under the additional assumption that the following holds: for all A∈A,A \in \mathcal{A}, any history HH, fail(A)=0  ⟹  unseen(A,H)=0\text{fail}(A) = 0 \implies \texttt{unseen}(A,H) = 0.

a. This implies that pointwise, for imperfect labeler h~:h~(A)⋅unseen(A,H)≤fail(A)⋅unseen(A,H)\tilde h: \tilde h (A) \cdot \texttt{unseen}(A,H) \le \text{fail}(A)\cdot \texttt{unseen}(A,H).

b. For the PAC labeling algorithm, Vt=P(Xt)Qt(Xt)ZtV_t = \frac{P(X_t)}{Q_t(X_t) }Z_t. Applying the above inequality and by change of measure, we get the desired result: E[Vt∣Ft−1]=E[P(Xt)Qt(Xt)h~(At)⋅unseen(At,Ht−1)∣Ft−1]≤EXt∼P[fail(A)⋅unseen(A,H)∣Ft−1]=Rt−1unseen{E}[V_t | F_{t-1}] = {E}[\frac{P(X_t)}{Q_t(X_t)}\tilde h (A_t) \cdot \texttt{unseen}(A_t,H_{t-1}) | F_{t-1} ] \le E_{X_t \sim P}[\text{fail}(A)\cdot \texttt{unseen}(A,H) \mid F_{t-1}] = R_{t-1}^{{unseen}}.

  1. We note that the setting of imperfect labels requires an additional weak technical condition: under the imperfect labeling algorithm, every true failure must have a positive probability of being audited with a ground-truth label at each round. This ensures that the true missing mass converges to 0 almost surely, even under an imperfect labeling history. Without this condition, the labeler could systemically miss failures, and we would not be able to bound the missing mass through the observed sequence. This is formalized in the updated manuscript and we remark the condition was satisfied by the algorithm in our original submission. We now state this condition explicitly to prevent confusion.
  2. Remarks on an alternate imperfect labeling extension: In our updated manuscript, we also provide an alternate imperfect labeling extension based on the prediction powered inference (PPI) framework, which uses ground-truth label audits to construct an unbiased estimator of the unseen failure probability. This approach has advantages over the original PAC labeling extension in that: (1) it satisfies the exact conditional mean equality assumed in the original theory, (2) it does not require the additional assumption on how the unseen indicator function handles false positives, and (3) on the SWE-chat dataset, it substantially reduces the number of ground-truth labels required before stopping. Please refer to the updated manuscript for additional details.

Thanks for your response, I have increased my overall score from 2 to 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to all reviewers

We thank the reviewers for their feedback. We are glad they appreciated the problem’s importance and technical novelty. As we see it, the criticisms of the paper common to multiple reviewers are as follows:

  1. A bug in the proof of the extension to imperfect failure labeling in the stopping time analysis.
  2. The desire for additional baselines and comparisons with existing approaches.
  3. Questions about ecological validity of our empirical application.

We address these comments as follows:

  1. We thank the reviewers for identifying the issue in the imperfect-labeling extension, which is an auxiliary generalization of our main result. The main results in the body of the paper were unaffected by this issue. We provide details for this straightforward correction in the response to reviewer R9Bv.
  2. We have added these baselines and comparisons with other approaches. We comment on the details in the responses to individual reviews.
  3. The point of the empirical section in our paper is to validate and show the usefulness of our methodological contribution by utilizing a public dataset and constructing a setting with known true missing mass, not to make claims about the safety of the distribution of sessions in SWE-Chat. Our approach lets us compare our method with other approaches and with a “ground truth” missing mass. We agree that our empirical results do not exactly match the real work-flow of an organization conducting pre-deployment testing of an AI model. Several of the choices we make, such as using publicly available data, LLM judges, and a fixed failure taxonomy, are in service of reproducibility of our results and apples-to-apples comparisons with the Good-Turing estimator. But these choices are in tension with ecological validity, since real pre-deployment testing would be conducted via real beta testing, particular application-specific notions of failure and mitigations.

We hope these responses, in addition to the undisputed strengths of the paper (including the important and novel problem setting, conceptually interesting solution, and technical novelty), help the program committee re-evaluate the merits of the paper. We have updated the framing in several places to clarify the questions and concerns raised. We also note that we have added an alternate imperfect labeling extension, an optimality result in the form of a lower bound that matches the asymptotic stopping time our procedure achieves, and streamlined proofs. None of these additions/changes substantively affect the contributions of the original submission, and we are happy for the submission to be judged solely on the content of the original submission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.