acceptodds
Under review as a conference paper at ICLR 2027

OptAPI: Optimally Active Powered Inference under Label Budget

Abstract

Gold-standard labeled data are essential for reliable statistical inference but are often severely limited by annotation budgets. Existing active inference methods use pretrained predictive models to exploit unlabeled data; however, they typically assume all observations are equally important and optimize efficiency for only a single parameter dimension. To effectively capture joint multivariate uncertainty, we propose , a two-stage optimally active powered inference framework that maximizes the utility of both labeled and unlabeled data under a strict budget. The first stage deploys an active sampling rule to identify highly informative observations, while the second stage fine-tunes the prediction-powered correction. Crucially, both stages are unified under a D-optimality criterion that minimizes the determinant of the asymptotic covariance matrix, thereby directly targeting multidimensional uncertainty. Furthermore, we establish the asymptotic normality of the estimator, which theoretically justifies our D-optimal design and guarantees the validity of our constructed confidence intervals. Extensive numerical evaluations on synthetic and real-world datasets demonstrate the framework's superior effectiveness and statistical efficiency.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Active Inference, Sampling, D-Optimality

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ySZxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes OptAPI, a framework combining active sampling and prediction-powered inference under a limited labeling budget. The authors introduce a D-optimality criterion to guide sampling and power tuning, derive optimality conditions, and develop a two-stage algorithm. Theoretical results and experiments are provided to support the proposed approach.

Critical strengths
  1. The paper addresses a practically relevant problem, particularly when obtaining reliable labels is expensive.

  2. D-optimality is a reasonable choice for multivariate inference, as it accounts for the joint covariance structure.

  3. The paper is generally well organized and includes theoretical analysis, simulations, and real-data experiments.

Critical weaknesses
  1. The methodological novelty is somewhat limited. The estimator largely follows the existing active inference and PPI framework, with D-optimality being the main extension. It is not entirely clear what new methodological insights are obtained beyond adapting a classical optimal design criterion.

  2. The theoretical optimality claims appear stronger than the results established. In particular, the conclusion states that the estimator attains the minimal asymptotic variance, while the main theoretical results characterize optimality under fixed values of the other design parameter.

  3. The empirical advantages are not fully convincing. The multivariate mean experiment is constructed to highlight scale disparities, and the main efficiency metric is directly aligned with the D-optimality objective. The evidence for broader improvements is less clear.

Questions for authors
  1. In Section 3.3, the sampling policy is constructed with λ0=1\lambda_0=1, although Theorem 1 shows that the optimal policy depends on λ\lambda. How sensitive is the final estimator to this initial choice? For example, would using λ0=0\lambda_0=0 or a pilot-based estimate lead to substantially different results?

  2. Theorem 2 considers unrestricted optimization over λ\lambda, whereas the implementation restricts λ\lambda to [0,1][0,1]. What motivates this choice? Could it lead to efficiency loss when the optimal λ\lambda lies outside this interval?

  3. In Section 3.3, the estimated covariance matrix combines a pilot covariance with an active-sampling correction. Is this matrix always positive semidefinite? If not, how is the log-determinant optimization handled when adding a small ϵI\epsilon I is insufficient?

  4. The paper motivates D-optimality through joint confidence ellipsoids, while Theorem 4 establishes marginal confidence intervals. Can the authors clarify the theoretical justification for the joint confidence regions evaluated in Section 4?

4 comments
AuthorsOpenReview ↗

Response to Reviewer ySZx (1/2, W1 - W3)

We thank the reviewer for the careful review and for recognizing D-optimality as a natural criterion for joint uncertainty. We clarify each point below.

W1: What is new beyond adopting D-optimality?

We do not claim novelty for D-optimality or for the prediction-powered estimator. Our contribution is the covariance-dependent design that arises when we combine them with active label acquisition. OptAPI chooses both which points to label and how much to rely on the prediction model ff. PPI (Angelopoulos et al., 2023), PPI++ (Angelopoulos et al., 2026), and active inference (Zrnic and Candès, 2024) are special cases (Section 3.1).

For a fixed λ\lambda, the D-optimal policy (Theorem 1) is πopt(x)∝[E{∥Δλ∥(Vλπ)−12∣X=x}]1/2.\pi^{\rm opt}(x)\propto[\mathbb{E}\lbrace \Vert \Delta_\lambda\Vert^2_{(\mathbf V^\pi_\lambda)^{-1}}\mid X=x\rbrace ]^{1/2}. The A-optimal rule uses a fixed weighting matrix, but this rule whitens the residual scores by the covariance that the policy itself induces. Thus, the policy has only an implicit definition, and we compute it with a budget-constrained MM algorithm with a closed-form water-filling step (Proposition C.1). Theorem 2 shows that the D-optimal power also depends on this covariance, which couples the two choices. The criterion also targets the volume of the joint Wald region, which is proportional to det⁡(Σ)1/2\det(\mathbf\Sigma)^{1/2} (see our response to Q4). We will state these contributions in Section 1.2 (see also our response to Reviewer ALYQ, W1).

W2: Optimality claims

We agree that "attains the minimal asymptotic variance" overstates our results. Theorems 1–2 characterize each design parameter given the other, not their joint optimum. The paper establishes the following.

  • For a fixed λ\lambda, no MM step increases the log-determinant, and the fixed points of the MM step are the policies of Theorem 1 (Proposition C.1(iii)–(iv)).
  • The one-step mixed policy is never worse than uniform sampling (Proposition C.1(v)).
  • After acquisition, Theorems 3–4 give valid inference for the resulting design (see our response to Reviewer ALYQ, W4).

We will replace the overstated claim with "a two-stage design that targets the D-optimal asymptotic covariance." Joint optimization of (π,λ)(\pi,\lambda) is future work.

W3: Empirical gains

Thank you for this point. Section 4.1 is constructed to show when D-optimality matters, not to claim universal superiority. Scale disparity is a change of units. The D-optimal design does not change under it, but the A-optimal design does, so Section 4.1 isolates this mechanism.

The two metrics answer different questions. MSE is a trace criterion, so it favors A-optimal designs. The effective sample size neffn_{\rm eff} measures the volume of the joint confidence region, which Section 4 evaluates (see our response to Q4). The real datasets have no constructed scale disparity. On the housing data, OptAPI has the smallest MSE at nearly all budgets and the largest neffn_{\rm eff} (Figure 4). On the census data, OptAPI has the largest neffn_{\rm eff} at every budget (Table A2):

neffn_{\rm eff} at nb/nn_b/n5%50%
OptAPI32,270225,700
A-optimal (Active Inference)30,360209,100
Uniform (PPI)24,300193,500

On the census data, the A-optimal methods have a slightly smaller MSE. This reflects the trade-off between joint volume and marginal accuracy (see our response to Reviewer ALYQ, W1). We will state this trade-off and report per-coordinate interval widths together with neffn_{\rm eff}, MSE, and coverage.

AuthorsOpenReview ↗

Response to Reviewer ySZx (2/2, Q1 - Q4)

Q1: Sensitivity to λ0\lambda_0

Thank you for the insightful comment. We would like to clarify that the final estimator is not sensitive to λ0\lambda_0. The initial power only shapes the design, and Stage II tunes λ\lambda again on the acquired labels (Section 3.3).

Appendix G.6 varies λ0∈{0,0.25,0.5,0.75,1}\lambda_0\in\lbrace 0,0.25,0.5,0.75,1\rbrace, including λ0=0\lambda_0=0, and efficiency and coverage are almost the same for all values (Figure A5). A pilot-based estimate restricted to [0,1][0,1] is also in this range, so we expect the same result. We use λ0=1\lambda_0=1 as the default because it matches active inference (Zrnic and Candès, 2024), where the Stage I residual is the prediction error (see also our response to Reviewer ALYQ, W3).

Q2: Why restrict λ\lambda to [0,1][0,1]?

Thank you for this important question. Validity does not require the restriction, but convexity does.

The estimating equation, Eq. (3.2), is unbiased for every λ\lambda, so Theorems 3–4 hold for any limit of λ^\widehat\lambda. For canonical GLMs, the curvature weights of the empirical Hessian are λ+(1−λ)ξi/πi\lambda+(1-\lambda)\xi_i/\pi_i. These weights are nonnegative for every labeling outcome only when λ∈[0,1]\lambda\in[0,1], which is the range from the classical estimator (λ=0\lambda=0) to the fully prediction-powered estimator (λ=1\lambda=1). PPI++ also clips λ\lambda to [0,1][0,1] and reports that the difference from the unrestricted estimator is usually small in practice (Angelopoulos et al., 2026, Section 6.2). Efficiency is lost only if the unrestricted optimum lies outside [0,1][0,1]. We will emphasize the constrained problem in the revised version, whose interior solutions satisfy Theorem 2.

Q3: Is the estimated covariance positive definite?

We appreciate this question. Yes, the matrix is positive semidefinite by construction. We have V^(λ)=V~+n−1∑iciΔ^λ,iΔ^λ,i⊤\widehat{\mathbf V}(\lambda)=\widetilde{\mathbf V}+n^{-1}\sum_i c_i\widehat\Delta_{\lambda,i}\widehat\Delta_{\lambda,i}^\top with weights ci=ξi(1−π^i)/π^i2≥0c_i=\xi_i(1-\widehat\pi_i)/\widehat\pi_i^{2}\ge0, because 0<π^i≤10\lt \widehat\pi_i\le1. Since V~\widetilde{\mathbf V} is an empirical covariance, V^(λ)⪰V~⪰0\widehat{\mathbf V}(\lambda)\succeq\widetilde{\mathbf V}\succeq0. Thus, V^(λ)\widehat{\mathbf V}(\lambda) can be singular but never indefinite.

For any ε>0\varepsilon\gt 0, V^(λ)+εI≻0\widehat{\mathbf V}(\lambda)+\varepsilon\mathbf I\succ0, so the log-determinant is always defined. The ridge only shifts λ^\widehat\lambda slightly, and Theorems 3–4 hold for any limit of λ^\widehat\lambda (see our response to Q2). No singular V^\widehat{\mathbf V} occurred in our experiments. For numerical stability, we will compute the log-determinant with a Cholesky factorization and increase ε\varepsilon only if it fails. We will include this argument in Section 3.3.

Q4: Joint confidence ellipsoids and Theorem 4

We agree that this justification should be explicit. Theorem 3 gives joint asymptotic normality of θ^\widehat\theta. With a consistent estimator Σ^\widehat{\mathbf\Sigma} whose limit is positive definite, the joint Wald region C1−α={θ: n(θ^−θ)⊤Σ^−1(θ^−θ)≤χp,1−α2},\mathcal C_{1-\alpha}=\big\lbrace \theta:\ n(\widehat\theta-\theta)^\top\widehat{\mathbf\Sigma}^{-1}(\widehat\theta-\theta)\le\chi^2_{p,1-\alpha}\big\rbrace, satisfies P(θ⋆∈C1−α)→1−α\mathbb P(\theta^\star\in\mathcal C_{1-\alpha})\to1-\alpha, and its volume is proportional to det⁡(Σ^)1/2\det(\widehat{\mathbf\Sigma})^{1/2}. This is the region that Section 4 evaluates and that the D-criterion minimizes. The marginal intervals of Theorem 4 do not give simultaneous coverage. We will update this joint region in Theorem 4 and Remark 4.

All corresponding clarifications, results, and discussion will be incorporated into the revised paper. We appreciate your thoughtful and constructive feedback, and welcome any further discussion.

References

  • PPI: Anastasios N. Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I. Jordan, and Tijana Zrnic. Prediction-powered inference. Science, 382(6671):669–674, 2023.

  • PPI++: Anastasios N. Angelopoulos, John C. Duchi, and Tijana Zrnic. PPI++: Efficient prediction-powered inference. The Annals of Applied Statistics, 20(3):2235–2252, 2026.

  • Active inference: Tijana Zrnic and Emmanuel J. Candès. Active statistical inference. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024.

I appreciate the authors' detailed response and clarifications. However, my main concern regarding optimality remains. Although the authors have acknowledged this limitation, the theoretical results establish only conditional optimality for each design parameter, rather than joint optimality of the proposed two-stage procedure. I believe this limits the strength of the paper's theoretical contribution.

Therefore, I maintain my original rating.

AuthorsOpenReview ↗

Response to Reviewer ySZx

Thank you for your continued engagement and for clarifying your remaining concern. We recognize that joint global optimality would strengthen the theoretical contribution. Our separation of the two stages is motivated by two considerations.

1. Distinct roles. Both stages target the same criterion, log⁡det⁡Vλπ\log\det\mathbf V^\pi_\lambda, but they operate with different information. The policy π\pi allocates the label budget before any new outcomes are observed, whereas λ\lambda calibrates how much the estimator relies on the predictions once the labels are collected. After acquisition, λ\lambda can still be retuned, but the probabilities under which the labels were acquired cannot. The working value λ0\lambda_0 therefore only shapes π\pi, and Figure A5 shows nearly identical efficiency and coverage for λ0∈{0,0.25,0.5,0.75,1}\lambda_0\in\lbrace 0,0.25,0.5,0.75,1\rbrace, which supports this design in practice.

2. Computational trade-off. For a fixed λ\lambda, the D-optimal policy has no closed form (Theorem 1). It is computed by the MM algorithm (Proposition C.1), where each step requires a covariance factorization and a budget-constrained water-filling pass over the whole pool. A profiled joint search over λ\lambda would solve this policy problem repeatedly, once per candidate value. The two-stage design avoids this outer loop while targeting the same criterion. In a multiwave extension, newly acquired labels could inform both the next policy and its prediction weight, and we will discuss this direction and its theoretical challenges as future work in Section 5.

We will make this rationale and the scope of our guarantees explicit in Section 3.3. We sincerely appreciate your time and hope these clarifications and revisions adequately address your concerns. We would be grateful if you would consider revisiting your assessment, and we welcome further questions and discussion.

Reviewer_Ho6uOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes OptAPI, a two-stage framework for active statistical inference under a label budget. Stage I chooses a sampling policy that minimizes the determinant of the asymptotic covariance (D-optimality) of an AIPW/PPI++-style estimator, computed by a majorize-minimize (water-filling) algorithm from quantities estimated on a labeled pilot set and stabilized by mixing with uniform sampling. Stage II tunes the power-tuning parameter λ by minimizing an estimated log-determinant on the actively sampled data. The authors prove asymptotic normality of the resulting estimator and validity of coordinatewise confidence intervals, and they show that the one-step mixed policy is never worse than uniform sampling. Experiments on synthetic mean estimation, linear, logistic, and Poisson regression, and on two real datasets (California Housing, census healthcare) report larger effective sample sizes than PPI, PPI++, Active Inference, and Confident Active Inference, with nominal coverage.

Critical strengths

Estimator (1.1) nests PPI, PPI++, and active inference as special cases of a choice of (π,λ)(\pi, \lambda), which makes the design space easy to reason about and positions the work clearly.

Critical weaknesses

Minor:

  1. There are some typos throughout the manuscript. “Normility,” “critiera,” and “Unlabel data” (Figure 1).

  2. Estimating a risk or parameter with an unbiased importance-weighted estimator from actively chosen labels has been studied in active risk estimation and active testing. While orthogonal in approach related works on active testing and label-efficient model evaluation shoulbe be included. Please position OptAPI relative to it.

Major:

Presentation:

  1. The Plot hygiene is simply bad. The figures are hard to read. Plots look squished especially coverage; it is hard to tell which one maintains nominal coverage because of overlapping.

  2. The manuscript can benefit from better writing. I feel at many places the basic ideas are being oversold covering space. For instance in Section 3.1, cutting redundant simplifications frees up nearly half a page of for meatier content—like expanded experimental ablations, intuition for D-optimality over A-optimality, or theoretical discussion in the main text. This text can easily go into the Appendix.

Novelty

For me novelty is somewhat limited relative to prior work. D-optimal design is classical and determinant-based subsampling for regression and GLMs is well developed. Please state clearly and cleanly what is technically new here beyond applying the log-det criterion to the AIPW covariance.

Theory

The theory does not support the paper’s main claims about the method as deployed. Some of them feel like overclaims.

i) Theorems 1–2 give first-order conditions only, with no convexity or global argument, and the two stages are not jointly optimized (π\pi is fit for λ0=1\lambda_0 = 1 before λ\lambda is retuned). The conclusion nonetheless claims the estimator “attains the minimal asymptotic variance.”

ii) The efficiency guarantee does not cover the deployed method. Prop. C.1(v) applies only to the one-step mixed policy; the experiments use K=10K = 10 iterations, which Remark 5 concedes is uncovered.

iii) The design is estimated from a fixed-size pilot, so the limiting policy is the pilot-based one, not πopt\pi_{\text{opt}}. The asymptotics establish validity, not optimality.

iv) Theorem 3 assumes P(η^≠η⋆)→0P(\hat\eta \neq \eta^\star) \to 0, which is implausible for a continuous, pool-dependent η^\hat\eta.

v) Consistency of λ^\hat\lambda is assumed, not proven. The appeal to PPI++’s ERM argument does not apply, because log⁡det⁡V^(λ)\log\det \hat V(\lambda) is not an empirical risk.

vi) I think the claimed generality to M-estimation is unimplemented. MM requires a matrix-valued nuisance, but only a scalar surrogate under a rank-one score factorization is given, and the surrogate for mean estimation (§4.1) is never specified. The weighting of the initial estimator θ^0\hat\theta_0 is also unspecified.

Experiments

The empirical evidence is not budget-matched and does not measure what users report.

i) Pilot labels are a free extra resource that needs explicit mentioning. OptAPI uses a labeled pilot on top of nbn_b that is never charged to the budget, and the pilot also feeds Stage II through V~\tilde V. Uniform baselines use no pilot, so i feel the reported gains are inflated; the headline “157.7% gain at nb=500n_b = 500” (§4.1) comes with 500 extra pilot labels. Realized label counts are reported only for the census experiment and the pilot ablation, although the budget holds only in expectation. Please report the realized number of labels (mean and spread) for every method, so that comparisons at a “matched budget” can be verified.

ii) The metrics do not match the inferential output. D-optimality is motivated by ellipsoid volume, but Theorem 4 delivers coordinatewise intervals. For marginal intervals, a trace criterion is the natural target. The paper should explain when the volume criterion is the right objective or maybe i am missing something? Plus interval width, the standard efficiency metric in this literature, is never reported. The primary metric neffn_{\text{eff}} is the determinant the method optimizes. Per-coordinate widths matter most here, because a volume criterion can shrink the ellipsoid while widening the interval for individual coefficients. The main text claims improved estimation accuracy, yet on a trace-based metric (MSE), A-optimal baselines match or beat OptAPI in Appendix G.4 and G.7. I feel this is a genuine trade-off which has not been highlighted.

iii) The experiments miss the scarce-label regime the prior papers in this field motivate. Budgets sweep up to 95% of nn, so most of each plot shows methods converging, and the smallest synthetic budget is matched by an equally large uncounted pilot.

iv) Cost and stability are unexamined. There is no complexity analysis or timing, even though Appendix F lists high-dimensional cost as a limitation. My understanding is the policy requires V−1V^{-1}, which costs O(p3)O(p^3) per factorization and O(np2)O(np^2) for the pool norms, and it is recomputed at every MM iteration. Every experiment has p≤4p \le 4, and V~\tilde V comes from a small pilot, so the claim that “a small pilot suffices” (Appendix G.7) is untested where V~\tilde V would be ill-conditioned. The paper would benefit from (i) a stagewise complexity breakdown, in big-O terms of n, n_l, p, K, and the size of the λ grid, covering the MM iterations, surrogate fitting, and the λ search; (ii) wall-clock comparisons against the baselines; (iii) scaling experiments in p, n, and K, including how quickly the MM iterations converge; and (iv) an analysis of behavior as p/n_l grows, including any regularization of Ṽ.

v) The uncertainty surrogate departs from established constructions without comparison. Zrnic & Candès (2024) build u(x)u(x) from the prediction error ∣Y−f(X)∣|Y - f(X)| for regression and from 2min⁡(p^,1−p^)2\min(\hat p, 1-\hat p) for binary classification. OptAPI instead fits a whitened conditional second moment by quasi-Poisson regression on the pilot, with unstated inputs. For logistic loss with a calibrated probabilistic proxy, its policy reduces to Wu et al.’s rule from Table 1 times a Mahalanobis leverage term, which the paper never states. Multiclass outcomes are not addressed.

vii) The method is batch only. It has no sequential variant, unlike Zrnic & Candès (2024), and the limitation goes unacknowledged. The pilot-then-sample structure is also essentially a two-wave design, so the omission of multiwave or sequential designs (e.g., Kluger & Bates, 2026) is notable. Please discuss whether the MM characterization and Stage II tuning extend to sequential or multiwave acquisition.

Questions for authors

Overall addressing my points would, as I understand the paper, require revising its main claims and framing, not just adding cosmetic clarifications . Of course, I welcome corrections by authors, if any. My genuinely friendly and honest recommendation to the authors is to incorporate these changes and resubmit.

6 comments
AuthorsOpenReview ↗

Response to Reviewer Ho6u (1/4)

We thank the reviewer for the detailed review. These comments help us align our claims, theory, and experiments, and we address each point below.

Minor

  1. Typos. Thank you for the careful reading. "Normility," "critiera," and "Unlabel data" (Figure 1) will be corrected, and we will proofread the full manuscript.
  2. Related work. Thank you for these references. Active risk estimation and active testing (Sawade et al., 2010; Kossen et al., 2021, 2022) use importance-weighted estimators on actively chosen labels to estimate the risk of a fixed model. Wu et al. (2026), already cited in our Table 1, apply this idea to finite-population LLM benchmark accuracy. OptAPI uses the same type of unbiased importance-weighted estimator but differs in target and design: it estimates the parameter θ⋆\theta^\star of a multivariate M-estimation problem, chooses labels by a D-optimal criterion on the joint covariance, and tunes how much the estimator relies on the predictions. Section 2 will cite these works and include the comparison.

Presentation

  1. Plots. Agreed. The revised figures will have larger panels with distinct markers and line styles. For coverage, the vertical axis will be zoomed around the nominal level, with a 95% reference line and Monte Carlo error bars, and an appendix table will report coverage at selected budgets.
  2. Writing. Thank you for this suggestion. We will tighten the writing throughout and move the special cases in Section 3.1 to the appendix, using the freed space for more discussion of D- versus A-optimality.

Novelty: What is new beyond classical optimal design?

We do not claim novelty for the D-criterion or for the prediction-powered estimator, a point we also make to Reviewer ySZx (W1). Unlike determinant-based subsampling, which selects from fully labeled data, OptAPI designs before the labels are observed. Our contribution is the design problem that this creates.

  • Implicit policy and MM algorithm. The D-optimal rule depends on the covariance Vλπ\mathbf V_\lambda^\pi that the policy itself induces, so it has no closed form (Theorem 1). Proposition C.1 gives a budget-constrained MM algorithm with closed-form water-filling steps and monotone descent.
  • Coupling with power tuning. The D-optimal power weights the scores by (Vλπ)−1(\mathbf V_\lambda^\pi)^{-1}, which depends on the design (Theorem 2), whereas the A-optimal power uses the fixed weight Hθ⋆−2\mathbf H_{\theta^\star}^{-2}. Under the D-criterion, the choices of π\pi and λ\lambda are therefore coupled.

Section 1.2 will state these contributions explicitly.

Theory

T(i): First-order conditions and joint optimization

The conclusion does overstate our results. Theorems 1–2 characterize each design parameter given the other, not their joint optimum. What we prove is that no MM step increases the log-determinant, that its fixed points are the Theorem 1 policies, that the one-step mixed policy is never worse than uniform sampling (Proposition C.1(iii)–(v)), and that Theorems 3–4 give valid inference (detailed in our reply to Reviewer ySZx, W2). The design is also insensitive to fitting π\pi at λ0=1\lambda_0=1 (Reviewer ySZx, Q1). In the revision, "attains the minimal asymptotic variance" becomes "a two-stage design that targets the D-optimal asymptotic covariance," and joint optimization of (π,λ)(\pi,\lambda) is listed as future work.

T(ii): K=10K=10 versus one step

Thank you for this precise observation. In the experiments, Stage I runs up to K=10K=10 MM steps from uniform sampling, while Proposition C.1(v) covers one step followed by uniform mixing. All experiments will switch to one MM step with uniform mixing, so that Proposition C.1(v) covers the deployed method. Additional steps reuse the same pilot and add no new label information. Algorithm 1, Remark 5, and Section 4 will be updated with the K=1K=1 results, and K=10K=10 will be reported as an ablation.

T(iii): The design comes from a fixed-size pilot

This distinction is correct. With a fixed pilot, the policy converges to a pilot-based limit rather than to πopt\pi_{\rm opt}, and Theorems 3–4 establish validity conditional on the pilot, not optimality. Proposition C.1(v) holds at the population design quantities, so we expect the policy to approach one that is never worse than uniform sampling as the pilot grows. Empirically, even a small pilot attains over 99% of an oracle's neffn_{\rm eff} (Table A3; see our response to Reviewer ALYQ, W2). The revision states the conditional validity explicitly and removes all claims that the deployed policy attains πopt\pi_{\rm opt}.

AuthorsOpenReview ↗

Response to Reviewer Ho6u (2/4)

T(iv): The condition P(η^≠η⋆)→0\mathbb P(\widehat\eta\neq\eta^\star)\to 0

Indeed, this condition is implausible for a continuous η^\widehat\eta. We replace it with η^→η⋆\widehat\eta\to\eta^\star in probability, which holds when the empirical budget equation converges uniformly and has a unique root (Reviewer ALYQ raised the same point in W4). The proof of Theorem 3 still goes through. η^\widehat\eta depends only on the pilot and the unlabeled pool, not on the labels, so the coupling in our proof still applies. Under a moment condition on uu and the scores, the two weighted scores differ by a term with conditional mean zero and variance O(∣η^−η⋆∣)O(\vert\widehat\eta-\eta^\star\vert), because πη\pi_\eta is Lipschitz in η\eta and bounded below by τnb/n\tau n_b/n. The proof will be updated accordingly.

T(v): Consistency of λ^\widehat\lambda

Correct: log⁡det⁡V^(λ)\log\det\widehat{\mathbf V}(\lambda) is not an empirical risk, so the PPI++ argument (Angelopoulos et al., 2026) does not apply. Consistency instead follows from uniform convergence (also discussed in our reply to Reviewer ALYQ, W4). Conditional on the pilot, V^(λ)\widehat{\mathbf V}(\lambda) is a quadratic in λ\lambda whose inverse-probability-weighted matrix coefficients converge by the law of large numbers and T(iv). Since V^(λ)⪰V~\widehat{\mathbf V}(\lambda)\succeq\widetilde{\mathbf V}, the matrix stays positive definite on [0,1][0,1], with a small ridge if needed (see Reviewer ySZx, Q3). Thus, the log-determinant converges uniformly on [0,1][0,1]. If the limit has a unique minimizer, λ^\widehat\lambda converges to it in probability, and this pilot-dependent limit suffices for Theorem 3. The empirical risk minimization claim on page 7 will be replaced by this argument, uniqueness will be stated as an assumption, and the full proof will appear in the appendix.

T(vi): Generality to M-estimation

  • A scalar nuisance suffices. For a fixed whitening matrix A\mathbf A, the MM step needs only the scalar uA(x)2=E{∥Δλ∥A−12∣X=x}u_{\mathbf A}(x)^2=\mathbb E\lbrace\Vert\Delta_\lambda\Vert^2_{\mathbf A^{-1}}\mid X=x\rbrace, not the matrix Mλ(x)\mathbf M_\lambda(x) (Proposition C.1(ii)). A general MM-estimator therefore needs one scalar regression of ∥Δ^i∥A−12\Vert\widehat\Delta_i\Vert^2_{\mathbf A^{-1}} on XiX_i, as in active inference (Zrnic and Candès, 2024). With one MM step (T(ii)), A\mathbf A is fixed, so one regression suffices. For GLMs, the target factorizes as v(x),x⊤A−1xv(x),x^\top\mathbf A^{-1}x, so only v(x)v(x) is learned. Our experiments cover mean estimation and GLMs, and Section 3.3 will state the general case.
  • Mean estimation. For the squared loss with λ0=1\lambda_0=1, Δλ0=f(X)−Y\Delta_{\lambda_0}=f(X)-Y, so u(x)2=E{∥f(X)−Y∥A−12∣X=x}u(x)^2=\mathbb E\lbrace\Vert f(X)-Y\Vert^2_{\mathbf A^{-1}}\mid X=x\rbrace. In Section 4.1, we regressed the norm ∥f(Xk)−Yk∥A−1\Vert f(X_k)-Y_k\Vert_{\mathbf A^{-1}} on f(Xk)f(X_k) with a quasi-Poisson model and a log link. By Jensen's inequality, this targets a lower bound on u(x)u(x). The revision fits the squared norm and states the model in Section 4.1.
  • Initial estimator. θ^0\widehat\theta_0 is the unweighted fit on the pilot. It only initializes the Newton (iteratively reweighted least squares) iterations that solve Eq. (3.2) to convergence. For λ∈[0,1]\lambda\in[0,1], the weights λ+(1−λ)ξi/πi\lambda+(1-\lambda)\xi_i/\pi_i are nonnegative, so the objective is convex for canonical GLMs (cf. our response to Reviewer ySZx, Q2). Section 3.1 will describe this procedure in place of the one-step update.
AuthorsOpenReview ↗

Response to Reviewer Ho6u (3/4)

Experiments

E(i): Pilot labels and budget

We appreciate the question and agree that the pilot labels should indeed be accounted for explicitly. The pilot is pre-existing labeled data, as in active inference with historical labels (Zrnic and Candès, 2024), so it is not charged to the label budget (as also explained to Reviewer ALYQ, W2).

  • Use of the pilot. The pilot enters only through the design, the starting value θ^0\widehat\theta_0, and, via V~\widetilde{\mathbf V}, the choice of λ^\widehat\lambda. Its labels are not used in the estimating equation, Eq. (3.2). All active baselines share the same pilot (Section 4), so the D- versus A-optimal comparisons are matched. The uniform baselines do not use the pilot.
  • Pilot charged to the budget. Suppose we charge the pilot to OptAPI and give the uniform baselines the same total number of labels, scaling their neffn_{\rm eff} proportionally, which favors them. The ratio of the neffn_{\rm eff} of OptAPI to that of the best uniform baseline is then (census: Table A2; ablation: Table A3):
Pilot / budgetCensus 0.6%Census 6.3%Ablation 2%Ablation 10%Ablation 20%Ablation 50%
OptAPI / best uniform1.161.251.301.211.120.89

Our OptAPI thus keeps its advantage when the pilot is at most 20% of the budget. In Section 4.1, the pilot equals the smallest budget, so the 157.7% gain is not matched in total labels.

  • Realized label counts. Each point is labeled independently with probability πi\pi_i, and ∑iπi=nb\sum_i\pi_i=n_b. The realized count therefore has mean nbn_b and variance ∑iπi(1−πi)≤nb\sum_i\pi_i(1-\pi_i)\le n_b, so its relative standard deviation is at most 1/nb1/\sqrt{n_b} (4.5% at nb=500n_b=500).

Section 4 will state the cost convention, qualify the headline gain, and report pilot-charged results and realized counts (mean and standard deviation) for every method.

E(ii): Metrics and the inferential output

We agree that the metrics should match the inferential output, and the trade-off should be stated. Section 4 evaluates the joint Wald region given by Theorem 3, whose asymptotic coverage is 1−α1-\alpha and volume is proportional to det⁡(Σ^)1/2\det(\widehat{\mathbf\Sigma})^{1/2}. This is the volume that neffn_{\rm eff} measures, and the D-criterion minimizes (derived in our response to Reviewer ySZx, Q4), and the revision adds this region to Theorem 4.

The volume criterion is the right objective for joint inference on θ⋆\theta^\star, such as simultaneous regions and joint tests. It does not guarantee a smaller MSE or narrower marginal intervals, and Appendices G.4 and G.7 show this trade-off. When only selected coefficients are of interest, the criterion log⁡det⁡(C⊤ΣC)\log\det(\mathbf C^\top\mathbf\Sigma\mathbf C) targets them directly, and C=ej\mathbf C=e_j gives the A-optimal rule for coefficient jj (Reviewer ALYQ, W1). The revision moves this trade-off to the main text, replaces the claim of improved estimation accuracy with improved joint efficiency, and reports per-coordinate interval widths alongside neffn_{\rm eff}, MSE, and coverage (see also Reviewer ySZx, W3).

E(iii): The scarce-label regime

The gains of OptAPI are largest at small budgets. On the census data, its neffn_{\rm eff} exceeds that of uniform sampling by 33% at nb/n=5n_b/n=5% and by 17% at 50% (Table A2; see also Reviewer ySZx, W3). Our theory covers any fixed fraction nb/n→ρ∈(0,1)n_b/n\to\rho\in(0,1), so only vanishing fractions fall outside its scope. In the synthetic experiments, the pilot is as large as the smallest budget, which E(i) addresses. The revision adds budgets below 5%, focuses the main-text plots on low budgets, and states the vanishing-fraction limitation in Appendix F.

E(iv): Cost and stability

Cost and stability deserve clearer documentation. With GG the number of λ\lambda values and rr the number of surrogate inputs, the costs are:

  • Pilot and surrogate: O(nlp2+p3)O(n_lp^2+p^3) per Newton step for the pilot fit, and O(nlr2+r3)O(n_lr^2+r^3) per iteratively reweighted least squares step for the surrogate.
  • Stage I: O(np2+p3)O(np^2+p^3) per MM step for the whitening factorization and the pool leverages, and O(nlog⁡n)O(n\log n) for η^\widehat\eta. We use one MM step (T(ii)).
  • Stage II: O(nbp2)O(n_bp^2) once for the coefficients of the quadratic V^(λ)\widehat{\mathbf V}(\lambda) (T(v)), then O(p3)O(p^3) per λ\lambda.
  • Final estimator: O(np2+p3)O(np^2+p^3) per Newton step.

Up to the number of Newton steps, the total cost is O((n+nl)p2+Gp3+nlog⁡n)O((n+n_l)p^2+Gp^3+n\log n), which is nearly linear in nn.

Our experiments use p≤4p\le4, so the claim that a small pilot suffices (Appendix G.7) applies to low dimensions. As p/nlp/n_l grows, V~\widetilde{\mathbf V} can become ill-conditioned. We compute the log-determinant with a Cholesky factorization and add a small ridge if it fails (see Reviewer ySZx, Q3). Appendix G.1 will add these costs and per-method wall-clock times, and Appendix F will qualify the small-pilot claim and list the behavior as p/nlp/n_l grows as a limitation.

Reviewer_ALYQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposed a two-stage optimally active powered inference framework that maximizes the utility of both labeled and unlabeled data under a label budget. The key idea is to derive the D-optimal sampling mechanism under AIPW estimator as in (1.1). In implmentation, the first stage deploys an active sampling rule to identify highly informative observations, while the second stage fine-tunes the prediction-powered correction. Overall, the paper is well-written and the topic is interesting. However, the novelty of the proposed method needs to be clarified. It seems that the major contribution is to focus on D-optimality rather than A-optimality, compared with existing literature, e.g., "Surrogate-Powered Inference: Regularization and Adaptivity" by J. Chen, H. Wang, etc.

Critical strengths
  1. The paper is well-written and the topic is interesting.
  2. The theoretical invetigation is thorough.
Critical weaknesses
  1. The key novelty seems to replace the A-optimality by D-optimality. Since most of the time, we mainly care about the inference of the parameters themselves; minimizing the entire volume of the covariance matrix may involve directions that we do not care about. Thus, the real utility of the D-optimality vs. A-optimality is questionable.

  2. The estimation of u(x)u(x) seems quite challenging. It involves a potential high-dimensional conditional expectation. How much sample should we allocate for two stages needs more discussion.

  3. The algorithm involves only one-time iteration between π\pi and λ\lambda. It is uncertain whether multiple iterations would be helpful.

  4. Theoretical results in Section 3.4 assume the convergence of the tuning parameters to the optimal parameters.

Questions for authors
  1. The mean estimation on page 4 seems inaccurate.

  2. How should people estimate u(x)u(x)? And what if the estimation is mis-specified or biased?

  3. The algorithm involves only one-time iteration between π\pi and λ\lambda. Will multiple iterations would be more helpful?

  4. The assumption in Theorem 3 that P(η^≠η⋆)→0P(\hat{\eta} \neq \eta^{\star}) \to 0 and λ^=λ⋆+op(1)\hat{\lambda} = \lambda^{\star} + o_{p}(1) are strong given the complexity of the proposed setup. Please provide further discussion and justification.

  5. Authors should compare with the method in "Surrogate-Powered Inference: Regularization and Adaptivity" to highlight the need of D-optimality.

4 comments
AuthorsOpenReview ↗

Response to Reviewer ALYQ (1/2)

We thank the reviewer for finding the paper well written and the theory thorough, and we clarify each point below.

W1: Why D-optimality?

We agree that D-optimality is not universally better than A-optimality. Our main contribution is the design problem, not the determinant. OptAPI chooses the sampling rule and the power together. Under the D-criterion, the optimal rule depends on the covariance that it induces (Theorem 1). We solve this implicit problem with a budget-constrained MM algorithm (Proposition C.1), and we then tune λ\lambda on the acquired labels.

Our target is joint inference on the full vector θ⋆\theta^\star, where D-optimality is a good choice.

  1. Joint uncertainty. The volume of the Wald region is proportional to det⁡(Σ)1/2\det(\mathbf\Sigma)^{1/2}, while tr⁡(Σ)\operatorname{tr}(\mathbf\Sigma) only sums the marginal variances. The D-optimal design also does not change under nonsingular linear reparameterizations of θ\theta.
  2. It does well when A-optimality is the better choice. For p=1p=1, the two criteria coincide. For mean estimation, the two rules coincide when the covariance is proportional to the identity. On the census data (p=2p=2, no scale disparity), the A-optimal methods have only a slightly smaller MSE, and OptAPI keeps the largest neffn_{\rm eff} (Appendix G.4).
  3. It recovers A-optimality when needed. If we replace V−1\mathbf V^{-1} with Hθ⋆−2\mathbf H_{\theta^\star}^{-2}, we get the A-optimal rule (Remark 1). If only C⊤θ\mathbf C^\top\theta is of interest, the MM step extends to log⁡det⁡(C⊤ΣC)\log\det(\mathbf C^\top\mathbf\Sigma\mathbf C). For C=ej,\mathbf C=e_j, this gives the A-optimal rule for coefficient jj. We will add this remark.

W2 & Q2: Estimation of u(x)u(x) and the pilot size nln_l

We appreciate the insightful comment. u(x)u(x) is a scalar regression, and a poor fit affects efficiency but not validity.

  1. Estimation. We estimate u(x)u(x) as active inference does (Zrnic and Candès, 2024). We regress a residual on the features over the pilot. For linear models and GLMs, the score factorizes, so u(x)2=v(x)x⊤A−1xu(x)^2=v(x) x^\top\mathbf A^{-1}x, where A\mathbf A is the pilot estimate of the score covariance under uniform sampling. The leverage term x⊤A−1xx^\top\mathbf A^{-1}x is exact. We learn only the scalar v(x)v(x), the conditional mean of the squared score residual, with a quasi-Poisson regression and a log link (Appendix G.4). For a general M-estimator, the target is the scalar ∥Δλ∥A−12\Vert \Delta_\lambda\Vert^2_{\mathbf A^{-1}}.
  2. Misspecification. We agree that a poor uu reduces efficiency. The uniform floor keeps each probability above τnb/n\tau n_b/n. Thus, the weights stay bounded, and Theorems 3–4 hold for any fitted uu. On the census data, a linear fit for vv was about 33% less efficient than uniform sampling, and the log link removes this problem.
  3. Allocation. The pilot is existing labeled data, so it costs no label budget. Stage II tunes λ\lambda on the actively collected labels, so the only choice is the pilot size. Appendix G.7 varies the pilot size with n=9,500n=9{,}500 and nb=4,750n_b=4{,}750 fixed (Table A3, below). Even a pilot of 2% of the budget reaches 99.3% of an oracle that fits vv with all labels. Thus, empirically, a small pilot is sufficient, and we will state this in Section 3.3.
nln_l952384759502,1382,375
neffn_{\rm eff} of OptAPI6,3036,3536,3346,3856,3806,357
% of oracle99.399.899.999.899.999.8

W3 & Q3: Multiple iterations between π\pi and λ\lambda

We agree that more iterations can help when new gold labels arrive. With a fixed pilot, one pass is enough.

  1. Why one pass is enough. The initial power λ0\lambda_0 only sets the design, and Stage II tunes λ\lambda again on the nbn_b active labels (Section 3.3). The design is not sensitive to λ0\lambda_0: for λ0∈{0,0.25,0.5,0.75,1}\lambda_0\in\lbrace 0,0.25,0.5,0.75,1\rbrace, efficiency and coverage are almost the same (Figure A5). Thus, a new estimate of λ\lambda on the small pilot would change the design very little.
  2. When more passes help. A change to π\pi for the tuned λ\lambda needs a second wave of labels, as in multiwave designs such as SPI (Chen et al., 2025). We develop this extension in subsequent work.
AuthorsOpenReview ↗

Response to Reviewer ALYQ (2/2)

W4 & Q4: Conditions on η^\widehat\eta and λ^\widehat\lambda in Theorem 3

Both conditions are needed only for validity, which requires η^\widehat\eta and λ^\widehat\lambda to converge to fixed limits, conditional on the pilot. These limits do not have to be optimal values.

  • The condition P(η^≠η⋆)→0P(\widehat\eta\neq\eta^\star)\to0 comes from Zrnic and Candès (2024, Theorem 5.2), where η\eta is chosen from a discrete grid. Our η^\widehat\eta is continuous, so we will replace the condition with η^→η⋆\widehat\eta\to\eta^\star in probability. This holds when the empirical budget equation converges uniformly and has a unique root. We will change the proof accordingly.
  • The condition λ^=λ⋆+op(1)\widehat\lambda=\lambda^\star+o_p(1) is the same as in PPI++ (Angelopoulos et al., 2026, Theorem 1). For our log-determinant objective, V^(λ)\widehat{\mathbf V}(\lambda) is quadratic in λ\lambda and satisfies V^(λ)⪰V~⪰0\widehat{\mathbf V}(\lambda)\succeq\widetilde{\mathbf V}\succeq0. Thus, log⁡det⁡V^(λ)\log\det\widehat{\mathbf V}(\lambda) converges uniformly on [0,1][0,1], and λ^\widehat\lambda converges if the limit has a unique minimizer. We will state uniqueness as an assumption.

Q1: Mean estimation on Page 4

Thank you for this correction. With wi=ξi/π(Xi)w_i=\xi_i/\pi(X_i), the solution of the score Eq. (3.2) for the squared loss is θ^λπ=∑i{λf(Xi)+wi[Yi−λf(Xi)]}∑i{λ+(1−λ)wi}.\widehat\theta^\pi_\lambda=\frac{\sum_i\lbrace \lambda f(X_i)+w_i[Y_i-\lambda f(X_i)]\rbrace }{\sum_i\lbrace \lambda+(1-\lambda)w_i\rbrace }. The displayed formula with the 1/n1/n normalization is a Horvitz–Thompson form. It equals this solution only when λ=1\lambda=1, as in the active mean estimator of Zrnic and Candès (2024). We will correct the example.

Q5: Relation to SPI

Thank you for this reference. SPI also labels adaptively, but it differs from our proposed OptAPI.

  • Design. SPI uses an A-optimal rule based on the conditional norm of its augmented influence function (Chen et al., 2025, Eq. (13)–(14)). OptAPI uses a policy-dependent D-optimal rule that targets joint uncertainty.
  • Augmentation. SPI combines surrogate estimators through a regularized weighting matrix. OptAPI tunes a scalar power on a black-box prediction in the loss.
  • Acquisition. SPI updates its labeling rule across multiple waves. OptAPI uses one active batch, followed by power tuning.

A direct comparison with SPI would mix these three differences. Our A-optimal baselines isolate the design criterion: they use the same type of trace rule as SPI, with the same estimator, pilot, and budget as OptAPI. D-optimality matters most under scale disparity. In Section 4.1, the A-optimal baselines are up to 24% less efficient than the classical estimator, while OptAPI gains up to 157.7% in neffn_{\rm eff}. We will cite SPI and include this comparison to Section 2 in the revised paper.

We thank the reviewer again and are happy to clarify further if needed.

References

  • PPI++: Anastasios N. Angelopoulos, John C. Duchi, and Tijana Zrnic. PPI++: Efficient prediction-powered inference. The Annals of Applied Statistics, 20(3):2235–2252, 2026.

  • SPI: Jianmin Chen, Huiyuan Wang, Thomas Lumley, Xiaowu Dai, and Yong Chen. Surrogate-powered inference: Regularization and adaptivity. arXiv:2512.21826, 2025.

  • Active inference: Tijana Zrnic and Emmanuel J. Candès. Active statistical inference. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024.

I appreciate authors’ efforts to address my concerns; however, my fundamental concern is whether adopting a new criterion alone can justify sufficient novelty in this work. Given all considerations, I would retain my score.

AuthorsOpenReview ↗

Response to Reviewer ALYQ

Thank you for your time and for engaging with our responses. We understand your view on novelty. Your feedback has helped us sharpen how we frame the contribution. We are happy to provide more clarification and would be very grateful if you would reconsider the assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary for AC and Reviewers

Dear Area Chair and Reviewers,

Thank you for your careful reviews and constructive feedback. Your comments have helped us sharpen the methodological contribution, clarify the scope of our guarantees, and identify where the evaluation needs strengthening. We summarize our responses and planned revisions below.

  • Methodological contribution and criterion. On the shared novelty concern, we clarified that our contribution is the covariance-dependent design problem rather than D-optimality itself. In OptAPI, the sampling policy determines its own covariance weighting, which leads to an implicit characterization, budget-constrained MM updates, and a coupling with power tuning. We will sharpen this distinction in Section 1.2, clarify the relationship to SPI raised by ALYQ, and explain how the inferential objective (joint or marginal) determines the appropriate criterion.
  • Theoretical guarantees and their scope. Following ySZx and Ho6u, we distinguish the conditional first-order characterizations, MM descent, and the one-step mixed-policy bound from joint global optimality. Because the one-step guarantee did not cover the reported multi-step implementation, the revision will deploy the one-step policy and report the multi-step version as an ablation. It will also state explicit conditions for the data-dependent η^\widehat\eta and λ^\widehat\lambda, which ALYQ also requested, and add a joint Wald region corollary with its consistency requirements.
  • Empirical evidence. The housing and census applications provide evidence beyond the constructed scale-disparity example noted by ySZx. OptAPI attains favorable MSE on housing and the largest joint region efficiency on census, while some A-optimal competitors achieve smaller census MSE. This reflects a trade-off between joint volume and marginal accuracy, not universal dominance. We will add coordinate wise interval widths to this discussion.
  • Implementation and sensitivity. In response to implementation questions from all three reviewers, we clarified the uncertainty surrogate, the role of the pilot, initialization, and power tuning. The existing ablations provide sensitivity evidence for the settings studied rather than universal robustness guarantees. We will highlight these results and align the method description with the implementation.
  • Related work, computation, and presentation. Following Ho6u's suggestions, we will expand the related work on active testing and multiwave sampling. We will also report computational costs and state that OptAPI currently targets moderate-dimensional, batch settings. Finally, we will improve the figures and move elementary derivations to the appendix.

We are grateful for your time and careful consideration, and we welcome further questions and discussion.

Sincerely,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.