- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies how to allocate a limited measurement budget when each target individual can be observed through only one variable subset. It assumes that the target is an exponential tilt of a frozen reference generator. GAMD uses a pilot sample to estimate the shift, evaluates panel information through conditional scores, and chooses an allocation to minimize the estimated first-order KL risk. It then refines the shift estimate using the acquired partial observations. Under correct specification and sufficient computational accuracy, the method attains the oracle’s asymptotic risk constant. Experiments cover synthetic distributions and controlled and natural shifts in gassensor data.
Critical strengths
-
A useful connection between partial measurements and full-distribution loss. I like that panel value includes information about unobserved features through conditional dependence. The resulting cost-aware criterion is directly tied to the local KL loss, and Theorem 1 gives a precise oracle benchmark. This makes the allocation rule well motivated within the assumed shift family.
-
Careful handling of simulation noise. I find the cross-completion estimator a useful part of the method: it avoids treating independent completion noise as panel information. I also appreciate the distinction between unbiased simulation noise and score bias in Appendix G. The synthetic risk approaching the oracle reference supports the proposed mechanism in a controlled setting.
Critical weaknesses
-
The estimation target can change under misspecification. I am concerned that partial-data likelihood can converge to an allocation-dependent parameter rather than the full-law KL projection . Greater Fisher information may therefore reduce variance without resolving the bias in distribution recovery. Appendix I.3 acknowledges this issue, but its effect is not quantified in the natural-drift experiments, where correct specification is not guaranteed. I would like to see whether this bias is small across the compared allocations.
-
Finite simulation may change the preferred allocation. GAMD optimizes , while Appendix G.4 shows that finite completions introduce panel-dependent score variance . This can change the allocation that minimizes final KL risk. I am not yet convinced that 16 or 32 completions make this effect negligible in the real-data experiments. A sensitivity analysis with larger simulation budgets, or a comparison with the finite-completion criterion, would help establish whether the ideal objective remains appropriate.
Questions for authors
-
For natural drift, how far is the population root of the acquired-panel score equation from the full-data projection ? Does this discrepancy change substantially across allocations?
-
How sensitive are the learned allocation and final loss to the completion count and the full-sample pool used for score centering? Does the ideal criterion select a similar allocation to the finite-completion criterion?
-
What explains the nearly flat controlled-shift loss in Table 1 as the budget increases? A complete-data estimation control would help distinguish model or computation error from limited target information.
4 comments
Author Rebuttal (2/2)
W2 / Q2: Sensitivity to finite conditional completions
We agree that finite conditional completions introduce panel-dependent score variance, potentially affecting both the final estimator and the preferred measurement allocation. We conduct two experiments to assess this effect.
Sensitivity of final recovery to simulation budgets. We fix the target data, pilot, initialization, and learned GAMD allocation, and vary the final-update simulation pool size $N_{\mathrm{upd}}\in\{8000,32000\}$ and the number of conditional completions per MAIN observation $L_{\mathrm{upd}}\in\{16,128\}$. Here, $N_{\mathrm{upd}}$ controls the shared feature-mean estimate used for score centering, while $L_{\mathrm{upd}}$ controls the Monte Carlo approximation of each conditional score. The mean recovery losses are:
| Target | Budget | (8000, 16) | (8000, 128) | (32000, 16) | (32000, 128) |
|---|---|---|---|---|---|
| Batch 7 | 800 | 0.2785 | 0.2785 | 0.2872 | 0.2864 |
| Batch 7 | 1600 | 0.2264 | 0.2264 | 0.2308 | 0.2297 |
| Batch 7 | 3200 | 0.2074 | 0.2074 | 0.2115 | 0.2115 |
| Batches 8–9 | 800 | 0.0704 | 0.0705 | 0.0705 | 0.0706 |
| Batches 8–9 | 1600 | 0.0398 | 0.0398 | 0.0398 | 0.0398 |
| Batches 8–9 | 3200 | 0.0325 | 0.0327 | 0.0325 | 0.0325 |
Each column denotes $(N_{\mathrm{upd}},L_{\mathrm{upd}})$. Increasing $L_{\mathrm{upd}}$ from 16 to 128 while fixing $N_{\mathrm{upd}}=8000$ changes the recovery loss by less than 0.7%. Increasing $N_{\mathrm{upd}}$ from 8000 to 32000 while fixing $L_{\mathrm{upd}}=16$ changes it by less than 3.2%. Thus, final recovery is relatively insensitive to these simulation-budget changes.
Ideal versus finite-completion allocation. We further examine whether accounting for finite-completion variance changes GAMD's selected allocation. The ideal criterion is the original design objective in Eq. (5), based on exact conditional-score information. The finite-completion-adjusted criterion additionally accounts for the variance introduced by finite conditional completions, characterized in Appendix G.4, and the shared finite-pool score-centering error analyzed in Appendix G.1. We use this adjusted objective as a sensitivity comparison, optimizing both criteria under identical candidate panels and coverage constraints.
We consider low- and high-computation settings $(N_{\mathrm{upd}},L_{\mathrm{upd}})=(8000,16)$ and $(32000,128)$, respectively. We measure the total variation (TV) distance between the allocations selected by the two criteria and evaluate the recovery gain of the adjusted allocation:
| Target | Budget | TV (low) | TV (high) | Finite-design gain |
|---|---|---|---|---|
| Batch 7 | 3200 | 0.0576 | 0.0253 | +0.03038% |
| Batches 8–9 | 800 | 0.0183 | 0.0087 | +0.00808% |
| Batches 8–9 | 3200 | 0.0109 | 0.0000 | -0.00233% |
The small TV distances indicate that the ideal and finite-completion-adjusted criteria select similar allocations in the tested settings. Moreover, the independent evaluation shows negligible additional recovery benefit from explicitly accounting for finite-completion variance.
Overall, these results suggest that the ideal criterion remains a reliable design objective at the tested simulation budgets: finite-completion noise has limited impact on final recovery and does not substantially change the preferred allocation.
Thank you again for your thoughtful questions and constructive suggestions. We hope these additional analyses and clarifications address your concerns. We would be grateful if you would consider revising your rating in light of these results.
Thank you for the additional experiments. The results quantify allocation-dependent bias, show limited sensitivity to finite completions in the tested settings, and help explain the flat trend in Table 1. I would welcome the new results in the manuscript, along with a brief explanation of the gap between the finite-budget and population-level losses for batches 8–9. My overall rating remains unchanged.
Thank you for the suggestion. We have incorporated the additional experimental results into the revised manuscript. For the gap between the finite-budget and population-point losses for batches 8–9, the two quantities evaluate different estimation targets. The finite-budget loss measures the actual estimator's recovery error, whereas the population-point loss measures the discrepancy of the allocation-dependent population partial-score solution from the full-data KL projection. Under model misspecification, the partial-data objective need not have the same population optimum as the full-data objective.
Author Rebuttal (1/2)
Thank you for the careful analysis and constructive suggestions. We have conducted additional experiments and analyses to address your concerns.
Q3: Controlled-shift performance
Upon further investigation, we found that the original VAEAC generator was insufficiently accurate. We therefore replaced it with a residual-aware principal-component kernel density estimator (PC-KDE). Specifically, we represent the 128-dimensional sensor data using 16 principal-component scores and 112 residual coordinates (7 per sensor). The reference density is modeled as a 512-component Gaussian kernel mixture in the score space, with component-dependent Gaussian residual means and shared residual covariance for each sensor.
The generator improvement is substantial: the standardized MSE for estimating missing shift features $\phi$ decreases from 0.253677 to 0.062178. We repeated the controlled-shift experiments using this generator with 30 replications. Following your suggestion, we also added a complete-data control, which observes all 128 sensor measurements for each target individual and fits the same exponential-tilt family. The results report within-family KL recovery loss:
GAMD's loss decreases by 64.96% as the budget increases from 400 to 3200, eliminating the nearly flat trend in the original results. The complete-data control achieves lower loss at every budget, as expected, while GAMD consistently outperforms all partial-measurement baselines.
W1 / Q1: Allocation-dependent estimation bias under natural drift
We agree that under model misspecification, the population partial-score solution may differ from the full-data KL projection and depend on the measurement allocation. To investigate this effect, we conducted additional analyses on six natural-drift settings using the improved reference generator.
Let $\beta^\dagger=\arg\min_{\beta\in\Theta}D_{\mathrm{KL}}(P|Q_\beta)$ denote the full-data KL projection, and define the target log-score (up to a constant) as
$$ R_P(\beta)=A(\beta)-\beta^\top\mathbb E_P
ϕ(X). $$
We report two complementary quantities:
Here, $\beta_p$ is the root of the allocation-weighted population partial-score equation characterized in Appendix I.3.
Finite-budget recovery. We first evaluate the actual estimators. Results are mean ± SE:
GAMD achieves the lowest mean recovery loss in all six settings.
Population-level allocation bias. We next evaluate $\Delta_p$ for each method, using its realized integer main-stage allocation:
GAMD also achieves the lowest population-point loss in all six settings. The nonzero $\Delta_p$ values confirm that measurement design cannot eliminate misspecification-induced bias. Nevertheless, GAMD achieves the lowest loss in both finite-budget recovery and population-level evaluation.