acceptodds
Under review as a conference paper at ICLR 2027

Generator-Aware Data Acquisition for Budgeted Distribution Recovery

Abstract

We study high-dimensional distribution recovery under a limited measurement budget. Target individuals are sampled from an unknown target distribution, and for each sampled individual only one costly subset of variables can be measured. A pretrained generator is available for a fully observed reference population, and the target is modeled as a low-dimensional exponential tilt of the reference law. We propose Generator-Aware Measurement Design (GAMD), which uses the generator to decide which variable subsets are most informative about how the target population differs from the reference. For each candidate subset, GAMD evaluates how much of the full target-shift signal can be recovered from the variables that would be observed; this quantity determines the Fisher information carried by that measurement. GAMD then combines these information values with measurement costs to allocate the budget and refines the target distribution from the acquired partial data. We prove that GAMD asymptotically matches the oracle design despite the unknown target shift and establish finite-budget guarantees. Synthetic and real-data experiments validate the proposed mechanism and demonstrate improved recovery under constrained measurements.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Active data acquisition, generative modeling, optimal experimental design

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_YcgyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to allocate a limited measurement budget when each target individual can be observed through only one variable subset. It assumes that the target is an exponential tilt of a frozen reference generator. GAMD uses a pilot sample to estimate the shift, evaluates panel information through conditional scores, and chooses an allocation to minimize the estimated first-order KL risk. It then refines the shift estimate using the acquired partial observations. Under correct specification and sufficient computational accuracy, the method attains the oracle’s asymptotic risk constant. Experiments cover synthetic distributions and controlled and natural shifts in gassensor data.

Critical strengths
  1. A useful connection between partial measurements and full-distribution loss. I like that panel value includes information about unobserved features through conditional dependence. The resulting cost-aware criterion is directly tied to the local KL loss, and Theorem 1 gives a precise oracle benchmark. This makes the allocation rule well motivated within the assumed shift family.

  2. Careful handling of simulation noise. I find the cross-completion estimator a useful part of the method: it avoids treating independent completion noise as panel information. I also appreciate the distinction between unbiased simulation noise and score bias in Appendix G. The synthetic risk approaching the oracle reference supports the proposed mechanism in a controlled setting.

Critical weaknesses
  1. The estimation target can change under misspecification. I am concerned that partial-data likelihood can converge to an allocation-dependent parameter rather than the full-law KL projection β†\beta^\dagger. Greater Fisher information may therefore reduce variance without resolving the bias in distribution recovery. Appendix I.3 acknowledges this issue, but its effect is not quantified in the natural-drift experiments, where correct specification is not guaranteed. I would like to see whether this bias is small across the compared allocations.

  2. Finite simulation may change the preferred allocation. GAMD optimizes tr⁡(FM−1)\operatorname{tr}(FM^{-1}), while Appendix G.4 shows that finite completions introduce panel-dependent score variance (F−IS)/L(F-I_S)/L. This can change the allocation that minimizes final KL risk. I am not yet convinced that 16 or 32 completions make this effect negligible in the real-data experiments. A sensitivity analysis with larger simulation budgets, or a comparison with the finite-completion criterion, would help establish whether the ideal objective remains appropriate.

Questions for authors
  1. For natural drift, how far is the population root of the acquired-panel score equation from the full-data projection β†\beta^\dagger ? Does this discrepancy change substantially across allocations?

  2. How sensitive are the learned allocation and final loss to the completion count and the full-sample pool used for score centering? Does the ideal criterion select a similar allocation to the finite-completion criterion?

  3. What explains the nearly flat controlled-shift loss in Table 1 as the budget increases? A complete-data estimation control would help distinguish model or computation error from limited target information.

4 comments
AuthorsOpenReview ↗

Author Rebuttal (1/2)

Thank you for the careful analysis and constructive suggestions. We have conducted additional experiments and analyses to address your concerns.

Q3: Controlled-shift performance

Upon further investigation, we found that the original VAEAC generator was insufficiently accurate. We therefore replaced it with a residual-aware principal-component kernel density estimator (PC-KDE). Specifically, we represent the 128-dimensional sensor data using 16 principal-component scores and 112 residual coordinates (7 per sensor). The reference density is modeled as a 512-component Gaussian kernel mixture in the score space, with component-dependent Gaussian residual means and shared residual covariance for each sensor.

The generator improvement is substantial: the standardized MSE for estimating missing shift features $\phi$ decreases from 0.253677 to 0.062178. We repeated the controlled-shift experiments using this generator with 30 replications. Following your suggestion, we also added a complete-data control, which observes all 128 sensor measurements for each target individual and fits the same exponential-tilt family. The results report within-family KL recovery loss:

BudgetGAMDUniformA-OSQDDisc.Complete data
4000.0524 ± 0.00590.0834 ± 0.01680.1097 ± 0.03010.1778 ± 0.07640.0191 ± 0.0025
8000.0313 ± 0.00190.1108 ± 0.03920.0656 ± 0.01540.0741 ± 0.01900.0118 ± 0.0015
16000.0197 ± 0.00090.0439 ± 0.00780.0626 ± 0.01970.0668 ± 0.01900.0061 ± 0.0008
32000.0184 ± 0.00070.0219 ± 0.00170.0290 ± 0.00460.0262 ± 0.00440.0027 ± 0.0004

GAMD's loss decreases by 64.96% as the budget increases from 400 to 3200, eliminating the nearly flat trend in the original results. The complete-data control achieves lower loss at every budget, as expected, while GAMD consistently outperforms all partial-measurement baselines.

W1 / Q1: Allocation-dependent estimation bias under natural drift

We agree that under model misspecification, the population partial-score solution may differ from the full-data KL projection and depend on the measurement allocation. To investigate this effect, we conducted additional analyses on six natural-drift settings using the improved reference generator.

Let $\beta^\dagger=\arg\min_{\beta\in\Theta}D_{\mathrm{KL}}(P|Q_\beta)$ denote the full-data KL projection, and define the target log-score (up to a constant) as

$$ R_P(\beta)=A(\beta)-\beta^\top\mathbb E_P

ϕ(X)\phi(X)

. $$

We report two complementary quantities:

  • Finite-budget recovery loss: $\mathcal L_B=R_P(\widehat\beta_B)-R_P(\beta^\dagger)$, evaluating the actual finite-budget estimator.
  • Population-point loss: $\Delta_p=R_P(\beta_p)-R_P(\beta^\dagger)$, evaluating the allocation-dependent population partial-score root.

Here, $\beta_p$ is the root of the allocation-weighted population partial-score equation characterized in Appendix I.3.

Finite-budget recovery. We first evaluate the actual estimators. Results are mean ± SE:

TargetBudgetGAMDUniformA-OSQDDisc.
Batch 78000.2785 ± 0.02610.3900 ± 0.03050.3630 ± 0.03010.4448 ± 0.0494
Batch 716000.2264 ± 0.02200.2617 ± 0.02540.2625 ± 0.02450.2743 ± 0.0311
Batch 732000.2074 ± 0.02020.2591 ± 0.02530.2560 ± 0.02710.2591 ± 0.0260
Batches 8–98000.0704 ± 0.01250.0798 ± 0.01400.0811 ± 0.01470.0762 ± 0.0119
Batches 8–916000.0398 ± 0.00860.0511 ± 0.00900.0500 ± 0.00940.0492 ± 0.0089
Batches 8–932000.0325 ± 0.00940.0429 ± 0.00980.0424 ± 0.00960.0409 ± 0.0080

GAMD achieves the lowest mean recovery loss in all six settings.

Population-level allocation bias. We next evaluate $\Delta_p$ for each method, using its realized integer main-stage allocation:

TargetBudgetGAMDUniformA-OSQDDisc.
Batch 78000.20830.42770.36380.7436
Batch 716000.23420.42850.36350.6407
Batch 732000.19720.42160.36430.6414
Batches 8–98000.25510.28800.33700.2576
Batches 8–916000.22350.28380.33670.2578
Batches 8–932000.24780.28410.33820.2637

GAMD also achieves the lowest population-point loss in all six settings. The nonzero $\Delta_p$ values confirm that measurement design cannot eliminate misspecification-induced bias. Nevertheless, GAMD achieves the lowest loss in both finite-budget recovery and population-level evaluation.

AuthorsOpenReview ↗

Author Rebuttal (2/2)

W2 / Q2: Sensitivity to finite conditional completions

We agree that finite conditional completions introduce panel-dependent score variance, potentially affecting both the final estimator and the preferred measurement allocation. We conduct two experiments to assess this effect.

Sensitivity of final recovery to simulation budgets. We fix the target data, pilot, initialization, and learned GAMD allocation, and vary the final-update simulation pool size $N_{\mathrm{upd}}\in\{8000,32000\}$ and the number of conditional completions per MAIN observation $L_{\mathrm{upd}}\in\{16,128\}$. Here, $N_{\mathrm{upd}}$ controls the shared feature-mean estimate used for score centering, while $L_{\mathrm{upd}}$ controls the Monte Carlo approximation of each conditional score. The mean recovery losses are:

TargetBudget(8000, 16)(8000, 128)(32000, 16)(32000, 128)
Batch 78000.27850.27850.28720.2864
Batch 716000.22640.22640.23080.2297
Batch 732000.20740.20740.21150.2115
Batches 8–98000.07040.07050.07050.0706
Batches 8–916000.03980.03980.03980.0398
Batches 8–932000.03250.03270.03250.0325

Each column denotes $(N_{\mathrm{upd}},L_{\mathrm{upd}})$. Increasing $L_{\mathrm{upd}}$ from 16 to 128 while fixing $N_{\mathrm{upd}}=8000$ changes the recovery loss by less than 0.7%. Increasing $N_{\mathrm{upd}}$ from 8000 to 32000 while fixing $L_{\mathrm{upd}}=16$ changes it by less than 3.2%. Thus, final recovery is relatively insensitive to these simulation-budget changes.

Ideal versus finite-completion allocation. We further examine whether accounting for finite-completion variance changes GAMD's selected allocation. The ideal criterion is the original design objective in Eq. (5), based on exact conditional-score information. The finite-completion-adjusted criterion additionally accounts for the variance introduced by finite conditional completions, characterized in Appendix G.4, and the shared finite-pool score-centering error analyzed in Appendix G.1. We use this adjusted objective as a sensitivity comparison, optimizing both criteria under identical candidate panels and coverage constraints.

We consider low- and high-computation settings $(N_{\mathrm{upd}},L_{\mathrm{upd}})=(8000,16)$ and $(32000,128)$, respectively. We measure the total variation (TV) distance between the allocations selected by the two criteria and evaluate the recovery gain of the adjusted allocation:

TargetBudgetTV (low)TV (high)Finite-design gain
Batch 732000.05760.0253+0.03038%
Batches 8–98000.01830.0087+0.00808%
Batches 8–932000.01090.0000-0.00233%

The small TV distances indicate that the ideal and finite-completion-adjusted criteria select similar allocations in the tested settings. Moreover, the independent evaluation shows negligible additional recovery benefit from explicitly accounting for finite-completion variance.

Overall, these results suggest that the ideal criterion remains a reliable design objective at the tested simulation budgets: finite-completion noise has limited impact on final recovery and does not substantially change the preferred allocation.

Thank you again for your thoughtful questions and constructive suggestions. We hope these additional analyses and clarifications address your concerns. We would be grateful if you would consider revising your rating in light of these results.

Thank you for the additional experiments. The results quantify allocation-dependent bias, show limited sensitivity to finite completions in the tested settings, and help explain the flat trend in Table 1. I would welcome the new results in the manuscript, along with a brief explanation of the gap between the finite-budget and population-level losses for batches 8–9. My overall rating remains unchanged.

Thank you for the suggestion. We have incorporated the additional experimental results into the revised manuscript. For the gap between the finite-budget and population-point losses for batches 8–9, the two quantities evaluate different estimation targets. The finite-budget loss measures the actual estimator's recovery error, whereas the population-point loss measures the discrepancy of the allocation-dependent population partial-score solution from the full-data KL projection. Under model misspecification, the partial-data objective need not have the same population optimum as the full-data objective.

Reviewer_2LHnOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a generator-aware measurement design algorithm for recovering a target distribution from partial observations under a limited budget, with theoretical guarantees and synthetic and real-data experiments.

Critical strengths

The authors address an interesting problem for which they provide both theoretical guarantees and experimental validation.

Critical weaknesses

The approach relies on a pre-trained reference generator and low-dimensional exponential-tilt model, which may limit its applicability when these assumptions are not satisfied.

Questions for authors

Could the authors clarify how sensitive the proposed method is to inaccuracies in the generator or misspecification of the target distribution?

2 comments
AuthorsOpenReview ↗

Author Rebuttal

Motivation and theoretical scope

Exponential tilting is a well-established framework in various applications, including case-control studies [1], entropy balancing for causal inference [2], and related KL-based distributional control of pretrained language models [3]. It also has a natural information-theoretic foundation [4], making it a principled and practically relevant model class. Proposition 5 (Appendix F.1) shows that unrestricted joint recovery from partial measurements is generally unidentifiable. Exponential tilting makes structured recovery tractable while preserving complex dependencies in the reference. GAMD's core contribution is to exploit these dependencies for budget-optimal measurement design targeting full-distribution KL recovery.

Under misspecification, Proposition 17 (Appendix I.3) shows that recovery can depend on measurement allocation. We therefore conduct additional experiments to evaluate GAMD under both target-model and reference-generator misspecification.

Real-data experiments: natural temporal drift

Our original submission already examined natural temporal drift, using later Gas Sensor batches directly as targets without imposed exponential tilting. We have updated this existing experiment after improving the original reference generator to a residual-aware PC-KDE: 16 principal-component scores and 112 residual coordinates for 128-dimensional observations, modeled by a 512-component Gaussian kernel mixture with component-dependent residual means. We report mean excess log-score against the full-data projection in a common fitted family.

TargetBudgetGAMDUniformA-OSQDDisc.
Batch 78000.27850.39000.36300.4448
Batch 716000.22640.26170.26250.2743
Batch 732000.20740.25910.25600.2591
Batches 8–98000.07040.07980.08110.0762
Batches 8–916000.03980.05110.05000.0492
Batches 8–932000.03250.04290.04240.0409

GAMD has the lowest mean loss in all six natural-shift settings.

Controlled misspecification simulations.

Consider six binary variables $x_i\in\{-1,+1\}$, $Q_0$ contains all 15 pairwise interactions:

$$ Q_0(x)=\frac{1}{Z_0}\exp\left\{\sum_{i=1}^6 h_ix_i+\sum_{1\le i<j\le6}J_{ij}x_ix_j\right\}. $$

We draw and fix $h_i\sim\mathcal N(0,0.12^2)$ and $J_{ij}\sim\mathcal N(0,0.32^2)$. The fitted tilt uses $\phi(x)=(x_1,\ldots,x_6,x_1x_2,x_3x_4,x_5x_6)^\top$: six main effects and only three adjustable pairwise interactions. The nominal target is $P_0(x)\propto Q_0(x)\exp{\beta_\star^\top\phi(x)}$. We form three perturbations $\psi_d$, $d\in\{0,1,2\}$, from Gaussian-weighted combinations of the 12 pairwise terms omitted from $\phi$ and all 20 three-way interactions. Regressing out the constant and nine fitted features under $P_0$, then standardizing the residuals, produces unit-variance changes outside the fitted feature space. We test: 1. Target misspecification: The true target includes extra interactions:

$$P_{\delta,d}(x)\propto Q_0(x)\exp\{\beta_\star^\top\phi(x)+\delta\psi_d(x)\}.$$

  1. Generator misspecification: The target is right, the algorithm uses an inaccurate reference:

$$Q_{0,\delta,d}(x)\propto Q_0(x)\exp\{\delta\psi_d(x)\}.$$

At $\delta=1$, we evaluate all 15 two-variable panels, budget $B=400$, and 12 replications per direction.

MisspecificationDirectionGAMDUniformA-OSQDDisc.
Target00.03570.08851.48090.0462
Target10.05480.49201.34580.0620
Target20.03730.52961.71710.0955
Generator00.03900.38231.07960.0524
Generator10.04780.47941.42960.0685
Generator20.05810.46491.31290.1675

These experiments show that GAMD is still better than the baselines under the evaluated misspecification settings.

Thank you again for your supportive assessment and suggestions. We hope these results clarify our work and strengthen your confidence in its evaluation.

References:

[1] Prentice, R. L., & Pyke, R. (1979). Logistic disease incidence models and case-control studies. Biometrika, 66(3), 403-411.
[2] Hainmueller, J. (2012). Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies. Political analysis, 20(1), 25-46.
[3] Khalifa, M., Elsahar, H., & Dymetman, M. (2020). A distributional approach to controlled text generation. arXiv preprint arXiv:2012.11635.
[4] Csiszár, I., & Shields, P. C. (2004). Information theory and statistics: A tutorial. Foundations and Trends® in Communications and Information Theory, 1(4), 417-528.

We thank the authors for their response. My overall rating and confidence remain unchanged.

Reviewer_Tzs4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors consider a class of distributions arising described by an exponential-tilt model and introduce a frozen generator based algorithm to estimate the distribution under budgeted measurements. Theoretical analysis on the algorithm convergence is provided with experimental results showing its performance.

Critical strengths

The authors provide asymptotic and non-asymptotic theoretical justifications of the algorithms. Experimental results also show that under this specific exponential tilted model, the algorithm performs well.

Critical weaknesses
  1. The main weakness of the paper is lacking motivations or proof of significance of the exponential tilt model. For instance, since the exponential tilt model reduces distribution estimation to estimating the parameter \beta, I would think this is a special case of previous literature treating budgeted distribution estimation problem. Thus, it is unclear from the paper why this specific formulation is more critical or more technically challenging compared to previous literature.

  2. Another weakness lies in the experiment section, where it is hard to argue if the proposed method is generally practical and useful. i) The authors compare methods assuming the ground truth oracle distribution is also exponential tilt, which is restrictive and for most practical purposes might not be true. ii) Per my limited research, the "A-OSQD" paper is considering a different formulation of multi-variant normal or multi-variant log normal distributions which are different from the exponential tilt model here. Having A-OSQD confined to the specific model considered by the author here and measure the gain within this specific model seems to be not a fair comparison. iii) The uniform SQD seems to be a randomized allocation, which is expected to be weak.

Questions for authors
  1. Why is exponential tilt model an important model to consider, any previous literature or SoTA methods that work under this assumption with practical significance? Furthermore, I think even the multi-variant normal (or log-normal) distribution and assuming the underlying oracle distribution this way is considerably restrictive and arguably not practical. Any motivation here?

  2. How well the algorithm performs practically if not assuming the underlying distribution is exponentially tilt? For example, one of the main significance of distribution estimation is to generate samples that resembles the oracle distribution, without assuming what the underlying distribution model is. Do we have experimental performance analysis without making assumptions the underlying distributions?

2 comments
AuthorsOpenReview ↗

Author Rebuttal (1/2)

Thank you for the constructive comments. We address your concerns below.

W1/Q1: Motivation and significance of exponential tilt

Exponential tilt models how the target changes relative to a reference distribution; it doesn’t impose a Gaussian or log-normal form on either distribution:

$$ \frac{dQ_\beta}{dQ_0}(x) = \exp\{\beta^\top\phi(x)-A(\beta)\}. $$

The reference $Q_0$ can retain complex, multimodal, and nonlinear dependencies; the feature map $\phi$ restricts the directions of change, not the complexity of the joint distribution.

Our problem is related to budgeted measurement design, including A-OSQD, but differs in its objective. Existing methods often estimate specified statistics, such as population means; we seek to recover a full joint distribution from partial observations. This is fundamentally more challenging: Proposition 5 (Appendix F.1) shows that unrestricted joint recovery is generally unidentifiable from the admissible panels. This motivates structural assumptions. Exponential tilt provides one such structure while retaining the complexity of the reference distribution. The technical challenge is to exploit nonlinear conditional dependencies in the generator to evaluate panels for full-distribution recovery.

This formulation is motivated by practical settings where a reference population is available but its composition or feature statistics change. It has been widely used in practical applications, including case-control studies [1], entropy balancing for causal inference [2], and controlled text generation with pretrained language models [3]. Exponential tilt also has a principled information-theoretic interpretation. Suppose we know certain feature statistics of the target population but not its full distribution. Among all distributions satisfying these constraints, we can choose the one that differs least from the reference distribution:

$$ \min_{Q\ll Q_0} D_{\mathrm{KL}}(Q|Q_0) $$

subject to

$$ \mathbb{E}_Q

ϕ(X)\phi(X)

= \mu. $$

Under suitable conditions, the solution has exactly the exponential tilt form. Intuitively, it preserves as much of the reference distribution as possible while satisfying the specified feature constraints [4]. Together with its broad practical applications, this principled formulation makes exponential tilting a natural and meaningful model class to study. We do not claim that arbitrary shifts satisfy this assumption. Our contribution is generator-aware, cost-constrained measurement design linked to full-distribution KL risk, with theoretical guarantees. We will clarify the motivation, contributions, and limitations in the revision.

References:

[1] Prentice, R. L., & Pyke, R. (1979). Logistic disease incidence models and case-control studies. Biometrika, 66(3), 403-411.

[2] Hainmueller, J. (2012). Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies. Political analysis, 20(1), 25-46.

[3] Khalifa, M., Elsahar, H., & Dymetman, M. (2020). A distributional approach to controlled text generation. arXiv preprint arXiv:2012.11635.

[4] Csiszár, I., & Shields, P. C. (2004). Information theory and statistics: A tutorial. Foundations and Trends® in Communications and Information Theory, 1(4), 417-528.

W2/Q2: Experimental validation and baselines

About baselines. A-OSQD and Uniform SQD are meaningful, complementary comparisons. We adapt A-OSQD's covariance-based Gaussian mean-estimation allocation rule to our setting, holding the downstream recovery procedure fixed across methods. This evaluates a classical covariance-based acquisition rule. Uniform SQD provides balanced random coverage without pilot-based optimization, giving a natural non-adaptive reference for measuring the benefit of information-driven design. Moreover, the original Disc. baseline is also task-relevant: it directly learns conditional scores with a neural network and optimizes the same recovery-oriented criterion.

We nevertheless agree that a more task-matched covariance-based comparison is valuable. We therefore introduce Linear-Score, inspired by A-optimal design. It uses the same pilot estimate, KL-oriented design objective, coverage constraints, and downstream estimator as GAMD, but replaces the generator-based conditional scores with their best linear predictions from observed variables. This linear approximation is characterized in Corollary 2 (Appendix F.2). Unlike the original A-OSQD formulation, Linear-Score does not assume a Gaussian or log-normal joint distribution. By keeping the remaining design and recovery procedures identical, this comparison isolates the benefit of nonlinear conditional information.

These baselines span balanced random allocation, classical covariance design, and task-matched linear and neural conditional-score alternatives, providing a reasonably comprehensive comparison.

AuthorsOpenReview ↗

Author Rebuttal (2/2)

We agree that evaluating GAMD beyond the exponential-tilt setting is important. We therefore conducted additional experiments.

Real-data experiments. Our original experiments included natural temporal drift, using subsequent Gas Sensor batches directly as targets without imposing exponential tilting. However, we found that the original reference generator was insufficiently accurate. We therefore replaced it with an improved residual-aware principal-component kernel density estimator (PC-KDE). Specifically, we represent the 128-dimensional sensor data using 16 principal-component scores and 112 residual coordinates (7 per sensor). The reference density uses a 512-component Gaussian kernel mixture in the score space, with component-dependent Gaussian residual means and shared residual covariance for each sensor. We then repeated the experiments using this improved generator.

We evaluated natural temporal drift using batch 7 and batches 8--9 directly as targets, without constructing them through exponential reweighting. Recovery is measured by excess log-score relative to the full-data projection within a common reference-based model family.

TargetBudgetGAMDUniformA-OSQDDisc.Linear-Score
Batch 78000.27850.39000.36300.44480.3253
Batch 716000.22640.26170.26250.27430.2647
Batch 732000.20740.25910.25600.25910.2492
Batches 8--98000.07040.07980.08110.07620.0744
Batches 8--916000.03980.05110.05000.04920.0404
Batches 8--932000.03250.04290.04240.04090.0339

GAMD achieves the lowest mean loss in all six settings.

Misspecification simulations. We consider six binary variables, $x\in\{-1,+1\}$, with all 15 two-variable measurement panels. The reference distribution $Q_0$ is an Ising model containing all 15 pairwise interactions:

$$ Q_0(x)=\frac{1}{Z_0} \exp\left\{ \sum_{i=1}^{6}h_ix_i+ \sum_{1\leq i<j\leq 6}J_{ij}x_ix_j \right\}. $$

The coefficients are sampled as $h_i\sim\mathcal{N}(0,0.12^2)$ and $J_{ij}\sim\mathcal{N}(0,0.32^2)$, then fixed. The fitted exponential-tilt model uses nine features: $ \phi(x)=(x_1,\ldots,x_6,x_1x_2,x_3x_4,x_5x_6)^\top. $ Thus, although the reference captures all pairwise dependencies, the fitted model can only adjust six main effects and three selected pairwise interactions. We first construct a nominal target $P_0(x)\propto Q_0(x)\exp\{\beta_\star^\top\phi(x)\}$.

To create misspecification, we introduce changes that cannot be represented by adjusting the nine fitted features. Specifically, we randomly combine the 12 pairwise interactions excluded from $\phi$ and all 20 three-way interactions, generating three perturbation functions $g_d(x)$, $d\in\{0,1,2\}$, using independent standard Gaussian coefficients. We then remove any component explained by the fitted features (via weighted regression under $P_0$) and normalize each residual to unit variance, obtaining $\psi_d(x)$. This ensures that the perturbations introduce changes outside the fitted feature space.

We use the same perturbations to study two distinct types of misspecification: 1. Target misspecification: The reference $Q_0$ is correct, but the true target contains additional interactions that the fitted model cannot represent: $$ P_{\delta,d}(x)\propto Q_0(x) \exp\{\beta_\star^\top\phi(x)+\delta\psi_d(x)\}. $$ 2. Generator misspecification: The target is correct, but the algorithm uses an inaccurate reference distribution: $$ Q_{0,\delta,d}(x)\propto Q_0(x) \exp\{\delta\psi_d(x)\}. $$

Here, $\delta$ controls the misspecification strength, and $d$ indexes three distinct perturbation directions. We set $\delta=1$ and evaluate three directions with 12 replications each. We report finite-budget recovery at $B=400$, measured by excess log-score relative to the full-data KL projection within the corresponding fitted model family.

MisspecificationDirectionGAMDUniformA-OSQDDisc.Linear-Score
Target00.03570.08851.48090.04620.0363
Target10.05480.49201.34580.06200.0589
Target20.03730.52961.71710.09550.0403
Generator00.03900.38231.07960.05240.0537
Generator10.04780.47941.42960.06850.1119
Generator20.05810.46491.31290.16750.1814

GAMD still outperforms the baselines in these misspecified settings.

We hope these additional experiments and clarifications address your concerns. We would sincerely appreciate it if you could reconsider your rating in light of these results. Thank you again for your time and thoughtful feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.