acceptodds
Under review as a conference paper at ICLR 2027

Spectra: Exact Component Transport for Test-Time Prior Adaptation in Simulation-Based Inference

Abstract

Simulation-based inference (SBI) has become a powerful approach to Bayesian inference in complex scientific models whose likelihoods are difficult or impossible to evaluate. Amortized SBI learns reusable inference models from simulated data, enabling rapid posterior inference for new observations, and modern generative models have made these models increasingly expressive. However, this reuse is limited to the prior distribution chosen during training, whereas scientific analyses often need revised priors as knowledge accumulates or alternative assumptions are tested. We introduce Spectra, a test-time adaptation method for diffusion-based SBI. Spectra uses an exact score-transport identity to obtain the adapted score from a frozen diffusion model in closed form for structured prior changes, without additional simulation or training. Across six SBI benchmarks, Spectra achieves accurate adaptation under strong prior shifts at low online sampling cost. This enables pretrained SBI models to incorporate updated prior information at test time.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Simulation-based inference, Prior adaptation, Test-time adaptation, Amortized inference, Diffusion models, Neural posterior estimation, Score-based generative models

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_fF2ROpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a new method for inference-time prior adaptation of diffusion models for simulation-based inference. Prior work has achieved this with approximate corrections to the score (assuming Gaussian p(x0∣xt)p(x_0 | x_t) structure). This paper shows that for a class of prior ratio changes (isotropic Gaussian-like functions), the corrections can be obtained exactly. They extend this to ratios that can be represented by positively weighted sums of such functions by sampling separately with each function in proportion to weights that are estimated via sampling. The method improves over previous methods on a wide variety of benchmarks, with significant speedups over existing diffusion inference-time adaptation methods.

Critical strengths
  • The paper is well written, and showcases it's core contributions clearly.
  • The method is a natural improvement over prior approaches (PriorGuide), and improve over it significantly, especially in runtime and with narrow target priors. The exact single-factor transport and component-weight estimation for mixture prior ratios are clever and conceptually clean tricks, although the single-factor transport is limited to isotropic ratio changes.
Critical weaknesses

It would be appropriate to include citations to some recent papers that have noted mathematically similar results as the "exact single-factor transport" in this paper:

[1] Moitra, Ankur, Andrej Risteski, and Dhruv Rohatgi. "Steering diffusion models with quadratic rewards: a fine-grained analysis.", 2026

[2] McDonald, Curtis. "Technical Note on Relating Scores of Tilted Distributions.", 2026

Questions for authors
  • How much upfront cost does the component weight estimation take (Eq. 16)? I would also like to see analysis on how the variance of the estimator changes with increasing problem dimensionality, given that importance sampling can become more difficult in that situation.
  • Do you think that [1] or [2] could suggest interesting extensions of the method in this work?
4 comments
AuthorsOpenReview ↗

Response to Reviewer fF2R (1/2)

(1/2)

We thank the reviewer for the positive assessment and for pointing out these relevant connections and practical questions.

Related work and possible extensions

We agree that the mathematical overlap with Moitra et al. [1] and McDonald [2] should be acknowledged explicitly. We have added both references in Section 3.3 and Appendix A.

Specifically, Moitra et al. [1] establish the exact score identity for linear exponential tilts, corresponding to the κ=0\kappa=0 case of our formulation. McDonald [2] extends this identity to isotropic exponential–quadratic tilts with nonnegative κ\kappa. After converting between variance-preserving and variance-exploding Gaussian-noising parameterizations, the latter identity is equivalent to the corresponding case of our single-factor score identity. Spectra applies this identity to prior adaptation of frozen amortized posteriors, extends it to κ<0\kappa<0 and positive mixtures, and characterizes the factors that admit exact transport (Theorem 2).

Regarding possible extensions, the Hubbard–Stratonovich construction of Moitra et al. [1] writes a low-rank positive-semidefinite quadratic tilt as a continuous mixture of linear tilts. Since each linear tilt is transportable, this suggests extending Spectra beyond the isotropic single-factor family. Appendix I now gives a closely related construction for target covariances that exceed an isotropic covariance along a few directions, whose number of components grows with the number of such directions and not with the dimension, and Appendix N.3 evaluates it in 10 and 20 dimensions. Their hardness result for rank-one negative-semidefinite tilts, which tighten the distribution along a single direction, also shows that the complementary case is intractable in the worst case when the tightening is very strong. The main practical challenges are discretizing the continuous mixture and reliably estimating its observation-specific component weights. We have noted this direction in Appendix A.

McDonald's result [2] also suggests extending the implementation to diffusion-SBI models trained with variance-preserving Gaussian noising.

Upfront cost of component-weight estimation (Eq. 16 → 17)

The path-space estimator is an upfront, observation-specific computation. Our practical configuration uses 4,096 proposal trajectories per component and estimator seed, three estimator seeds, 400 reverse time points, and no Langevin corrector. Each transition requires two frozen-score evaluations, amounting to approximately 19.6 million trajectory-wise score evaluations for K=2K=2. For a fixed per-component budget, the weight-estimation cost scales linearly with KK.

The table reports measured times in seconds for K=2K=2 on one A100 GPU, excluding compilation, as medians over the two prior–observation pairs of Appendix M.2. Online sampling generates 1,000 posterior samples using (N,L)=(100,0)(N,L)=(100,0). PriorGuide refers to the benchmarked implementation and includes its per-observation precomputation.

TaskWeights (Eq. 17)OnlineSpectra-PS totalPriorGuide total
Two Moons4.700.064.760.78
SLCP17.980.1118.093.28
OUP41.370.2041.586.58
GL-10D29.520.1729.695.79
GL-20D65.510.3365.8416.57

Weight estimation dominates first-use latency, making Spectra-PS more expensive than PriorGuide for the first 1,000 mixture samples. However, the estimated normalizers can be reused for a fixed observation, pretrained model, and factor dictionary when only the mixture coefficients change, without repeating weight estimation. Appendix M.2 now also reports the setup–sampling trade-off against PriorGuide (VJP), including the number of repeated batches after which Spectra has lower cumulative cost.

AuthorsOpenReview ↗

Response to Reviewer fF2R (2/2)

(2/2)

Estimator variance versus dimensionality

We agree that dimension matters, and we have added Appendix H.3 to address it. It computes the error of the path-space estimator exactly in a Gaussian setting and compares it with runs of the implemented estimator. In short, the error grows with dimension when every parameter receives the same factor, more reverse steps offset this growth, and the implemented estimator follows the exact error wherever that error is moderate.

For one estimator seed with nPn_P proposal trajectories, the estimate Z^k\widehat Z_k is unbiased for the finite-chain normalizer Z~k\widetilde Z_k. When its second moment is finite,

Var⁡(Z^k)Z~k2=χ2(Pϕk(R)∥Qk(R))nP,\frac{\operatorname{Var}(\widehat Z_k)}{\widetilde Z_k^2} = \frac{\chi^2(P_{\phi_k}^{(R)}\Vert Q_k^{(R)})}{n_P},

where Qk(R)Q_k^{(R)} is the proposal path law and Pϕk(R)P_{\phi_k}^{(R)} is the base path law tilted by the factor at its endpoint. The square root of this ratio is the root-mean-square relative error of Z^k\widehat Z_k, which we call its relative error below.

We evaluate the divergence exactly for a base posterior N(0,0.05Id)\mathcal N(0,0.05I_d) and a zero-centered factor exp⁡(−κ∥θ∥2/2)\exp(-\kappa\Vert\theta\Vert^2/2), with exact scores and the paper's noise schedule. The values κ=40\kappa=40 and κ=240\kappa=240 shrink the posterior variance by factors of 3 and 13. The table gives the relative error for nP=4096n_P=4096 and the population ESS benchmark at a budget of 4,096 trajectories, 4096/(1+χ2)4096/(1+\chi^2).

κ\kappaddTime pointsχ2\chi^2Relative error (nP=4096n_P=4096)Population ESS (of 4,096)
40204000.2170.73%3,364
40644000.8771.5%2,182
240204001.541.9%1,612
2406440018.86.8%207
2406412801.501.9%1,637

Both path laws factorize over the dd coordinates, so 1+χ2=(1+χ12)d1+\chi^2=(1+\chi_1^2)^d, where χ12\chi_1^2 is the divergence for a single coordinate. With the same factor on every parameter, the error therefore grows with dd. More reverse steps reduce the mismatch per coordinate. With 20 time points per parameter, the error stays at 1.9% up to d=64d=64 (last row), at a proportional increase in score evaluations. The growth reflects the total shift between component and base posterior. If κ\kappa is lowered with dd so that the KL divergence of the component from the base posterior stays fixed, the error falls slightly, from 1.9% at d=20d=20 to 1.7% at d=64d=64. Direct estimation from base samples, the alternative estimator in the paper, has a far larger error at the same budget, which barely changes as reverse steps are added (Figure 5 in Appendix H.3).

We also ran the implemented estimator with 63 seeds in 18 configurations of this setting (Table 4 in Appendix H.3). Its error follows the same trends and agrees with the exact value to within 25% wherever the exact error is moderate, including the paper's 400 time points up to d=64d=64. When the exact error is large, the spread across seeds is usually smaller than the exact error and the empirical ESS is higher than the population benchmark, because most runs miss the rare trajectories with very large weights. The exact calculation therefore shows where a run's own diagnostics would mislead. For the component weights that Spectra uses, which average log⁡Z^k\log\widehat Z_k over three seeds, a two-component version with weights near 0.73 and 0.27 gives a mean weight error of at most 0.0091 for d≤64d\le64 with 400 time points.

These results assume exact scores and zero-centered factors. They do not cover learned scores or the discretization gap between Z~k\widetilde Z_k and ZkZ_k.

We hope these revisions address the reviewer’s questions and would be glad to clarify any remaining points.

I thank the authors for the detailed reply! I am happy with the answers, and maintain my score.

Thank you for your time and constructive feedback. We are glad our responses addressed your questions.

Reviewer_RapyOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Diffusion-SBI offers expressive conditional posterior estimation, treating conditional parameter posterior as the target distribution to be reconstructed from Gaussian noise through a diffusion model. Training this model is, however, expensive, making this technique impractical when training and test priors are different.

This work aims at overcoming this issue: "a test-time method for adapting pretrained diffusion-SBI models while leaving both the simulator and the trained model unchanged". Specifically, the authors contribute two main theoretical items: (i) a closed-form expression to correct model inference in cases with specific ratios between training and test priors, and (ii) a method to reduce more complex scenarios as a linear combination of cases as in (i). These two items are then thoroughly investigated and benchmarked on classical SBI problems (e.g., Two Moons, SLCP, OUP), reporting considerable performance improvements w.r.t. the state of the art (e.g., PriorGuide), especially for nonlinear tasks.

The paper concludes by summarizing the main results as well as highlighting some of the main limitations of the work.

Critical strengths

The paper is technical but remarkably well-written. The text is terse, even when a general picture of rather technical details is given (not easy). In particular, I appreciated:

  • the very nice overview of the different uses of SBI across different disciplines;
  • that the intended contributions are clearly stated;
  • that the formal derivations are clear, correct, and illustrated in logical order;
  • the high reproducibility grade. The proposed idea is, albeit maybe not universally applicable, valuable and nifty, and could be used as a starting point for further research in this direction.
Critical weaknesses

The only weakness I can see in this work is the practical applicability of the proposed method to real-world cases, where prior shifts do result in quadratic exponential density ratios. How frequent are these situations? Are they just edge scenarios, or do they represent the bulk of the diffusion-SBI use case? I do believe that the presented contribution is valuable, but, as a potential user, I'd appreciate it if the authors could briefly highlight when I can or cannot use their method.

Questions for authors

Major questions:

  • line 197:

    • could you give some intuition on what r(θ)∝ϕ(θ)r(\theta) \propto \phi(\theta) means practically? I understand the maths but not the practical implications;

    • from what I understand, if the above condition is true, then $q(\theta|x) = q^\phi (\theta|x)$, and, most importantly, every r(θ)r(\theta) identifies a unique πnew\pi_{new} (this seems to be related to the later statement on the cc canceling out). I struggle to understand what the more general framing r(θ)∝ϕ(θ)r(\theta) \propto \phi(\theta) obtains in this case;

  • section 3.2: the derivation is sound, but I think I could be of help to add just a bit more context about the general picture. I understand that "the key observation" is indeed that Gaussian and quadratic exponential combine well algebraically (they are "conjugate" in some sense), but it is a bit difficult to follow (or keep in mind) where this would enter the testing pipeline. I think just a couple more sentences reminding "where we are starting from" and "where we want to arrive" could be helpful, especially since this seems to be one of the main results of the paper, and really deserves clarity/ease of reading (minor: line 212 "the integral definition ..." I'm not sure if I understand which equation; I'd just put the eq. number);

  • line 229: how is eq. (12) related to (9) + (10)? In (10), $m_\tau(z) \propto z + \tau a$, but in (12) we have the additional scaling AτzA_\tau z - just one sentence briefly mentioning the assumptions could be very helpful;

  • section 3.6: it would be good to briefly mention how expensive the sampling step (16) is (just anticipating the results from Sec. 4.4 and M.2);

  • fig. 2: if I understand correctly, tt confidence intervals means using a Student's tt distribution to estimate the 95% CI. As far as I know, this relies on the normality of the underlying data, which is not present in either C2ST nor MMTV differences. It would be good to have an additional plot (in the supplements is also fine) where nonparametric statistics are used, e.g., median, IQ, or order-statistic CI for median ("inverting" the sign test - Conover, Practical Nonparametric Statistics). The 95% CI for the median with 10 observations would be $ [X_{(2)}, X_{(8)} ] $

Minor questions:

  • line 212: what would be the use case when κ<0\kappa < 0? Would it be suitable for priors defined on a compact domain?

  • does the uniqueness in thr 2 come from the fact that we are working with diffusion models, and therefore we need a "conjugate" of a Gaussian kernel? Would this change if the stochastic process underlying the model were not a Wiener process?

  • line 250: it would be very valuable if some necessary/sufficient conditions for the assumption $Z_k (x) < \infty $ were provided

3 comments
AuthorsOpenReview ↗

Response to Reviewer Rapy

We thank the reviewer for the positive assessment and for the detailed suggestions on Section 3, which have made the derivation and its role in inference clearer.

Practical applicability

Although we have not quantified their frequency across applications, the exact single-factor class includes standard operations in prior-sensitivity analysis: shifting the mean of a Gaussian prior, tightening or relaxing an isotropic Gaussian prior, and replacing a uniform prior with an isotropic Gaussian restricted to the same domain. These conditions apply in the model's parameter coordinates and impose no Gaussian assumption on the posterior. They can also hold for non-Gaussian priors on physical parameters, as illustrated by the fixed reparameterization in the diffusion MRI model of Section 4.5. As a brief guide for users:

  • Directly applicable, with no setup cost: mean shifts of Gaussian priors, isotropic tightening or relaxing of isotropic Gaussian priors, and isotropic Gaussian restrictions of uniform priors.
  • Applicable through mixtures: multimodal priors and anisotropic or correlated Gaussian changes, using positive mixtures of transported components with observation-specific weight estimation. The number of components can grow quickly in high dimensions, except for structured cases such as covariances that exceed an isotropic level along a few directions (Appendices I and N.3).
  • Limited by the backbone: adaptation relies on accurate scores at the transformed states and noise levels, as we now state after Proposition 1, and cannot extend beyond the support of the training prior.

The new Section 4.5 illustrates these cases on three scientific models, an SIRD epidemic, a MEG source-power model, and a diffusion MRI model, where Spectra adapts the frozen model as accurately as PriorGuide without new simulations or retraining.

Questions

Major questions

  1. Line 197 → 197. The constant cc in Eq. (8) already absorbs any positive scale, so the proportionality was unnecessary. We now write r(θ)=ϕ(θ)r(\theta)=\phi(\theta), which makes qϕq^\phi the target posterior. Theorem 2 and the later derivations are unchanged.
  2. Section 3.2 → lines 206–208. We now state where the derivation starts, the frozen model trained under πtr\pi_{\rm tr}, and where it leads, the score of the target posterior needed for reverse sampling. The noisy-posterior integral and its transformation are written out in Eq. (11), followed by the derivatives that give Eq. (12). This also replaces the indirect reference at line 212 with an equation number.
  3. Line 229 → 242–246. Section 3.3 now explains that it allows a general affine query Aτz+uτA_\tau z+u_\tau to characterize which factors admit transport with a scalar Gaussian variance. The characterization equation, formerly Eq. (12), is now Eq. (13). Theorem 2 then forces Aτ=I/DτA_\tau=I/D_\tau and uτ=ρτau_\tau=\rho_\tau a, which recovers Eqs. (9)–(10).
  4. Section 3.6 → lines 317–318. We now note that path-space weight estimation scales linearly with KK at a fixed per-component budget and can dominate first-use latency, with pointers to Section 4.4 and Appendix M.2 for measured timings.
  5. Fig. 2. Appendix K.4 adds Figure 6, which reports median paired gaps with conservative 95% distribution-free intervals, obtained by inverting the sign test across the same ten observations. It shows the same pattern as Figure 2.

Minor questions

  1. Line 212 → 216. A negative κ\kappa increases the variance of an isotropic Gaussian prior, for example to test sensitivity to prior concentration, as now noted in lines 203–204. On a shared compact support the factor is bounded, so the reweighted posterior is normalizable. Transport still requires 1+κτ>01+\kappa\tau>0 and transformed noise levels within the range supported by the pretrained model.
  2. Theorem 2. Yes, the conjugacy intuition is right. Theorem 2 requires the Gaussian kernel to stay Gaussian under an affine query and a scalar variance change, which forces log⁡ϕ\log\phi to be isotropic quadratic. This is Gaussian kernel closure, since ϕ\phi need not be a density. A corruption process with non-Gaussian kernels could admit a different factor family and would need a new characterization. VP/OU corruption reduces to the same Gaussian kernel after rescaling and keeps the same characterization.
  3. Line 250 → 269–271. Section 3.4 and Appendix F now give sufficient conditions. Zk(x)<∞Z_k(x)<\infty holds automatically for κk>0\kappa_k>0 or compact posterior support, and for an exact positive representation it follows from normalizability of the target posterior, since bkZk(x)≤Zr(x)b_kZ_k(x)\le Z_r(x).

We hope these revisions address the reviewer's questions and would be glad to clarify any remaining points.

Thank you very much for the clarification. I am also satisfied with the response and am keeping my score (clear accept).

Thank you for taking the time to review our responses and for your positive assessment and constructive feedback!

Reviewer_383kOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Spectra, a test-time prior adaptation method for simulation-based inference that adapts a diffusion-based posterior model trained under an existing prior to a new prior without retraining or additional simulations. The central idea is to derive an exact score transport identity for isotropic exponential-quadratic prior ratios, enabling computation of the new posterior score through a single transformed score evaluation of a frozen diffusion model. The authors extend this approach to positive mixtures of transportable components to accommodate more complex prior shifts and reduce sampling costs by selecting only one component per reverse diffusion trajectory. Experiments on six distinct SBI benchmarks show improved posterior accuracy over PriorGuide, particularly for nonlinear posteriors and strong prior shifts.

Critical strengths

S1. This paper derives an exact score transport identity that enables test-time prior adaptation through a single transformed query to a frozen diffusion model. The result does not require Gaussian assumptions on the base posterior or additional Jacobian evaluations, providing an elegant and theoretically grounded approach to prior adaptation.

S2. It also extends the exact single factor transport to more expressive prior changes through positive mixtures of transportable components, offering an efficient way to handle mixture priors.

S3. The proposed method is evaluated on six SBI benchmarks, including nonlinear and multimodal posterior distributions, under different prior shifts. The experiments demonstrate improved posterior accuracy over PriorGuide in several challenging settings.

Critical weaknesses

W1. Although this paper evaluates the proposed method on six SBI benchmarks, the experiments are restricted to relatively low-dimensional parameter spaces. This raises concerns about the scalability and practical applicability of the proposed method to more challenging, high-dimensional SBI problems. In particular, it remains unclear whether the reported accuracy and computational advantages would persist for complex posterior distributions with strong dependencies and high-dimensional prior shifts. Evaluating Spectra on more realistic, high-dimensional scientific inference tasks would provide stronger evidence of its practical usefulness.

W2. While Spectra achieves efficient online sampling by selecting a single mixture component per trajectory, estimating the observation-specific mixture weights introduces substantial computational overhead. As shown in Appendix M.2, the total latency for generating the first 1,000 posterior samples is higher than that of PriorGuide across the evaluated tasks. Moreover, this estimation cost scales linearly with the number of mixture components, potentially limiting the efficiency of the proposed method for more complex prior distributions. The results in Appendix N.1 also reveal cases where inaccurate component-weight estimation leads to significant degradation in posterior accuracy. These things raise concerns about both the computational efficiency and robustness of mixture prior adaptation.

W3. The exact score transport identity is limited to isotropic exponential-quadratic prior-ratio factors, which restricts the class of prior changes that can be handled exactly. More general prior shifts (including anisotropic or correlated Gaussian priors) require approximation through positive mixtures of transportable components. However, such approximations introduce additional errors and potentially require a large number of components. In particular, the tensor product quadrature construction (Appendix I) can lead to an exponential increase in the number of components with dimension. This raises concerns about whether the proposed framework can efficiently accommodate general high-dimensional prior changes.

Questions for authors

Please see the weaknesses section.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 383k (1/3)

(1/3)

We thank the reviewer for the careful reading and for raising scalability, cost, and the scope of exact transport. We have revised the paper on all three points and address each below.

W1. Higher-dimensional and scientific problems

We have added Section 4.5 and Appendix O, which evaluate Spectra on four problems that address each part of this concern.

  • High-dimensional parameters and prior shifts. The SIRD epidemic, MEG source-power, and MGL-32D problems have 30–32 parameters, and in each the prior shift tightens all of them at once.
  • Complex posteriors with strong dependencies. MGL-32D has a multimodal posterior whose components have differently oriented, correlated covariances, and its analytic reference allows exact evaluation in 32 dimensions. In diffusion MRI, two qualitatively different tissue solutions explain the same measurements (Jelescu et al., 2016), so the posterior is bimodal, and the target prior favors one solution, which moves posterior mass toward it.
  • Realistic scientific inference. The SIRD model infers a time-varying contact rate from 75 noisy measurements of a mechanistic epidemic, with a target prior that favors lower contact rates during part of the outbreak. The MEG model infers regional cortical source power through the lead field of a real head model, with a target prior that favors higher auditory power. The diffusion MRI model is an established two-compartment model of neural tissue. References come from NUTS or sequential Monte Carlo on the exact likelihood.

All methods use (N,L)=(100,0)(N,L)=(100,0). The table reports mean C2ST against the target posterior, online time for 1,000 samples, and one-time setup per observation, measured on one A100 after compilation.

ProblemddBasePriorGuideSpectraOnline (s), PG (VJP) / SpectraSetup (s), PG (VJP) / Spectra
MGL-32D, K=1K=1320.9870.5200.5131.47 / 0.6014.3 / 0
MGL-32D, K=2K=2320.9770.5230.5201.46 / 0.6014.3 / 110.1
SIRD epidemic300.9600.5270.5282.75 / 1.1728.6 / 0
MEG source power320.8890.6820.6849.74 / 4.13103.6 / 0
Diffusion MRI50.6400.5330.5321.62 / 0.6615.7 / 0

Accuracy. Spectra adapts the frozen models as accurately as PriorGuide, which receives the exact prior ratio. Without adaptation, the frozen models depart clearly from the target posteriors, and both methods reduce this gap to a similar level. On MEG, adaptation lowers C2ST from 0.89 to 0.68. The adapted posteriors also recover the quantities of scientific interest. The error in the posterior mean contact rate falls from 0.40 to 0.14 reference standard deviations, and the error in the posterior probability of the favored diffusion MRI solution falls from 0.233 to 0.025.

Computational cost. The online advantage persists in these problems. Spectra samples 2.4–2.5 times faster than PriorGuide (VJP), at the speed of the unadapted model. For single-factor updates, it also needs no setup, whereas the released PriorGuide implementation estimates a reverse-conditional variance for each new observation, which takes 14–104 seconds here. Mixture targets instead require component-weight estimation, 110 seconds for the two-component MGL-32D target, and we discuss this setup–sampling trade-off under W2.

Appendix O gives the full problem definitions, reference validation, and additional metrics, including a diffusion MRI target that only tightens the prior.

AuthorsOpenReview ↗

Response to Reviewer 383k (2/3)

(2/3)

W2. Cost of mixture adaptation

We agree that weight estimation makes the first mixture samples more expensive than PriorGuide and that its cost grows linearly with the number of components KK. Spectra trades this observation-specific setup for cheaper online sampling. Once the weights are available, each trajectory follows one component, keeping reverse-sampling cost independent of KK. Single-factor adaptation requires no weight estimation.

Let WSW_{\mathrm S} denote the weight-estimation cost and WPGW_{\mathrm{PG}} the precomputation of the released PriorGuide implementation, which estimates a reverse-conditional variance for each new observation, and let tSt_{\mathrm S} and tPGt_{\mathrm{PG}} denote online time per batch of 1,000 samples. Both setups are paid once per observation, so the cumulative costs after BB batches are

\qquad T_{\mathrm{PG}}(B)=W_{\mathrm{PG}}+B t_{\mathrm{PG}}.$$ When $t_{\mathrm{PG}}>t_{\mathrm S}$, Spectra has lower cumulative cost whenever $$B> \frac{W_{\mathrm S}-W_{\mathrm{PG}}} {t_{\mathrm{PG}}-t_{\mathrm S}}.$$ The following measurements use one A100 and the sampler setting $(N,L)=(25,8)$ of Table 1, so the MGL-32D times are higher than the $(100,0)$ times in our response to W1. Each cost entry is **PriorGuide (VJP) / Spectra**, and the final column gives the first integer batch satisfying this inequality. | Configuration | Setup (s) | Online per batch (s) | First-batch total (s) | First cheaper batch | |---|---:|---:|---:|---:| | MGL-32D, $K=1$ | 14.26 / 0 | 3.21 / 1.28 | 17.47 / 1.28 | 1 | | MGL-32D, $K=2$ | 14.26 / 110.13 | 3.19 / 1.28 | 17.45 / 111.42 | 51 | | GL-20D, $K=7$ | 8.58 / 229.61 | 1.96 / 0.69 | 10.54 / 230.30 | 176 | | GL-20D, $K=49$ | 8.58 / 1608.76 | 1.96 / 0.69 | 10.54 / 1609.46 | 1,267 | *Timings exclude compilation and use one observation and checkpoint per configuration. Crossover batches use unrounded timings and project repeated sampling for the same observation, backbone and target at the measured per-batch rates. The GL-20D rows use the low-rank correlated priors of our response to W3.* Online sampling is $2.5$–$2.8\times$ faster than PriorGuide (VJP) in these configurations, and single-factor Spectra is cheaper from the first batch. The trade-off also includes posterior accuracy. On the original mixture benchmarks at $(25,8)$, Spectra achieves mean C2ST of **0.521 versus 0.561** for PriorGuide on Two Moons and **0.570 versus 0.618** on SLCP. Mixture adaptation therefore offers improved accuracy on these nonlinear tasks alongside lower marginal sampling cost, with the crossover workload determining when it also reduces total computation. The estimated normalizers can also be reused across prior changes that share the same factor dictionary, when only the mixture coefficients change. ### W2. Reliability of component-weight estimation As the reviewer notes, Appendix N.1 shows a significant failure of practical component weights. It occurs for one of ten observations in one anisotropy orientation, where path-space estimation places nearly all mass on components with negligible reference weight, increasing C2ST by **0.417** at $K=52$ and **0.405** at $K=104$. On the other nine observations, the C2ST difference from reference-weight sampling is below 0.003 in every setting. Our diagnostics do not separate Monte Carlo estimation error from error inherited from the learned finite base chain (Appendix G). Our new low-rank experiments on GL-10D and GL-20D (Appendix N.3 and our response to W3) compare estimated and analytically known weights with the same component samplers. Median weight $L_1$ errors are 0.042–0.075, and replacing reference weights with estimated weights increases mean C2ST by less than 0.007 in all four settings, keeping Spectra comparable to PriorGuide (VJP). The measured weight errors therefore have a modest effect on posterior accuracy in these structured cases. Appendix H.3 separately analyzes the estimator's Monte Carlo error under exact Gaussian scores and shows when a run's own effective sample size can understate it.
AuthorsOpenReview ↗

Response to Reviewer 383k (3/3)

(3/3)

W3. Number of components for anisotropic and correlated priors

We agree that this is a real limitation. The exact single-factor class covers isotropic precision changes (Section 3.2), and for general anisotropic or correlated changes the full-rank tensor-product quadrature of Appendix I needs a number of components that grows exponentially with dimension. However, priors with suitable covariance structure avoid this growth, and we have added the construction (Appendix I) and experiments (Appendix N.3) for such a subclass.

When the target covariance exceeds an isotropic covariance only along a few directions, Σ=λI+BB⊤\Sigma=\lambda I+BB^\top with r=rank⁡(B)r=\operatorname{rank}(B), the target prior is exactly a continuous mixture of isotropic kernels,

\mathbb E_{\xi\sim\mathcal N(0,I_r)}\left[\mathcal N(\theta;\mu+B\xi,\lambda I)\right],$$ and only the $r$-dimensional variable $\xi$ needs to be discretized. With $m$ Gauss–Hermite nodes per direction, this gives $K=m^r$ components independently of $d$. For the isotropic Gaussian training priors used here, each kernel induces an exactly transportable factor. For the rank-one and rank-two targets below, $K=7$ and $K=49$ in both 10 and 20 dimensions, whereas the full-rank quadrature would need at least 1,024 components in 10 dimensions and more than $10^6$ in 20. In terms of the prior ratio, this subclass is an exactly transportable isotropic factor multiplied by a low-rank positive-semidefinite quadratic tilt, the case that Moitra et al. (2026) treat with a related auxiliary-variable construction. We evaluated it on GL-10D and GL-20D with target covariance $0.02I+0.05UU^\top$, where $U$ has one or two orthonormal columns, using seven nodes per direction (new Appendix N.3). Spectra-Ref uses the analytic component weights, Spectra-PS the path-space estimates, and PriorGuide (VJP) the exact correlated prior ratio. All methods use three learned backbones and $(N,L)=(25,8)$. | Task | Rank | $K$ | Prior TV | Spectra-Ref | Spectra-PS | PriorGuide (VJP) | |---|---:|---:|---:|---:|---:|---:| | GL-10D | 1 | 7 | 0.0091 | 0.5380 | 0.5406 | 0.5411 | | GL-10D | 2 | 49 | 0.0118 | 0.5249 | 0.5315 | 0.5318 | | GL-20D | 1 | 7 | 0.0091 | 0.5504 | 0.5539 | 0.5546 | | GL-20D | 2 | 49 | 0.0118 | 0.5468 | 0.5520 | 0.5543 | *Mean C2ST against the exact target posterior over ten observations. Prior TV is the total variation between the quadrature prior and the exact target prior.* The finite quadrature changes the prior by a total variation of about 0.01, and several components keep posterior mass (median effective number 2.6 for $K=7$ and 6.6 for $K=49$). With estimated weights, Spectra achieves C2ST comparable to PriorGuide. The cost of this subclass lies in weight estimation, which grows linearly with $K$. On GL-20D it takes 230 seconds for $K=7$ and 1,609 seconds for $K=49$, while online sampling stays at 0.69 seconds per 1,000 samples, and the break-even points are given under W2. The number of components therefore grows exponentially with the rank of the covariance structure, not with the parameter dimension. We hope these revisions address the reviewer's questions and would be glad to clarify any remaining points.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of revisions

We thank all reviewers for their constructive feedback. We have uploaded a revised paper and summarize the main changes below.

  • Higher-dimensional and scientific problems (383k, Rapy). New Section 4.5 and Appendix O apply Spectra to a 32-dimensional multimodal task and three scientific models: an SIRD epidemic, MEG source power, and diffusion MRI.
  • Cost of mixture adaptation (383k, fF2R, Rapy). Section 3.6 notes that weight estimation grows linearly with the number of components. Appendix M separates setup from online cost and gives break-even points against PriorGuide (VJP).
  • Structured anisotropic priors (383k, fF2R). Appendix I adds a low-rank construction whose number of components depends on the rank of the covariance structure, not on the dimension, and new Appendix N.3 evaluates it in 10 and 20 dimensions.
  • Estimator variance (fF2R). New Appendix H.3 analyzes how the path-space estimator error depends on dimension and the number of reverse steps.
  • Related work (fF2R). Section 3.3 and Appendix A cite Moitra et al. (2026) and McDonald (2026) and state the correspondence with our score identity.
  • Clarity of Section 3 (Rapy). We expanded the single-factor derivation and the transition to the characterization, added conditions for finite component normalizers and a use case for negative κ\kappa, and added a nonparametric companion to Figure 2 (Appendix K.4).

Dear Reviewers, The authors have now submitted their responses to the reviews. There is only a limited amount of time left for you to review their replies and provide any further comments.

Your feedback is crucial to ensure that the review process proceeds smoothly. We would greatly appreciate it if you could share your reactions before the discussion period closes.

Thank you very much for your work. Best regards.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.