acceptodds
Under review as a conference paper at ICLR 2027

HAPERA: Residual Allocation for Amortized Posterior Estimation in Hierarchical Populations

Abstract

Applications from device fleets to patient cohorts often require inferring the global parameters of a hierarchical model from short early observation windows. Amortized simulation-based inference trains a network once on simulated populations and reuses it without retraining. Because populations evolve at very different speeds across the prior, simulators may set windows to a random fraction of the population lifetime implied by the global parameters. We show that an ideally trained estimator then treats the window length as evidence about these parameters, even though their marginal prior is unchanged. At deployment, where windows are chosen independently of the parameters, this evidence is spurious, so the learned posterior is biased. Simulation-based calibration (SBC) on the training simulations cannot reveal this bias, because the ideal estimator is calibrated for those simulations. We propose HAPERA, a flow-matching estimator of the joint posterior over global and unit-level parameters. First, it trains on windows drawn from their marginal distribution, independently of the parameters. Second, it adds an allocation residual, a vector-field term dedicated to shifting the population location and every unit's deviation from it equally in opposite directions. Such a shift leaves each unit's parameter, the location plus its deviation, and hence the likelihood, unchanged, so only the prior can tell these allocations apart. On a synthetic benchmark, a classifier two-sample test places HAPERA's posterior samples clearly nearer NUTS reference samples than those of the hierarchical amortized estimators HNPE and TFMPE, trained under the same window design, at every window length and seed. On two real-world datasets, LED degradation and drug pharmacokinetics, HAPERA is nearly indistinguishable from the NUTS reference and closer than both estimators. In ablations, parameter-dependent windows pull the posterior away from the reference, most for the shortest windows. Removing the residual also pulls it away at every training seed.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
amortized inference, simulation-based inference, hierarchical Bayesian models, posterior estimation, conditional flow matching, calibration, observation-window design, mixed-effects models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_SqGjOpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies amortized simulation based inference for hierarchical population models and proposes HAPERA, a flow matching method for estimating joint posteriors over global and unit level parameters. The authors first identify a bias caused by coupling observation windows with the true model parameters and propose a decoupled simulation design to address it. HAPERA then introduces a sensitivity based vector field and an allocation residual that adjusts global and local parameters while preserving unit level parameters to first order. The method is evaluated on synthetic degradation data, an LED dataset, and an indomethacin dataset, using NUTS as the reference posterior. The authors report that HAPERA outperforms existing methods on the synthetic benchmark and achieves near chance C2ST on the real datasets. Ablation studies show that coupled training introduces posterior bias, while the allocation residual provides a smaller improvement in posterior accuracy.

Critical strengths
  1. The residual has a clear mathematical construction. Equations~5 to 7 establish the directional cancellation and normalize the correction across population sizes and scales.

  2. The window design ablation supports Proposition 1. With matched pairs at three seeds, coupled training raises C2ST by 0.125 at f = 0.05 and by 0.068 when pooled over fractions.

Critical weaknesses
  1. The theory is correct but thin. As the authors acknowledge, Proposition 1 applies the proposal prior identity of Papamakarios and Murray (2016) to the observation window. Corollary 1 follows directly from the fact that the ideal estimator equals the true conditional of its own training joint.

  2. The paper does not report the simulation and tuning budgets for HNPE and TFMPE alongside those for HAPERA. This omission matters because HAPERA uses fresh simulations throughout training. It remains unclear whether its advantage would persist under matched budgets.

  3. The paper does not show that the coupled design is common in practice. It says only that simulators "may" set windows this way, and cites no prior work that sets the window from the true lifetime, so it is hard to judge how often this problem arises. All evidence of harm comes from a simulator the authors built themselves, so the practical significance of the first contribution is unclear.

Questions for authors
  1. Why are the prior anchor and design gate needed? Explaining their roles before introducing the equations would make the method easier to assess.

  2. What accounts for HAPERA's large gain over HNPE and TFMPE? The methods share the same window design, and the residual ablation shows only a small improvement. What C2ST does the ported TFMPE architecture achieve under the same training pipeline? This comparison would help separate the contributions of the architecture and training procedure.

  3. How many simulated populations and how much GPU time did each method use for training? I am also curious how long HAPERA and NUTS take per population at inference. NUTS seems to work well at the current sizes, so it is hard to see when amortization pays off.

  4. Are the real data results stable across independent training runs when model selection uses only the fit cohort? Since lifetime predictions remain worse than the naive baseline, what practical benefit does the closer agreement with NUTS provide?

6 comments
AuthorsOpenReview ↗

Comment 1 of 5 : Response to Reviewer SqGj

Thank you for your comments on our paper. We have uploaded a revised PDF, which incorporates the comments of all reviewers, reviewed together.

Our benchmarks are models whose likelihood, the probability of the recorded data given all parameters, has a closed form. Their posterior can be sampled directly with the No-U-Turn sampler (NUTS), a Markov chain Monte Carlo method, and these samples, drawn under the same model, prior and window as every network, are the NUTS reference (§4.1). An amortized estimator such as HAPERA is trained on simulations alone and never evaluates the likelihood, so this kind of estimator also applies to simulators whose likelihood cannot be written down, where NUTS cannot be run; the closed-form benchmarks are what let us score it against a reference. The paper's main score is C2ST, the accuracy of a classifier that tries to separate a network's draws of the global parameters (the population-level parameters shared by all units) from the NUTS draws: it is 0.5 when the classifier cannot tell them apart, and higher the farther the network lies from the reference (§4.1). The window fraction f, from 0.05 to 0.20 on the synthetic benchmark, is the window's length as a fraction of a population lifetime drawn from the prior independently of the population observed.

The revised paper reports the simulation and training budgets side by side (Table 7 (a); Tables 13 and 20). Every amortized comparator of Tables 1–2 is now trained under HAPERA's regime: fresh simulations at every optimizer step, with HAPERA's per-epoch simulation budget, epoch cap and stopping rule (§4.1). The advantage persists under these rules. On the synthetic benchmark, HNPE (19.8M; M: millions of network parameters) reads 0.640 and TFMPE-Direct (17.8M; TFMPE without its surrogate stage) reads 0.620, against 0.528 for HAPERA (17.6M) (Table 1). HAPERA is nearer the reference in 24 of 24 cells, one per comparator, training run and window fraction (four values of f), each beyond twice its standard error (Table 9).

On tuning, HAPERA's optimization settings and network size (Table 3) were fixed in advance, and no run varied them to select them. Under HAPERA's regime, HNPE and TFMPE keep the optimizer, step size and batch size of their authors' recipes (Tables 13, 20), except that a short probe before training chose the batch size and step size of TFMPE at its published size on indomethacin (App. E.7).

On the theory, Proposition 1 starts from the identity of Papamakarios and Murray (2016) for training on parameters drawn from a proposal instead of the prior, and adds the case in which the window rule, not the parameter sampler, creates the proposal. Under the coupled design, which sets each training window to a fraction of the simulated population's own lifetime, the simulated parameters still follow the model's declared prior, so the standard correction, the ratio of the declared prior to their distribution, equals 1 and changes nothing (§3.2). Yet a long window signals a slow population, so an ideally trained estimator treats the window length as evidence. It then returns a different posterior from the one wanted at deployment, where the window is chosen independently of the parameters; HAPERA's decoupled design draws its training windows that way. The standard calibration check, a rank check on the training simulations, cannot detect the difference (Corollary 1).

The new Figure 2 shows both results on a scalar model: for the same observation, the means of the two designs' posteriors lie 0.36 apart, 0.81 standard deviations, and both rank histograms are uniform. On the synthetic benchmark, training under the coupled design, all else equal, places HAPERA farther from the NUTS reference at every window fraction and training seed, one seed per training run (Figure 3a); at the shortest window fraction, f = 0.05, its posterior of the first population-location coordinate lies farther by 0.55–0.60 standard deviations of the NUTS draws at 3 of 3 training seeds (§4.4; Table 16).

AuthorsOpenReview ↗

Comment 2 of 5 : Response to Reviewer SqGj

On whether the coupled design arises in practice, the introduction of the revised paper (§1) cites published simulators in which the simulated parameters set what is observed, for example an amortized pharmacokinetic estimator trained on simulated studies whose sampling times depend on the peak time and half-life implied by each study's sampled rate constants, and evaluated on clinical studies with their own schedules (Ojeda et al., 2026), and the C-MAPSS engine trajectories, which end at simulated failure in training while test trajectories are cut short (Saxena et al., 2008). In the first, the sampling times play the role of our window and the clinical studies that of deployment: in training the times follow from the simulated parameters, and at evaluation they are the studies' own. The coupled design builds the same dependence into training, and a designer has a reason for it: a fixed window covers more of a fast population's lifetime than of a slow one's, whereas a random fraction of each population's own lifetime makes the observed stages comparable (§1; Figure 1a).

Because your point is that all evidence of harm comes from a simulator we built, an additional test for this response repeats the comparison of the two posteriors of Proposition 1 on a published drug model that is not ours, mrgsolve's pk1 one-compartment model, with a population prior we chose and windows set by the coupled design. No network is trained: for the same data, NUTS samples the posterior under the declared prior, which an estimator should return, and the posterior under the prior conditional on the window length, which the ideally trained estimator of the coupled design returns. At f = 0.05, a classifier separates the two posteriors with an accuracy of 0.723, against 0.500 when both sets of draws come from one posterior (two halves of one NUTS run), on average over the 188 of 200 populations whose NUTS runs passed their convergence thresholds.

The revised paper is also rearranged to read more easily, and every main-text symbol is explained where it first appears or earlier.

Our direct answers to your questions follow.

Q1. Why are the prior anchor and design gate needed, and can their roles be explained before the equations?

A1. The prior anchor lets the network learn only how the data move the posterior away from the prior. The design gate damps the learned output on coordinates the records say little about: those to which the simulated trajectories barely respond at the recorded times (§3.3).

HAPERA moves random starting draws along a vector field, a rule that gives each draw a direction of movement, and flow matching learns this rule (§2.2). The prior anchor is the fixed part of the rule that, by itself, carries standard normal starting draws to the prior. The design gate is computed from the simulator's sensitivity at the recorded times, not from the recorded values, and multiplies each coordinate of the learned output by a value between 0 and 1 before the output is added to the anchor (Eq. 4).

Both are there because a short window carries little information about some parameters, and where the records say little, the posterior stays close to the prior. The anchor builds the prior into the rule, so the network does not have to learn the prior as well. Without the gate, the learned output would act with full strength on every coordinate; with it, the flow stays close to the anchor on the coordinates the records say little about, unless the learned output is large. With no observation, the gate closes and the flow returns the prior (§1; §3.3–§3.4).

On the synthetic benchmark, removing the prior anchor with all else unchanged moves HAPERA (17.6M) farther from the NUTS reference at 3 of 3 training seeds: C2ST pooled over the window fractions increases by 0.0139 on average (Table 17 (c)). Removing the design gate moves it farther at 2 of 3 training seeds, by 0.0115 and 0.0497, and nearer at the first training seed, by 0.0178 (Table 17 (c); §4.4; App. E.6).

The introduction of the revised paper explains both roles before any numbered equation, and §3.3 introduces them in words before Eq. 4.

AuthorsOpenReview ↗

Comment 3 of 5 : Response to Reviewer SqGj

Q2. What accounts for HAPERA's large gain over HNPE and TFMPE, given the shared window design and the small effect of the residual ablation?

A2. The gain comes from the paper's second contribution, HAPERA's estimator itself, that is, its network architecture together with the three components of its vector field. The comparators share its window design and, under HAPERA's regime, its simulator, prior and training-budget rules, and still lie farther from the NUTS reference.

Three comparisons in the paper size this. The difference to HNPE is about half as large when HNPE's training procedure changes from its authors' recipe to HAPERA's regime: HAPERA's C2ST is lower than that of HNPE (19.8M) by 0.2271 under the recipe, as in the submitted paper, and by 0.1118 when both train under HAPERA's regime (Tables 12 (a), 1; App. E.3). Under one regime, the differences between complete estimators are this 0.1118 and, to TFMPE-Direct (17.8M), 0.0917 (Table 1), and the order holds in the smaller size class, where HAPERA (1.66M) reads 0.557 against 0.601 for HNPE (1.67M) and is nearer at 3 of 3 training seeds (Table 19 (a); App. E.7). Within HAPERA, Table 17 removes one component at a time with the other two in place, and pooled C2ST then changes by at most 0.0497, at one training seed (§4.4; Table 17 (a), (c)).

The differences of 0.1118 and 0.0917 are between the estimators as built and trained: each keeps its own architecture, objective, optimizer and batch size (App. D.3; Table 13 (a)). HAPERA's network is one joint transformer that reads the observations, the units' local offsets and the global parameters as tokens, with 12 layers at token width 256 (§2.2; Table 3). HNPE factors the posterior into a global and a local part, each a normalizing flow (App. D.3). TFMPE-Direct is a tokenized transformer with 5 layers at token width 768, trained by flow matching as HAPERA is (Table 7 (b); App. D.3).

The allocation residual is one of the three components of the estimator, and the three work together in one vector field: the prior anchor supplies the prior, the design gate scales the learned output, and the residual, a learned correction, makes explicit how each unit's parameter is split between population and unit (§3.3–§3.4). The residual head has 526 parameters in a network of 17.6M (Table 3), and on the synthetic benchmark removing it alone moves HAPERA farther from the reference at 3 of 3 training seeds: pooled C2ST increases by 0.0160, 0.0058 and 0.0025, each beyond twice its standard error (Table 17 (a)). On the population location, the only scored group of coordinates that the residual acts on, the coordinate-wise Wasserstein distance to the reference is lower with the residual at 3 of 3 training seeds, by 0.0314, 0.0048 and 0.0032 standard deviations of the NUTS draws (Table 17 (d)).

Q3. What C2ST does the ported TFMPE architecture achieve under the same training pipeline?

A3. The ported TFMPE architecture is scored on calibration, not on C2ST; the C2ST of TFMPE's own architecture under HAPERA's regime is 0.620, against 0.528 for HAPERA (Table 1). That value belongs to TFMPE-Direct (17.8M), TFMPE's posterior network from its authors' implementation, trained directly on simulator-generated populations under the same simulator, prior, window design and training-budget rules as HAPERA (§4.1). TFMPE's published network (821,473 parameters) trained the same way reads 0.615, against 0.557 for HAPERA (1.66M) (Tables 21 (a), 19 (a)).

The port is a different network. It is the port of App. D.5, built to test the window design outside HAPERA's architecture, with token width 256 and 22 encoder layers, where TFMPE-Direct has 768 and 5 (App. D.5; Table 7 (b)). It is trained once under each window design and then checked on windows drawn independently of the parameters: at f = 0.10, its 90% intervals for the first population-location coordinate cover the generating value at a rate of 0.820 under the coupled design, against 0.895 under the decoupled design, with 0.90 nominal (Table 14, rows 3 and 5).

Reviewer_xm6pOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies amortized posterior estimation for hierarchical populations observed through short trajectory windows. It identifies how choosing training windows using the same latent parameters that generate the observations can change the posterior target, and proposes decoupled window sampling that preserves the marginal window distribution. The paper proposes HAPERA, a tokenized conditional flow estimator combining a prior anchor, a design gate, and a residual allocation. The residual redistributes change between population location and local offsets while satisfying a cancellation identity at a common flow state.

Empirical study covers a synthetic degradation benchmark and two measured systems, using NUTS as a posterior reference, and comparisons with hierarchical amortized estimators.

Critical strengths
  1. The paper identifies a concrete simulation-base inference failure case and gives an actionable correction. The paper explains why parameter-dependent observation windows can change the learned conditional target, even when calibration under the training joint is satisfactory. The decoupled construction offers a practical remedy when deployment windows are independent of the inferred parameters.

  2. The complete estimator shows strong and consistent global-posterior fidelity on the evaluated problems. On the synthetic benchmark, HAPERA achieves better performances than hierarchical baselines such as HNPE and TFMPE. Results on the two measured systems are also close to NUTS

Critical weaknesses
  1. It's unclear whether the emprical comparison is conducted in a fair way. The paper reports architectural configurations and includes capacity-matched variants, but does not sufficiently document baseline setup, including simulation budgets, optimization steps, stopping criteria, and convergence to establish training comparability. Moreover, the main TFMPE comparison retains its learned simulator-surrogate stage, whereas the original TFMPE paper also considers a "Direct" variant trained on simulator-generated populations.

  2. The evaluation of the allocation residual is not complete. Its independent contribution is evaluated only on the synthetic benchmark, while the complete estimator also includes the prior anchor, design gate, and initialization choices. The reported application results do not show whether this particular addition really contributes to their strong posterior fidelity. Given its central role in the proposed method, this limits the evidence for a substantial architectural improvement.

Questions for authors
  1. What actual setup (e.g. simulation budgets, optimization steps, etc) was used for each baseline relative to HAPERA?

  2. Does the allocation residual improve posterior fidelity on a measured system when the rest of HAPERA is held fixed?

3 comments
AuthorsOpenReview ↗

Comment 1 of 2 : Response to Reviewer xm6p

Thank you for your comments on our paper. We have uploaded a revised PDF, which incorporates the comments of all reviewers, reviewed together.

On the fairness of the comparison, every amortized comparator of Tables 1–2 of the revised paper is now trained under HAPERA's regime, and the TFMPE comparator is TFMPE-Direct (17.8M), the "Direct" variant you name, trained directly on simulator-generated populations without the surrogate stage. Under this regime, mean C2ST to the NUTS reference on the synthetic benchmark (0.5: chance) is 0.528 for HAPERA against 0.640 for HNPE (19.8M) and 0.620 for TFMPE-Direct (Table 1), and HAPERA is nearer beyond two paired standard errors (SE) in 24 of 24 training-seed-by-window-fraction cells across both comparators (Table 9). With TFMPE's published network (821,473 parameters per network) under the same regime, the Direct route reads 0.615 and the surrogate route 0.737 (Table 21 (a)).

On the allocation residual, the revised paper presents it as one of three estimator components, alongside the prior anchor and the design gate, and removes each one alone on the synthetic benchmark, in pairs of networks that share the training seed and differ only in the vector field (App. D.5; Table 17). Removing the prior anchor moves HAPERA farther from the NUTS reference at 3 of 3 training seeds, by 0.0139 on average in C2ST pooled over window fractions; removing the design gate moves it farther at 2 of 3, by 0.0115 and 0.0497, and nearer at the first training seed, by 0.0178 (Table 17 (c)). The removal of the residual alone, on the synthetic benchmark and on both measured systems, is taken up under your second question.

Our direct answers to your questions follow.

Q1. What actual setup (e.g. simulation budgets, optimization steps, etc.) was used for each baseline relative to HAPERA?

A1. Under HAPERA's regime, HAPERA, HNPE (19.8M) and TFMPE-Direct (17.8M) share the simulator, the prior, the decoupled window design and one rule for budget and stopping, with populations drawn afresh at every optimizer step. Every estimator draws 192,000 fresh populations per epoch on the synthetic benchmark (640,000 on a measured system), for at most 500 epochs, and stops after 80 epochs without a gain in validation loss; a synthetic epoch is 500 optimizer steps for HAPERA and 1,920 for the comparators (App. D.3). Across the three synthetic runs this gave 53.0 to 95.2 million simulated populations for HAPERA, 43.6 to 81.6 million for HNPE and 43.4 to 78.1 million for TFMPE-Direct, not counting the validation populations, 19,200 drawn afresh after every epoch (Tables 7 (a), 13 (a)). In its first two runs HNPE drew more than HAPERA did and still lies farther from the reference at every window fraction (Tables 9, 13 (a)). Each estimator keeps its own architecture, objective, optimizer and batch size (App. D.3; Table 13 (a)). The stopping rule ended every run of Tables 1–2 before the cap; Table 13 (a) and Appendices E.2 and E.4 record each run.

On the measured systems, HNPE and TFMPE-Direct each lie farther from the reference than HAPERA by distance of C2ST to 0.5 in 5 of 5 conditions, beyond two SE over scoring seeds in 4 of 5 and 3 of 5 (Tables 2, 11 (a); App. E.2).

In the submitted paper, the comparators of Tables 1–2 were trained under their authors' recipes, on fixed simulation sets whose sizes and epoch caps we set for this problem: one pool of 80,000 simulated populations for HNPE, 8,000 of them held out for validation, and 100,000 single-unit simulations followed by 100,000 surrogate-generated populations for TFMPE (Table 7 (a); App. D.3). There HNPE (19.8M) reads C2ST 0.755, with HAPERA lower by 0.2271 (Table 12 (a)); under HAPERA's regime it reads 0.640 and the difference is 0.1118 (Table 1). In the ≈1.6M size class under the same regime, HAPERA (1.66M) reads 0.557 against 0.601 for HNPE (1.67M) and is nearer at 3 of 3 training seeds, each beyond two paired SE (Tables 19 (a), 20; App. E.7).

AuthorsOpenReview ↗

Comment 2 of 2 : Response to Reviewer xm6p

Q2. Does the allocation residual improve posterior fidelity on a measured system when the rest of HAPERA is held fixed?

A2. In an additional test for this response, we trained HAPERA without the allocation residual on LED and indomethacin, at the training seeds of runs 1 to 3 of Table 23. Every other setting, the scoring and the NUTS reference are those of the paper's networks, so the two networks of a pair differ in the residual alone. In each cell, the nearer of the two networks is the one whose C2ST, the mean over three scoring seeds, lies closer to 0.5, the rule by which Table 2 marks its bold entries. The network with the residual is the nearer on LED in 7 of 9 training-seed-by-condition cells and on indomethacin in 1 of 6; on indomethacin every value of either network lies within 0.014 of 0.5. The table gives C2ST with the residual / without it; the values with the residual are the cells of runs 1 to 3 of Table 23.

System, window fractionTraining seed 1Training seed 2Training seed 3
LED, f = 0.050.510 / 0.5680.534 / 0.5460.527 / 0.512
LED, f = 0.100.487 / 0.4950.508 / 0.5370.504 / 0.508
LED, f = 0.150.509 / 0.5160.511 / 0.5340.508 / 0.510
indomethacin, f = 0.150.506 / 0.5050.514 / 0.5040.505 / 0.502
indomethacin, f = 0.200.487 / 0.4950.501 / 0.4950.496 / 0.497

On LED the two cells that go the other way are f = 0.10 at the first training seed and f = 0.05 at the third; the differences in distance to 0.5 reach 0.058 in favor of the residual and 0.015 against it. On indomethacin the network with the residual is the nearer in one cell, f = 0.20 at the second training seed, and the differences against it reach 0.010.

At each training seed, the comparison on a measured system rests on the three conditions of LED or the two of indomethacin, on one recorded population per system, whereas on the synthetic benchmark it rests on 800 condition pairs, each one evaluation population at one window fraction. There the two networks of a pair share their initialization: the network without the residual starts from HAPERA's backbone weights and HAPERA's residual head starts at 0, so both start from the same initial field (App. D.5). Removing the residual moves HAPERA farther from the NUTS reference at 3 of 3 training seeds: pooled C2ST rises by 0.0160, 0.0058 and 0.0025, with paired SEs of 0.0016, 0.0007 and 0.0006 (Table 17 (a)), and the sliced Wasserstein distance by 0.0223, 0.0082 and 0.0028 (Table 17 (b)). With SEs clustered by population, each of these C2ST differences stays beyond two SE (App. D.4). By window fraction, HAPERA is nearer with the residual in 11 of 12 seed-by-fraction cells in C2ST and in 12 of 12 in sliced Wasserstein distance (Table 17 (a), (b); App. E.6). The comparison of Table 1 holds without the residual: at the first training seed, where the removal moves HAPERA most, HAPERA reads 0.549 with the residual (Table 10 (a)) and 0.0160 higher without it (Table 17 (a)), that is, 0.565, against means of 0.640 and 0.620 for the comparators, neither of which reads below 0.590 at any training seed and window fraction (Table 9).

On the synthetic benchmark, of the scored coordinate groups only the population location μ receives the residual term, and its coordinate-wise Wasserstein distance to the reference is lower with the residual at 3 of 3 training seeds, by 0.0314, 0.0048 and 0.0032 standard deviations of the NUTS draws, each beyond two paired SE. The log-scales log τ are also nearer at 3 of 3 seeds, beyond two paired SE at the first two, and on the correlation and the noise scale the sign changes across seeds (Table 17 (d); App. E.6).

The paper's claim on the measured systems is about the estimator as a whole, with its three components: across six training runs per system, HAPERA's C2ST ranges from 0.487 to 0.545 on LED and from 0.487 to 0.520 on indomethacin (Table 23; App. E.8).

I want to thank the authors for preparing the additional results, which addressed some of my initial concerns. I am not sure if it's only from my side: I can only see the latest version of the pdf and could not access the previous versions. It is therefore a bit difficult to see what has been changed without explictly specifying the new content (e.g. use different colors). After giving the manuscript a second in-depth read, I also noticed some further issue about its readablity that I didn't notice before, e.g. the sloppy and heavy usage of symbols, some awkward mentioning of result values in the main text, etc.

I believe the current manuscript needs a proper rewriting, which I encourage the author to continue doing it. I tend to keep my initial rating for now.

Reviewer_YTR3OpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The authors introduce HAPERA: a hierarchical flow matching neural posterior estimator that is trained on simulations where the observed window size of a changing population is sampled independently from the parameters and an allocation residual . The authors evaluate their method on one synthetic and two real-world settings, comparing to both existing state-of-the-art amortized methods as well as a gold-standard NUTS solver. They also present two ablations to evaluate the impact of their central contribution's parts. In all settings, the method consistently demonstrates good performance.

Critical strengths
  • The paper identifies a clear potential weakness in modeling evolving populations with hierarchical SBI and addresses it appropriately.
  • The authors cover significant relevant related work and position the paper well within it.
  • The method consistently performs better than existing state-of-the-art for hierarchical NPE in the covered benchmarks.
Critical weaknesses
  • The practical relevance of the method's motivation is unclear - the authors claim that simulators may set observation window size based on the parameters, but they cite no prior work that could verify this is indeed how it is handled in practice.
  • The addition of the allocation residual to the method is presented as a central contribution, but its impact is shown to be almost negligible in the ablation in Figure 4B.
  • The presentation quality, especially of the math and figures could be improved:
    • The paper makes use of a large set of notational conventions, which are only sometimes explained alongside relevant equations. This makes the math difficult to follow and requires excessive jumping between the main text and Appendix A, most notably in section 3.2.
    • Figures 1 and 2 contain exclusively text boxes connected with arrows, which makes them no faster to read than equivalent prose. The meaning of the blue boxes in these figures is unclear to me and never described.
Questions for authors
  • Could the authors make an argument for the practical relevance claim that simulators set the window size depending on the parameters or identify related work that shows this?
  • Could the authors polish Figure 1 to explain the method visually rather than through organized text?
  • Could the authors clarify the meaning of the blue boxes in Figures 1 and 2?
  • Could the authors simplify the math and notation in section 3? I would suggest moving implementation details such as Equations 5-7 into the appendix, replacing section 3.2 with a higher-level overview.

Nitpicks:

  • Please highlight the best values in Table 1 in bold.
  • The legend of the two subplots in Figure 4 is largely repeated - why not simplify and use a single joint legend?
4 comments
AuthorsOpenReview ↗

Comment 1 of 3 : Response to Reviewer YTR3

Thank you for your comments on our paper. We have uploaded a revised PDF, which incorporates the comments of all reviewers, reviewed together.

We rearranged the revised paper so that it reads more easily and repeats less. Every symbol in the main text is now defined or explained in words where it first appears or earlier, so going back and forth to Appendix A is no longer needed, and Appendix A is now a single notation list grouped by topic. Table 1 shows the best amortized values in bold, and Figure 3, which replaces the submitted Figure 4, labels the two window designs directly and uses a single legend.

The allocation residual is one of three estimator components, alongside the prior anchor and design gate, and the title of the revised paper accordingly names the setting, hierarchical populations observed over short windows, rather than one component. The paper's numbers show the weight of the residual in C2ST on the synthetic benchmark, pooled over its four window fractions. C2ST is the accuracy of a classifier separating an estimator's posterior draws from those of the NUTS reference, the No-U-Turn sampler's posterior for the same data, and 0.5 means that the two cannot be told apart.

Table 1 compares complete estimators; for HAPERA, that is its network architecture together with the three components of its vector field. HAPERA's mean C2ST over three training seeds is 0.528, against 0.640 for HNPE (19.8M) and 0.620 for TFMPE-Direct (17.8M; TFMPE without its surrogate stage), both trained under the same simulator, prior, window design and training-budget rules: differences of 0.1118 and 0.0917 (Table 1). With only the window design changed, HAPERA trained on coupled windows lies 0.0682 farther from the reference on average over the three training seeds (Table 15 (a)).

With only the residual removed, HAPERA lies farther from the reference at 3 of 3 training seeds, by 0.0160, 0.0058 and 0.0025, each beyond two paired standard errors, and at 11 of 12 seed-by-fraction cells (§4.4; Appendix E.6). This is the comparison that the submitted Figure 4B plotted; Table 17 (a) of the revised paper gives it by training seed and window fraction. The comparison of Table 1 holds without the residual: at the first training seed, where the removal moves HAPERA most, HAPERA reads 0.549 with the residual (Table 10 (a)) and 0.0160 higher without it (Table 17 (a)), that is, 0.565, against means of 0.640 and 0.620 for the comparators, neither of which reads below 0.590 at any training seed and window fraction (Table 9).

Our direct answers to your questions follow.

AuthorsOpenReview ↗

Comment 2 of 3 : Response to Reviewer YTR3

Q1. What argument or related work supports the practical relevance of simulators setting observation windows from the parameters?

A1. The closest published precedent has both sides of the mismatch we study: an amortized pharmacokinetic estimator is trained on simulated studies whose sampling times depend on the peak time and half-life implied by each study's sampled rate constants, and is evaluated on clinical studies with their own schedules (Ojeda et al., 2026). §1 of the revised paper cites it with two related examples: a survival-data simulation framework sets each scenario's censoring parameter from the other parameters to reach a predefined censoring rate (Wan, 2017), and in the C-MAPSS turbofan benchmark each training trajectory ends when its engine's simulated degradation reaches failure, while test trajectories are truncated (Saxena et al., 2008).

The coupled design of Eq. 2 builds the same dependence into training, and a designer has a reason for it: a fixed window covers more of a fast population's lifetime than of a slow one's, whereas a random fraction of each population's own lifetime makes the observed stages comparable (§1; Figure 1a). In missing-data terms, this design violates the distinctness condition: the parameters governing what is observed are not separate from those being estimated (Rubin, 1976; Seaman et al., 2013; §1). We study deployment in which the window is chosen independently of the parameters, as when it closes at a decision date. The concern is this mismatch between training and deployment, not parameter-dependent observation as such: a dependence that also holds at deployment, as in gravitational-wave population inference (Leyde et al., 2024), must be modeled in the likelihood (§5).

Under the coupled design, a long window signals a slow population, so an ideally trained estimator treats the window length as evidence about the parameters (Proposition 1); in the scalar model of the new Figure 2, the two designs' posterior targets for the same observation have means 0.36 apart, 0.81 standard deviations. On the synthetic benchmark, coupled training places HAPERA farther from the NUTS reference at every window fraction and training seed, most at the shortest window fraction, f = 0.05, where its C2ST rises by 0.1246 on average over the three training seeds (Table 15 (a)). At that window fraction, coupled training places HAPERA's posterior for the first coordinate of the population location 0.55–0.60 standard deviations of the NUTS draws farther from the reference at 3 of 3 training seeds (Table 16).

Q2. Could Figure 1 explain the method visually rather than through organized text?

A2. Figure 1 is redrawn as three panels. Panel (a) plots the degradation paths of a fast and a slow population with their observation windows shaded: under the coupled design the fast population's window is the shorter one, so the window's length reveals the population's speed, and under the decoupled design one window, independent of the parameters, serves both. Panel (b) is the estimator diagram, with a small plot beside each of the three components showing what it does. Panel (c) plots one population's posterior at f = 0.05 under decoupled training (HAPERA), under coupled training and from the NUTS reference.

AuthorsOpenReview ↗

Comment 3 of 3 : Response to Reviewer YTR3

Q3. What do the blue boxes in Figures 1 and 2 mean?

A3. Blue marks what belongs to HAPERA: its decoupled window design, the components of its estimator and its results. The caption of Figure 1 of the revised paper now opens by saying what blue marks in it, and the opening of Appendix C states the convention in full. The submitted Figure 1 was an overview of training and deployment; its blue boxes were the observation window, HAPERA, the posterior and the note on the lifetime that scales the window. It is now the redrawn Figure 1. The submitted Figure 2 showed how the allocation residual attaches to the vector field, with blue on the residual's correction and the coordinates that receive it. It is now Figure 5 in Appendix C.1, whose caption names the blue corrections and the blue-outlined slots that receive them.

Q4. Could Section 3 use simpler notation and an overview, with implementation equations moved to the appendix?

A4. In the revised paper, the section on the allocation residual (submitted §3.2, now §3.4) opens with an overview in words. Of the submitted Eqs. 5–7, the two implementation equations have moved to Appendix C.1 (Eqs. 8–9), and one remains in the main text, as Eq. 5. Each symbol is explained in words where it appears. For Proposition 1 and Corollary 1 (§3.2 of the revised paper), the new Figure 2 shows on a scalar model what each one says, and their technical conditions and proofs are in Appendix B.

The overview says what the residual does. A unit's parameter is the population location plus a multiple of the unit's own offset, and the records speak about that parameter as a whole, not about how it is split between population and unit. The training loss scores the location and the offsets separately, so the estimator has to learn the split, and a wrong split can shift the posterior of the population location even where every unit's parameter fits the records (§1). The residual moves the population location one way and every unit's offset the other way by the matching amount, so each unit's parameter stays where it was and only the split changes. Figure 4a in Appendix C.1 shows this with numbers. With a multiple of 2, moving the location from 3 to 5 and a unit's offset from 2 to 1 keeps the unit's parameter at 7 (3 + 2·2 = 5 + 2·1).

Reviewer_YTR3OpenReview ↗

Rebuttal Follow-Up

Thank you for the detailed rebuttal and the revised manuscript. My main points have all been addressed; the new Figure 1 presents a significant improvement. I maintain some concern over the density of the paper and its heavy notation. I have therefore raised my score to 2 (weak rejection).

Comments on the paper

1 comment

Dear authors and reviewers,

The author-reviewer discussion window will close tomorrow on November 18.

@authors: If not done already, please respond to the reviewers' comments before the discussion window closes. I would advise short and concise responses over lengthy explanations.

@reviewers: Please check the authors' responses and provide any additional comments before the discussion window closes. Consider updating your initial review if necessary, and in particular your score if your opinion has changed. You may increase your score if you feel the authors' responses warrant it, maintain your score if your opinion remains the same, or even decrease your score if you feel the other reviews or the authors' responses reveal new shortcomings. At the very least, please acknowledge the authors' responses.

Thank you for your attention and cooperation.

The AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.