acceptodds
Under review as a conference paper at ICLR 2027

Simulation-Free Variational Inference for Latent Continuous-Time Markov Chains

Abstract

Latent continuous-time Markov chains (CTMCs) are a widely-used model class in systems biology, epidemiology, and spatial statistics, but learning them from noisy observations requires inference over unobserved continuous-time dynamics and can become prohibitively expensive in large state spaces. We introduce CTMC Matching, a simulation-free variational inference method for latent CTMCs that directly parameterizes the posterior's time marginals. Given a marginal curve, the KL-optimal path measure realizing those marginals has an explicit structure: its transition rates are an exponential tilt of the prior rates by a potential that solves a concave dual problem. We prove that this projection is exact: at the exact smoothing marginals it recovers the smoothing posterior. We develop efficient algorithms that couple this inner optimization with end-to-end learning of both the variational posterior, amortized with a neural network, and the model parameters. For structured CTMCs such as interacting particle systems and chemical reaction networks, we exploit model structure to obtain scalable implementations without enumerating the full joint state space. Across a range of systems, CTMC Matching achieves competitive inference and parameter-learning performance while avoiding latent-trajectory simulation during optimization.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
variational inference, latent stochastic processes, latent dynamics models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_teWSOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

Inferring the continuous latent states of latent stochastic differential equations (SDEs) is very challenging for high-dimensional state spaces and long time horizons. Recent work has shown that variational inference for latent SDEs can be made simulation-free (Course & Nair, 2023a; Bartosh et al., 2025; Smith et al., 2026). Smith et al. (2026) address this problem in a simulation-free way with the Helmholtz-SDE, which optimizes the variational objective over path laws compatible with the prescribed marginals. Built on that work, the authors developed a discrete-state counterpart of this idea for the continuous-time Markov chain (CTMC), which has a discrete state space, continuous time, and the Markov property. They introduced CTMC Matching, a simulation-free variational inference method for latent CTMCs.

Critical strengths

This paper is clearly written, well-organized and has Math rigor in derivation and proofs. The authors explicitly acknowledge the close conceptual connection to Smith et al. (2026) and appropriately credit the mathematical foundations underlying their proofs to Mielke et al. (2014), Renger (2018), and Patterson & Renger (2019).

Critical weaknesses
  1. The authors worked on Bayesian smoothing problem but they did not provide a convergence guarantee or a quantitative error bound between its learned variational posterior and the true Bayesian smoothing posterior, which is a standard practice in theoretical papers of this field.

  2. The authors solved the globally KL-optimal generator problem in Equation (5). However, for interacting particle systems, the authors replaced it with independent local optimization problems conditioned on neighborhood configurations in Equation (12). The authors proved that these local solutions produce a globally compatible generator and that the resulting path cost decomposes in Proposition 3.5. But they did not establish that this generator minimizes the original global KL objective over all generators compatible with the product-form marginals.

Questions for authors
  1. Can you provide a rigorous convergence guarantee or a quantitative error bound between its learned variational posterior and the true Bayesian smoothing posterior?

  2. Can you resolve the CTMC local matching issue for global KL-optimal?

  3. The high-dimensional analysis uses the product form of q_t^{\phi}, but the exact smoothing distribution generally does not factorize in practice. Therefore, how could you guarantee the exact marginals belong to the proposed variational family?

  4. The theoretical assumptions seems very restrictive and excludes many important CTMC examples.

3 comments
AuthorsOpenReview ↗

Response to Reviewer teWS (1/2)

We thank the Reviewer for the careful review of our results, and for pointing out two results that the paper was missing. Both are now in the revision.

Q1. "Can you provide a rigorous convergence guarantee or a quantitative error bound between its learned variational posterior and the true Bayesian smoothing posterior?"

The revision provides both (new Theorem 3.5, proved in Appendix B.5), which we briefly describe next. Let qtq_t be the variational marginals, pt⋆p_t^\star the exact smoothing marginals, Q\mathbb{Q} be the variational posterior, and P(⋅∣Y)\mathbb{P}(\cdot \mid \mathcal{Y}) the smoothing posterior, as laws on paths. We measure the error of the marginals and of their time derivatives by

δ2=∫0T(∣q˙t−p˙t⋆∣12+∣qt−pt⋆∣12) dt.\delta^2 = \int_0^T \Big( | \dot q_t - \dot p_t^\star |_1^2 + | q_t - p_t^\star |_1^2 \Big)\, \mathrm{d}t . We show the following properties:

  • Unique optimum. The objective is a convex function of the marginal curve, and it is minimized if and only if qt=pt⋆q_t = p_t^\star for all tt.
  • Quantitative upper bound. For a sequence of variational parameters (ϕn)n≥1(\phi_n)_{n\ge 1}, we show that if δϕn→0\delta_{\phi_n}\to 0 then DKL(Qϕn∥P(⋅∣Y))≤K δϕn2→0D_{\mathrm{KL}}\big(\mathbb{Q}_{\phi_n} \Vert \mathbb{P}(\cdot \mid \mathcal{Y})\big) \le K \,\delta_{\phi_n}^2 \to 0 for all nn large enough, where the constant KK is explicit and depends only on the smoothing posterior. This requires showing that the path-space KL divergence can be upper bounded by the error of its marginals and their time derivatives when the rates are those optimized with the inner solve in CTMC Matching. Proposition B.1 in Appendix B bounds the same KL divergence for arbitrary marginals, by the error of the initial distribution plus a weighted integral of the same two errors.
  • Optimization error. Remark B.2 splits the KL divergence of the learned posterior into the optimization error and the smallest KL divergence attainable in the family.

As is common in these types of results, we do not prove that stochastic-gradient training of a neural network converges to these solutions, but rather that the distance to the true smoothing posterior can be bounded in terms of the error of the marginals under CTMC Matching.

Q2. "They did not establish that this generator minimizes the original global KL objective over all generators compatible with the product-form marginals." / "Can you resolve the CTMC local matching issue for global KL-optimal?"

The Reviewer is correct in this observation: in general it does not, and the revision now states exactly what holds and how to close the gap. Let Qloc\mathbb{Q}^{\mathrm{loc}} be the posterior with the local rates, and Q\mathbb{Q} the posterior with the globally optimal rates at the same marginals.

In the new Proposition 3.7, we show: DKL(Qloc∥P(⋅∣Y))=DKL(Q∥P(⋅∣Y))+DKL(Qloc∥Q).D_{\mathrm{KL}}\big(\mathbb{Q}^{\mathrm{loc}} \Vert \mathbb{P}(\cdot \mid \mathcal{Y})\big) = D_{\mathrm{KL}}\big(\mathbb{Q} \Vert \mathbb{P}(\cdot \mid \mathcal{Y})\big) + D_{\mathrm{KL}}\big(\mathbb{Q}^{\mathrm{loc}} \Vert \mathbb{Q}\big). The last term is zero only in special cases, for example when the sites do not interact.

We measure the contribution of each component on a small Ising model, where the exact smoother is tractable, and find that on this example improving the posterior family leads to much higher gains than solving for the rates globally. See Appendix C.1 and the response to the next question for more details.

Q3. "The exact smoothing distribution generally does not factorize in practice. Therefore, how could you guarantee the exact marginals belong to the proposed variational family?"

We cannot, and we do not assume it for Local CTMC Matching. Theorems 3.4 and 3.5 concern CTMC Matching with an unrestricted family, and we explicitly address restricted families, such as the ones considered in Local CTMC Matching, in the new Remark B.2. There, the KL divergence to the exact posterior is the optimization error plus the smallest KL divergence attainable in the family.

We measured this second term on a model with an exact posterior (Figure 2) and, most importantly, generalize our Local CTMC Matching posterior to factorize across non-overlapping blocks of sites. This simple addition, which leaves the algorithm unchanged, allows for an "interpolation" between the Local CTMC Matching and CTMC Matching (see Figure 2), which can be tuned based on the complexity of the problem and the compute budget.

On a 4×44\times 4 Ising lattice, block marginals of 2×22\times 2 sites roughly halve the KL divergence of product marginals to the exact posterior at every coupling strength. Following up on the previous question, on the same model the term DKL(Qloc ∥ Q)D_{\mathrm{KL}}\big(\mathbb{Q}^{\mathrm{loc}} \,\Vert\, \mathbb{Q}\big) accounts for only 0.25%0.25\% to 2.3%2.3\% of the KL divergence to the exact posterior (Appendix C.1), which is much smaller than the gain from a more expressive marginal family.

AuthorsOpenReview ↗

Response to Reviewer teWS (2/2)

Q4. "The theoretical assumptions seems very restrictive and excludes many important CTMC examples."

We would like to go through the assumptions (Appendix A.1) one by one, since our experiments already include the cases that each of them seems to exclude.

  • Finite state space. It is used for the general theory. Reaction networks with unbounded counts are covered by the construction of Section 4.2, where Lemma E.1 gives an example of compatible rates in closed form on the infinite state space.
  • Positive prior rates on a strongly connected graph. To the best of our knowledge, this assumption is required to make the potential unique. However, cases not covered by this assumption can be addressed with simple fixes in practice, and we explicitly show this in our experiments: irreversible dynamics are handled with a small rate floor 0<ϵ≪10< \epsilon \ll 1 in the prior at training time, as stated in Appendix A.1: the wildfire model (unburned →\to burning →\to burned) and the enzyme and Lotka–Volterra models are of this kind, and their exact references use the same floor.
  • Positive likelihoods. These are needed only for the exactness and convergence theorems. The flashing ratchet has nearly noiseless observations (likelihood 0.9990.999), yet our method is competitive with the baselines.

If the reviewer has specific examples in mind of CTMCs that may not be covered by our framework, we would be glad to discuss them.

Thank you for the rebuttal and the additional clarifications. After considering the paper and the authors’ responses as a whole, I still find the overall level of methodological innovation limited. Therefore, I have decided to maintain my original score.

Reviewer_MZyZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes CTMC Matching, a simulation-free variational inference method for latent CTMCs (CT-HMMs). An amortized network directly parameterizes the posterior time marginals. At each queried time, the posterior rates are chosen as the KL-optimal rates consistent with the master equation, which are an exponential tilt of the prior rates by a potential obtained from a concave dual problem. The envelope theorem removes the need to differentiate through this inner solve, and the construction recovers the exact smoothing posterior when the marginals are exact. For large structured systems, a mean-field “local” variant solves small per-site problems. Experiments cover a discrete flashing ratchet, three synthetic chemical reaction networks plus real transcription data, and wildfire spread modelled as an interacting particle system.

Critical strengths
  1. Choosing KL-optimal rates for given marginals means the only approximation lies in the marginal family, and the exactness result (Theorem 3.4) makes this precise. The proofs are careful, and the inner solvers come with explicit acceptance checks. This gives the method a solid foundation beyond a heuristic parameterisation.
  2. On 64×64 and 128×128 wildfire grids, where forward–backward smoothing is impossible, the local variant trains with inner-solve cost below 2% of a training step and gives competitive forecasts. This shows the approach can reach regimes that matter in practice.
Critical weaknesses
  1. The paper is motivated by the cost of simulation-based inference, yet it reports no end-to-end runtime or memory comparison with any baseline, and no scaling study in the time horizon or number of observations. Many baseline numbers are copied from prior work. For the enumerated CRNs, one inner solve takes 1.7–7.5 s, and an exact smoother is available. This contradicts the main text’s statement that enumeration is infeasible for these benchmarks, and leaves out exact maximum likelihood as a natural baseline where it is tractable.
  2. Results are mixed: DFR parameter error is worse than FIM (5.33% vs 4.67%), low-illumination transcription CRPS is worse than both baselines, daily wildfire AP and recall are below UTAE, and weekly wildfire prediction is within error bars of TwistNet–KL. The near-zero reconstruction error is not informative, since the recognition network receives the emission log-likelihoods as input. In addition, scalability depends on a mean-field family that stays product-form at all times and cannot represent inter-site correlations, and each experiment needs a custom marginal parameterisation, which limits how general the method is.
Questions for authors
  1. Please report end-to-end wall-clock time and peak memory to reach a given accuracy (ELBO, posterior MSE or parameter error) for CTMC Matching and the main baselines (NeuralMJP, EP, TwistNet–KL / NAS-X). How does cost per step scale with the horizon TT and number of observations KK compared with an ODE-based method?
  2. Section 4 presents the CRNs as having state spaces where enumeration is infeasible, but Appendix E.2.1 truncates them to at most 10,201 states, where exact forward–backward smoothing is used as the reference. On these truncated models, exact maximum likelihood by gradient descent is also feasible. How does CTMC Matching compare with it in relative parameter error and time?
  3. Local CTMC Matching keeps the posterior product-form at all times. On a small interacting system where exact smoothing is feasible, how large is the gap in posterior accuracy and ELBO relative to the exact posterior?
  4. Since the recognition network receives the emission log-likelihoods and uses basis functions centred at the observation times, is the near-zero reconstruction BCE informative? Could you report reconstruction on held-out (masked) days instead?
  5. Outside the DFR, how often do inner solves fail the acceptance checks (and receive the fixed penalty with no gradient)? Relatedly, does the large standard error for Lotka–Volterra (0.618 ± 0.523) come from a few failed trajectories?
3 comments
AuthorsOpenReview ↗

Response to Reviewer MZyZ (1/3)

We thank the Reviewer for the positive assessment of our work, and for the excellent points raised. We follow the order of the questions, and then address the remaining points of W1 and W2.

Q1. "Please report end-to-end wall-clock time and peak memory to reach a given accuracy for CTMC Matching and the main baselines (NeuralMJP, EP, TwistNet–KL / NAS-X). How does cost per step scale with the horizon and number of observations compared with an ODE-based method?"

We now report all of them in Appendix F.

  • NeuralMJP (ODE-based), discrete flashing ratchet. Using the same encoder, initialization, batch size, and GPU, the relative parameter error reaches below 10%10\% in 0.250.25 hours in median for CTMC Matching over three runs, versus 6.1 hours for NeuralMJP (our method is more than 20 times faster). For the transformer encoder used in our main experiment, our method converges to <10%<10\% in 0.450.45 hours.
  • Scaling. If we multiply trajectory length by a factor of 11, 22 and 44, a training step takes 0.450.45, 0.740.74 and 1.311.31 seconds for CTMC Matching, and 2.762.76, 5.175.17 and 9.659.65 seconds for NeuralMJP. Both grow roughly linearly, due to the cost of encoding all observations in the smoothing posterior. Note that one could achieve speedups in CTMC Matching by only conditioning on a window around tt, but we do not investigate the option in this work. We measure peak memory at 2.12.1, 4.04.0 and 8.08.0 GB, against 3.63.6, 6.56.5 and 12.312.3 GB for the baseline. With the Transformer encoder of our main experiment (rather than the controlled comparison), CTMC Matching is still faster per step, but uses more memory than NeuralMJP (see Appendix F).
  • TwistNet–KL, TwistNet–DRE, NAS-X, NASMC, weekly wildfires. We measure the time taken to train each method, with the same setup and number of iterations as those in Table 4: 1.61.6 hours for CTMC Matching, against 10.610.6, 12.012.0, 10.210.2 and 8.68.6 hours for the baselines. Peak memory for our method is at 2.72.7 GiB, versus 7.27.2 to 8.78.7 GiB.
  • EP for chemical reaction networks. EP is cheaper to train: minutes of CPU time, against 1.11.1 to 6.46.4 GPU-hours for CTMC Matching. After training, our amortized posterior takes 77 to 3333 milliseconds per trajectory, against 44 to 99 seconds for EP.

Q2. "Section 4 presents the CRNs as having state spaces where enumeration is infeasible, but Appendix E.2.1 truncates them… exact maximum likelihood by gradient descent is also feasible. How does CTMC Matching compare with it in relative parameter error and time?"

This is correct on both counts, and we provide a clarification for both points here and in the revised paper.

  • Wording. Section 4 now says that the simulated benchmarks are truncated, and "enumeration is infeasible" is reserved for the live-cell data and the wildfires. The truncated benchmarks are there because they admit an exact reference, hence validating the accuracy of each approximate inference method.
  • Exact maximum likelihood. We ran it on all three models (new row of Table 2, Appendix F.2). On the enzyme and gene models our estimates agree with it within a fraction of its standard error: 0.50170.5017 against 0.5016±0.01500.5016 \pm 0.0150, and 9.9979.997 against 10.010±0.07510.010 \pm 0.075. On Lotka–Volterra our three estimates are 1.11.1 to 1.71.7 standard errors below the exact ones, with a parameter error of 3.6%3.6\% against 1.4%1.4\%.
  • Time. Exact maximum likelihood is fast on these benchmarks: 88 seconds, 3.63.6 minutes and 3434 minutes, against 1.11.1, 3.33.3 and 6.46.4 hours to train our model. We do not claim that one should prefer our method or other approximate inference schemes to exact maximum likelihood when it is feasible, and we use it as an exact reference. Moreover, note that exact maximum likelihood is not available for the transcription model (unbounded counts) or the wildfires (340963^{4096} configurations), which are the settings our method is intended for.
AuthorsOpenReview ↗

Response to Reviewer MZyZ (2/3)

Q3. "On a small interacting system where exact smoothing is feasible, how large is the gap in posterior accuracy and ELBO relative to the exact posterior?"

We measured it on a stochastic Ising model on a 4×44\times 4 lattice, with parameters fixed at the truth (new Figure 2, Appendix C.1). The KL divergence from the product-form posterior to the exact smoothing posterior, which is equivalent to the ELBO gap, strongly depends on the coupling strength. Interestingly, the gap is at its largest at intermediate coupling: the posterior is close to a product both for weak coupling, where the sites are nearly independent, and for strong coupling, where strong dependencies make transitions less likely and the posterior concentrates on a single configuration.

Most importantly, the product form is also no longer the only option for high-dimensional state spaces: we generalize our framework to block marginals (new Proposition 3.8), blocks of 2×22\times 2 sites lower the gap at every coupling, which roughly halves the KL to the exact marginals compared to the product posterior.

Q4. "Is the near-zero reconstruction BCE informative? Could you report reconstruction on held-out (masked) days instead?"

We agree that it is not informative about predictive skill: it measures the fit to the observed days, which the baselines nevertheless do not attain. Assessing reconstruction on held-out days provides an interesting picture, and we have added the new comparison in Table 10 and Figure 13 in Appendix E.3. Since the masked configuration is out-of-distribution for the recognition network, we fine-tune the posterior of each model (we also train the baselines, using the settings provided in the codebase of Migliorini and Smyth, 2026). We hide days 11, 33 and 55 from the posterior of every method, and simply train the posterior to reconstruct days 00, 22, 44, and 66 (note that we do not train for masked reconstruction, in order to simulate truly missing observations). CTMC Matching attains 0.99±0.340.99 \pm 0.34 BCE (AUPRC 0.420.42), while the four baselines attain 1.821.82 to 2.292.29 (AUPRC 0.030.03 to 0.140.14), close to predicting no fire at all times (1.981.98). This is despite maintaining their skill at reconstructing observed days, which stays in the range of 1.201.20 to 1.521.52 BCE.

Q5. "Outside the DFR, how often do inner solves fail the acceptance checks? Relatedly, does the large standard error for Lotka–Volterra (0.618±0.5230.618 \pm 0.523) come from a few failed trajectories?"

  • Failed solves are rare: across the reaction-network runs, between 0.004%0.004\% and 0.027%0.027\% of the inner solves fail the acceptance checks (2626 to 530530 out of 0.50.5 to 22 million). On the wildfires, every local solve in the calibration of Appendix E.3 passes the tolerance within eight sweeps.
  • The standard error in LV is due to an outlier: a single trajectory in which our amortized posterior fails (MSE 52.452.4, against 0.170.17 for EP), and without it our MSE is 0.096±0.0130.096 \pm 0.013. Over all 100100 trajectories our error is lower than EP's on 9898 of them.
AuthorsOpenReview ↗

Response to Reviewer MZyZ (3/3)

W1. "Many baseline numbers are copied from prior work."

The posterior errors of all baselines in Table 2 (stated in its caption) are reproduced, and we reproduced NeuralMJP and all sequential Monte Carlo baselines for time and memory measurements. We take from the publications the parameter estimates of FIM (a pretrained foundation model), the multi-seed NeuralMJP results, the parameter errors of EP in Table 2, and the multi-seed weekly wildfire accuracy of the sequential Monte Carlo baselines.

W2. "Results are mixed…"

We agree with each of the four facts, and our claim is not that CTMC Matching is the most accurate method on every benchmark. Rather, we show that inference without simulating latent trajectories is competitive with methods that do simulate, is cheaper than the simulation-based amortized baselines, and scales to high-dimensional settings where they are too expensive to apply. We would like to remark that:

  • Flashing ratchet: FIM is a pretrained foundation model, and its error (4.67%4.67\%) and ours (5.33%5.33\%) differ by less than one percentage point.
  • Low-illumination transcription: the three scores are within one standard error (14.6±1.014.6 \pm 1.0, 14.4±1.414.4 \pm 1.4, 14.2±1.714.2 \pm 1.7), and in the high-illumination condition ours is the lowest (19.219.2 against 24.824.8).
  • Daily wildfires: UTAE is a supervised forecaster trained directly for the task of one-day-ahead prediction. Yet, our generative model is better on four of the six metrics (F1, precision, IoU, cross-entropy).

W2. "Each experiment needs a custom marginal parameterisation, which limits how general the method is."

We partly disagree, since the only requirement for the recognition network is to have differentiable marginals. It is exactly the generality of the method that allows for custom marginal parameterization for each data type. The revision adds two data points: on the flashing ratchet experiment, CTMC Matching trains with two different encoders (Transformer and ODE-RNN, Appendix F.1), and on the Ising model it works with no network at all, with (non-amortized) marginals that are piecewise linear in time.

Reviewer_ipEKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper provides a new way to do simulation-free VI for continuous time markov chain (CTMC). Traditional VI approaches typically parameterize the posterior rate matrix; however, to compute the ELBO, one needs to solve for the variational posterior marginal by solving an ODE. This paper instead parameterizes the variational posterior marginal, and choses the rate that optimizes the KL to the prior. To solve for that rate, a dual problem needs to be solved via an inner-loop optimization. The proposed method is evaluated on discrete flashing ratchet, chemical reaction networks and wildfire modeling tasks, where it demonstrates promising performance.

Critical strengths
  • The paper proposes a novel and principled framework for simulation-free VI for CTMC.

  • The paper also develops a tractable algorithm to solve the constrained rate optimization.

  • The paper also provides theoretical guarantees that rate recovery would not introduce additional gap, which seems to be stronger then standard marginal-to-rate construction.

  • The empirical validation is broad and intensive.

Critical weaknesses
  • The conceptul novelty is quite similar to [1], though [1] was developed for SDE and this work is for CTMC.

  • The paper lacks empirical evidence for computational efficiency. In particular, I'd appreciate comparison to simulation-based approaches across different settings.

  • In high dimensional interacting CTMCs, posterior factorization is used for the marginal which seems to be a severe constraint for interacting systems.

[1] Henry D Smith, Brian L Trippe, and Scott W Linderman. Closing the approximation gap in simulation-free latent SDEs. arXiv preprint arXiv:2606.16138, 2026.

Questions for authors

See weakness. I'd expect more discussion / empirical studies around these points.

2 comments

We thank the reviewer for the positive assessment and for three clear points, which we answer in turn.

W1. "The conceptual novelty is quite similar to Smith et al., 2026, though it was developed for SDE and this work is for CTMC."

We agree on the connection, and the paper states it openly (Sections 1 and 2.2), which Reviewer teWS lists as a strength. What we share with Smith et al. is the principle: that one can learn the posterior marginals of a continuous-time stochastic process, then choose the best dynamics compatible with them. What it takes to realize this principle on discrete states is different, and it is the content of the paper:

  • Different optimization target. At fixed marginals, the freedom is in how probability flux is distributed over the transitions of a graph, which is directly determined by the transition rates of the posterior CTMC (in place of the drift of an SDE). We show that the KL-optimal choice is an exponential tilt of the prior rates by a potential that solves a concave problem, which we solve exactly at every queried time, with gradients that do not pass through the solver (Propositions 3.2 and 3.3).
  • Extension to high-dimensional state spaces. Local CTMC Matching for interacting particle systems (Section 3.2), the closed-form rates for reaction networks with unbounded counts (Lemma E.1), and score-function gradients for discrete neighborhoods have no counterpart in the SDE setting.

Moreover, the revision adds theory that is specific to this setting: the convergence result (Theorem 3.5), the local-versus-global decomposition (Proposition 3.7), and block marginals (Proposition 3.8).

W2. "The paper lacks empirical evidence for computational efficiency. In particular, I'd appreciate comparison to simulation-based approaches across different settings."

We now provide it, in two settings with simulation-based baselines (Appendix F).

  • Discrete flashing ratchet, against NeuralMJP, which integrates the master equation at every step. With the same encoder, initialization, batch size and GPU, CTMC Matching stays below a 10%10\% parameter error after about 0.250.25 hours, against about 66 hours or more for NeuralMJP (our method is more than 20 times faster). Each training step is 66 to 77 times faster, with 35%35\% to 43%43\% less memory.
  • Weekly wildfires, against four sequential Monte Carlo methods, which simulate particles during training. CTMC Matching trains in 1.61.6 hours, against 8.68.6 to 12.012.0 hours, with about a third of the memory.

On the truncated reaction networks the baseline EP is cheaper to train than our method, since it doesn't require training a recognition network (Appendix F.2, and point 2 of the global response). However, we are two orders of magnitude faster at posterior inference once the network is trained, and achieve lower error in our comparison.

W3. "In high dimensional interacting CTMCs, posterior factorization is used for the marginal which seems to be a severe constraint for interacting systems."

We agree that it is a constraint, so we added a variant that overcomes this limitation: a variational posterior factorized over non-overlapping blocks, which can represent all correlations inside each block while keeping the algorithm unchanged. We measured its effect on an Ising model where the exact smoother is computable: blocks of 2×22\times 2 sites roughly halve the KL divergence to the exact posterior at every coupling strength (Figure 2). We also want to clarify one point: unlike in classical mean-field approximations for CTMCs, the factorization is on the time marginals only, and the posterior rates of a site depend on the state of its neighbors.

I would like to thank the authors for their reply. My concerns are all resolved and I maintain my positive evaluation of the paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global response

We thank the reviewers for their careful reading, constructive feedback, and overall positive assessment of our work. We are glad that they found our framework "novel and principled", with "broad and intensive" empirical validation (ipEK); that the exactness result and the proofs give the method "a solid foundation beyond a heuristic parameterisation", and that it "can reach regimes that matter in practice" (MZyZ); and that the paper is "clearly written, well-organized" and has "math rigor in derivation and proofs" (teWS).

Two concerns were shared by several reviewers, and we address both in the revision. All changes are in blue, and the details are in the individual replies.

1. Beyond product-form marginals (ipEK, MZyZ, teWS). We show that the product form is not required, and we measure what it costs. More specifically:

  • We generalize Local CTMC Matching to block marginals (new Proposition 3.8): we show that one can factorize the marginals over non-overlapping blocks of sites, representing all correlations inside each block, while keeping the algorithm unchanged.
  • We measure the gap on a model with an exact posterior (new Figure 2, Appendix C.1): a stochastic Ising model on a 4×44\times 4 lattice, across a grid of coupling strengths. In terms of nats to the exact posterior, we show that the new block marginals interpolate between Local CTMC Matching and CTMC Matching (see Figure 2). We illustrate that the gains are steady as we increase the size of the lattice (Figure 7).

2. Computational cost (ipEK, MZyZ). We directly compare the computational cost of each method (across Section 4, with details in the new Appendix F). Here is a summary of the results:

  • Against NeuralMJP, a simulation-based amortized inference method, in a controlled comparison on the discrete flashing ratchet: CTMC Matching reaches a relative parameter error below 10%10\% more than 20 times faster, in median over three runs (0.25 against 6.1 hours). A training step of CTMC Matching is 66 to 77 times faster and uses 35%35\% to 43%43\% less memory.
  • Against the sequential Monte Carlo baselines on the weekly wildfires: CTMC Matching trains to the accuracy reported in the paper in 1.61.6 hours, against 8.68.6 to 12.012.0 hours for the baselines, with about a third of the memory.
  • Against expectation propagation on the chemical reaction networks: EP is cheaper to train, since it has no recognition network. However, our estimates are closer to the exact maximum likelihood estimate, which we added as a reference, on all three models. Once trained, our posterior is also two orders of magnitude faster to compute: 77 to 3333 milliseconds per trajectory, against 44 to 99 seconds for EP.

Moreover, we have added new results to our theory, as advised by Reviewer teWS. The revision adds a convergence result with a quantitative error bound (Theorem 3.5), and an exact characterization of what the local rates lose with respect to the globally-optimized ones (Proposition 3.7). The details are in our reply to Reviewer teWS.

We also ran the analyses that Reviewer MZyZ asked for: a comparison with exact maximum likelihood on the truncated reaction networks and the masked reconstruction of held-out days on the weekly wildfires experiment (Table 11, Figure 13).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.