acceptodds
Under review as a conference paper at ICLR 2027

Structured Neural SDEs for Functional Calibration

Abstract

Neural Stochastic Differential Equations (Neural SDEs) provide flexible continuous-time generative models, but generic neural drift and diffusion networks are costly to simulate on long horizons and can give unstable gradients when the training signal is a path functional rather than a pointwise observation. We introduce SLiSDE, a family of Neural SDE models built from structured linear stochastic layers. Parallel-in-time simulation is obtained at the layer level, while expressivity is recovered by gated in-flow stacking: previous-layer paths modulate the next layer's latent flow through learned gates. For functional calibration tasks in which rare paths dominate the loss, we add an optional Girsanov tilt that acts as a learned importance sampler with an exact likelihood-ratio correction. We prove well-posedness, a discretisation error bound, validity of the change of measure, and a universality result: the terminal laws of the gated stack are dense in the space of square-integrable laws. Experiments on functional calibration benchmarks show that the structured model outperforms fully neural SDE baselines while retaining parallel-time simulation and stable importance weights.

Primary area:
learning on time series and dynamical systems
Keywords:
Neural SDEs, Functional Calibration, Time-series

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_USAbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SLiSDE, a Neural SDE architecture that uses structured linear stochastic layers to reduce simulation costs. Each discretized layer forms an affine recurrence compatible with parallel associative scan, while gated in-flow stacking preserves nonlinear expressivity by allowing preceding-layer paths to modulate the next layer’s transition and offset. For functional calibration, the authors introduce an optional Girsanov tilt that increases the frequency of rare paths under an auxiliary sampling distribution. An exact likelihood-ratio correction enables estimation of expectations under the original model, with the aim of reducing Monte Carlo error and improving tail-sensitive calibration. The paper presents theoretical results on well-posedness, discretization error, validity of the change of measure, and terminal-law universality, together with experiments on a synthetic path-functional benchmark and the DAX and SPX option-surface benchmarks.

Critical strengths
  1. Scan-compatible structured dynamics. The structured linear SDE layers yield affine recurrences compatible with associative scan, enabling parallel-in-time simulation without introducing an additional approximation to the discretized recurrence.
  2. Expressive nonlinear coupling. Gated in-flow stacking introduces nonlinearity through inter-layer coupling while preserving affine dependence within each layer. This enhances expressivity without sacrificing scan compatibility.
  3. Targeted rare-path sampling. The Girsanov tilt addresses the difficulty of sampling rare paths in functional calibration by learning a proposal that visits relevant tail regions more frequently. Likelihood-ratio correction allows these samples to estimate expectations under the original model, offering a way to improve tail-sensitive estimation.
Critical weaknesses
  1. The architectural contribution needs clearer differentiation from prior work in selective state-space models. The principle of input-dependent gates modulating linear recurrences is present in structured sequence models such as the structured linear CDE SLiCE (Walker et al., 2025). While SLiSDE successfully adapts this to Brownian-driven SDE layers, the paper would benefit from explicitly stating what uniquely emerges from the stochastic setting that a deterministic gated recurrence could not achieve. For instance, does the independent Brownian driver per layer enable qualitatively different dynamics than a deterministic gated CDE? A brief discussion would clarify the novelty.
  2. The gating mechanism is not isolated by ablation. All reported SLiSDE variants use gated in-flow (Appendix C), and Table 5 ablates only noise type and block size. The contribution of the gate itself, inter-layer nonlinearity via path-dependent modulation, is not directly compared against alternatives such as: (a) a simple concatenation-based coupling without learned gates, (b) untrained (random or fixed) gates to isolate the effect of learning, or (c) a single deep structured layer with matching parameter count to isolate the effect of depth. Without such comparisons, the reported improvements cannot be attributed specifically to the gating design.
  3. The main-text presentation should better reflect the qualified empirical results. The Girsanov tilt is presented as a contribution, but its quantitative comparisons and controller ablations appear only in Appendix E, with a brief summary in Sect. 6.2. It may be better to list the results of Girsanov tilt in main text.
  4. Please check the reference section carefully. The citation 'H. Smith et al. Calibrating generative models to distributional constraints' is not completed and remains revision marks.
  5. The paper uses θ\theta for both model parameters and Girsanov controller parameters (θc)(\theta_c). While clearly differentiated in Appendix C.8, the main text could be clearer.
  6. Table 1 title shows "Computational cost of composing affine transition maps", but the Neural SDE column reports recurrent cost, not scan cost.
Questions for authors

Please refer to 'Weaknesses'.

2 comments
AuthorsOpenReview ↗

Response to Reviewer USAb (1/2)

We thank the reviewer for the careful reading. The current version of the work addresses all the remarks.


Weakness 1: Differentiation from selective state-space models

Input-dependent gates modulating linear recurrences are central to selective state-space models, including SLiCE (Walker et al., 2025). We have clarified this connection in Section 3.3 and made the distinction from our architecture more explicit.

The main difference is not simply that SLiSDE is stochastic: our SLiCE baseline is also driven by Brownian motion and therefore produces random paths. Rather, the distinction concerns how stochasticity enters the architecture and how it interacts with the gated layers.

In the SLiCE baseline, each successive block processes the path produced by the preceding block, without introducing an additional stochastic driver. In SLiSDE, each layer is instead a linear SDE with its own diffusion coefficients and Brownian driver. The previous-layer path modulates the coefficients through causal gates, but does not itself serve as the driving signal. This provides two additional capabilities:

  • Layerwise stochastic dynamics. The Brownian drivers can be shared, correlated, or independent, and Theorem 4 establishes well-posedness for any such correlation structure. When a layer has an independent Brownian component, its output retains genuine conditional randomness given the preceding layers. This distinguishes it from a deterministic transformation of the preceding path, as in the SLiCE baseline considered here.
  • Layer-specific changes of measure. The Girsanov construction in Section 4 acts only on the innovation driving the final layer. The innovation tilt leaves the prefix dynamics unchanged, while the additional initial-state shift modifies their starting distribution. The resulting likelihood ratio remains available in closed form (Theorem 5). This separation permits targeted importance sampling without changing the prefix Brownian driver.

Independent drivers. The reviewer asks whether independent drivers produce qualitatively different dynamics. In law they do, as the first point shows; in calibration they make no statistically significant difference (improvement) for the experiments we have considered. The original experiments use a shared Brownian motion on TOY and SPX and, on DAX, drivers correlated with the first through a learned coefficient initialised at 0.95 (now documented in Appendix D). We retrained the two- and three-layer configurations of Tables 2–4 with an independent Brownian motion in each layer, all other settings unchanged, with seven paired seeds. The table reports the loss ratio between the independent-driver and original configurations (geometric mean; p from a paired t-test on the log-ratios).

TOY-2TOY-3DAX-2DAX-3SPX-2SPX-3
Loss ratio1.070.731.021.031.020.97
p0.780.140.700.720.690.41

No configuration shows a statistically significant difference. A layer can therefore be given a driver of its own at no measurable cost, which is precisely what the last-layer Girsanov tilt requires.

AuthorsOpenReview ↗

Response to Reviewer USAb (2/2)

Weakness 2: Gate ablation

We ran the three ablations the reviewer proposes, and two more that separate the two gates. All follow the paper's protocol on the SLiSDE cells of Tables 2–4 (two and three layers on each dataset), with seven seeds, and each variant is compared with the full model seed by seed.

  • (a) Coupling without a learned gate: the previous layer's state enters the next layer through a plain linear map.
  • (b) Untrained gates: both gates are kept, frozen at their random initialisation.
  • (c) A single layer: one structured layer at widths 32 to 256, the widest with more parameters than the two-layer stack.
  • Each gate alone: the flow gate without the offset gate, and the reverse.

The table gives the loss relative to the full model (geometric mean over seeds; above 1 means the full model is better; p from a paired t-test on the log-ratios).

Cell(a) Linear coupling(b) Frozen gatesFlow gate onlyOffset gate only
TOY, 2 layers1.38 (p = 0.09)6.90 (p < 0.001)0.93 (p = 0.60)0.84 (p = 0.43)
TOY, 3 layers0.98 (p = 0.94)7.91 (p < 0.001)0.97 (p = 0.88)1.26 (p = 0.14)
DAX, 2 layers1.79 (p = 0.002)1.09 (p = 0.61)0.87 (p = 0.06)1.22 (p = 0.12)
DAX, 3 layers3.47 (p = 0.001)2.21 (p = 0.007)1.54 (p = 0.01)1.50 (p = 0.003)
SPX, 2 layers1.42 (p = 0.05)2.08 (p < 0.001)1.44 (p = 0.001)1.19 (p = 0.26)
SPX, 3 layers1.63 (p = 0.02)1.85 (p < 0.001)1.06 (p = 0.35)1.01 (p = 0.94)

Learning the gates matters in five of the six cells (column a and b). In short, the gated stack earns its place on the two real option surfaces, while on the synthetic benchmark a single structured layer is already enough.

For ablation (c), the same ratio for the best single layer, relative to the better of the two- and three-layer full models:

(c)TOY (2 layers)DAX (3 layers)SPX (2 layers)
Best single layer1.01 (p = 0.98)1.49 (p = 0.003)1.95 (p < 0.001)

Weaknesses 3–6

We appreciate these suggestions and have followed them. The Smith et al. citation is completed, the controller parameters are now ϕ\phi throughout, with θ\theta reserved for the backbone, and Table 1 separates the sequential work, the parallel time and the total work of the scan. The principal results of the Girsanov tilt are stated in Section 6, and we intend to move the DAX calibration results (Table 9) there in the final version together with a more detailed explaiation of the experiments and performance.

As for Table 1, the Neural SDE (parallel) scan cost is not reported because Neural SDEs do not admit a parallel scan (no associative flows). We have clarified this in the manuscript.

Reviewer_A68bOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

Introduces a Neural SDE model class where each layer is an SDE with linear-in-state drift and diffusion functions. Each layer modulates the next by updating the drift and diffusion coefficients via predicted gating parameters. This allows for parallel-in-time evaluation layerwise. The model is applied to functional calibration tasks and compared to Neural SDE and CDE models.

Presentation

The paper is nicely written and easy to follow in general. There are a few improvements that could be made, listed below.

  • Section 2.1 introduces the linear SDE, which indicates that the model is a single-layer construction. It could be clearer that this model has LL layers where each layer is Eq (2). For example, an overview figure would provide the reader with some assistance here.
  • Section 3.3 is difficult to follow without Appendix C. It would be clearer if the causal features were defined along with the diagonal scale and offset's functional form. This appears in the Appendix but is relatively key to the method description.
  • Section 3.5 describes an algorithm in text. A line-by-line algorithm description would be significantly more readable.

Informativeness

  • The model construction of linear-in-state SDE layers with non-linearity introduced by gating parameters from the previous layer's state is clean.
  • The functional calibration objective is a clear characterisation of the problem. This is appreciated as a special case of the Neural SDEs as GANs work, where in the functional calibration setting the path functionals are known. The connection and extension to generative modelling through GANs is hinted at in the future work.
  • The Girsanov tilt on the final layer is an interesting approach for up-weighting rare events but is not found to produce a statistically significant improvement in Table 9.

## Soundness My main concern about the papers soundness is empirical. Tables 2 and 3 do not match parameter counts for each method. This becomes particularly unsettling when it is shown directly in Table 2 that the loss appears to have no correlation on parameter count in general. We are therefore unable to determine whether the SLiSDE method outperforms SLiCE and Neural SDE by virtue of its construction or because it has found a favourable parameter configuration.

To assess the predictions of each method, a fair hyperparameter search should be performed over the model size with all other training settings fixed. The paper reports a search over two "width" sizes (32, 64) corresponding to the latent dimension but fixes the width to 128 for the Neural SDE drift and diffusion networks. Depth is searched over sizes (2, 3) but it is not specified what depth means for the Neural SDE model.

Additionally, the timing results are not comparable due to parameter count differences. In Table 2, the ms / epoch for the Neural SDE is (1.3x and 1.6x) slower than SLiSDE while it's parameter count is (1.2x and 2.1x) larger. The timing difference could therefore be a result of the model size rather than model class. Further, as evidenced by Appendix E Table 6/7, the proposed speedup of SLiSDE vanishes at relatively modest batch sizes and training horizons.

Critical strengths

The model construction is novel and an interesting contribution. Specifically, the linear-in-state SDE hidden layers and non-linear gating between layers provides a useful separation that allows parallel-in-time layer-wise integration while maintaining expressivity. This promises decreased runtime and potentially improved performance over baselines.

Critical weaknesses

The benefit of this construction is not experimentally validated in terms of improved performance or decreased runtime over general Neural SDE baselines. The empirical results do not set each model class on an equal footing, or perform a sufficient hyperparameter search to identify optimal architectures for each baseline. It is therefore difficult to evaluate whether the improved loss and decreased runtime for SLiSDE is a result of the model or a favourable hyperparameter selection. See the questions below for specific recommendations.

Questions for authors
  • Can you report iso-parameter results for the Neural SDE, SLiCE and SLiSDE in Tables 2 and 3?
  • Can you extend the hyperparameter selection to: include single-layer (depth L=1L=1), vary Neural SDE drift and diffusion network widths and report results for all hyperparameter combinations?
  • Can you clarify the Neural SDE implementation: what does depth correspond to? Why should one expect the linear-in-state model to outperform the nonlinear-in-state Neural SDE for matched architectures and parameter counts?
  • Following from the previous question: can the Neural SDE model be implemented with an equivalent form of depth to the linear-in-state models, where the previous layer's state modulates the vector fields of the current layer? If so, is this not the fair baseline to compare to rather than a single-layer model?
  • Sequential SLiSDE evaluation displays a shorter runtime than the parallel-in-time scan evaluation at moderate batch sizes. What is the benefit of the parallel-in-time scan over increasing the batch size of a sequential implementation (with potentially nonlinear drift and diffusion matrices)?
2 comments
AuthorsOpenReview ↗

Response to Reviewer A68b (1/2)

We thank the reviewer for the detailed report and for precise recommendations. We ran the search they ask for. In short: on the two option surfaces the best Neural SDE of the search, whatever its size, stays behind the best SLiSDE; on the synthetic benchmark a well-chosen Neural SDE draws level with it; and the speed advantage holds at equal parameter count. Below the detailed answer to each question.


Questions 1–2: the model classes on an equal footing

Each row of Tables 2–4 is the best cell of its family in a search over the latent dimension (Appendix D). The hidden width of the Neural SDE was searched as well (64 and 128 hidden width). To answer the reviewer's questions in full, we have rerun the grid search of the Neural SDE overits whole architecture under the protocol of the tables, with every other setting fixed (optimiser, schedule, batch size, epochs, seeds, loss): latent dimension 32 or 64, hidden width 64, 128 or 256, and 1, 2 or 3 hidden layers. This gives 18 architectures per dataset, from 23K to 445K parameters. We also trained single-layer (L = 1) SLiSDE and SLiCE models.

Loss ×10⁻⁴, mean ± standard error over seven seeds, parameter counts in brackets:

Neural SDE of Tables 2–4 (L = 2 / L = 3)Best Neural SDE of the 18SLiSDE
TOY1.56 ± 0.21 / 3.02 ± 0.340.77 ± 0.08 (54K)0.85 ± 0.17 (103K), 0.66 ± 0.06 (74K)
DAX1.97 ± 0.12 / 1.91 ± 0.181.82 ± 0.12 (355K)1.40 ± 0.24 (three layers), 1.62 ± 0.07 (one layer, 21K)
SPX11.90 ± 0.11 / 10.29 ± 0.139.36 ± 0.16 (355K)6.44 ± 0.40 (18K)

Timing at equal parameter count

The comparison at equal size is in the paper. Tables 6–7 time a SLiSDE of 136K parameters against a Neural SDE of 137K. Per training step SLiSDE is 5.7–7.2× faster at batch size 64 and 1.3–1.4× faster at 1,024, the batch size of all our calibration runs.

The advantage does not vanish with the horizon; it grows with it, from 5.7× at T = 512 to 7.2× at T = 8,192 (batch size 64). It does shrink with the batch size: at 4,096 paths, where the batch alone saturates a single GPU, the Neural SDE step is the faster one, and the paper reports it.

Model size does not explain the epoch times of Table 2. In the search, the time per epoch of the Neural SDE is set by its number of hidden layers and hardly by its parameter count: 63–78 ms with one hidden layer, 89–135 ms with three. On TOY its best architecture has fewer parameters than either SLiSDE cell (54K against 74K and 103K) and still takes longer per epoch (75 ms against 65–67 ms, timed in the same pass). A single SLiSDE layer of 21K parameters takes 13 ms, against 65–66 ms for the Neural SDE of 23K. The one cell slower than the Neural SDE of its block is the three-layer DAX stack, as the paper states.

AuthorsOpenReview ↗

Response to Reviewer A68b (2/2)

Question 3: depth of the Neural SDE, and what to expect at equal size

Depth. For SLiSDE and SLiCE the depth L is the number of stacked layers. The Neural SDE is a single SDE: its L is the number of hidden layers of its drift and diffusion networks, and its width is its latent dimension. Section 6 now states this.

What to expect. Our claim is not that a layer which is linear in its state is more expressive than a nonlinear one. It is twofold. First, nothing is lost: by Theorem 6 the terminal laws of a two-layer gated stack are dense among the laws with finite second moment, so the restriction excludes no target. Second, this comes at a lower cost, because the nonlinearity moves from every time step to the coupling between layers. Whether the structured model also fits better is then an empirical question, which the search above answers: the best SLiSDE is ahead of every Neural SDE on the two option surfaces and level with the best one on TOY.


Question 4: a Neural SDE with the same form of depth

Such a model can be built, but it is not a different baseline. A stack of Neural SDE layers in which the state of each layer modulates the vector fields of the next is itself one Neural SDE, on the joint state of its layers, with a block-triangular vector field. It is a constrained member of the class the baseline already represents, with a larger latent state, and it costs L sequential nonlinear simulations instead of one. For SLiSDE the situation is the opposite: a single layer is a linear SDE, so stacking is its only source of nonlinearity. The natural counterpart of a deeper SLiSDE is therefore a Neural SDE with larger networks, which is the comparison of Questions 1–2.


Question 5: the scan against a larger batch

A larger batch and the scan use the hardware in different directions. A larger batch simulates more paths at once: it lowers the Monte-Carlo error of the loss, but each path still needs its T steps one after the other, so the pass is not shorter. The scan composes the T steps in O(log⁡T)O(\log T) rounds: the pass is shorter for the same paths, in exchange for more arithmetic (Remark 1 and Table 1 now state both). The two compete for the same device. When the paths that the loss needs already fill it, as with the 1,024 paths of our training runs on a single GPU, the recursion is the efficient mode, and it is the one all our experiments use. When capacity is left, with few paths, transitions that are cheap to compose, or more than one device, the scan turns it into a shorter pass: in Table 7 the forward pass is 1.4× shorter at 64 paths and T = 512.

None of the speed-ups we report over the Neural SDE relies on the scan. They come from linearity in the state (Section 3.3): the transition maps (Fk,gk)(F_k, g_k) of a layer depend only on the Brownian increments and on the path of the previous layer, so all of them are computed at the beginning of the pass in batched tensor operations, and what remains is a cheap structured recurrence zk+1=Fkzk+gkz_{k+1} = F_k z_k + g_k. A Neural SDE cannot do this: its networks read the current state, so their evaluations are inherently sequential. However, we would like to point out that for the application we target, the O(log⁡T)O(\log T) depth is a very desirable property and indeed constitutes a contribution of the paper.


The Girsanov tilt (Table 9)

Table 9 gives means and standard errors over seeds, which hide the pairing. Against the vanilla model the far-call error falls by about a third (0.64×), and it is lower on six seeds of seven (paired t-test, p = 0.03); the total held-out loss falls by 15–24%. What is not significant at seven seeds is the gain over the untilted model given the same path budget, and the paper says so. The clearest evidence is the controlled study of Table 10: at equal budget the tilt reduces the error of rare-event estimates by a factor of 2 at probability 10⁻² and of 4 at 10⁻⁴, where plain Monte Carlo returns zero in 90% of the replications.


Presentation

All three suggestions are implemented: an overview figure of the L-layer stack; the gate input and the two gate equations in the main text; and a line-by-line algorithm of one forward pass (in Appendix C).

We hope this puts the model classes on an equal footing, and that the outcome, a lower loss than the Neural SDE on both real datasets and a lower cost at equal size, addresses the reviewer's main concern.

Reviewer_fw4LOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Neural SDEs are stochastic differential equations whose drift term and diffusion coefficients are represented by unconstrained neural networks. These equations are very useful for learning continuous-time dynamics in financial and physical applications due to their modelling flexibility as they do not require to specify in advance the particular form the drift and diffusion coefficients. However, these models are computationally expensive to run as they require sequential simulations of (nonlinear) stochastic dynamics. To overcome this disadvantage, the authors introduce a novel family of Neural SDE architectures, which they call SLiSDE, designed for learning stochastic dynamics from expectations of path functionals (aka functional calibration). The idea is to bring the nonlinearity from the drift and diffusion coefficient evaluated at each time step in standard Neural SDEs into gated in-flow coupling between structured linear stochastic layers. Importantly, this idea allows to keep each layer affine within its own state and use parallel associative scans. Even though each individual stochastic layer remains affine, the nonlinear expressivity remains thanks to a linear decoder and the modulation of the coefficients across the layers. With this new family, the authors achieve a reduction in the sequential simulation depth from O(T)O(T) to O(log(T))O(log(T)), where TT denotes the number of time steps. Gate in-flow coupling is in essence a particular modulation of the next layer affine flow which is used for stacking the layers and keep the scan structure. In addition, the authors provide a last(final)-layer Girsanov change of measure inside the architecture to improve the estimation of calibration losses for rare events and reduce the variance of rare events estimates. A significant number of theoretical results are established in the manuscript including well-posedness, strong order 1/2 convergence of the Euler-Maruyama numerical scheme, validity of the last-layer Girsanov change of measure, Wasserstein density of terminal laws, and so on. On the validation side, the experiments shows that the novel model SLiSDE improve ccalibration accuracy w.r.t. neural SDE baseline on the path-dependent TOY benchmark and DAX/SPX option surfaces. In comparison with the structured linear controlled differential equation (SLiCEs), SLiSDEs show good improvement on path-dependent TOY benchmark but modest improvement on the DAX/SPX benchmarks.

Critical strengths
  1. With the proposed SLiSDE backbone, the authors manage to achieve O(log(T))O(log (T)) parallel depth in the sequence length via a parallel associative scan, compared with the O(T)O(T) sequential depth of standard neural SDE solvers (here TT denotes the number of time steps). This is an important and substantial architectural contribution (as training neural SDEs is computionally very expensive in long horizon simulations). The gate in flow stacking mechanism is another an important and substantial contribution as it allows tohave parallel scan structure within each layer. Another strength is the harmonious way in which parallel simulation and nonlinear expressivity are combined together in the proposed SLiSDE architecture.

  2. The theoretical guarantees (well-posedness, strong order 1/2 convergence of the Euler-Maruyama numerical scheme, validity of the last-layer Girsanov change of measure, Wasserstein density of terminal laws) are substantial and significant to justify the proposed model.

  3. A particular mention is the presentation of the theoretical results presented in Section 5 which find a good balance in providing the needed technical detail for the stochastic analysis audience, and, at the same time, makes the results accessible to the ML audience with a good high-level overview.

Critical weaknesses
  1. It seems that some of theoretical computational advantages of SLiSDE like O(log(T))O(log(T)) parallel depth, do not always lead to better computational performance compared to Neural SDE baselines, e.g., in the 3 layer DAX experiment. A detailed discussion on the result of that experiment is needed to clarify this point.
Questions for authors

Clarifications:

  1. In the proof for the Validity of last layer Girsanov tilt (Proof of Theorem 5 in Appendix B.5) at page 26, paragraph Step 0. The factor eμθTz0−1/2∥μθ∥2e^{\mu_{\theta}^T z_0 - 1/2 \|\mu_{\theta}\|^2} is stated to be independent of $E_T (u_{\theta}) $ under P\mathbb{P} (lines 1392-1393). The statement of the theorem assume that the last-layer controller uθ,tu_{\theta,t} is progressively measurable w.r.t the final-layer filtration and it seems that $u_{\theta} $ can depend on the starting state z0z_0 (equation (6)) and as a consequence also ET(uθ)E_T(u_{\theta}). How can the factor eμθTz0−1/2∥μθ∥2e^{\mu_{\theta}^T z_0 - 1/2 \|\mu_{\theta}\|^2} still be independent of ET(uθ)E_T(u_{\theta})?

  2. Point raised in the weakness section.

  3. Appendix E, page 44, paragraph "Budget and diagnostics". The self-normalised estimator is reported to perform worse than the vanilla one, does it mean that the proposed Girsanov controller is quite sensitive for rare events? This point is not very clear from the explanation in Appendix C.7.

Suggestions: Please fix the following typos

  1. Line 138 page 3 appears to be cut off "state; this is the only name used for the scan in the". Moreover there is an additional "." in line 142 after Remark 1.

  2. The format of the paper is from ICLR 2026. The "AI use statement" has changed this year.

  3. The title in Section 3.2 has an additional bracket.

2 comments
AuthorsOpenReview ↗

Response to Reviewer fw4L part1

We thank the reviewer for the careful reading, the positive assessment, and the useful comments. All three questions are addressed below.


Q1: Proof of Theorem 5

We thank the reviewer for pointing this out. The independence assertion is indeed incorrect: the controller reads the prefix path, which depends on z0z_0, so ET(u)\mathcal{E}_T(u) need not be independent of z0z_0. However, independence is not required, and the conclusion of Theorem 5 remains valid. We have corrected the argument as follows.

Define the initial-state likelihood factor

E0μ:=exp⁡(μ⊤z0−12∥μ∥2).\mathcal{E}_0^\mu := \exp(\mu^\top z_0 - \frac{1}{2}\lVert\mu\rVert^2) .

Under the stated Novikov condition, (Et(u))t≤T(\mathcal{E}_t(u))_{t \le T} is a strictly positive P\mathbb{P}-martingale with E0(u)=1\mathcal{E}_0(u) = 1. In particular,

EP[ET(u)∣F0]=1.\mathbb{E}^{\mathbb{P}}[\mathcal{E}_T(u) \mid \mathcal{F}_0] = 1 .

Since E0μ\mathcal{E}_0^\mu is F0\mathcal{F}_0-measurable, for every A∈F0A \in \mathcal{F}_0,

EP[E0μET(u)1A]=EP[E0μ1A EP[ET(u)∣F0]]=EP[E0μ1A].\mathbb{E}^{\mathbb{P}}[\mathcal{E}_0^\mu \mathcal{E}_T(u) \mathbf{1}_A] = \mathbb{E}^{\mathbb{P}}[\mathcal{E}_0^\mu \mathbf{1}_A\ \mathbb{E}^{\mathbb{P}}[\mathcal{E}_T(u) \mid \mathcal{F}_0]] = \mathbb{E}^{\mathbb{P}}[\mathcal{E}_0^\mu \mathbf{1}_A] .

Taking A=ΩA = \Omega shows that E0μET(u)\mathcal{E}_0^\mu \mathcal{E}_T(u) defines a probability density. Taking A={z0∈B}A = \lbrace z_0 \in B \rbrace, for any Borel set B⊆RdB \subseteq \mathbb{R}^d, establishes that z0∼N(μ,Id)z_0 \sim \mathcal{N}(\mu, I_d) under Q\mathbb{Q}.

The remaining conclusions follow from a two-stage change of measure. Define the intermediate measure Pμ\mathbb{P}^\mu by

dPμdP=E0μ.\frac{d\mathbb{P}^\mu}{d\mathbb{P}} = \mathcal{E}_0^\mu .

Under Pμ\mathbb{P}^\mu, the initial state has law N(μ,Id)\mathcal{N}(\mu, I_d), while (Wprefix,ϵ)(W^{\mathrm{prefix}}, \epsilon) remains a 2m2m-dimensional Brownian motion independent of F0\mathcal{F}_0. Moreover, for 0≤s≤t≤T0 \le s \le t \le T,

EPμ[Et(u)∣Fs]=EP[E0μEt(u)∣Fs]E0μ=Es(u).\mathbb{E}^{\mathbb{P}^\mu}[\mathcal{E}_t(u) \mid \mathcal{F}_s] = \frac{\mathbb{E}^{\mathbb{P}}[\mathcal{E}_0^\mu \mathcal{E}_t(u) \mid \mathcal{F}_s]}{\mathcal{E}_0^\mu} = \mathcal{E}_s(u) .

Consequently, Et(u)\mathcal{E}_t(u) is also a martingale under Pμ\mathbb{P}^\mu, and Girsanov's theorem applies to

dQdPμ=ET(u).\frac{d\mathbb{Q}}{d\mathbb{P}^\mu} = \mathcal{E}_T(u) .

It follows that

(Wprefix,ϵQ),ϵtQ=ϵt−∫0tus ds,(W^{\mathrm{prefix}}, \epsilon^{\mathbb{Q}}), \qquad \epsilon_t^{\mathbb{Q}} = \epsilon_t - \int_0^t u_s\ ds ,

is a 2m2m-dimensional Brownian motion under Q\mathbb{Q}, independent of F0\mathcal{F}_0. In particular, z0z_0, WprefixW^{\mathrm{prefix}}, and ϵQ\epsilon^{\mathbb{Q}} are mutually independent under Q\mathbb{Q}, as claimed in Theorem 5.

The corresponding Radon–Nikodym derivative is

dQdP=E0μET(u),\frac{d\mathbb{Q}}{d\mathbb{P}} = \mathcal{E}_0^\mu \mathcal{E}_T(u) ,

and hence

log⁡dPdQ=−μ⊤z0+12∥μ∥2−∫0Tus⊤ dϵsQ−12∫0T∥us∥2 ds.\log\frac{d\mathbb{P}}{d\mathbb{Q}} = -\mu^\top z_0 + \frac{1}{2}\lVert\mu\rVert^2 - \int_0^T u_s^\top\ d\epsilon_s^{\mathbb{Q}} - \frac{1}{2}\int_0^T \lVert u_s\rVert^2\ ds .

In discrete time, this gives

log⁡dPdQ=−μ⊤z0+12∥μ∥2−∑k=0N−1uk⊤ΔϵkQ−12∑k=0N−1∥uk∥2Δt.\log\frac{d\mathbb{P}}{d\mathbb{Q}} = -\mu^\top z_0 + \frac{1}{2}\lVert\mu\rVert^2 - \sum_{k=0}^{N-1} u_k^\top \Delta\epsilon_k^{\mathbb{Q}} - \frac{1}{2}\sum_{k=0}^{N-1} \lVert u_k\rVert^2 \Delta t .

This is precisely the likelihood ratio used in the implementation, including both the initial-state and innovation contributions.

We have revised Appendix B.5 accordingly: Step 0 now uses the conditional martingale argument, and Step 4 states independence under Q\mathbb{Q} with respect to the shifted innovation ϵQ\epsilon^{\mathbb{Q}}. The statement of Theorem 5, the implemented likelihood ratios, and the numerical results remain unchanged.

Q2: Computational performance

The observation is correct, and the submission did not explain it carefully enough. Three points, all now stated in the revision.

The experiments use the sequential recursion. Because of our limited GPU resources, all calibration runs (the three-layer DAX one included) evaluate each layer by the sequential recursion rather than by the parallel associative scan (which has O(log⁡T)O(\log T) depth and pays off when parallel capacity is left to spare). The epoch times of Tables 2–4 therefore do not measure the O(log⁡T)O(\log T) depth. This is now highlighted in the paper. However, we would like to point out that for the application we target, the O(log⁡T)O(\log T) depth is a very desirable property and indeed constitutes a contribution of the paper.

Where the speed of our model comes from (besides the parallel associative scan). A key point of the paper, understated in the submission and now highlighted, is that linearity in the state pays off also before any scan is applied. The affine transition maps (Fk,gk)(F_k, g_k) of a layer (its flows) depend only on the Brownian increments and on the path of the previous layer, both known before the layer is run. All TT of them, gate-network evaluations included, are therefore computed at the beginning of the pass in batched tensor operations over time steps and paths; what remains is a cheap structured recurrence

zk+1=Fkzk+gk,z_{k+1} = F_k z_k + g_k ,

which is then run step by step or by the parallel scan. A Neural SDE cannot do this: its networks read the current state, so their TT evaluations are inherently sequential. Gated in-flow stacking preserves both properties: the exact associative scan and the batched computation of the flows.

This is what Table 6 measures: evaluated sequentially, SLiSDE is 6–7× faster per training step than a Neural SDE of the same size at B=64B = 64 and 1.3–1.4× faster at B=1024B = 1024; only at B=4096B = 4096, where the batch saturates the GPU, is the Neural SDE step faster.

The three-layer DAX row. Each row of Tables 2 to 4 is the best cell of a grid search over the width of the model (latent dimension 32 or 64, Appendix D), so the models of a block need not have the same width. In that block the search selects the widest SLiSDE (latent dimension 64) and the narrower Neural SDE (latent dimension 32): the Neural SDE of latent dimension 64 did not learn the surface better. The cost of a stack grows with its width and depth, while that of the Neural SDE is set by its hidden layers – hence 129 against 114 ms. At equal latent dimension 32, the three-layer SLiSDE of the same grid takes 77 ms per epoch against 114 ms, at a lower mean loss.


Q3 – Self-normalised against vanilla estimator

The difference between the two estimators does not, by itself, indicate instability of the Girsanov controller. Both estimators use the same tilted paths and the same likelihood-ratio weights w=dP/dQw = d\mathbb{P}/d\mathbb{Q}, and differ only in their normalisation. The vanilla importance-sampling estimator uses the exact normalising constant EQ[w]=1\mathbb{E}^{\mathbb{Q}}[w] = 1, established in Theorem 5, whereas the self-normalised estimator replaces it with the empirical mean of the weights.

The initial-state shift can produce occasional paths with large likelihood-ratio weights that do not reach the target event. Such paths contribute nothing to the numerator of either estimator, but introduce additional variability into the denominator of the self-normalised estimator.

The effect can also be quantified asymptotically. Let EE denote the rare event and I=P(E)I = \mathbb{P}(E). Under suitable moment conditions, the delta method gives

lim⁡n→∞n (Var(I^SN)−Var(I^IS))=I2 VarQ(w)+2I2(1−EP[w∣E]).\lim_{n\to\infty} n\ (\mathrm{Var}(\hat I_{\mathrm{SN}}) - \mathrm{Var}(\hat I_{\mathrm{IS}})) = I^2\ \mathrm{Var}^{\mathbb{Q}}(w) + 2 I^2 (1 - \mathbb{E}^{\mathbb{P}}[w \mid E]) .

In particular, self-normalisation has greater asymptotic variance whenever EP[w∣E]≤1\mathbb{E}^{\mathbb{P}}[w \mid E] \le 1.

Crucially, when combined with the vanilla importance-sampling estimator, the same controllers reduce the relative RMSE by approximately a factor of 2–4 compared with plain Monte Carlo at n=1024n = 1024, with the improvement increasing as the event becomes rarer. We retain self-normalisation in the calibration training objective, where weights are handled in log-space, and use vanilla importance sampling as the primary estimator for the rare-event evaluation in Tables 10–11. We have expanded Appendix C.7 to clarify this distinction.


Typos

Thank you for spotting the typos and the wrong format, which have now been fixed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.