acceptodds
Under review as a conference paper at ICLR 2027

From Distributions to Stochastic Processes: Neural Approximation of Measure-Valued Maps

Abstract

Learning mappings between probability distributions arises naturally in settings where both inputs and outputs are represented by populations of samples rather than individual observations. This paper develops an approximation-theoretic framework for such distribution-to-distribution learning problems and extends it to mappings between stochastic processes. For continuous operators acting on -compact families of finite-dimensional input laws, we establish uniform neural approximation in the 2-Wasserstein metric. Our construction relies on finite law statistics as input representations, a simplex-valued neural map, and a shared atomic output support that guarantees the output is always a valid probability measure. We then extend this approximation principle to probability laws on separable Hilbert spaces via fixed finite-rank orthogonal projections. Together, these results establish the representational feasibility of learning transformations whose inputs and outputs are probability laws rather than deterministic vectors or functions. To assess practical relevance, we construct numerical problems whose target transformations are naturally defined at the level of distributions or probability laws: predicting the first-passage-time distribution of an Ornstein–Uhlenbeck process and the nonlinear response-path laws of a Duffing oscillator. Since the theoretical framework is deliberately model-agnostic and covers a substantially broader class of operators than can be efficiently represented by any single practical architecture, our experiments instantiate the framework using task-adapted neural models rather than reproducing the theoretical construction verbatim. Both models outperform a fixed-feature MLP baseline and distribution-space kernel regression. These proof-of-concept demonstrations complement the approximation theory by supporting the learnability and practical relevance of distribution-to-distribution transformations for random systems.

Primary area:
learning theory
Keywords:
distribution-to-distribution learning, probability measures, stochastic processes, neural networks, approximation theory, universal approximation, measure-valued operators, operator learning, random fields, Wasserstein distance.

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_pV5sOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies approximation of continuous measure-to-measure operators, which is a timely topic in machine learning and computational mathematics. It develops approximators based on classical measure quantization, covering, and partition of unity arguments. The results are worst case over Wasserstein-2 compact sets and qualitative only. They hold both for state spaces \R^d as well as Hilbert spaces; the latter setting is handled via projection arguments. The paper is complemented by three numerical experiments on nonlinear measure-to-measure operators where estimators not following the theory are learned from a collection of empirical measure labeled pairs. Comparisons to baselines are also given.

Critical strengths

The choices and setup for the three numerical experiments are very interesting and highly non-trivial. These could be benchmarks for future work in this area. I really like the exit time example.

Critical weaknesses

The main weakness is that the approximation theory seems to just be done for approximation sake, and is not relevant to practical methods. The authors acknowledge this several times. However, it feels like the approximation results are halfway through an interesting proof, then the authors just stop at an intermediate step and call that the approximator. For example, the covering mixture approximation could be further approximated by the practical architectures used in the numerics, or by e.g., transformers or graph NNs. Without the additional connection to new or existing architectures for measure-to-measure maps, the theory becomes a less interesting exercise in metric space approximation theory, which is well understood.

Overall, this makes the paper feel incomplete and not ready for publication.

Another weakness is the heavy AI use. The paper displays extremely prevalent use of AI to write long passages of text. While readable, it is unnatural and sometimes confusing. The paper would be much improved if human input could push the boundaries of the theory further to architectures that are actually implementable.

The actual implemented method is not much better than the MLP baseline, which is another minor weakness

Questions for authors

On the top of page 4, it says "Two kinds of input statistics are needed" regarding the bounded continuous test functions and second moment. Is this necessary and sufficient? Or only sufficient?

Reviewer_cAS3OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how neural networks can learn mappings from one probability distribution to another. The authors develop a framework for neural approximation of distribution-to-distribution operators, establishing uniform approximation in the 2-Wasserstein distance for continuous operators on W2W_2-compact families of input laws. They extend this result to mappings between laws of stochastic processes or paths in separable Hilbert spaces and analyze the additional error incurred when input distribution features are estimated from finitely many query samples. The main experiments consider prediction of Ornstein–Uhlenbeck first-passage-time distributions, represented by finite-time bins and a censoring category, and prediction of response-path distributions for a Duffing oscillator. A supplementary experiment considers a controlled Gaussian distributional mapping. These experiments use neural models adapted to the individual tasks.

Critical strengths
  1. Clear and unified problem formulation The paper treats probability distributions as both inputs and outputs and explicitly addresses settings in which each input–output law pair is observed through independently sampled ensembles without trajectory-level pairing. This formulation is well motivated by applications such as uncertainty propagation and extends naturally to distributions of paths or random fields.

  2. Clear theoretical structure, with extensions to process laws and empirical query features Theorem 2.1 establishes approximation of operators between finite-dimensional probability laws. Theorem 2.2 extends this result to laws on separable Hilbert spaces while separately controlling input projection, finite-dimensional approximation, and output reconstruction errors. Proposition B.5 further analyzes the effect of estimating input features from finitely many query samples. The authors clearly distinguish representational guarantees and query-sampling error bounds from training or optimization guarantees.

  3. Valid probability-valued outputs without requiring target densities Finite statistical encodings, neural simplex-valued weights, and a shared atomic support form a complete approximation construction that guarantees nonnegative output mass summing to one. The construction does not require the target distributions to admit densities and therefore accommodates discrete or singular output laws.

Critical weaknesses
  1. Limited theoretical novelty and significance Prior work has already studied distribution-to-distribution regression, including Oliva et al. (2013), and established neural approximation results for probability-measure-valued maps and more general metric-space-valued maps, including Kratsios (2021) and Kratsios et al. (2023). The contribution here primarily combines uniform W2W_2 approximation, finite statistical encodings that can be estimated from samples, empirical query-error analysis, and an extension to Hilbert-space laws within one framework. The main arguments use standard tools such as compactness, finite covers, partitions of unity, approximation by finitely supported measures, and orthogonal projections. In my assessment, this provides a clear unified construction, but the resulting new theoretical insights remain limited. In addition, the main text presents bounded weak-convergence tests and a separate second-moment coordinate as two necessary types of statistics. This necessity claim seems too strong. As far as I understand, on a W2W_2-compact set KK, the weak and W2W_2 topologies coincide. Compactness then allows finitely many bounded continuous test functions to distinguish laws up to any prescribed W2W_2 tolerance. A separate second-moment coordinate is therefore not necessary for the uniform approximation result on this compact set, and its presentation as essential additional information overstates its role.

  2. Incomplete theoretical support for the implemented models The OU classification head is broadly consistent with the simplex-weight output construction, and Proposition C.5 and Corollary C.6 provide general approximation results for generative outputs. However, these results require additional conditions, including a continuous selection of generators in L2L^2, and the paper does not establish how they apply to the specific conditional RealNVP architecture used in the experiments. The experiments use global PCA transformations fitted on the training data and then held fixed across inputs. What remains missing is uniform reconstruction-error control for the selected finite-dimensional representations over the relevant families of laws. Retaining 99% of the variance describes aggregate variance retention on the training data; it does not directly establish the uniform W2W_2 reconstruction guarantee used in Theorem 2.2. The connection between the general representational results and the actual implementations therefore remains incomplete.

  3. Limited empirical evidence and insufficiently controlled neural-model comparisons The OU, Duffing, and supplementary controlled Gaussian experiments demonstrate the feasibility of the selected models on these tasks. However, the limited range of tasks and baselines provides insufficient evidence of the framework's added value relative to existing approaches. In particular, Appendices D.2.2–D.2.3 specify that the main Duffing model is trained for up to 1,000 epochs, whereas the MLP baseline is trained for 200 epochs. Their principal hidden-layer widths are 256 and 64, respectively; their PCA preprocessing differs; and their Sinkhorn losses use blur parameters of 0.01 and 0.05. Because these factors vary simultaneously, the comparison does not isolate the effect of the proposed modeling design from differences in training budget, capacity, preprocessing, and loss configuration. The attribution of the reported performance gains is consequently insufficiently supported.

Minor issue The manuscript does not follow the ICLR 2027 formatting template. Please revise the formatting to conform to the required conference template.

Questions for authors
  1. What is the key theoretical contribution beyond the closest existing results? Please compare the paper with the closest measure-valued approximation results in terms of the input and output spaces, information required by the encoder, error metric, and approximation guarantees. Which steps require new arguments beyond the existing frameworks? In particular, what additional theoretical content is provided by the finite encoding estimated from empirical samples and by the extension to Hilbert-space laws?

  2. Which theoretical results apply to the complete experimental models, and how are their assumptions satisfied? Please explain how the experimental families of laws and target operators satisfy the compactness, continuity, and open-neighborhood assumptions of the relevant theorems. For PCA fitted on the training data, is there analysis or evidence supporting uniform reconstruction accuracy over the relevant families of laws? For the conditional RealNVP used in the Duffing experiment, how is the continuous L2L^2 generator-selection condition in Proposition C.5 and Corollary C.6 satisfied, and how do the general generator approximation results apply to the chosen architecture?

  3. Is there evidence separating the effect of the modeling design from configuration differences? Please explain the rationale for using different training budgets, capacities, PCA preprocessing, and loss parameters for the main Duffing model and the MLP baseline. Are there comparisons with aligned budgets, evidence of sufficient convergence, or relevant ablations supporting the conclusion that the observed advantage primarily comes from the proposed modeling design?

Reviewer_M8uFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies neural approximation of maps from input probability laws to output probability laws in the 2-Wasserstein metric. For W2W_2-compact families of finite-dimensional input laws, it proves uniform approximation using finitely many law statistics and a neural network that assigns weights to a common finite set of output atoms. It extends the result to laws of processes on separable Hilbert spaces through finite-dimensional projections and gives a bound for estimating input features from empirical samples. The experiments predict Ornstein–Uhlenbeck first-passage-time laws and Duffing oscillator response-path laws, with an additional Gaussian example in the appendix.

Critical strengths

The topic is potentially interesting. Learning maps whose inputs and outputs are probability laws, observed only through finite ensembles and without pairing between input and output samples, is a natural formulation for population-level data and could be relevant to settings where individual samples cannot be tracked.

Critical weaknesses

(W1) The law-to-law learning problem is clearly formulated, but the practical gap relative to existing methods remains unclear. Although the paper cites Oliva et al. (2013), it does not identify a concrete setting with intrinsically unpaired observations in which existing distribution-on-distribution regression methods are insufficient, or explain how the proposed guarantees address that limitation. This makes the practical significance of the results difficult to assess.

(W2) The main universal approximation results are qualitative. Proposition B.5 gives a query-sample concentration bound for a fixed network, and Proposition C.2 bounds the output support size under an additional compact-support assumption. However, the paper does not provide an end-to-end bound on the number of input features, network size, or path-space projection ranks required to achieve a prescribed W2 error, even in its structured special cases. This makes the approximation's complexity difficult to assess.

Questions for authors

(Q1) Could the authors clarify the main theorem’s novelty relative to prior results on finite-statistic representations of measures and neural approximation of measure-valued maps? In particular, which step requires a new argument, or which class of maps is covered here that would not follow from combining existing results? Could the authors add a comparison table for the closest prior approximation results?

(Q2) The paper notes that convergence of second moments upgrades weak convergence to W_2) convergenceт (lines 167-168). However, since (\mathcal K) is assumed to be (W_2)-compact, weak convergence within (\mathcal K) already implies (W_2) convergence. Is the separate second-moment coordinate needed in Lemma B.1 or Theorem 2.1, or would bounded test-function expectations alone suffice under the stated assumptions?

(Q3) Could the authors quantify the approximation complexity under additional regularity assumptions? In particular, how would the number of input features, output atoms, and network parameters scale with the target W2W_2 error?

(Q4) As far as I understand, both process experiments are induced by a fixed pathwise map, with input and output trajectories sampled independently to remove their correspondence. Thus, they demonstrate learning a pushforward from unpaired ensembles, while the nonlinear Gaussian task in Appendix is synthetic. Could the authors evaluate the method on a practically motivated law-to-law map with nonlinear dependence on the input law, or on data for which pairing is unavailable because of the measurement process? This would help assess the broader motivation for distribution-level learning.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.