- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
Prior work in the evolution of point clouds has used a microscopic approach, where the evolution of each particle is tracked and predicted. The authors of the submission counsel for a macroscopic approach, where the quantiles of the initial and final distribution are monitored. The authors show that for simple problems, summarizing the population using such an empirical measure and compressing this into a finite-dimensional token leads to good performance.
Critical strengths
The strength of the method lies in its alleviation of the computational burden for problems where one has to predict how populations evolve under an unknown transformation. When the detailed trajectories of individual points are not required, this reduces to a measure-to-measure (M2M) regression problem. Vandergrift et al have detailed this approach. The authors advance this approach and in their macroscopic formalism the attention mechanism operates on measures instead of individual particles. This leads to savings in computational complexity. One has to note, that this reduction in complexity has been achieved by simplifying the problem to a macro-scale. In my opinion, Baptista et al have also considered this, but eschewed a quantile approach due to its problems in scaling to higher dimensions.
Critical weaknesses
-
As the Quantile Transformer relies on quantiles, how do the authors deal with scaling to higher dimensional point clouds? There the authors are obligated to resort to simplifications such as Sliced Projections. In higher dimensions, the universal approximation guarantees established by the authors break down in my opinion.
-
In my opinion, the method should not work in cases where the distribution has a disconnected support.
-
The quantile / measure approach ignores any localized interactions between the particles.
-
The approach is very sensitive to smaller sizes of samples. This limitation is exacerbated as the dimensionality of the space increases.
Questions for authors
-
As the Quantile Transformer relies on quantiles, how do the authors deal with scaling to higher dimensional point clouds? There the authors are obligated to resort to simplifications such as Sliced Projections. In higher dimensions, the universal approximation guarantees established by the authors break down in my opinion. Please provide addiitonal proof for this.
-
In my opinion, the method should not work in cases where the distribution has a disconnected support. How does the method extend to such cases.
-
The quantile / measure approach ignores any localized interactions between the particles. What are the ramifications of this?
-
The approach is very sensitive to smaller sizes of samples. This limitation is exacerbated as the dimensionality of the space increases. What is the minimum number of samples for R^d and how does this compare with the M2M approach of Vandergrift et al?
4 comments
Response – Part II: On the absence of localized interactions and the minimum number of samples
-
On possible ramifications of ignoring localized particle interactions.
-
From a formal approximation standpoint, there are no ramifications in the sense that, whenever measures evolve through local particle interactions and this evolution gives rise to a -continuous map between the compact sets allowed by Thms. 3.1 and Thm 4.1, there exists a Quantile Transformer that can approximate it for a sufficiently large M with high probability.
-
From a practical standpoint, a possible ramification concerns learning efficiency, i.e., how many training sequences are needed to reach a certain test error level. Specifically, a particle transformer with neighborhood attention applies a shared rule directly to nearby particles, which can make such local dynamics easier to learn from finite data. In contrast, since the Quantile Transformer attends over measure tokens, it learns how local interactions affect the next distribution without explicitly updating each particle from its neighbors. It may therefore require more training sequences to attain the same next-measure error on unseen sequences. We include this discussion in App. A.2.
-
-
On a universal, -dependent minimum .
-
We address the following reviewer question
The approach is very sensitive to smaller sizes of samples. This limitation is exacerbated as the dimensionality of the space increases. What is the minimum number of samples for R^d and how does this compare with the M2M approach of Vandergrift et al?
There is no universal minimum for depending exclusively on . For a given error and failure probability , Thm 4.1 gives only a sufficient threshold that also depends on the distribution class, the continuity modulus of in Sec. 2.1--Step 1, and the elliptical decoder stability. Likewise, Vandergrift et al. [3] give no minimum .
There is, however, a dimensional limitation shared by both methods: for , approximating a single distribution within can require particles. Fournier and Guillin [2] show that for a fixed confidence, the worst-case error of an -particle empirical measure scales as , even for laws supported on the unit ball. In particular, the uniform law on the unit ball is relevant since it belongs to our elliptical class: here, a matching lower bound of can be extracted from [2], point c) just below their Thm. 1. Put differently, even when the target law is known, the error cannot decrease past for this example. Thus, at fixed , any method outputting an -particle empirical measure needs exponentially-in- many particles to approximate the target law. Our in Thm 4.1 may be larger, since it further depends on other problem-related quantities as mentioned above.
-
We add dimension-dependent degradation tests for the Quantile Transformer and M2M- of [3] on the Gaussianity-preserving McKean Vlasov dynamics (see revised Fig. 3, Sec. 5 and App. H.8). Specifically, we update our experiments with new results for and (besides the prior , ). Due to lack of space, the and results are left to the appendix section. Fig. 3 (e-f) depicts the average test normalized by to account for dimensional scaling. Reducing from to increases mean normalized error by for QT versus for M2M- at , and by versus at . Increasing from to decreases QT's error by but increases M2M-'s by at . At , errors increase by and , respectively. Furthermore, at fixed , both methods’ normalized errors increased from to . QT has a consistent advantage and a smaller variance across all settings. We hypothesize the latter is due to QT avoiding intermediate particle reconstruction (via token-based training), thus preventing particle-level error accumulation during recursive rollout. We also note the two methods' markedly different training times: across the 6 settings, M2M M2M- requires the mean training time of QT for updates on an NVIDIA L4, including periodic validation.
-
Our new experiments, discussions, and clarifications cover the reviewer's raised concerns in full. If the reviewer deems our response satisfactory, we kindly request reconsideration of the score. Otherwise, we are happy to answer any further questions.
[1] Fraiman et al. (2023), A Cramér–Wold theorem for elliptical distributions.
[2] Fournier & Guillin (2015), On the rate of convergence in Wasserstein distance of the empirical measure.
[3] Vandergrift et al. (2026), Measure-to-measure Regression with Transformers.
Response to the authors
Thank you for the response. Whilst my questions were addressed in letter, I retain grave concerns about this approach requiring many, many additional samples as it completely ignores any localized interactions between the particles, which are part and parcel of any real world application. However, an intransigent stance would defeat the purpose of a discussion period between authors and reviewers. More importantly, I asked questions and the authors responded to most of them. I shall duly increase my score.
We thank the reviewer for engaging with our response and reconsidering their assessment. We wish to provide further clarification regarding the concern that QT can potentially require "many, many additional samples".
If this concern refers to the number of particles per measure, the exponential-in-dimension limitation applies to every method returning an -particle measure (as per our prior example of the uniform law on the unit ball). Furthermore, any method aiming to learn a map between measures is bound to pay for potential ill-conditioning of the target and its interplay with .
Separately, if this concerns the number of training sequences per problem instance, particle-based attention (which models localized interactions) could improve learning efficiency for some subset of dynamics. Our discussion identifies this as a hypothesis requiring empirical investigation. This does not remove the computational disadvantage providing the core motivation of our work: a particle-based transformer operating on -length sequences of -atom empirical measures with dense attention has attention cost and requires expensive Optimal Transport computations for training. QT instead attends over measure tokens, with the forward-pass cost stated in Sec. 4, and trains using cheap Euclidean token losses.
Our QT therefore offers a point of tradeoff, whose value has to be established per application. In particular, our experiment comparing QT with M2M- (Sec. 5, App. H.8) highlights QT's computational advantage in this case: QT achieves lower error and lower variance in all settings when training on the same set of sequences and with the same sample rate . Thus, here, QT requires neither additional particles per measure nor additional training trajectories. We provide the full experiment statistics below, which clearly highlights the computational advantage of QT across several dimensions and sample rates.
| M2M time (min) | QT time (min) | M2M/QT time | M2M peak memory (GiB) | QT peak memory (GiB) | ||
|---|---|---|---|---|---|---|
| 3 | 32 | 77.86 | 19.58 | 3.98× | 0.370 | 0.266 |
| 3 | 128 | 117.36 | 18.41 | 6.38× | 0.690 | 0.409 |
| 3 | 500 | 773.29 | 19.96 | 38.74× | 2.063 | 0.980 |
| 6 | 32 | 80.57 | 19.58 | 4.12× | 0.370 | 0.388 |
| 6 | 128 | 125.08 | 19.36 | 6.46× | 0.694 | 0.903 |
| 6 | 500 | 855.16 | 24.01 | 35.62× | 2.062 | 2.926 |
| Mean | — | 338.22 | 20.15 | 16.79×* | — | — |
*Ratio of mean times, computed before rounding. The arithmetic mean of the six individual time ratios is 15.88×. Memory denotes PyTorch's recorded peak allocated GPU memory. The experiments were run on an NVIDIA L4 GPU
Response – Part I: On the scope of our theoretical results
Dear reviewer FDwh,
Thank you for your time, comments, and clarification questions, which helped us improve our manuscript. We address concerns point by point below. In addition, we summarize manuscript changes and some common points raised by reviewers in a general response. Note that revised manuscript additions are in blue. References are in the final comment.
Regarding the scope of our universal approximation result in higher dimensions.
We address the reviewer's claim that
We respectfully note there is a misunderstanding of our results on this point. We devote Sec. 4 entirely to describing a slicing-based architecture which is provably universal whenever the measures are elliptical, with PD covariance, and belong to P2(Rd) (see Prop. 4.2 and Thm. 4.1). We prove this result by relying on the finite Cramer-Wold type result of [1], which ensures that elliptical measures are identifiable from a finite number of carefully chosen slices such as Vd in Eq. (15). All claims are rigorously proven in the respective appendix sections.
If, however, the reviewer refers to generic measures in Rd, then indeed, the finite set of slices we consider is not identifiable for such generic measures. We emphasize that we do not claim such a result. Indeed, in the Conclusions Sec. 6 we explicitly state that the general case remains open.
Regarding connectedness of measure support.
We address the reviewer's claim that
We respectfully point out that our theoretical results disagree with the reviewer's claim. In particular, Thms 3.1 and 4.1 make no assumption of the connectedness of the support, and include measures with disconnected support whenever they satisfy our assumptions, i.e., when the 1D measures belong to P2(R), and the higher-dimensional measures are elliptical, with PD covariance and belong to P2(Rd) for d≥2. For example, Thm. 3.1 accommodates all finitely supported measures with at least two distinct support points (which have disconnected support by definition) since they belong to P2(R). For the elliptical case in Rd,d≥2, take e.g., X=RU, where U∼Unif(Sd−1), R⊥U, and P(R=1)=P(R=2)=21. Then Law(X) is spherically symmetric, hence elliptical, has PD covariance, disconnected support Sd−1∪2Sd−1, and belongs to P2(Rd). This type of measure thus satisfies all assumptions needed by Thm 4.1. In short, our method includes measures with disconnected support as long as they comply with the conditions of Thms. 3.1 and 4.1.