- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes PaPi, a parallel-in-time integration method designed to accelerate ODE solving in neuro-spectral architectures. The key idea is to decompose the dynamics into linear and nonlinear components, and combine Picard–Duhamel integration with parallel scans to replace the conventional sequential RK4 solver. Experiments on multiple PDE benchmarks demonstrate substantial improvements in training efficiency while maintaining competitive predictive accuracy.
Critical strengths
- Well-motivated problem. This work addresses the computational bottleneck of sequential RK4 integration by introducing a parallel-in-time approach, potentially improving both training and inference efficiency.
- Principled formulation. The proposed method is mathematically well motivated, with convergence analysis and error bounds providing theoretical support.
- Strong empirical results. PaPi achieves significant speedups across multiple PDE benchmarks while maintaining competitive predictive accuracy.
Critical weaknesses
- Limited methodological novelty. The core components, including Picard iteration, Duhamel's principle, exponential integration, and parallel scans, are well-established techniques. The contribution appears to lie primarily in their combination and adaptation to NeuSA.
- Insufficient ablation studies. The individual contributions of key design choices, such as warm-starting, temporal loss weighting, and different quadrature schemes, are not sufficiently isolated.
- Limited generality. The method heavily relies on decomposing the dynamics into linear and nonlinear components, with an efficiently computable linear propagator. Its effectiveness for strongly nonlinear dynamics or systems without a tractable linear component remains unclear.
- Limited inference evaluation. Although the paper emphasizes inference acceleration, comprehensive end-to-end inference latency comparisons with existing solvers are lacking.
Questions for authors
See weakness.
2 comments
Thanks for the reply and my concern has been addressed. But I decided to keep my current score.
We thank the reviewer for the constructive feedback, and appreciate that they have found the method to be principled and well motivated. We have clarified PaPi’s contributions and expanded the evaluation in the revised manuscript. Answers to each remark are below.
Weakness 1 (Limited methodological novelty):
We have added a more in-depth literature review, discussing in greater detail Picard methods for diffusion-model sampling and training, as well as related work in numerical analysis. We would argue that, although each of the integration method's components are themselves classical, PaPi's approach to the combination of exponential integration and parallel Picard iteration in the spectral domain are, to the best of our knowledge, unique. In particular, they differ from related work in how they compute the linear flow via the diagonalization of differential operators in the spectral basis, offloading any non-diagonal corrections to the nonlinear component. This makes it easy to compose, differentiate, and scale, as well as conducive to PDE applications with NeuSA.
Still, the reviewer's input has motivated us to sharpen our claim to novelty in the revised manuscript. PaPi's main contribution is to combine these pre-existing methods and extend them to scientific machine learning applications. In particular, PaPi's unique implementation allows it to fully leverage the massively parallel capabilities of modern ML, along with the natural linear-nonlinear splitting induced by PDEs in the spectral domain and NeuSA's formulation. This specialization lies behind the strong experimental results.
We have also emphasized the role of our second contribution, which are the procedures for the use of PaPi in training. The orders-of-magnitude speedups in the training of these Neural DEs are only possible due to the use of warm-starting, which to the best of our knowledge is new. We further emphasize this contribution by adding a full theoretical and experimental treatment of this feature, proving that it provides a reliable training strategy and showing experimentally that the induced gradients point in the right direction.
Weakness 2 (Insufficient ablation studies):
This is a reasonable concern that has been voiced by the other reviewers. We have added to the revised draft a substantial number of new ablations, including warm-starting, temporal loss weighting, and different integration methods.
Weakness 3 (Limited generality):
We believe the remark about strongly nonlinear systems is due to suboptimal presentation on our end. In the new version of the draft we clarify the discussion of the role of nonlinearity and state plainly that PaPi converges unconditionally for all Lipschitz systems; it simply converges faster when its Lipschitz constant is smaller. We have removed all references to epsilon (which in the first draft implied smallness) and state all results in terms only of the nonlinear factor’s Lipschitz constant. To illustrate this, we have included an additional experiment on the inverse problem for the Navier-Stokes equation, a strongly nonlinear PDE from fluid dynamics. PaPi still achieves state-of-the-art performance with substantially faster training times:
Moreover, we would argue that the existence of a linear/nonlinear decomposition is a reasonable assumption in PDE applications; a large portion of the PDEs of physical interest seem to be decomposable in these terms. Still, this is a valid concern, and we have addressed it more explicitly in the revised draft's section 4.2, where we describe general procedures to extract linear-nonlinear decompositions from fully nonlinear terms, making the method more broadly applicable.
Weakness 4 (Limited inference evaluation):
We believe these concerns are answered in the revised drafts expanded ablation section.
Misc. (Presentation)
We have used the additional space in the rebuttal to substantially clean up and clarify the writing, introducing more explanations where the previous draft was terse and including more figures. We hope the current draft is accessible and easier to read.