- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper presents CABLE, an approach for reduced order modeling in a latent space. It is presented as an improvement over parameterized neural ODEs, and differs in the way the latent dynamics are modeled and learned. Instead of using a neural network for the latent dynamics like their PNODE baseline they use a neural network only for a weight function that selects from a set of Affine Experts, that is trained along with this weight function during training and frozen during inference.
Critical strengths
I like the presentation in general: the language is clear and concise, and the figures are visually appealing and not overloaded. I also like the approach of isolating and demonstrating the various model properties through experiments and not just through arguing them.
Critical weaknesses
To me the significant weak point is the experimental section:
- There is exactly one experimental result where CABLE is numerically compared to Baselines. When comparing Uniform CABLE to PNODE in this table they are within the stated SDs of each other for all but one row, where it is quite close. This makes the abstract sound slightly overstated, even though it is technically correct.
- It is not clear to me, whether the adaptive sampling that most of the claimed numerical improvement comes from, is actually a property of the CABLE architecture or orthogonal to it, see also my question below.
- Most of the experimental results are discussed in the appendix, with just a short conclusion on page 7. A comparison to baselines is only done in Table 1 (PNODE, GPLaSDI, U-Net, FNO) and D.2 (just PNODE). So depending on how you look at it there is only one or maybe two datasets / problems with an actual comparison, and in Table 1 the margins are slim, as stated above. This is simply too little to get a conclusive impression of CABLEs performance against the stated baselines.
I believe the work has potential, but as of now the experimental results are just too thin to get a decent picture of CABLEs performance. I would like to see more experiments on different datasets and problems, preferably also problems the other methods were tested on originally (because that usually means they are well tuned on those). Depending on what those experiments show, I see to paths forward: either the pure numerical performance against the baselines already justify the approach, or if the margins stay as slim, the additional properties (error bounds, stability by construction, …) could justify the approach. In this case I would like a more detailed discussion and demonstration of those properties especially compared against the baseline. Of course a negative result (no significant improvement over baselines) could also be interesting to the community, but for a conclusion like that more experiments are needed as well.
Questions for authors
- Paper and appendix mention error bounds and stability by construction as advantages. The experimental section refers most of the discussion on this to the appendix. I am missing a discussion of these properties for the baselines. E.g. is enforcing stability structurally impossible for the Baselines?
- l. 326-327: „For CABLE, we leverage the fact that our model defines a latent vector field … at every training and candidate input parameter.“ -> would this not also apply to other baselines (at least PNODEs)? Why is there no adaptive-PNODE column in Table 1?
- Introduction mentions „parameter studies, uncertainty quantification, and optimization.“ as potential use-cases. Are there benefits to the CABLE in any of those apart from inference time? Maybe the convex blend of experts has advantages against a pure Neural ODE when it comes to using this model in an optimization problem?
3 comments
My biggest concerns were addressed:
- The authors added an ablation on the adaptive sampling scheme to PNODEs as well.
- The experimental section was expanded with two more problems, where Cable is compared against PNODE and GPlasDI
I still have some objections: p6l323 „These approaches guarantee stability around a single equilibrium, and require evaluating the learned function’s gradient during rollout. CABLE instead imposes stability directly on its expert matrices, with an explicit learned metric P.“ I think (have not checked) stability is also enforceable structurally through weight limits, in this case no explicit function has to be learned. More importantly: is the explicit learned metric P of cable not also a Lyapunov function?
p17l894 „ All models train for 20,000 epochs, with CABLE/PNODE using the derivative-matching loss while GPLaSDI uses its own model loss.“ If I recall correctly the PNODE paper uses solver based training. The original Vlasov-Fokker-Planck experiment uses both derivative and solver based training if I recall correctly. I agree that using similar training strategies for CABLE and PNODE makes sense for a fair comparison, but the authors should clarify why they chose to omit the solver training for those examples. An ablation that shows, that this training regime doesn’t unfairly favor one of the methods would also be good.
We thank the reviewer for their follow-up and are glad that our revisions addressed the main concerns. We address the remaining objections below.
Stability (Section 4). We agree that our wording was too strong. CABLE's learned is a Lyapunov function, and stability can also be enforced structurally in neural ODEs. We removed the mention of gradient evaluations (end of section 4) and clarify that CABLE uses the established common-Lyapunov construction as an optional constraint on its expert library. We do not claim that the common-Lyapunov construction itself is exclusive to CABLE.
PNODE training (Appendix B.2). The reviewer is correct that the original PNODE uses solver-based training. For the added Burgers benchmarks, we chose derivative matching for both CABLE and PNODE to provide a common training procedure and avoid the additional cost of differentiating through ODE solves during joint autoencoder/dynamics training. Derivative matching is also commonly used in equation-based ROM approaches, including Champion et al. (2019) and LaSDI variants. We have clarified this choice in Appendix B.2.
We note that PNODE and CABLE do receive short-horizon rollout fine-tuning on the Vlasov problem (Appendices A and B.3), with gradients detached between steps. Unfortunately, we do not think we will be able to run solver-based training ablations before the end of the rebuttal period. We instead clarify our training choices and explicitly acknowledge that we use a different training approach than the original one the Burgers' problems.
We thank the reviewer for their comments and concrete suggestions. We believe that addressing the points raised in this review has substantially improved the empirical argument of the manuscript.
Adaptive PNODE (Section 6.3, Table 2, Appendix B.3). The reviewer is correct that our sampling criterion can be applied to any dynamics model with a learned vector field. We added adaptive PNODE with the same 49-trajectory budget applied to CABLE (Table 2). PNODE's 99th-percentile error decreases from 42±6% to 37±3%, while maximum error changes from 68±9% to 72±17%, with no clear improvement. CABLE's improvement is much larger from adaptive sampling, as its maximum error decreases from 50±9% to 26±3%. Figures 3 and 8 show that CABLE, unlike PNODE, concentrates acquisitions in the difficult low-temperature region, leading to the lower tail errors.
Additional benchmarks (Section 6.2, Table 1, Appendix B.2). On the published 1D Burgers' benchmark from PNODE's manuscript, CABLE matches PNODE's accuracy (1.11±0.52% versus 1.04±0.69% mean error) at roughly half the inference time (52.1 ms versus 110.4 ms). On the published 2D benchmark from GPLaSDI's manuscript, all three ROMs are tested with adaptive sampling. GPLASDI is the most accurate and fastest model, but CABLE and GPLaSDI achieve mean errors below 1%. CABLE's inference time is still quite fast 19.3 ms compared to GPLaSDI's 4.6 ms.
Stability and error analysis (Section 4). Stability is not exclusive to CABLE, and stable neural ODEs exist, which we now cite and discuss. However, the existing approaches we are aware of require learning a Lyapunov function or energy function, and require evaluating that function’s gradient during rollout. CABLE instead imposes stability directly on its expert matrices, with an explicit learned metric P. This makes every mixture of experts contractive, and adds no neural-network or gradient evaluations during prediction.
Optimization and uncertainty quantification. We have not tested these applications directly, but it remains an interesting future direction. We note that gradient-free optimization methods such as Bayesian optimization or differential evolution often only require a large number of model runs, so any advantage in inference time is valuable for those problems. Explicit operators may support sensitivity analysis or constrained surrogate design, but these applications remain future work.