- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper introduces Data-Anchored Response Spaces (DARS), a method to improve the stability of neural parameterization of neural surrogates applied to the scenario of atmospheric forecasting. The key contribution is the idea of using training prototypes to constrain the output of the neural network to be more physically realistic. The paper also proposes a method for closed loop adaptation of the neural network using the non-differentiable simulator. The results compare DARS to an unrestricted network, a componentwise limiter, and a simulator only forecast and find that DARS remains stable for 72 hour rollouts.
Critical strengths
- The idea of using multivariate response profiles rather than controlling each component independently is clever and seems like a good way to improve stability.
- The main result in Table 1 is interesting as it shows that DARS is most stable in comparison to the compared baselines across 3 seeds.
Critical weaknesses
- The baselines are not strong enough in comparison to DARS as it is only compared to a unconstrained NN and a simple limiter. The authors should compare to a standard kernel regression model as well as a convex mixture router. In addition, ablations of the convex restriction, prototype selection, and learned routing methods would strengthen the paper.
- The writing of the paper was unclear and difficult to understand. It took significant effort to understand the methods and results. Mathematical notation and terms are written before defining them, for example in the Introduction section.
- The transfer experiments on the 22 initializations include multiple extra steps detailed in Appendix Q, so it is unclear if the DARS method is responsible for the positive results or if the extra steps cause the improvment.
Questions for authors
Have you compared DARS to a convex-combination model using the same response dictionary or other standard learned methods that use the same inputs?