- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposed a new method for long-horizon prediction on physical systems where low-frequence presents a strong signal in prediction than high-frequency signals. This frequence-dependent behaviors then inspired a simple design of making use of intermmediate readouts from the neural operator to correct low-frequency predictions in the final readout. The idea is simple. On the tested examples, its improvements over long horizon are significant.
Critical strengths
The proposed research can be seen as effort in multi-scale modeling in neural operators. Also, long-horizon prediction has been particularly difficult for prediction on physical systems with strong perturbations. So the importance of the targeted problem has been established and heavily studied.
The propose solution is lightweight. This is an advantage in that it will not add excessive computational workload.
The results are improved greatly in long-horizon prediction.
Critical weaknesses
When positioned within the context of multi-scale modeling in neural operators, it is difficult to see why PAIR is categorically different from existing work. The limitation of existing work seems to be 'At these hidden layers, the features typically have no direct mapping to the physical field and no additional prediction constraint. The final prediction loss acts on them through the remaining decoder. ' But I am afraid that this just points out a difference, not a problem or limitation. In other words, why is it a fundamental limitation such that existing mutl-scale operators cannot predict frequence-dependent signal well? Research like [1-3] explicitly trains models on multi-resolution. Admittedly, some focus on the spatial resolution, but they could natually deal with learning at different frequencies in the examples employed in this paper.
Also, the paper shows the intermediate features are useful. However, which intermediate features? Is this architecture specific? If so, this is a strong limitation on the generality of the proposed method; if not, why are these features selected and based on what criteria? I assume there must be architectural constraints implicitly assumed here, e.g. if it is upsampling operations, the features need to be a super-resolution representation of part of the whole solution space. In any way, it is hard to find the explaination and justificaiton.
Theorem 1 seems to be crucial for the justification of the methodological design and the guidance on later experiments. I have one question here. 'Assume that, over a local interval [0, τ ], K_{base} and K_{coarse} are fixed positive-semidefinite matrices that commute', what architectural or other constraints are assumed for this assumption to hold? Is there any guarantee that K_{base} and K_{coarse} can commute? Theoretically, it is possible construct two postive-semidfinite matrices that do not commute and do not share eigenvectors. I assume those matrices are not covered by Theorem 1. However, this brings the question of the validity of this assumption.
The evaluation is interesting. The choice of baselines is open to debate. Allow me to observe that the baseline methods employed here are not really designed for multi-scale modelling. The U-Net is supposed to be designed for capturing multi-scale features but the employed baseline is an unmodified U-Net which is not designed to deal with functions.
There seems to be a lack of verification of the proposed PAIR. There is a controlled training on coarse and full. However, it is difficult to see why this is not simply because the increase of the model capacity, or specifically attributed to the way the intermediate features are selected and used for corrections.
- Multiwavelet-based Operator Learning for Differential Equations
- Multi-Grid Tensorized Fourier Neural Operator for HighResolution PDEs
- HiLNO: A Hierarchical Latent Neural Operator with Multi-Scale Supervision for PDEs on General Geometries
Questions for authors
- What are the criteria of feature selection? Is this architecture specific? If not, is any constraint in selecting the intermediate features?
- What guarantees the validity of the assumption in Theorem 1 in 'Assume that, over a local interval [0, τ ], K_{base} and K_{coarse} are fixed positive-semidefinite matrices that commute'? While K_{base} and K_{coarse} are positive-semidienite, how do we know they commute for any or arbitrary neural networks?
- Can you employ baseline methods that are specifically designed to handle multi-scaling signals?
- How do we know PAIR works not simply because of the increase of the model compacity?
- How would you position the research with respect to multi-scale modeling in neural operators?
Thank you for your response. I have carefully read the rebuttal and appreciate your clarifications.