- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SPEAR is a mixture-of-experts neural operator for pretraining across heterogeneous PDE datasets. An AFNO branch is meant to model shared low-frequency dynamics, and a local convolutional MoE branch with expert-specific low-rank adapters (top-2 routing) is meant to capture PDE-specific high-frequency structure. A knowledge-guided aggregation step projects dataset similarities and dataset–expert routing preferences into expert space, clusters the experts and merges their adapters. Experiments cover twelve PDE datasets, task-specific fine-tuning, transfer to turbulence and wave benchmarks, and component ablations.
Critical strengths
- Combining a shared AFNO backbone with lightweight routed adapters is a sensible way to share capacity across PDE tasks while keeping task-specific behaviour, and projecting dataset similarity into expert space is an interesting idea for organizing experts.
- The evaluation is broad: twelve datasets, downstream transfer, expert-count variations and aggregation variants. Removing the additional branch clearly hurts (Table 5), and fewer routing destinations retain accuracy after fine-tuning (Table 1).
Critical weaknesses
- The supplied code does not implement the claimed spectral split. With the settings in Table 8 (128×128 input, patch 8, 32 modes) the latent grid is 16×16 and its rFFT is 16×9, so AFNO2D.forward, which slices [:32, :32], processes the entire spectrum, and _fft_high_mask zeroes only the leading 9×9 block, which is not a magnitude high-pass because the first axis holds both positive and negative frequencies (cf. Eqs. 3–4). The ablation in Table 5 thus shows the value of an extra branch, not of spectral disentanglement, and the relation to FreqMoE, which already combines low-frequency processing, high-frequency experts and low-rank adaptation, is not discussed.
- The aggregation and compression claims are not reproducible from the materials. The supplied checkpoint routes to six destinations per layer but still stores twelve adapter pairs per layer (about 1.57M parameters over four layers, the pre-merge figure in Table 7, not the 0.786M reported after merging), and the available merge_similar_experts averages adapters by weight similarity rather than implementing Eqs. 9–12; the similarity and routing matrices, reused from NESTOR (Appendix A.6), are not provided. Merged-model accuracy is reported only after further fine-tuning, so the loss caused by merging itself is not measured.
Questions for authors
- (Clarification) Which implementation produced the reported results, and what are the effective spectral cutoff and latent resolution? Do the main results hold with a frequency-consistent mask, and against a parameter-matched local branch without filtering?
- (Clarification) Can you release the dataset-similarity and routing matrices, their construction, and the code that produced the merging groups, and report accuracy immediately after merging, before fine-tuning?
- (Clarification) After physically removing merged experts, what are the stored and activated parameter counts, checkpoint size under identical serialization, memory and latency?
- (Clarification) Were all baselines in Table 3, in particular Poseidon, re-evaluated under the same data size, horizon, fine-tuning budget and L2RE metric? Poseidon's original evaluation uses final-time relative L1 error.