- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes FlowGEM, a CFD neural surrogate with 2 core components: multi-scale attention (MSA) and physics enrichment with anchor decoder features. Evaluated on DrivAerML and NASA-CRM, FlowGEM achieves the best results on 4/6 reported metrics compared to recent baselines. Prediction error across graph-frequency bands are presented and analyzed to show which frequency bands FlowGEM can best resolve compared to a baseline.
Critical strengths
strengths:
-
Strong empirical result on evaluated 2 datasets: overall winning on 4/6 reported metrics in DrivAerML and NASA-CRM, compared to state-of-the-art models
-
Frequency-resolved error analysis provides a useful additional evaluation perspective. Even though the spectral analysis does not directly establish the classical training-dynamics notion of spectral bias that the paper claims to investigate, analyzing prediction error across graph-frequency bands is still valuable and relatively uncommon in related work.
Critical weaknesses
weaknesses:
- MSA novelty and design are weakly justified.
Proposed MSA is computed conceptually close to RNN, with the time dimension being replaced by scale, so it's not inherently novel architectural design. The paper does not include intuition or justifications for such design. Besides, scale-specific representation learning is delegated to FiLM.
A natural baseline to justify the choice is, how is this formulation compare to an alternative and intuitive approach where each scale uses stacked MSA layers? Plus in this case FiLM modulation is conceptually not needed because each layer already encodes scale information by learning transformation on its assigned scale. This should be included as a control for the recurrent MSA design.
-
Impact of physics enrichment is not isolated. Since both anchor and queries are point samples, and both ultimately use decoder feature learned to predict CFD quantity, the query decoder already learns physics, and I find it a stretch to call the inclusion of decoder feature from the anchor part "physics-enriched". The only difference between anchor/query decoder feature is that the anchor feature is trained to be helpful to the Phys-Queries, but the paper does not ablate whether improvements come from auxiliary supervision, extra transformation depth to the anchor feature by the decoder, or the full proposed mechanism. A controlled physics-anchor ablation is therefore necessary.
-
Missing hyperparameter sensitivity ablations: The paper does not study the impact of geometry-pool size and anchor count even though they are the hyperparam to the core technical proposals, and their choices directly affect fidelity, capacity, and cost.
-
Missing more context on experiment background. For example what's the parameter size for all models, and their training/inference cost as in latency and FLOPs.
-
Limited dataset coverage: as both AB-UPT and Transolver-3 includes AHMEDML in the evaluation, it would be highly justified for FlowGEM to include this dataset in the evaluation.
-
Inconsistent result for claims of network failure in high-frequency learning: the paper claimed that high-frequency features are much more challenged to be learned compared to low-frequency, but Fig 3 shows that the largest benefit of FlowGEM over its (L=1) variant and ABUPT occurs in the mid-frequency range, with much smaller improvement at the highest frequencies.
Questions for authors
Please see each weakness for requests for additional evidence or writing clarification.
Dear reviewers and area chairs,
We thank you sincerely for your thorough reading of our work and your suggestions to improve it. Unfortunately, we will not be able to address all of the comments at this time, so we have decided to withdraw the submission. We have already started improving the work, including additional datasets, baselines and more thorough ablation studies. We will take all of your feedback into account in the next version of this work.
Best regards, The authors