- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces AutoMF, an ensemble workflow for multifidelity field prediction built around nine adapted surrogate models, including neural operators, convolutional networks, and reduced-basis models. For each dataset, five reserved high-fidelity examples determine one scalar weight per model, shared across inputs and spatial locations. The authors compare model selection, inverse-error weighting, and jointly optimized ensemble weights. The fitted ensemble achieves lower evaluation error than selecting a single model using the same fitting examples on 17 of 22 datasets. The paper provides detailed comparisons of individual surrogates and ensemble strategies across the benchmark.
Critical strengths
The evaluation includes a diverse collection of datasets and model types, providing evidence for the workflow’s usefulness across a broad range of tested multifidelity prediction settings. The variation in individual model performance across tasks also motivates considering multiple surrogate approaches rather than relying on one architecture throughout the benchmark.
The detailed performance comparisons and field visualizations help readers understand the benefits and limitations of combining predictions. In particular, the visualization of predicted fields and spatial errors illustrates how an ensemble can improve on individual models in a successful example. Comparisons with simpler weighting rules and smaller model subsets further help assess which aspects of the workflow contribute to its gains.
Critical weaknesses
The paper does not fully quantify the additional training time, memory requirements, and inference costs associated with maintaining and evaluating multiple surrogate models. Although the accuracy gains are demonstrated, a more complete computational comparison would help establish whether these gains justify the additional resources. This is particularly relevant because selecting a single model and constructing an ensemble have different deployment costs, even when both begin with the same trained library.
The paper includes useful comparisons with inverse-error weighting, uniform averaging, and smaller fitted ensembles. However, these controls largely retain the same adapted surrogate library. Comparisons with ensembles constructed from the implemented baseline models, or repeated training runs of a single architecture, would help isolate the benefit of this particular library. Evaluating such alternatives under comparable computational budgets would strengthen the practical conclusions.
Questions for authors
1.Beyond the reported weighting and subset controls, have you evaluated ensembles constructed from the implemented baseline models or repeated training runs of a single architecture? If such results are available, how do they compare with AutoMF?
2.Can you provide computational measurements, such as total training time, inference latency, and memory requirements, to contextualize the accuracy improvements over selecting a single model?
3.For cases such as Poisson II, where the fitted mixture performs worse than the selected model, what weights are assigned to the individual models? Does this deterioration appear only on evaluation examples, or also on the fitting examples when measured using the reported mean relative error? This would help distinguish generalization effects from the difference between the fitting and reporting objectives.