- Rating
- 3
- Confidence
- 2(−1)
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper primarily investigates the question: under what conditions can differences between models be transformed into actual improvements in predictive performance? By exploring various ensemble methods, the paper focuses on distinguishing between potential combinatorial gains existing in the pool of experts and actual gains that can be achieved on unseen cases with limited development data. Experiments show that having diverse models does not always mean they work well together, and simply adding more combinations does not always improve how well the models perform on new cases.
Critical strengths
- The paper clearly states its research problem and presents a systematic experimental analysis. It verifies the effectiveness of equal-weight ensembles and also examines the sources and limitations of ensemble benefits. The study uses computational control, compares different model architectures, analyzes error-space distributions, and tests across cases, creating a thorough experimental analysis framework.
- It is methodologically valuable to distinguish between potential gains and gains that can actually be achieved. The paper uses an oracle as an ideal reference to show the possible combinatorial benefits within a fixed group of experts. At the same time, it shows through limited data learning and independent testing that these gains may not be possible, which helps prevent confusing model differences or training set improvements with real generalization gains.
Critical weaknesses
- The main contribution focuses on empirical analysis, but there is still no combination method that works well in all cases. The article clearly explains when current combination strategies work and where they fall short. However, the experiments depend on specific expert groups and task settings, and using more complex selection strategies does not always lead to better results than simpler ones. As a result, the findings do not show that the conclusions can be applied to other PDE tasks, and they do not offer a general algorithm that can reliably deliver the possible benefits of combining methods.
Questions for authors
- The paper compares equal-weight averaging, fixed convex combination, learnable combination with different spatial granularities, and risk-based expert selection strategies. However, it does not fully explain how these methods are better than other learnable ensemble methods. Could the authors add comparative experiments with methods such as stacking and input-conditioned gating, and evaluate them on the same dataset and with the same computational budget?
- The point-wise oracle for goal perception in the paper shows much higher gains than learnable combination methods on the Elasticity and Pipe tasks. However, since the oracle depends on evaluation goals, it cannot be used directly for real prediction. Could the authors discuss the main reasons why ensemble gains are hard to achieve, such as limited development data, not enough expressive power in the combination rules, or weak generalization across cases? This would help clarify the practical meaning and limits of the achievable complementarity concept in the paper and also help test the methods in more varied scenarios to strengthen the experiments, such as by evaluating performance on cross-resolution or geometric generalization tasks. Please also consider whether changes in grid resolution and discretization (like mesh and point cloud) affect error complementarity between experts and the gains from equal-weight averaging and learnable combination.
5 comments
Response to Reviewer EQ4B, Question 1: Comparisons with Stacking and Input-Conditioned Gating
We thank the reviewer for this suggestion. Our study evaluates constrained linear stacking, input-conditioned gating, and nonlinear prediction-level correction. We further conducted controlled experiments to strengthen these comparisons.
Stacking and input-conditioned gating. Our fixed convex combination serves as a constrained linear stacker, learning shared weights across cases. The Global combiner predicts input-dependent weights from coordinates and expert predictions, while regional and pointwise variants allow spatially varying weights. These gating methods use the same frozen experts, fitting data, model capacity, and optimization budget within each task. We also investigate nonlinear prediction-level correction using a larger residual MLP (Sec. 5.3, Tab 5), with matched centre-only and expert-aware controls to evaluate the additional value of individual expert predictions.
Further evaluation of learned combination. We evaluated gating models with widths of 32, 128, and 256 using the same expert pools and data splits. At 100 fitting cases, the gains over equal averaging are 2.77%, 2.83%, and 2.85% on Elasticity, and 3.35%, 3.71%, and 4.40% on Pipe, respectively. We further used validation data to select the best-performing combination strategy, including the option of retaining the fixed convex combination. On Elasticity, validation-selected gating further reduces error by 0.62% compared with the fitted fixed convex combination, with a paired-case 95% interval of [0.47%, 0.78%]. This shows that learned weighting can provide additional accuracy even beyond a strong fixed combination.
Response to Reviewer EQ4B, Question 2: Factors Affecting Realizable Gains and Discretization
We thank the reviewer for highlighting these factors. We further investigated how fitting data, learner capacity, the starting aggregate, and generalization influence the realization of neural operator complementarity.
(a) Fitting-data availability and learner capacity.
We evaluated gate widths of 32, 128, and 256 with 25, 50, and 100 fitting cases, using the same frozen experts and data splits. At 100 fitting cases, the mean relative- improvements over equal averaging are:
| Gate width | Elasticity: Point | Pipe: 2×2 |
|---|---|---|
| 32 | +2.77% | +3.35% |
| 128 | +2.83% | +3.71% |
| 256 | +2.85% | +4.40% |
Additional fitting data also improves realized gains. With width 32, increasing the fitting cases from 25 to 100 improves Elasticity from 2.48% to 2.77%, while increasing from 50 to 100 cases improves Pipe from 0.64% to 3.35%. These results show how data availability and learner capacity can influence the accuracy recovered through learned combination.
(b) Starting aggregate and the value of expert information.
We extended the nonlinear correction experiments in Tab 5 by interpolating between equal averaging and the fitted convex combination:
\qquad \lambda\in\{0,0.25,0.5,0.75,1\}.$$ Using matched centre-only and expert-aware correction models, **the additional error reductions from individual expert information are 3.00%, 1.93%, 1.10%, 0.70%, and 0.66%**, respectively. **These results reveal that the incremental value of individual expert predictions depends on the starting aggregate.** Their additional contribution decreases along the tested path from equal averaging to the fitted convex combination, offering further insight into how expert complementarity can be exploited beyond an existing aggregate. **(c) Cross-case generalization.** Our five-fold transfer experiments (Tab 3) further evaluate whether learned combinations retain their gains on unseen cases. On Elasticity, fitted convex combinations achieve **2.83% improvement under random holdout and 2.82% under input-statistic holdout**, demonstrating successful cross-case transfer in these settings. **(d) Resolution and discretization.** We additionally evaluated three frozen Darcy models on the same 200 physical cases. Equal averaging achieves **8.87% improvement** over the best fixed constituent at the native $85\times85$ resolution and retains **8.40% improvement** when predictions are interpolated to a $421\times421$ evaluation grid. **These results demonstrate that the ensemble benefit is largely preserved under the tested output-resampling operation**, providing evidence beyond evaluation at a single scoring resolution. **Together, these experiments provide concrete insights into how available complementarity translates into realized accuracy**, highlighting the roles of fitting data, learner capacity, the starting aggregate, and generalization.
Response to Reviewer EQ4B, Weakness 1: Scope and Generality of the Findings
We thank the reviewer for recognizing our systematic analysis and the distinction between available and realizable complementarity. Below, we provide the response point by point to address the reviewer's concern.
Response to Weakness 1: Scope and Generality of the Findings
Our study demonstrates substantial recoverable accuracy in competitive neural operators and systematically investigates when further gains can be realized. Simple averaging achieves these improvements without architectural modifications or learned aggregation weights.
Equal averaging improves over the strongest constituent in all six fixed LinearNO pools, including reductions of 18.46% on Elasticity and 17.80% on Navier–Stokes (Table 1). On Elasticity, seven-model LRSA averaging reduces error by 23.80% relative to the strongest constituent (Sec. 3.2).
Additional analyses of member-disjoint triples support robustness to expert-pool composition. Two member-disjoint LinearNO triples achieve gains of 18.46% and 21.64%, while two member-disjoint LRSA triples achieve 16.63% and 18.30%. Training configurations remain fixed within each architecture, and all comparisons use the same Elasticity evaluation cases.
Our compute-controlled experiments also demonstrate improvements beyond the tested single-model alternatives. The supplementary Elasticity controls show reductions of 4.79% relative to long training and 5.06% relative to SWA, at the same counted training-FLOP budget.
Beyond these accuracy gains, our analyses investigate how additional combination complexity and expert information contribute to realized performance. Fig 4 compares learned combinations against strong fixed references, while Table 5 separates residual correction from the incremental value of individual expert predictions. In the heterogeneous Pipe study, case-level advantage selection further improves over a fitted fixed convex combination by 6.46%, with a paired-case 95% interval of [1.95%, 11.72%] (Tab 4).
These findings provide empirical insights into both the potential and the practical realization of neural operator complementarity. They motivate strong fixed references, matched centre-only controls, and explicit evaluation of whether additional combination flexibility provides reliable gains. We will highlight these broader implications and supplementary robustness results in the revision.