- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to predict high-fidelity (HF) outputs when many low-fidelity (LF) observations and only a few HF observations are available. The proposed model CA-MFPD first fits a Gaussian process (GP) to the LF data. It then fits a second GP to the difference between the HF observations and the adjusted LF predictions. Together, these models form a teacher that predicts HF outputs and their uncertainty. Next, they use leave-one-out errors on the HF data to adjust that uncertainty. They then train a neural student to learn the teacher's predictive distribution. Real HF observations also guide the student's predicted values. An optional HF-only GP can help at prediction time. The paper analyzes this approach and evaluates it on ten benchmark problems, comparing prediction accuracy, uncertainty and computational cost with other methods.
Critical strengths
-
The CA-MFPD takes joint predictions from a multi-fidelity GP, calibrates their uncertainty using scarce HF data and trains a student from them.
-
The empirical study is substantial and relatively transparent about its tradeoffs: ten benchmarks, 20 seeds, seven nested HF budgets, paired comparisons, uncertainty metrics, ablations and inference timings.
Critical weaknesses
The paper's distinctive empirical claim is covariance-aware distillation, but the bencemark evaluation reports only the point accuracy and marginal uncertainty. Figure 4 illustrates the covariance issue with a two-point toy example, but the paper does not directly report how closely the student matches the teacher’s cross-input covariance on held-out benchmark inputs. That makes it harder to judge whether the extra joint training helps for its stated purpose.
Questions for authors
Please give the precise weighted-ridge objective and weights for fitting , the ridge value and the GP hyperparameter fitting procedure. These choices affect both the teacher covariance and its LOO calibration.
4 comments
Dear Reviewer,
Thank you for your comments on the contribution and experimental evaluation of our paper. We have added an explanation of how multi-fidelity posterior transfer can be used in a cross-input engineering task.
We evaluate near-hole stress screening on SimuStruct perforated plates. For each geometry, we fit the models using a coarse-mesh low-fidelity stress field and six high-fidelity stress measurements. We then draw 4,000 samples from the student's full joint predictive distribution over near-hole locations to estimate the probability that the maximum near-hole von Mises stress exceeds 585.31 MPa.
We compute the class-balanced Brier score by first calculating the mean Brier score separately for exceedance and non-exceedance cases, and then averaging the two class-specific scores with equal weight. For each plate, we first average the predicted probabilities across five seeds. The results are as follows:
| Number of plate geometries | Exceedance cases | Joint KL | Marginal KL |
|---|---|---|---|
| 10 | 2/10 | 0.0472 | 0.0941 |
| 9 | 5/9 | 0.1540 | 0.1699 |
| 100 | 28/100 | 0.2300 | 0.2655 |
Among the 100 additional geometries, joint KL detects 16 of the 28 exceedance cases, compared with 10 for marginal KL.
This experiment illustrates one use of multi-fidelity posterior transfer in spatial screening: using a small number of high-fidelity measurements and a low-fidelity stress field to predict the joint stress distribution across multiple near-hole locations, and then estimate the probability of a spatial maximum exceeding the threshold.
Thank you again for your valuable comments.
Thank you for providing the new ablation results and these addresses my concern about direct covariance evaluation. I'm willing to raise the score.
Thank you for your thoughtful follow-up and for reconsidering your score. We are glad that the additional ablation results and direct covariance evaluation addressed your concern. We appreciate your careful reading and constructive feedback, which helped us strengthen the paper.
Response to Reviewer ZxYM
Dear Reviewer,
Thank you for your careful review and for recognizing the method's use of joint multi-fidelity GP predictions, HF-based calibration, and the scope of the empirical study. We address the main weakness and the methodological question below.
W1. Direct evaluation of cross-input covariance. We agree that the two-point example in Fig. 4 illustrates why marginal matching cannot identify cross-input covariance, but does not by itself show how closely the trained student matches the teacher on benchmark inputs. We added a matched comparison of joint Gaussian KL and marginal Gaussian KL across six benchmarks, three nested HF budgets per benchmark, and five paired seeds. The variants share the teacher, data splits, calibration, student architecture and rank, initialization, Sobol augmentation points, loss weights, optimizer, training budget, and HF anchors. Marginal Gaussian KL matches the teacher and student means and marginal variances at each input but omits cross-input covariance.
The batch-mean metric evaluates predictive variance for an equal-weight average over a held-out input batch. These results separate pointwise predictive performance from direct cross-input dependence measures.
Q1. Weighted-ridge fit for (b,ρ) and GP hyperparameter fitting. At HF inputs, let Ai=(1,μL(xiH)), let A be the matrix with rows Ai, let wi=(vL(xiH)+10−10)−1, and set W=diag(wi). We estimate (b,ρ) by minimizing ∑iwi(yiH−b−ρμL(xiH))2+λρρ2, where λρ=10−6max{21tr(A⊤WA),10−12}. The intercept is unpenalized, and the fitted slope is clipped to [−5,5] before fitting the discrepancy GP.
The exact GPs use a constant-scaled ARD Matérn-5/2 kernel, normalized targets, and a fixed diagonal nugget of 10−6. The signal-variance initialization is 1 with bounds [10−6,106]; each length scale is initialized at 0.2 with bounds [0.02,2]. We maximize the exact log marginal likelihood using L-BFGS-B, with eight restarts for the main fits and one restart for each HF-LOO fit. For each HF observation, the LF GP is held fixed, and the affine and discrepancy components are refit after removing that observation; the resulting LOO residuals determine the covariance scale. These implementation details are now specified in the reproducibility appendix.
Thank you for pointing out the need for a direct benchmark-level covariance audit and more precise fitting details. These additions make the empirical test of the central claim and the model-fitting procedure clearer.