- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
To address the issue of "inconsistency between response prediction accuracy and nonlinear design metrics" in electromagnetic structure screening, a frozen surrogate model and gradient-free two-stage framework is proposed.
Critical strengths
The problem is accurately identified and has practical value. "Model freezing, access to only a few frequency points, yet downstream metrics being threshold/extreme-value types" represents a real constraint in RF and antenna engineering.
Critical weaknesses
- Key control missing, making it impossible to disentangle the contributions of the two mechanisms. The paper only performs score ablations within DARC (Table 3), and does not graft "scenario scoring" onto other uncertainty-aware reconstructors. The empirical response GP (§4.2, Table 5) itself has posterior covariance; using it for posterior sampling + the same gg scoring would be a readily available control that requires no surrogate model. Without this control, readers cannot determine whether the advantages in Table 2 stem from the idea of "decision-aware scoring" or from DARC's specific propagation operator; the latter defines the contribution boundary of this paper. (Locations: §4.2 baseline list, §4.4, Table 3)
- It claims "a clean sweep in all 90 settings," yet the appendix provides only 6 aggregate columns.
- §4.1 Each setup has only one kk and three nested layouts. The method centered on "practical testing with a small number of frequency points" does not report stability under noisy or repeated measurements, so the practical effect of γγ regularization remains unexamined.
Questions for authors
- Please provide per-setting win/loss statistics for the 90 settings (e.g., in how many settings DARC achieves the best/worst UMSE/regret rank, along with counterexamples), and provide numerical tables for Figures 2 and 3 (including means and standard deviations).
- Please report the proportion of the 90 settings in which the “no-correction” fallback option was selected, as well as the distribution of the choices for , , and . If the fallback option was selected in a significant number of cases, please provide the results excluding those instances.
- Please clarify the consequences of and checkpoint selection sharing the same 256 structures: Can you provide sensitivity results for estimating using independent subsets (or cross-fitting)?
3 comments
Response to Reviewer fSkD [2/2]
W3. §4.1 Each setup has only one kk and three nested layouts. The method centered on "practical testing with a small number of frequency points" does not report stability under noisy or repeated measurements, so the practical effect of γγ regularization remains unexamined.
A3. We extended the evaluation to noisy observations and added the results to Appendix B.8, Table 11.
-
DARC retains the lowest mean selection regret at every tested noise level across all three tasks. It outperforms fine-tuning, latent adaptation, and Gappy AE under noisy observations. Table B summarizes DARC's performance from noise-free observations to the highest tested noise level.
-
Parameter selection favors stronger regularization as noise increases. At the highest tested noise level, the calibration set selects a larger γ than in the noise-free control in 211 of 300 configurations. The remaining choices are unchanged. Selection records are provided in
07_measurement_noise/gamma_selection.csvin the supplementary materials.
Table B. Mean Top-13 selection regret of DARC (ours) under measurement noise (↓). Noise standard deviation is σ times the training-data response scale.
| Task | σ=0 | σ=0.01 | σ=0.05 | σ=0.10 |
|---|---|---|---|---|
| HGA | 2.483 | 2.485 | 2.495 | 2.540 |
| RFF | 1.186 | 1.195 | 1.261 | 1.429 |
| Hairpin | 2.019 | 2.022 | 2.051 | 2.112 |
Q2. Please report the proportion of the 90 settings in which the “no-correction” fallback option was selected, as well as the distribution of the choices for 𝜌 , 𝜏 , and . If the fallback option was selected in a significant number of cases, please provide the results excluding those instances.
A4. We have added the complete parameter-selection records to the supplementary materials.
-
DARC's gains do not rely on the no-correction fallback. The fallback was never selected, so excluding fallback cases leaves all results unchanged.
-
Adaptive scenario weighting was selected in 94% of configurations. The remaining 6% used fixed reference weights . Complete selection records are provided in
01_original_data/parameter_selections.csvin the supplementary materials.
Q3. Please clarify the consequences of 𝐶 and checkpoint selection sharing the same 256 structures: Can you provide sensitivity results for estimating 𝐶 using independent subsets (or cross-fitting)?
A5. We repeated the evaluation with an independent reference set, separate from the data used for checkpoint selection.
DARC maintains its selection performance with an independent reference set. We replaced the shared reference set with 256 independent structures while keeping the surrogate and evaluation setup unchanged. The differences are small and show no consistent degradation across tasks, indicating that DARC's selection benefits do not depend on sharing reference data with checkpoint selection.
Table C. Mean Top-13 selection regret with shared and independent reference sets (↓).
| Task | Shared reference set (256) | Independent reference set (256) |
|---|---|---|
| HGA | 2.601 | 2.561 |
| RFF | 1.296 | 1.300 |
| Hairpin | 2.417 | 2.341 |
Thank you again for your constructive feedback. We would appreciate your reconsideration of the score in light of the additional experiments and evidence.
Official Comment by Reviewer fSkD
Thank you for adding the GP posterior-scoring controls and analyses of measurement noise and independent reference sets; these substantially address my empirical concerns, and I am raising my rating from 1 to 2 (weak rejection). However, I remain unconvinced that the proposed residual transformation and weighting provide a sufficiently substantial advance over established residual-based stochastic decision methods, despite the demonstrated practical gains.
Response to Reviewer fSkD [1/2]
Thank you for your careful review and for recognizing the practical value of the problem. We address your concerns point by point below.
A1. We added GP posterior-scoring comparisons to Appendix B.4, Table 7.
DARC outperforms response-GP posterior scoring on all three tasks. Following your suggestion, we sample from the empirical response GP's joint posterior and average the same design objective over these samples. Table A includes both direct scoring and posterior averaging for comparison.
Using the same surrogate predictions and historical residuals does not close the gap. We additionally evaluate a residual GP with these inputs, paired with direct scoring or posterior objective averaging. These controls show that DARC's advantage extends beyond equipping classical GP reconstruction with posterior scoring.
Table A. Mean Top-13 selection regret for GP controls and DARC (↓).
A2. We have expanded the numerical supplement to make the per-setting results and figure data easier to inspect.
DARC outperforms all three model-tuning baselines in UMSE across all 90 settings. These comparisons cover fine-tuning, latent adaptation, and Gappy AE. Per-setting results are provided in
01_original_data/setting_metrics.csvin the supplementary materials.DARC's selection advantage extends beyond response accuracy. DARC achieves the lowest regret in 83/90 settings, including every HGA and Hairpin setting. In the Hairpin cases where the empirical GP achieves lower response error, DARC still reduces selection regret by 24.4%–26.6%, demonstrating that more accurate reconstruction does not necessarily yield better design choices. The remaining cases occur only on RFF at higher observation budgets, but the average regret gap is only 0.026.
Numerical results for Figures 2 and 3 are now available. We provide them as
01_original_data/figure2_results.csvand01_original_data/figure3_results.csvin the supplementary materials.