acceptodds
Under review as a conference paper at ICLR 2027

Decision-Aware Residual Calibration: Bridging the Prediction–Decision Gap in Electromagnetic Design with Sparse Feedback

Abstract

Sparse response feedback provides direct evidence for correcting surrogate predictions during electromagnetic design. Converting such feedback into reliable design decisions poses two coupled challenges. First, errors observed at only a few response locations must be propagated to reconstruct the complete response. Second, errors that remain after correction must be accounted for when evaluating nonlinear objectives, as they can alter candidate rankings. We propose Decision-Aware Residual Calibration (DARC), a gradient-free framework that uses sparse feedback to correct surrogate responses and account for remaining errors in nonlinear design scoring, without updating the surrogate. DARC first uses historical cross-location error dependencies to propagate observed discrepancies across the full response. DARC further identifies residual errors that may still affect design scoring after response correction. It applies the feedback-propagation operator used for the current prediction to each historical error pattern. The unrecovered component represents a possible remaining error and is added to the corrected response to form a scenario. Together, these scenarios yield a design score that accounts for how post-correction errors affect the nonlinear design objective. Across all six task–surrogate combinations, DARC achieves the lowest average response-reconstruction error and selection regret among the compared methods. In particular, DARC achieves stronger calibration performance with fewer feedback observations. Controlled ablations further show that accounting for post-correction errors improves selection under nonlinear objectives beyond scoring a single corrected response.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Surrogate Modeling, Data reconstruction, Computational Electromagnetics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 5 reviews, above 27% of the others.
Reviewer_fSkDOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

To address the issue of "inconsistency between response prediction accuracy and nonlinear design metrics" in electromagnetic structure screening, a frozen surrogate model and gradient-free two-stage framework is proposed.

Critical strengths

The problem is accurately identified and has practical value. "Model freezing, access to only a few frequency points, yet downstream metrics being threshold/extreme-value types" represents a real constraint in RF and antenna engineering.

Critical weaknesses
  1. Key control missing, making it impossible to disentangle the contributions of the two mechanisms. The paper only performs score ablations within DARC (Table 3), and does not graft "scenario scoring" onto other uncertainty-aware reconstructors. The empirical response GP (§4.2, Table 5) itself has posterior covariance; using it for posterior sampling + the same gg scoring would be a readily available control that requires no surrogate model. Without this control, readers cannot determine whether the advantages in Table 2 stem from the idea of "decision-aware scoring" or from DARC's specific propagation operator; the latter defines the contribution boundary of this paper. (Locations: §4.2 baseline list, §4.4, Table 3)
  2. It claims "a clean sweep in all 90 settings," yet the appendix provides only 6 aggregate columns.
  3. §4.1 Each setup has only one kk and three nested layouts. The method centered on "practical testing with a small number of frequency points" does not report stability under noisy or repeated measurements, so the practical effect of γγ regularization remains unexamined.
Questions for authors
  1. Please provide per-setting win/loss statistics for the 90 settings (e.g., in how many settings DARC achieves the best/worst UMSE/regret rank, along with counterexamples), and provide numerical tables for Figures 2 and 3 (including means and standard deviations).
  2. Please report the proportion of the 90 settings in which the “no-correction” fallback option was selected, as well as the distribution of the choices for ρ\rho, τ\tau, and πj\pi_j. If the fallback option was selected in a significant number of cases, please provide the results excluding those instances.
  3. Please clarify the consequences of CC and checkpoint selection sharing the same 256 structures: Can you provide sensitivity results for estimating CC using independent subsets (or cross-fitting)?
3 comments
AuthorsOpenReview ↗

Response to Reviewer fSkD [1/2]

Thank you for your careful review and for recognizing the practical value of the problem. We address your concerns point by point below.

W1. Key control missing, making it impossible to disentangle the contributions of the two mechanisms. The paper only performs score ablations within DARC (Table 3), and does not graft "scenario scoring" onto other uncertainty-aware reconstructors. The empirical response GP (§4.2, Table 5) itself has posterior covariance; using it for posterior sampling + the same gg scoring would be a readily available control that requires no surrogate model. Without this control, readers cannot determine whether the advantages in Table 2 stem from the idea of "decision-aware scoring" or from DARC's specific propagation operator; the latter defines the contribution boundary of this paper. (Locations: §4.2 baseline list, §4.4, Table 3)

A1. We added GP posterior-scoring comparisons to Appendix B.4, Table 7.

  • DARC outperforms response-GP posterior scoring on all three tasks. Following your suggestion, we sample from the empirical response GP's joint posterior and average the same design objective over these samples. Table A includes both direct scoring and posterior averaging for comparison.

  • Using the same surrogate predictions and historical residuals does not close the gap. We additionally evaluate a residual GP with these inputs, paired with direct scoring or posterior objective averaging. These controls show that DARC's advantage extends beyond equipping classical GP reconstruction with posterior scoring.

Table A. Mean Top-13 selection regret for GP controls and DARC (↓).

ReconstructionScoringHGARFFHairpin
Response GPDirect3.013 ± 0.0481.420 ± 0.0283.409 ± 0.055
Response GPPosterior MC2.970 ± 0.0561.450 ± 0.0363.438 ± 0.803
Residual GPDirect2.533 ± 0.1411.281 ± 0.0232.424 ± 0.077
Residual GPPosterior MC2.830 ± 0.4741.469 ± 0.1892.108 ± 0.068
DARC (ours)Weighted scenarios2.492 ± 0.0961.180 ± 0.0082.022 ± 0.084

W2 & Q1. Please provide per-setting win/loss statistics for the 90 settings (e.g., in how many settings DARC achieves the best/worst UMSE/regret rank, along with counterexamples), and provide numerical tables for Figures 2 and 3 (including means and standard deviations).

A2. We have expanded the numerical supplement to make the per-setting results and figure data easier to inspect.

  • DARC outperforms all three model-tuning baselines in UMSE across all 90 settings. These comparisons cover fine-tuning, latent adaptation, and Gappy AE. Per-setting results are provided in 01_original_data/setting_metrics.csv in the supplementary materials.

  • DARC's selection advantage extends beyond response accuracy. DARC achieves the lowest regret in 83/90 settings, including every HGA and Hairpin setting. In the Hairpin cases where the empirical GP achieves lower response error, DARC still reduces selection regret by 24.4%–26.6%, demonstrating that more accurate reconstruction does not necessarily yield better design choices. The remaining cases occur only on RFF at higher observation budgets, but the average regret gap is only 0.026.

  • Numerical results for Figures 2 and 3 are now available. We provide them as 01_original_data/figure2_results.csv and 01_original_data/figure3_results.csv in the supplementary materials.

AuthorsOpenReview ↗

Response to Reviewer fSkD [2/2]

W3. §4.1 Each setup has only one kk and three nested layouts. The method centered on "practical testing with a small number of frequency points" does not report stability under noisy or repeated measurements, so the practical effect of γγ regularization remains unexamined.

A3. We extended the evaluation to noisy observations and added the results to Appendix B.8, Table 11.

  • DARC retains the lowest mean selection regret at every tested noise level across all three tasks. It outperforms fine-tuning, latent adaptation, and Gappy AE under noisy observations. Table B summarizes DARC's performance from noise-free observations to the highest tested noise level.

  • Parameter selection favors stronger regularization as noise increases. At the highest tested noise level, the calibration set selects a larger γ than in the noise-free control in 211 of 300 configurations. The remaining choices are unchanged. Selection records are provided in 07_measurement_noise/gamma_selection.csv in the supplementary materials.

Table B. Mean Top-13 selection regret of DARC (ours) under measurement noise (↓). Noise standard deviation is σ times the training-data response scale.

Taskσ=0σ=0.01σ=0.05σ=0.10
HGA2.4832.4852.4952.540
RFF1.1861.1951.2611.429
Hairpin2.0192.0222.0512.112

Q2. Please report the proportion of the 90 settings in which the “no-correction” fallback option was selected, as well as the distribution of the choices for 𝜌 , 𝜏 , and πj\pi_j . If the fallback option was selected in a significant number of cases, please provide the results excluding those instances.

A4. We have added the complete parameter-selection records to the supplementary materials.

  • DARC's gains do not rely on the no-correction fallback. The fallback was never selected, so excluding fallback cases leaves all results unchanged.

  • Adaptive scenario weighting was selected in 94% of configurations. The remaining 6% used fixed reference weights πj\pi_j. Complete selection records are provided in 01_original_data/parameter_selections.csv in the supplementary materials.

Q3. Please clarify the consequences of 𝐶 and checkpoint selection sharing the same 256 structures: Can you provide sensitivity results for estimating 𝐶 using independent subsets (or cross-fitting)?

A5. We repeated the evaluation with an independent reference set, separate from the data used for checkpoint selection.

DARC maintains its selection performance with an independent reference set. We replaced the shared reference set with 256 independent structures while keeping the surrogate and evaluation setup unchanged. The differences are small and show no consistent degradation across tasks, indicating that DARC's selection benefits do not depend on sharing reference data with checkpoint selection.

Table C. Mean Top-13 selection regret with shared and independent reference sets (↓).

TaskShared reference set (256)Independent reference set (256)
HGA2.6012.561
RFF1.2961.300
Hairpin2.4172.341

Thank you again for your constructive feedback. We would appreciate your reconsideration of the score in light of the additional experiments and evidence.

Reviewer_fSkDOpenReview ↗

Official Comment by Reviewer fSkD

Thank you for adding the GP posterior-scoring controls and analyses of measurement noise and independent reference sets; these substantially address my empirical concerns, and I am raising my rating from 1 to 2 (weak rejection). However, I remain unconvinced that the proposed residual transformation and weighting provide a sufficiently substantial advance over established residual-based stochastic decision methods, despite the demonstrated practical gains.

Reviewer_RWBYOpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

My apologies in advance if this is due to my ignorance of the field. But I found the paper very difficult to follow. Specifically, in Section 2 (Problem Formulation), What does a structure mean? Is the so-called structure x ever used in the decision problem (2)? Similarly, I have confusion around the definitions of "design space", "response", "frequencies", etc. I feel like for an application-driven paper like this, it might be better to start with a general introduction of the problem of interest before diving into the details as the problem context may not be familiar to the ICLR community. Again, I apologize if these are common sense. But for this reason, I can not provide a meaningful summary or review for this work.

Critical strengths

See above

Critical weaknesses

See above

Questions for authors

See above

2 comments
AuthorsOpenReview ↗

Response to Reviewer RWBY

Thank you for your feedback. We clarify the problem setting and how physical designs are used in the final selection.

  • A structure xx is a physical design, and we compare structures based on their performance. For example, a smartphone uses an antenna to send and receive invisible electromagnetic waves. These waves oscillate at different rates. A familiar analogy is a piano: keys on the right produce higher-pitched sounds because their strings vibrate faster. This rate of vibration is called frequency. Electromagnetic waves also have frequencies, and an antenna's performance can vary across them. In our formulation, a structure xx represents a specific design defined by its shape, size, or layout. The design space contains all allowable designs. A design's response yy records its performance at different frequencies.

  • A pretrained surrogate model reduces evaluation costs, but prediction errors can lead to incorrect design rankings. Obtaining the complete response for every candidate design is costly. Instead, the surrogate model fθ(x)f_\theta(x) predicts this response from the design. A predefined function gg maps a response yy to a design score g(y)g(y). Applying gg to a predicted response gives an estimated score. However, the model is trained on a limited set of previously evaluated designs. It may therefore fail to capture how new designs perform at different frequencies, leading to inaccurate scores and rankings.

  • DARC uses a few observations to improve design selection at a much lower cost than full evaluation. Observing a candidate's performance at a few frequencies is less costly than obtaining its complete response. These observations help identify prediction errors. DARC combines them with historical prediction errors to correct the predicted response and obtain a more reliable score using gg. Equation (2) then selects the LL highest-scoring candidates. Each candidate ii corresponds to a design xix_i. Its score is derived from the surrogate prediction fθ(xi)f_\theta(x_i) after correction, so the design participates in selection even though xix_i does not appear explicitly in Eq. (2).

We will revise the Introduction and Section 2 to clarify these concepts before presenting the mathematical formulation.

Thank you for your clarification. I really think the paper needs substantial changes to improve the presentation to make it meet the bar for ICLR, and more importantly, make it understandable by humans. For this reason, I keep my rating.

Reviewer_d1eiOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Decision-Aware Residual Calibration (DARC), a gradient-free framework that uses sparse feedback to correct surrogate responses and account for remaining errors in nonlinear design scoring, without updating the surrogate model. The method consists of two components. First, residual-conditioned reconstruction leverages cross-frequency correlations in historical prediction errors to correct the response at unobserved frequencies. Second, residual-aware design scoring refines design scores by accounting for the errors that may remain after correction. These scenarios yield a design score that accounts for how post-correction errors affect the nonlinear design objective. Experiments across six task–surrogate combinations demonstrate improvements in response reconstruction and selection regret over several existing baselines.

Critical strengths

1.The idea and pipeline are clear and easy to follow. The overall framework is relatively lightweight, and the work accounts for both response‑correction errors and remaining prediction errors after calibration.

2.The empirical evaluation is comprehensive, with diverse electromagnetic design tasks and a good set of representative baselines covering both model‑tuning and data‑reconstruction families.

3.The method holds practical application value for surrogate‑aided electromagnetic design under sparse measurement budgets. Table 2 demonstrates that DARC achieves the lowest average selection regret across all six tasks and surrogate architecture. Figures 2 and 3 further show that DARC remains competitive across different feedback budgets and surrogate training sizes, supporting its practical effectiveness under limited observations.

Critical weaknesses

1.The proposed framework relies on cross-frequency dependencies estimated from historical surrogate prediction errors. However, these correlations may vary across different surrogate models and data distributions. The paper provides limited analysis of their stability and generalizability, leaving it unclear whether historical error patterns remain reliable for unseen designs, particularly under distribution shifts.

2.The proposed weighting mechanism assigns higher importance to historical error patterns that are more similar to the observed feedback. While this design is intuitive, the paper does not sufficiently explain why similarity in observed errors provides a reliable basis for weighting the remaining-error scenarios after correction.

3.The experiments mainly consider noise-free observations and randomly sampled frequency locations, while the method additionally relies on fully observed historical reference responses. Its robustness under measurement noise, nonuniform observation patterns and limited historical reference data remains unclear.

Questions for authors
  1. How stable are the historical cross-frequency error correlations across different surrogate models and data distributions?

  2. Could you explain what the theoretical or empirical justification is for using the similarity of observed errors to determine the weights of post-correction residual scenarios?

1 comment
AuthorsOpenReview ↗

Response to Reviewer d1ei

Thank you for recognizing the clarity of our framework, the breadth of our evaluation, and its practical value. We address your concerns point by point below.

W1 & Q1. How stable are the historical cross-frequency error correlations across different surrogate models and data distributions?

A1. We conducted additional experiments and summarize the results below.

  • DARC achieves the lowest mean selection regret across all six task–surrogate combinations. Table A summarizes DARC’s results. Historical error statistics are estimated separately for each surrogate. We added detailed results in the numerical supplement.

  • DARC retains its selection advantage under the tested distribution shifts. We shift the test candidate distribution toward high-performing designs by increasing their probability of being sampled. At shift levels 0, 1, and 2, the top-performing 25% of designs receive 1×, 2×, and 3× the sampling weight of other designs, respectively. DARC achieves the lowest mean selection regret at every shift level in Table B.

Table A. DARC’s mean Top-13 selection regret across surrogate architectures (↓).

TaskSurrogateDARC (ours)
HGAMLP2.46
HGAResNet2.53
RFFMLP1.19
RFFResNet1.17
HairpinMLP2.04
HairpinResMLP2.00

Table B. Mean Top-13 selection regret under distribution shift (↓).

Task / Shift levelSourceFine-tuningGappy AEDARC (ours)
HGA-09.474.765.022.45
HGA-17.803.924.242.05
HGA-27.133.684.001.97
RFF-012.363.432.221.19
RFF-113.644.792.791.46
RFF-214.255.313.031.55
Hairpin-03.732.833.162.03
Hairpin-13.722.783.162.12
Hairpin-23.602.733.152.17

W2 & Q2. Could you explain what the theoretical or empirical justification is for using the similarity of observed errors to determine the weights of post-correction residual scenarios?

A2. We explain the rationale for the weighting and provide supporting ablation results below.

  • Our weighting uses observed errors as clues to what may remain after correction. If two designs have similar overall errors, correcting them in the same way leaves similar errors behind. We use similarity at the observed locations as a clue to this shared pattern.

  • The scoring ablation provides empirical support for this weighting. Table 3 shows that accounting for remaining-error scenarios improves selection over direct scoring of the corrected response. Adapting the weights further lowers regret on RFF and Hairpin, supporting the use of observed feedback to refine scenario scoring.

W3. Robustness under practical conditions

A3. We conducted additional experiments to assess DARC under noisy observations, nonuniform sampling, and limited historical reference data.

  • DARC retains its selection advantage under noisy observations. To test robustness beyond noise-free feedback, we added Gaussian noise with different standard deviations . Table C reports DARC’s results with and without noise. DARC achieves the lowest mean regret at every tested noise level in the full comparison in Appendix B.8, Table 11.

  • DARC retains its selection advantage under nonuniform sampling. We tested low-, middle-, and high-frequency preferences using Beta(2,5), Beta(2,2), and Beta(5,2) sampling over the normalized frequency range, respectively. Table D reports DARC’s results. DARC achieves the lowest mean regret across all tasks and sampling patterns. Full comparisons are provided in Appendix B.9, Table 12.

  • DARC remains useful with fewer historical reference structures. Table E examines the effect of reducing the reference set from 256 to 128 and 64 structures while keeping the calibration selection set fixed. Even with 64 reference structures, DARC reduces mean selection regret by 42.8%–88.5% across the three tasks relative to the uncalibrated surrogate. Details are provided in Appendix B.10, Table 13.

Table C. DARC (ours): mean Top-13 selection regret under measurement noise (↓).

Taskσ=0σ=0.01σ=0.05σ=0.10
HGA2.482.482.502.54
RFF1.191.191.261.43
Hairpin2.022.022.052.11

Table D. DARC (ours): mean Top-13 selection regret under different observation sampling patterns (↓).

TaskUniformLowMiddleHigh
HGA2.453.724.635.39
RFF1.191.771.714.71
Hairpin2.032.662.291.70

Table E. DARC (ours): mean Top-13 selection regret by historical reference set size (↓).

Task64128256
HGA2.822.682.56
RFF1.501.401.30
Hairpin2.782.612.34
Reviewer_Uz3dOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposed a framework for selecting electromagnetic design without updating their given fixed surrogates, by leveraging residual-aware design scoring. However, the problem formulation and technical presentation are mathematically unclear.

Critical strengths

None

Critical weaknesses
  1. Code is not provided for reproducible, their submitted supplemental material only contains json, csv files for data, no codes for the algorithms. The author should provide their actual algorithm implementation in either supplemental material or an anonymous link
  2. The problem interpretation is unclear, the surrogate fθf_\theta maps design xx to a real vector yy then the objective value is computed as g(y)g(y). How this procedure relate to the electromagnetic domain, and why does the objective value need to be computed via yy as output of a frozen model fθf_\theta rather than computed directly from the design space? This setting seems to be unnatural when evaluating a design depends on the a pre-trained surrogate. This surrogate is also not introduced intuitively in the problem formulation. Why the surrogate was not trained to learn the direct mapping from design xx to objective values instead?
  3. The technical presentation is highly inconsistent and incoherent. For example, CC in problem formulation is a set of designs but then being used as a matrix in the method section. In Eq(2), it is unclear that where C come from or just a fixed given set of designs. The estimated full response qq in problem formulation is also not introduced, whether it could be q=Hpq=Hp or not, and why it need to be in the form of Eq(5). Optimizing the regret in Eq(3) over A^\hat{A} does not depend on the first term which is the oracle’s scores.
  4. The problem is not novel, their formulation leads to an optimization problem, in which the method try to find L optimal designs from a set of candidates CC by ranking them via their proposed residual-aware design scoring. There was neither comparison nor discussion with various optimization methods such as Bayesian optimization, surrogate-based or generative-based optimization methods,…. The experimented baselines are also not sufficient.
  5. The method is unclear , how the author chose the scale matrix SS and the reference weights π\pi, how these weights are sensitive to the final performance? The author wrote several disjointed equations, neither sufficient explanation nor clear motivation for computing this. Table 3 ablated in scoring rules, but there is not concrete theoretical or conceptual explanation to show why their proposed rules are better
Questions for authors

See Weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer Uz3d [1/2]

Thank you for your feedback. We address your concerns below.

W1. Code is not provided for reproducible, their submitted supplemental material only contains json, csv files for data, no codes for the algorithms. The author should provide their actual algorithm implementation in either supplemental material or an anonymous link

A1. We will publicly release the complete implementation and datasets upon acceptance, as stated in Appendix A.3. The current submission provides Algorithm 1, Eqs. (4)–(9), and implementation details and hyperparameter search spaces in Appendix A. The supplementary material provides machine-readable results, including per-setting and per-repetition statistics and random seeds. These materials will support reproduction and verification of our experiments.

W2. The problem interpretation is unclear, the surrogate fθf_\theta maps design 𝑥 to a real vector 𝑦 then the objective value is computed as g(y)g(y) . How this procedure relate to the electromagnetic domain, and why does the objective value need to be computed via 𝑦 as output of a frozen model fθf_\theta rather than computed directly from the design space? This setting seems to be unnatural when evaluating a design depends on the a pre-trained surrogate. This surrogate is also not introduced intuitively in the problem formulation. Why the surrogate was not trained to learn the direct mapping from design 𝑥 to objective values instead?

A2.

  • Electromagnetic design objectives are evaluated from frequency responses. For a given design xx, full-wave simulation produces its response yy. The objective g(y)g(y) then calculates metrics such as bandwidth and response maxima. Since obtaining yy requires expensive simulation, while evaluating g(y)g(y) is inexpensive, we use a surrogate model to predict the former and compute the objective from its prediction.

  • This response-based modeling choice is well established and theoretically grounded.

    Yu et al. [1] provide a comprehensive review of response-based surrogate modeling in AI-assisted electromagnetic design. More broadly, Astudillo and Frazier [2] establish the theoretical value of this setting by showing that it can improve optimization efficiency.

  • We study calibration of an existing response surrogate. Given a pretrained surrogate, we use sparse response observations to improve its predictions and design selection. The setting does not require model parameters to remain frozen.

[1] Yu, Y., Zhang, Z., Cheng, Q. S., Liu, B., Wang, Y., Guo, C., & Ye, T. T. (2022). State-of-the-Art: AI-Assisted Surrogate Modeling and Optimization for Microwave Filters. IEEE Transactions on Microwave Theory and Techniques, 70(11), 4635–4651.

[2] Astudillo, R., & Frazier, P. I. (2019). Bayesian Optimization of Composite Functions. Proceedings of the 36th International Conference on Machine Learning, 97, 354–363.

AuthorsOpenReview ↗

Response to Reviewer Uz3d [2/2]

W3. The technical presentation is highly inconsistent and incoherent. For example, 𝐶 in problem formulation is a set of designs but then being used as a matrix in the method section. In Eq(2), it is unclear that where C come from or just a fixed given set of designs. The estimated full response 𝑞 in problem formulation is also not introduced, whether it could be q=Hpq=Hp or not, and why it need to be in the form of Eq(5). Optimizing the regret in Eq(3) over A^\hat A does not depend on the first term which is the oracle’s scores.

A3. We clarify the notation, the given candidate set, and the roles of reconstruction, selection, and evaluation below.

  • We have clarified the notation. The original submission uses C\mathcal C for the candidate set and CC for the residual covariance. To avoid confusion, we have renamed the covariance matrix as Σe\Sigma_e in the revised paper.

  • The candidate set is given, not generated by DARC. Section 2 explicitly introduces C\mathcal C as a given candidate index set. DARC estimates scores and selects LL designs from this set.

  • qq denotes the full reconstructed response. Section 2 defines q∈Rdq\in\mathbb R^d, whereas Hp∈RmHp\in\mathbb R^m contains only predictions at observed coordinates. Equation (5) specifies our reconstruction method, which uses historical error correlations to propagate sparse observed discrepancies to unobserved coordinates. It is a methodological choice, not a requirement imposed by the problem definition.

  • The oracle term measures the gap to the best possible selection. It is indeed constant with respect to the selected set. Equation (2) selects candidates using estimated scores, since ground-truth scores are unavailable at selection time. Equation (3) then evaluates this selection against the oracle using ground-truth scores.

W4. The problem is not novel, their formulation leads to an optimization problem, in which the method try to find L optimal designs from a set of candidates 𝐶 by ranking them via their proposed residual-aware design scoring. There was neither comparison nor discussion with various optimization methods such as Bayesian optimization, surrogate-based or generative-based optimization methods,…. The experimented baselines are also not sufficient.

A4. We clarify the scope of our contribution and the baseline comparisons below.

  • DARC addresses surrogate calibration, not design optimization or generation. To our knowledge, this is the first study of selection-oriented electromagnetic surrogate calibration from sparse response feedback. DARC improves scores for a given candidate pool. Comparing complete Bayesian optimization or generative design pipelines would address a different task involving candidate acquisition or generation, beyond the scope of this study.

  • Our baselines directly test alternative ways to exploit the same sparse feedback. They include Fine-tuning, Latent adaptation, Gappy AE, Residual spline, and Response GPs. Table 3 isolates the contribution of scenario scoring. We additionally compare response and residual GPs with direct or posterior Monte Carlo scoring in Appendix B.4, Table 7. DARC achieves lower mean selection regret than all four GP alternatives on all three tasks.

W5. The method is unclear , how the author chose the scale matrix 𝑆 and the reference weights 𝜋 , how these weights are sensitive to the final performance? The author wrote several disjointed equations, neither sufficient explanation nor clear motivation for computing this. Table 3 ablated in scoring rules, but there is not concrete theoretical or conceptual explanation to show why their proposed rules are better

A5. We clarify the roles of SS and π\pi and the rationale behind scenario scoring.

  • SS and π\pi follow fixed rules rather than requiring additional tuning. SS uses training-data normalization scales to make response errors comparable across coordinates. The reference weights π\pi give each reference structure equal total importance. They are distinct from the scenario weights adapted to the current feedback.

  • Scenario scoring captures effects that averaging responses can lose. Remaining prediction errors can change bandwidth or band extrema. DARC therefore evaluates the objective on each plausible corrected response before averaging the resulting scores. Sparse observations favor relevant error patterns, while the effective-sample-size constraint prevents excessive weight concentration.

  • Table 3 directly tests this scoring rationale. DARC outperforms Direct and Weighted mean on all three tasks. Weighted mean uses the same scenario weights and achieves more accurate responses, yet worse selections. Fixed-weight scenarios also outperform Direct on all three tasks, showing that scenario evaluation provides benefits even without adaptive weighting.

Reviewer_Uz3dOpenReview ↗

Response to Author Rebuttal

I thank the authors for their detailed rebuttal. However, I believe the original submission still requires substantial revisions to improve the clarity and coherence of its technical presentation. As also noted by other reviewers, several terms are not commonly used within the ICLR community, and there are inconsistencies in the notation that make the paper difficult to follow. While I appreciate the authors' efforts to address these concerns, I believe the paper would benefit from a more thorough revision and another round of peer review.

Reviewer_KMkSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses a practical problem in electromagnetic (EM) design using surrogate models, which is how to leverage sparse response feedback to correct a fixed model's response and to produce better rankings under nonlinear design objectives without updating the surrogate parameters. The proposed framework, DARC uses a two stage post-training method: residual conditioned reconstruction and residual aware design scoring. Evaluation covers three EM tasks and two neural network architectures, totaling 90 settings; showing good performance on several metrics.

Critical strengths

Good motivation of a real prediction–decision gap. The paper emphasizes that nonlinear objectives make ranking sensitive to residual errors that remain after correction.

Principled reuse of the same correction operator. Constructing remaining-error scenarios by applying the identical operator GG used for the current sample is well-designed as it avoids a separate residual model and keeps the method coherent.

Strong empirical results. The empirical results show significant gains over baselines and the ablations helps isolate the contribution of scenario scoring from response correction.

Critical weaknesses

Limited novelty. I think the biggest weakness of DARC is its novelty of the reconstruction stage. The response-correction module is essentially regularized optimal interpolation on historical residuals. The overall method can appear as a combination of classical techniques rather than a fundamentally new conceptual and algorithmic idea.

Assumption of a good reference set of historical responses. It seems to me that the performance depends on the historical error covariance being a good model of the errors that appear at test time. In practice, when the surrogate is weak or the design space is highly non-stationary, this covariance may be unreliable under distribution shifts. The authors should provide additional experiments exploring distribution shift between reference and test structures.

Hyper-parameter tuning burden Several hyper-parameters are needed for DARC (γ\gamma, τ\tau, ρ\rho). Although selected on a calibration set, the paper needs to provide a thorough sensitivity study or guidance on how robust the method is when the calibration set is small. Moreover, it would be helpful if the authors could provide some intuition or justification on how to select the hyper-parameters.

Computational costs. For each test sample one must form and invert an m×mm \times m matrix and evaluate the nonlinear objective on MM scenarios. How does DARC scale when the reference set becomes large (in the thousands) or when many candidates should be scored simultaneously?

Questions for authors

See above.

4 comments
AuthorsOpenReview ↗

Response to Reviewer KMkS [1/2]

Rebuttal to Reviewer KMkS

Thank you for recognizing our motivation, DARC's principled design, and the contribution of scenario scoring. We address your concerns point by point below.

W1. Limited novelty. I think the biggest weakness of DARC is its novelty of the reconstruction stage. The response-correction module is essentially regularized optimal interpolation on historical residuals. The overall method can appear as a combination of classical techniques rather than a fundamentally new conceptual and algorithmic idea.

A1.

Our application-driven study contributes methodological insights and empirically established knowledge of practical value.

  • Our novelty lies in selection-oriented EM surrogate calibration from sparse feedback. We first study how to turn a small number of costly frequency observations into better design choices using surrogate models. The central challenge is that design selection depends on nonlinear objectives such as bandwidth and band extrema. Consequently, a more accurate predicted response does not necessarily lead to a better design choice.

  • Our key finding is that sparse feedback can improve design selection beyond what response reconstruction alone achieves. In Table 3 of the paper, we compare DARC with the Direct and Weighted mean variants, which remove its scenario-scoring mechanism to isolate its contribution to design selection. DARC outperforms both variants on all three tasks, showing that even more accurate response reconstruction cannot substitute for our scenario-scoring mechanism.

  • DARC also improves reconstruction accuracy rather than trading it for better selection. Across all 90 settings, it achieves lower test UMSE than fine-tuning, latent adaptation, and GappyAE, even when those baselines are tuned explicitly for development-set UMSE. These results establish the value of DARC's reconstruction stage itself.

  • DARC outperforms combinations of classical reconstruction and scoring methods. To assess whether combining classical reconstruction and scoring methods is sufficient for this task, we compare response and residual GPs using Gaussian conditioning [1], followed by direct scoring or Monte Carlo estimation of the expected objective under the joint posterior. DARC achieves lower mean regret than these alternatives on all three tasks, demonstrating gains beyond these combinations of classical methods.

Table A. Top-13 selection regret for classical combinations and DARC (↓).

ReconstructionScoringHGARFFHairpin
Response GPDirect3.013 ± 0.0481.420 ± 0.0283.409 ± 0.055
Response GPPosterior MC2.970 ± 0.0561.450 ± 0.0363.438 ± 0.803
Residual GPDirect2.533 ± 0.1411.281 ± 0.0232.424 ± 0.077
Residual GPPosterior MC2.830 ± 0.4741.469 ± 0.1892.108 ± 0.068
DARC (ours)Weighted scenarios2.492 ± 0.0961.180 ± 0.0082.022 ± 0.084

We have added these results and experimental details to Appendix B.4 (Table 7) of the revised paper and the updated supplementary material.

[1] Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian Processes for Machine Learning. MIT Press.

AuthorsOpenReview ↗

Response to Reviewer KMkS [2/2]

W2. Assumption of a good reference set of historical responses. It seems to me that the performance depends on the historical error covariance being a good model of the errors that appear at test time. In practice, when the surrogate is weak or the design space is highly non-stationary, this covariance may be unreliable under distribution shifts. The authors should provide additional experiments exploring distribution shift between reference and test structures.

A2. We conducted additional experiments and added the results and experimental details to Appendix B.5 (Table 8) of the revised paper.

DARC retains its selection advantage despite increasing residual-covariance mismatch. Our new experiment constructs candidate pools that increasingly favor high-performing designs, while keeping the reference set, surrogate, and calibration parameters fixed. Shift levels 0/1/2 assign the best-performing 25% of test designs 1×/2×/3× the sampling weight of other designs. DARC achieves the lowest mean selection regret at every tested shift level across all three tasks.

Table B. Top-13 selection regret under distribution shift (↓).

Task / Shift levelFine-tuningGappy AEDARC (ours)
HGA-04.7575.0242.454
HGA-13.9194.2392.051
HGA-23.6763.9991.968
RFF-03.4252.2201.194
RFF-14.7922.7881.457
RFF-25.3063.0341.553
Hairpin-02.8283.1592.029
Hairpin-12.7843.1572.115
Hairpin-22.7333.1482.173

W3. Hyper-parameter tuning burden Several hyper-parameters are needed for DARC ( 𝛾 , 𝜏 , 𝜌 ). Although selected on a calibration set, the paper needs to provide a thorough sensitivity study or guidance on how robust the method is when the calibration set is small. Moreover, it would be helpful if the authors could provide some intuition or justification on how to select the hyper-parameters.

A3. We have added experiments with smaller calibration selection sets to Appendix B.6 (Table 9) of the revised paper.

  • Parameter selection uses small, predefined search spaces. DARC selects among seven correction options using unobserved-response MSE, then among 17 scoring options using objective MSE on the calibration selection set.

  • DARC maintains selection performance with 75% fewer calibration structures. We conducted additional experiments with calibration selection sets of 32, 64, and 128 structures, reselecting DARC's parameters at each size with all other settings unchanged. Reducing this set by 75% increases mean selection regret by at most 1.64% across the three tasks, showing that DARC maintains its selection performance with substantially fewer calibration structures.

Table C. Top-13 selection regret by calibration selection set size (↓).

Task3264128
HGA2.450 ± 0.1192.453 ± 0.1042.454 ± 0.111
RFF1.205 ± 0.0181.191 ± 0.0081.194 ± 0.009
Hairpin2.062 ± 0.0642.038 ± 0.0472.029 ± 0.051

W4. Computational costs. For each test sample one must form and invert an 𝑚 × 𝑚 matrix and evaluate the nonlinear objective on 𝑀 scenarios. How does DARC scale when the reference set becomes large (in the thousands) or when many candidates should be scored simultaneously?

A4. We added scalability experiments covering larger reference sets and candidate pools to Appendix B.7 (Table 10) of the revised paper.

  • Larger reference sets do not enlarge the correction system. Response correction solves an m×mm\times m system determined by the sparse observations, so its O(m3)O(m^3) factorization cost is independent of reference-set size. Scenario scoring scales linearly with the number of reference scenarios for fixed response dimensions and solver iterations.

  • DARC efficiently handles thousands of references and candidates. With 2,048 reference structures, correcting and scoring 4,096 candidate designs takes 11.72–13.87 seconds across the three tasks on a single RTX 3090, with peak allocated GPU memory below 1.90 GiB.

Table D. Computation time across reference and candidate set sizes (seconds).

ReferencesCandidatesHGARFFHairpin
2562560.591.070.68
1,0242560.630.890.76
2,0482560.650.920.80
2,0481,0242.903.534.75
2,0484,09611.7213.8712.80

Thank you again for your constructive feedback. We would appreciate your reassessment in light of the additional experiments and clarifications.

AuthorsOpenReview ↗

Looking Forward to the Response from Reviewer KMkS

Dear Reviewer,

We sincerely thank you for your valuable feedback and constructive suggestions, which have helped us improve our manuscript. We have provided detailed responses to your comments.

We would be happy to address any further questions or concerns you may have.

Best regards, The Authors

I appreciate the authors' detailed rebuttal and their efforts to address the raised concerns. Nevertheless, I am still unconvinced by the conceptual novelty of the proposed moethod. Moreover, I believe the method still requires substantial effort for hyper-parameter tuning. Therefore, I decide to keep my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.