acceptodds
Withdrawn from ICLR 2027

Stable Test-Time Adaptation for High-Dimensional Simulation Regression

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Machine learning surrogates are increasingly used in engineering to accelerate costly simulations of partial differential equations (PDEs), yet distribution shifts between training and deployment often cause severe performance degradation, e.g., under unseen geometries or configurations. Test-Time Adaptation (TTA) can mitigate such shifts, but existing methods are largely developed for classification or image-based tasks on regular grids. When applied to high-dimensional regression on unstructured meshes, a standard setting in engineering simulation, estimating distributional statistics becomes severely ill-conditioned, leading to unstable adaptation. In this work, we propose a stable test-time adaptation method for high-dimensional simulation regressors. Our method uses E-optimal source subset selection to stabilize KL-based alignment between source and target representation distributions. The retained maximally informative source samples further enable stable source-risk regularization and parameter selection at test time. When applied to pretrained simulation surrogates, our method yields up to 12% improvement in out-of-distribution RMSE with modest computational overhead. To the best of our knowledge, this is the first systematic demonstration of effective TTA for high-dimensional simulation regression and generative design optimization, validated across diverse state-of-the-art benchmark datasets.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
test-time adaptation, distribution shift, neural surrogates, high-dimensional regression

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bQBROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ETA, which combines E-optimal source sample selection, feature alignment, source-risk regularization, and importance-weighted model selection to adapt pretrained simulation models without target labels. It analyzes the stability of the adaptation objective and evaluates accuracy, stability, and computational cost across seven simulation regression and generative engineering design tasks.

Critical strengths
  1. The paper addresses a relevant deployment problem: adapting simulation models to new configurations with limited source data and no target labels.

  2. Experiments span mechanical and thermal simulations and generative design tasks, showing performance improvements in multiple settings.

  3. Ablation studies provide evidence that source sample selection, regularization, and model selection improve adaptation accuracy and stability.

Critical weaknesses
  1. Whether feature alignment preserves physical responses remains insufficiently validated. The method aligns latent feature means and variances, although changes in load or thickness can produce meaningful physical changes. The stability analysis concerns statistical perturbations; response curves or sensitivity analyses are needed to assess whether adaptation preserves correct physical trends.

  2. The core sampling design needs stronger comparisons. Although component ablations are provided, E-optimal selection is mainly compared with random sampling. Comparisons with other representative sampling strategies under matched budgets would help establish its relative advantage.

  3. The distribution shifts remain relatively controlled. All four SIMSHIFT tasks use medium-difficulty shifts. Target samples are grouped by known shift parameters, with one update followed by a model reset for each group. These results do not establish robustness to simultaneous changes in multiple parameters, more distant extrapolation, or continuously changing deployment conditions.

Questions for authors

Please address the concerns raised in the Weaknesses above.

Reviewer_4DXXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identified a research gap in addressing distribution shifts for high-dimensional simulation regression problems and proposed ETA, a method for stable test-time adaptation of neural simulation surrogates based on E-optimal source sample selection, combined with source-risk regularization and importance-weighted model selection. Experimental results generally show improvement over existing TTA baselines.

Critical strengths
  1. The research gap between existing TTA applications (i.e. mostly on classification…) and high-dimensional simulation regression on unstructured meshes is clearly motivated. The authors provide sufficient background in the related work section and clearly distinguish Test-Time Adaptation (TTA) from other related concepts such as TTT, Domain generalization, meta-learning, and active learning.
  2. Section 4 and the Appendix provide theoretical support and formal proofs for the proposed method.
Critical weaknesses
  1. The presentation needs improvement, and several key design choices need clarification.
  • The main idea of TTA and the intuition for why E-optimal sample selection would help is not clearly explained in the introduction section.
  • In Section 3 (Setting), although the authors have defined "limited information retained from the source domain" as "m retained samples", the overall storage budget or any formal definition of what information should be retained is not clearly discussed in the paper.
  • Regarding Figure 2 and the paragraph starting from Line 179, it is not very clear to me why E-optimal sampling is stable implies better adaptation performance than random sampling.
  • The authors emphasized a setup in which only a small subset of source samples is available at test time, but the authors did not clearly explain why this setup is important or provide any real-world examples. The reference to "proprietary constraints or the dataset size" in the introduction section needs explanations in more detail or some examples.
  • Minor issue: the text labels in Figure 1 are small and crowded, making them difficult to read.
Questions for authors
  1. The paper defines objective as to “stably (w.r.t. sample variations in the target batch) minimize the surrogate upper bound in Eq. 4”. Could the authors clarify why not choosing the stability of prediction after adaptation?
  2. In the discussion section, the authors identified "incorporating physics-informed constraints and priors" as a future direction. Could the authors provide a more detailed discussion on how these can be integrated into the current ETA framework?
  3. In Table 10, ETA and random sample selection achieve near identical RMSE on most datasets. Could the authors provide further discussion for these comparative results (Section 5.5 (Line 445-453)).
Reviewer_c3XmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes E-optimal Test-Time Adaptation (ETA) for high-dimensional simulation regression under distribution shift. A small retained source subset is used for feature-distribution alignment, source-risk regularization, and learning-rate selection without target labels. A covariance-based bound motivates the source-selection criterion. The method is evaluated on seven datasets spanning simulation regression and generative design.

Critical strengths

The engineering setting is worthwhile, and the evaluation covers several different simulation tasks rather than one benchmark. The method can improve a pretrained surrogate using unlabeled target inputs and a small set of retained source samples.

The high-resolution Plate results give the clearest evidence for the stability argument. ETA avoids the deterioration seen in the source model and SSA as resolution increases (Figure 3). Table 10 also shows a substantial improvement over random source selection at the largest Plate seting

Critical weaknesses

The specific contribution of E-optimal selection is less clear on the other datasets. In Table 10, random selection gives nearly the same mean RMSE on Rolling and Forming, although E-optimal selection can reduce variability. The highest-resolution Plate result is a meaningful exception. Since source regularization and learning-rate selection also contribute to ETA, the paper needs a clearer account of when the E-optimal subset is responsible for the gain and when the other components do most of the work.

The theoretical and practical notions of stability need to be separated. Theorem 1 bounds how the adaptation objective changes with target-batch statistics at fixed model parameters; it does not directly guarantee stable predictions after adaptation. The implementation also uses PCA, pivoted-QR initialization, and weighted channel-wise KL alignment (Appendix C). The theorem is useful motivation, but its connection to this implemented procedure is not sufficiently clear.

Questions for authors
  1. What explains the large E-optimal advantage at the highest Plate resolution but the small mean-RMSE differences on several other tasks? With regularization and learning-rate selection held fixed, is the main benefit better average predictions, lower variability, or both?

  2. Which parts of Theorem 1 apply to the PCA/QR-based selector and weighted channel-wise loss used in the experiments? Please distinguish the stability guaranteed for the objective from the prediction stability supported empirically.

Comments on the paper

1 comment

We thank the reviewers for their time and feedback. Given all considerations, we will withdraw our submission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.