- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents BLISS, a dataset and benchmark for predicting the buckling behavior of imperfect spherical shells. The dataset contains millions of nonlinear FEM simulations, with three prediction targets: collapse load, displacement field, and failure site. The main motivation is to study how ML surrogates generalize when nearby geometric defects begin to interact and change the dominant buckling mechanism. The work consists of three evaluation protocols, ranging from a standard random split to extrapolation across defect depths and a sparse-to-dense defect distribution shift. Existing models perform poorly when transferred from isolated to interacting defects.
Critical strengths
-
The paper raises an interesting and thought-provoking question: can deep learning models generalize across different physical mechanisms within the same problem? The results are quite surprising. For example, model performance on B1 and B3 is even negatively correlated. I believe these findings offer useful insights for future AI4S research. The authors also provide a high-quality dataset of considerable scale.
-
The paper provides a fairly comprehensive benchmark, including not only the dataset but also a wide range of baselines, evaluation metrics, and experimental protocols. This makes it a useful resource for further research.
Critical weaknesses
The setting considered in this paper is rather specific. It remains unclear how well the findings and potential improvements developed for this particular task would generalize to other physical systems or be useful in practice.
Questions for authors
-
In B3, the number of defects changes from 11–18 in training to 69–81 in testing, alongside the change in minimum defect spacing. The distribution of failure-site locations also changes. Therefore, the significant performance degradation may be partly caused by the increased geometric complexity and changes in the input distribution?
-
The appendix mentions that the simulations use Abaqus/Standard with a neo-Hookean material model and S4R shell elements. Could the authors clarify the underlying shell formulation (e.g., Kirchhoff–Love or Reissner–Mindlin)? How sensitive are the buckling loads and failure modes to this choice?
Response to Reviewer gad3
We thank the reviewer for the positive assessment and for the precise question about B3, which led to the most informative new experiment in the revision. Below, κ is the knockdown factor (the normalised collapse load), and a κ error is the median relative error over the test shells, averaged over three seeds.
Is the B3 drop partly due to more defects and a different input distribution? (Q1) This is a fair concern. In B3 the number of defects, their spacing and the failure-site distribution all change together. We separated them with controlled tests on the dense shells (App. N.9, Table 45).
Removing defects down to the sparse count keeps the full dense error. The paired layouts hold the same 11–18 defects, placed 25° apart in one and in clusters at 10° in the other. Changing only the spacing raises the κ error from 0.6–1.6% to 6.8–8.0%. Per shell the solver κ changes by a median of 7.7% between the two layouts, while the models' predictions change by only 0.8–2.0%. The models miss the change that the spacing causes. These controls identify the local spacing of the defects as the dominant cause of the B3 drop on hemispheres, and the count and the failure-site distribution contribute little. The error also persists on the 1,950 dense test shells whose κ lies inside the range of the training labels (4.2–9.3% over six families). Models trained on B1, which includes dense shells, also miss an isolated close pair. They predict changes of −1.1 to +0.3% when a single pair is moved from 25° to 10° on 100 B1 test shells (99 sparse), whereas the solver gives −14% (Sec. 5.2). On cylinders, count and spacing contribute about equally for four of the five families (spacing share 0.40–0.59, and 1.05 for DeepONet, App. P.5), and the revision reports both. The spacing share is the fraction of the sparse-to-dense error increase already present at the sparse count with dense spacing.
Shell formulation and sensitivity (Q2). S4R is a finite-strain Reissner-Mindlin (shear-deformable) shell. On 80 re-solved shells, the Kirchhoff-type element S4R5 changes the median κ by 1.23%, fully integrated S4 by 0.11%, and the continuum shell SC8R with three elements through the thickness by 0.61%. All keep a median field R² ≥ 0.997 and every clear failure site (Table 34, App. N.1). Where two candidate sites deflect almost equally, the site is kept on 65–85% of the shells. The finite element solver (Abaqus) does not allow hyperelastic laws with S4 and S4R5 elements, so these two are compared with linear elastic S4R runs. These changes are well below the 4.72% error of the best learned B3 model, and the B1 ranking of the eight families keeps a Spearman correlation of 0.86 (S4R5) and 0.98 (S4) with the released one (App. N.5). With a single element through the thickness, SC8R raises κ by 5.0%, mostly a through-thickness discretisation effect that three elements remove.
Generality and practical use (W1). We agree the setting is specific by design. It fixes the equations, material and load so that only the defect layout changes, and the abstract and Sec. 6 now state the scope. The practical lesson is that an in-distribution leaderboard can pick the wrong surrogate when an interaction between inputs decides the output. Over eight families, the B1 ranking on κ predicts the B2 ranking (deeper defects, Spearman 0.83) and is unrelated to the B3 ranking (+0.02), and SFNO goes from first on B1 to last of the six geometry models on B3. A second system, axially compressed cylinders, shows the same failure with the opposite interaction sign (App. P). Neighbours of the deepest defect raise κ by a median of 6.4% on hemispheres and lower it by 15.1% on cylinders (neighbour probe, Table 44 and App. P.5). Sparse-trained models miss both. They predict a decrease of 0.5–4.8% on hemispheres and capture 4.9–13.1% of the drop on cylinders. Five families trained on sparse cylinders err on κ by 1.3–1.6% on sparse and by 6.5–17.4% on dense test cylinders (three seeds, Table 49), and by 3.8–11.6% on the 107 dense ones with κ inside the range of the training labels (Table 50). The paired control and the neighbour probe need only a solver, so other benchmarks can apply them to their own interactions.