acceptodds
Under review as a conference paper at ICLR 2027

BLISS: A Dataset and Benchmark for the Buckling of Imperfect Spherical Shells

Abstract

Machine-learning (ML) surrogates are increasingly used to predict how a structure responds to its geometry. However, existing scientific-ML benchmarks rarely use a change in the physical mechanism itself as the source of a distribution shift. Their shifts come from random splits, new initial conditions or held-out ranges of sampled inputs. Here, we introduce BLISS, a dataset and benchmark of 8,528 imperfect hemispherical shells simulated with finite elements until collapse, in which an instability creates the shift. A small change in geometry, two defects moving close enough to interact, changes how the shell collapses. Models map the surface with its defects to three outputs of the collapse, the collapse load (knockdown factor), the displacement field and the failure site where collapse starts. Within distribution, as on existing benchmarks, the best models reach a field of 0.985 and find 98% of clear failure sites. With deeper defects they lose the failure site where defects interact. Trained on shells with far-apart defects and tested on shells with close defects, every model that reads the shell geometry scores below a constant prediction of the average training shell on the field. Moving a single pair of defects from to apart lowers the collapse load by 14% in the solver, while the models predict almost no change. Five dense examples recover most of the field of two models, and the choice between two nearly equal failure sites stays close to chance. Our simulations match eight physical tests within 3.3%. Code, scorer and data-access instructions are available at https://anonymous.4open.science/w/BLISS-3388/.

Primary area:
datasets and benchmarks
Keywords:
benchmark dataset, shell buckling, computational physics, geometric deep learning, neural operators, surrogate models, simulation, imperfection sensitivity, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gad3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents BLISS, a dataset and benchmark for predicting the buckling behavior of imperfect spherical shells. The dataset contains millions of nonlinear FEM simulations, with three prediction targets: collapse load, displacement field, and failure site. The main motivation is to study how ML surrogates generalize when nearby geometric defects begin to interact and change the dominant buckling mechanism. The work consists of three evaluation protocols, ranging from a standard random split to extrapolation across defect depths and a sparse-to-dense defect distribution shift. Existing models perform poorly when transferred from isolated to interacting defects.

Critical strengths
  1. The paper raises an interesting and thought-provoking question: can deep learning models generalize across different physical mechanisms within the same problem? The results are quite surprising. For example, model performance on B1 and B3 is even negatively correlated. I believe these findings offer useful insights for future AI4S research. The authors also provide a high-quality dataset of considerable scale.

  2. The paper provides a fairly comprehensive benchmark, including not only the dataset but also a wide range of baselines, evaluation metrics, and experimental protocols. This makes it a useful resource for further research.

Critical weaknesses

The setting considered in this paper is rather specific. It remains unclear how well the findings and potential improvements developed for this particular task would generalize to other physical systems or be useful in practice.

Questions for authors
  1. In B3, the number of defects changes from 11–18 in training to 69–81 in testing, alongside the change in minimum defect spacing. The distribution of failure-site locations also changes. Therefore, the significant performance degradation may be partly caused by the increased geometric complexity and changes in the input distribution?

  2. The appendix mentions that the simulations use Abaqus/Standard with a neo-Hookean material model and S4R shell elements. Could the authors clarify the underlying shell formulation (e.g., Kirchhoff–Love or Reissner–Mindlin)? How sensitive are the buckling loads and failure modes to this choice?

1 comment
AuthorsOpenReview ↗

Response to Reviewer gad3

We thank the reviewer for the positive assessment and for the precise question about B3, which led to the most informative new experiment in the revision. Below, κ is the knockdown factor (the normalised collapse load), and a κ error is the median relative error over the test shells, averaged over three seeds.

Is the B3 drop partly due to more defects and a different input distribution? (Q1) This is a fair concern. In B3 the number of defects, their spacing and the failure-site distribution all change together. We separated them with controlled tests on the dense shells (App. N.9, Table 45).

test on dense B3 shellsheld fixedchangedκ error, 5 families
full dense test (2,200)––4.7–6.8%
cut to the sparse count (200 shells)spacing 10°count → 11–186.3–7.7%
paired layouts, spread (100 shells)count, depths, footprintspacing 25°0.6–1.6%
same 100 layouts, clusteredcount, depths, footprintspacing 10°6.8–8.0%
reweighting the failure sites (2,200)layoutssite distribution → sparsechanges ≤ 0.3 points

Removing defects down to the sparse count keeps the full dense error. The paired layouts hold the same 11–18 defects, placed 25° apart in one and in clusters at 10° in the other. Changing only the spacing raises the κ error from 0.6–1.6% to 6.8–8.0%. Per shell the solver κ changes by a median of 7.7% between the two layouts, while the models' predictions change by only 0.8–2.0%. The models miss the change that the spacing causes. These controls identify the local spacing of the defects as the dominant cause of the B3 drop on hemispheres, and the count and the failure-site distribution contribute little. The error also persists on the 1,950 dense test shells whose κ lies inside the range of the training labels (4.2–9.3% over six families). Models trained on B1, which includes dense shells, also miss an isolated close pair. They predict changes of −1.1 to +0.3% when a single pair is moved from 25° to 10° on 100 B1 test shells (99 sparse), whereas the solver gives −14% (Sec. 5.2). On cylinders, count and spacing contribute about equally for four of the five families (spacing share 0.40–0.59, and 1.05 for DeepONet, App. P.5), and the revision reports both. The spacing share is the fraction of the sparse-to-dense error increase already present at the sparse count with dense spacing.

Shell formulation and sensitivity (Q2). S4R is a finite-strain Reissner-Mindlin (shear-deformable) shell. On 80 re-solved shells, the Kirchhoff-type element S4R5 changes the median κ by 1.23%, fully integrated S4 by 0.11%, and the continuum shell SC8R with three elements through the thickness by 0.61%. All keep a median field R² ≥ 0.997 and every clear failure site (Table 34, App. N.1). Where two candidate sites deflect almost equally, the site is kept on 65–85% of the shells. The finite element solver (Abaqus) does not allow hyperelastic laws with S4 and S4R5 elements, so these two are compared with linear elastic S4R runs. These changes are well below the 4.72% error of the best learned B3 model, and the B1 ranking of the eight families keeps a Spearman correlation of 0.86 (S4R5) and 0.98 (S4) with the released one (App. N.5). With a single element through the thickness, SC8R raises κ by 5.0%, mostly a through-thickness discretisation effect that three elements remove.

Generality and practical use (W1). We agree the setting is specific by design. It fixes the equations, material and load so that only the defect layout changes, and the abstract and Sec. 6 now state the scope. The practical lesson is that an in-distribution leaderboard can pick the wrong surrogate when an interaction between inputs decides the output. Over eight families, the B1 ranking on κ predicts the B2 ranking (deeper defects, Spearman 0.83) and is unrelated to the B3 ranking (+0.02), and SFNO goes from first on B1 to last of the six geometry models on B3. A second system, axially compressed cylinders, shows the same failure with the opposite interaction sign (App. P). Neighbours of the deepest defect raise κ by a median of 6.4% on hemispheres and lower it by 15.1% on cylinders (neighbour probe, Table 44 and App. P.5). Sparse-trained models miss both. They predict a decrease of 0.5–4.8% on hemispheres and capture 4.9–13.1% of the drop on cylinders. Five families trained on sparse cylinders err on κ by 1.3–1.6% on sparse and by 6.5–17.4% on dense test cylinders (three seeds, Table 49), and by 3.8–11.6% on the 107 dense ones with κ inside the range of the training labels (Table 50). The paired control and the neighbour probe need only a solver, so other benchmarks can apply them to their own interactions.

Reviewer_vJGLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces a new physics dataset that models collapse of imperfect shells. The goal of the dataset is to explore how small shifts in geometry of the input (the locations of defects) impact the outcome of the physics simulation (collapse load). The dataset is demonstrated to meet physical tests within a reasonable tolerance. Baseline machine learning models trained on the data demonstrate that extrapolation to out-of-distribution tasks is an open challenge

Critical strengths

The paper is thorough in describing this new dataset. The context of the dataset is well-placed in literature and the unique features of the dataset are carefully described in relationship to 14 other benchmarks. The description of the how the data is generated and why it is important is well-written. Baseline results are provided that indicate that the dataset is challenging for 10 machine learning architectures and compared against 3 non-learned baselines like the training mean.

Critical weaknesses

My primary concern is that this is the wrong audience for the work. A single dataset is primarily of interest to the application area itself because it is impossible to draw general conclusions about the ability of machine learning models to learn instabilities from a single dataset. Any conclusions drawn will be unique to this dataset. A suite of datasets with these properties would be more informative to the machine learning community. Similar datasets cited in the Related Work are published in engineering venues (Computers & Structures or Communications Engineering). Only dataset suites that contain multiple datasets like “The Well” are published in machine learning venues (NeurIPS, KDD, ICLR); or the cited papers introduce a novel machine learning method along with the dataset (“MGN Plate”, “Geo-FNO solids”, etc).

The results indicate that the challenge is in out-of-distribution prediction, which is not surprising. The dataset claims to be the only of the similar datasets to provide train/test splits to test out-of-distribution (either B2 deeper-defects or B3 sparse-to-dense scenarios). The prediction of buckling of imperfect spherical shells itself (in-distribution) does not appear to be challenging. However, the benchmark suites do test situations where physics changes across datasets, thereby making it easy to train on one set of physics and test on different physics - indeed this is the motivation for creating these suites.

Questions for authors

Are there models that work well for the other types of “shift source” in Table 1 but fail here? What is the lesson for the model developers in the machine learning community?

Does the paragraph labeled “Scale” gives the total compute for the entire paper (i.e., does “the simulations” mean the computation for all of the train/validation/test data)? If so, it would be helpful to break this down into something can be interpreted as how the computation scales. How much compute does each simulation take? Presumably, inference time of machine learning needs to be much faster than this to be useful. How much RAM and how many cores per node (as this affects compute time)?

1 comment
AuthorsOpenReview ↗

Response to Reviewer vJGL

We thank the reviewer for asking what an ML researcher learns from a single physical system. In-distribution accuracy selects the wrong model when the interaction regime shifts, and the best of the six geometry models in distribution is the worst on dense shells. A semi-analytical shell model, derived with an AI agent (Claude Opus), predicts the shifted collapse loads from the same sparse training labels.

Out-of-distribution failure is unsurprising (W2). We agree that some degradation out of distribution is expected and that in-distribution prediction is easy. Suites change the equations between datasets. BLISS keeps them fixed and moves the defects closer, and it is the only benchmark in Table 1 whose shift comes from an interaction between inputs. The result we highlight is about model selection. Over eight families, the B1 (in-distribution) ranking on κ (the knockdown factor) predicts the ranking on deeper defects (B2, Spearman 0.83, p = 0.015) but not on the sparse-to-dense shift (B3, +0.02, p = 0.98). Of the six geometry models, SFNO is first on B1 (0.58%) and last on B3 (9.37%), and DeepONet is last on B1 (1.32%) and first on B3 (4.72%).

This raises a question. Can this shift be predicted at all from the training data? If it could not, the failure of the models would teach little. It can: we share the predictions obtained by a semi-analytical model that was derived from a problem statement that contains only what the paper discloses, with no simulation results and no information on how many defects a shell has or how far apart they lie. The task asked for a theory that includes the interaction of nearby defects, and the model is calibrated with three coefficients on the same sparse training κ as the ML models. It predicts κ on the 2,200 dense test shells to 1.41%, against 4.72% for the best learned model (App. O, Table 46). It is less accurate than the best learned models on B1 and B2 (1.44% vs 0.53% and 2.74% vs 1.08%) and three times more accurate across the sparse-to-dense shift. The shell model represents the interaction explicitly, while the learned surrogates miss its effect under the shift.

On 100 paired layouts with the same defect count, depths and footprint, five B3 families err by 0.6–1.6% with defects 25° apart and by 6.8–8.0% with them 10° apart. The solver κ changes by a median of 7.7% between the layouts, and the predictions by 0.8–2.0% (App. N.9). Models trained with dense shells miss it too. Moving one pair from 25° to 10° lowers the solver κ by a median of 14%, and the B1 models predict −1.1 to +0.3% (Sec. 5.2).

Models that handle other shifts but fail here; lesson for developers (Q1). Yes. Three of our baselines report robustness to other shifts in their own papers, Transolver the best accuracy on AirfRANS, which tests unseen flow conditions, SFNO stable rollouts over a simulated year, and MeshGraphNet generalisation to larger meshes. On B3, every geometry model except DeepONet is less accurate than the training-mean κ (6.80%, 9.37% and 5.99% for these three, against 5.11%, Table 2). For ML benchmarking, in-distribution accuracy alone cannot select a surrogate when an interaction between inputs governs the target. We recommend that developers report the rank correlation between in-distribution and shifted accuracy, and probe the interaction that decides the output with paired solver counterfactuals.

Audience; a suite would be more informative (W1). We agree that a suite gives broader conclusions, and BLISS is one controlled component of one. Physical experiments validate the simulations, with collapse pressures within 3.3% (Sec. 3). Single-system benchmarks are established in ML venues (AirfRANS and DrivAerNet++, NeurIPS 2022 and 2024). What transfers is the protocol, a shift defined by a change of interaction regime with the equations, material and load fixed, and paired counterfactuals that isolate it. We added axially compressed cylinders (App. P). There, neighbouring defects lower κ instead of raising it, sparse-trained models fail on dense shells (κ error 1.3–1.6% sparse vs 6.5–17.4% dense, and 3.8–11.6% with the dense κ inside the training range, Table 50), and the rank correlations repeat the pattern (B1–B2 +0.60, B1–B3 −0.90, Table 52).

Compute (Q2). The Scale paragraph covers the released dataset and benchmark, with 21,600 core-hours for the simulations (8 cores and about 6 GB per solve), 2,130 GPU-hours for the field models, 2,900 CPU core-hours for the tabular models and 2,400 for verification (Table 3). A solve takes 18 min on sparse and dense shells, and 44 min with four times the elements. Inference takes 0.3–26 ms, 4×10⁴ to 4×10⁶ times faster than a solve (Table 4). The cylinders added 3,537 core-hours (Table 53).

BLISS therefore tests whether a surrogate captures a physical interaction, beyond in-distribution accuracy.

Reviewer_HuBqOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This is a quite interesting paper where the authors constructed a dataset specifically for the spherical thin shells with many small inward dents. Each shell is loaded by external pressure until it first collapses. The main idea is to evaluate whether ML models can capture the interaction between nearby defects. When defects are far apart, collapse usually starts at the deepest defect. However, when defects are sufficiently close, they can interact and change the buckling behavior. The authors design three evaluation splits, including a sparse-to-dense generalization test, and several controlled interventions. The results shows that while the existing models achieve good accuracy under a random split, their performance drops significantly when transferring from sparse to dense shells. In particular, all geometry-based models perform worse than the training mean on the displacement field. More interestingly, the authors shows that only moving one pair of the defects closer leads to most of the models failing to predict.

Critical strengths

I appreciate the benchmark design in the paper. Instead of introducing a distribution shift by simply changing the range of input parameters, the authors focus on a specific physical mechanism: the interaction between nearby defects. This makes the benchmark more interesting than a standard out-of-distribution evaluation. I particularly like the observation that models can learn the shortcut that buckling starts at the deepest defect. This works well for sparse shells but becomes unreliable when defects interact. The authors also perform a controlled experiment by moving one pair of defects closer while keeping the other defects unchanged. I find this experiment quite convincing, since it shows that even models trained with dense shells can fail to capture the interaction between nearby defects.

The other thing I like is that the authors provide mesh convergence test, which is always lack in the simulation dataset. Also the paper also includes an analysis of near-tie failure sites. Within that analysis, the authors do not simply treat every simulated failure site as an exact ground truth, especially when two possible sites have similar displacement magnitudes. This is important for a benchmark involving buckling instabilities.

Critical weaknesses

One part which bothers me a little is that all simulations use the same S4R shell element and arc-length continuation setup. I would like to see whether the results remain consistent with another shell element formulation or a smaller arc-length step size. The displacement field changes noticeably between consecutive solver states near the collapse point, so I think a more direct convergence analysis of the peak displacement field would be useful.

Besides, the dataset uses a single neo-Hookean material model, together with fixed shell thickness, slenderness, defect width, and boundary conditions. These assumptions are reasonable for constructing a controlled benchmark, but it is unclear how sensitive the results are to them. For example, would a different hyperelastic material law produce similar displacement fields and failure sites? I would also like to know the typical strain levels at the first limit point to better understand the importance of the constitutive model.

Finally, I have some concern of the generalization of the proposed benchmark. The whole dataset considers only one family of hemispherical shells with inward Gaussian defects, fixed physical parameters, and a clamped boundary. The interaction distance between defects is related to the buckling wavelength, which depends on the shell geometry and thickness. Therefore, the specific sparse and dense regimes in this benchmark may not directly apply to other shell configurations.

Overall, I think BLISS provides a useful dataset and a well-motivated benchmark for scientific machine learning.

Questions for authors
  1. Could the authors extend the mesh convergence study to more dense shells, particularly those with near-tie failure sites? I would also like to know whether the displacement field and failure site remain consistent under a different shell element formulation or smaller arc-length steps.

  2. Does authors consider other simulators with different finite-element models, such as triangle meshes? I think different simulators with different discretization will generalize a more reliable data distribution.

  3. Have the authors considered varying the shell slenderness or expressing the defect spacing relative to the buckling wavelength? I think this would help establish whether the observed generalization failure is specific to the current shell configuration or a more general phenomenon.

1 comment
AuthorsOpenReview ↗

Response to Reviewer HuBq

We thank the reviewer for the positive assessment and for questions that made the labels more trustworthy. We re-solved 80 shells (40 sparse and 40 dense, half of them near-ties) under each change of element, arc-length step and material, and re-meshed 195 test shells (App. N). Here κ is the knockdown factor (the normalised collapse load), and a near-tie is a shell whose second site deflects more than 90% as much as the failure site. The changes in the table move the median κ by at most 1.23%, against errors of 4.72–9.37% for the six geometry models on the dense test, and keep every clearly defined failure site. The conclusions on the models hold as well. Scored against the labels of each setting in Table 37, the model ranking keeps a Spearman correlation of 0.93–1.00 with the released one (0.86 for S4R5). This is by design. We chose a geometry-dominated problem, which makes it a clean test for surrogate modelling with ML.

change (80 shells)median |Δκ|field R²near-tie site kept
arc length ×1/2, ×1/40.06%, 0.09%0.998, 0.99978%, 70%
S3R triangles0.40%1.00075%
S4 full integration*0.11%1.00085%
S4R5 Kirchhoff*1.23%1.00075%
SC8R continuum, 3 layers0.61%0.99765%
Mooney-Rivlin0.09%1.00092%
linear elastic0.39%1.00082%
finer mesh (195 shells)0.28%0.999780%

*Abaqus does not allow hyperelastic laws with S4 and S4R5, so these two rows compare linear elastic runs.

Element formulation, arc-length step, mesh, peak field (W1, Q1). See Table 34 and App. N.1–N.2. The peak field converges when the maximum arc-length increment is halved and quartered (R² 0.998 and 0.999). The finer mesh (four times the elements) changes κ by 0.30% on sparse and 0.25% on dense shells and keeps the site on all 39 clear shells and 80% of the 50 near-ties. Two further settings move κ more. A compressible material with ν\nu = 0.45 lowers it by 7.2% on sparse and dense shells alike. The classical buckling pressure falls by 6.3% at this ν\nu, so about 1% remains. SC8R with one element through the thickness raises κ by 5.0% (0.61% with three elements). One label does depend on the solver. On the 2,200 dense test shells, the solver states one increment before and after the peak keep the normal deflection to R² 0.984–0.989 (Table 42) and the site on only 59–61% of the 428 near-ties (Table 43). The near-tie site still converges with the step, and the half and quarter arc lengths agree on 90% of near-ties (Table 35). The benchmark therefore scores the failure site only on clear shells and reports near-tie accuracy against this solver floor.

Other discretisations (Q2). Yes. Triangles (S3R) change κ by 0.40%, fully integrated S4 by 0.11% and the continuum shell SC8R by 0.61%, and all keep every clear failure site. All runs use one solver, whose κ matches eight physical tests within 3.3% (Sec. 3).

Material law, strain levels, defect width (W2). The Mooney-Rivlin hyperelastic material and a linear elastic one change κ by 0.09% and 0.39%, with the field unchanged (R² 1.000). At the limit point the maximum principal strain is 0.89% in median (largest 1.22%) and the minimum −1.19% (Table 36), so geometric nonlinearity governs the collapse. At 0.75, 2 and 3 times the released defect width, the dense-to-sparse difference in κ persists and changes sign for the two wider defects, which overlap (Table 40).

Slenderness and buckling wavelength (W3, Q3). In buckling wavelengths lcl_c (19.8° of arc at R/t = 110), the minimum spacing is 0.51 lcl_c on dense and 1.26 lcl_c on sparse shells. Re-solved at R/t = 200 and 300 with positions scaled by lcl_c, 100 sparse and 100 dense test shells keep the higher κ of dense shells (dense-to-sparse ratio 1.050, 1.064 and 1.067 at R/t = 110, 200 and 300, Table 39). The regimes therefore scale with lcl_c. We agree that the benchmark itself is at one slenderness and one boundary condition, and the abstract and Sec. 6 now say so. The clamped rim follows Lee et al. (2016), and the defects lie within 60° of the pole, where the rim leaves κ unchanged (Derveni et al., 2023).

Other shells (W3). We added axially compressed cylinders, with another load and other boundary conditions, validated against Ubamanyu et al. (2026) (mean κ 0.638 vs 0.639). Neighbours lower κ on cylinders (−15.1%) instead of raising it (+6.4% on hemispheres), and sparse-trained models fail on dense cylinders in the same way (App. P).

cylinders, median κ error5 familiestraining mean
sparse test (100)1.3–1.6%5.9%
dense test (200)6.5–17.4%20.2%
dense, κ inside the training range (107)3.8–11.6%11.7%
dense, trained with dense shells (100, Table 52)2.3–3.2%14.9%

Together, these results show that the labels and the ranking of the models hold across the formulations, meshes and materials of the first table.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers and the AC

We thank the three reviewers for careful and constructive reviews. Reviewers HuBq and gad3 found the benchmark design well motivated, and all three rightly asked how far the results generalise beyond one shell configuration and what the ML community learns from them.

Fit to ICLR (vJGL). Reviewer vJGL asked whether this work belongs in this community. Our entire motivation to conduct this work was to expose limitations in state-of-the-art machine learning models that have been permeating physics and engineering domains of application. The field models BLISS evaluates were developed in the ML community (MeshGraphNet at ICLR, SFNO and Transolver at ICML), and every result in the paper concerns how they learn. The main finding concerns model selection under distribution shift, a question the community studies (Miller et al., 2021; Teney et al., 2023). An in-distribution leaderboard predicts the order of these models under a shift to deeper defects, but not under a shift of the interaction regime (item 4), even though shell theory calibrated on the same training data predicts the shifted response (item 5). This indicates that the limitation lies in the learned models rather than in the data. We believe this is under-appreciated and can motivate surrogates with physical inductive biases, and the protocol that exposes it (a shift with the physics fixed, and paired solver counterfactuals) transfers to other systems.

We have uploaded a revised PDF (changes in blue). We also edited the main text for clarity and moved some details to the appendices to stay within the page limit. The main additions are:

  1. Label robustness (HuBq, gad3). We verified that the finite element labels are robust by re-solving 80 shells with smaller arc-length steps, four other shell elements (including triangles and a Kirchhoff element) and two other material laws, plus a finer mesh on 195 test shells. The median knockdown factor κ (the normalised collapse load) moves by at most 1.23%, against a 4.72% error for the best learned model on the dense test. The median field R² stays at 0.996 or above and every clearly defined failure site is kept (Table 34, App. N.1–N.2). Two further settings move κ more and are reported. A compressible material (ν\nu = 0.45) lowers κ by 7.2% on sparse and dense shells alike, and the classical buckling pressure itself falls by 6.3% at this ν\nu, so about 1% remains. A continuum shell with one element through the thickness raises κ by 5.0%, and with three elements the change falls to 0.61%.
  2. Separating the causes of the B3 accuracy drop (gad3). B3 trains on sparse shells (defects at least 25° apart) and tests on dense shells (defects down to 10° apart). Controlled tests show that the local spacing between defects, more than their number or where the shells fail, is the dominant cause of the drop (App. N.9). On 100 paired layouts with the same defect count, depths and footprint, five families err on κ by 0.6–1.6% with the defects 25° apart and by 6.8–8.0% with the same defects 10° apart (Table 45).
  3. Scope (HuBq, gad3, vJGL). Re-solves at R/t = 200 and 300 with spacing scaled by the buckling wavelength keep the higher κ of dense shells (App. N.6). A second structural system (axially compressed cylinders, App. P) shows the same loss of accuracy, although neighbouring defects lower κ there instead of raising it. Five families trained on sparse cylinders err on κ by 1.3–1.6% on sparse and by 6.5–17.4% on dense test cylinders (three seeds, Table 49), and the cylinder data will be released. The abstract and Sec. 6 now state the scope (clamped hemispheres at one slenderness, R/t = 110).
  4. The lesson for ML (vJGL). Over eight model families, the in-distribution ranking on κ predicts the ranking under deeper defects (Spearman 0.83, p = 0.015) and is unrelated to the ranking under the sparse-to-dense shift (+0.02, p = 0.98, Sec. 5.1). An in-distribution leaderboard would have selected SFNO (0.58%), the least accurate of the six geometry models on the dense test (9.37%, App. N.9). On cylinders the two correlations are +0.60 and −0.90 over five families (Table 52).
  5. Separating mechanics from ML (vJGL). A semi-analytical shell model, derived with an AI agent (Claude Opus) from a problem statement that gives neither the number of defects of a shell nor their spacing, and calibrated with three coefficients on the same sparse training shells as the ML models, predicts κ on the 2,200 dense test shells to 1.41%, against 4.72% for the best learned model (Table 46). The task, reproduced with the statement in App. O, asked for a theory that includes the interaction of nearby defects. The shift is therefore predictable with shell theory and the same training shells. Published analytical models treat defects that do not interact (Baizhikova et al., 2026), and we found no published model of this form. We report the model as a reference target with released per-shell predictions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.