acceptodds
Under review as a conference paper at ICLR 2027

Reliability-Aware Constitutive Learning for Materials Discovery

Abstract

Accurate characterization of material behavior at elevated temperatures is essential for engineering design but remains costly and time-consuming due to the extensive experimental testing required across multiple thermo-mechanical conditions. Recent advances in high-throughput mechanical testing workflows coupled with machine learning have enabled rapid prediction of stress-strain behavior from limited experimental measurements. However, existing approaches often lack interpretability and reliability assessment, provide limited support for out-of-distribution generalization to unseen alloy families, and offer no guarantees of physically meaningful stress-strain trajectories. We present a physics-aware constitutive learning framework that combines trajectory-aware training, a shape-aware loss, and a per-temperature mean stress loss to improve trajectory fidelity while preserving temperature-dependent material response. Beyond prediction accuracy, we introduce a reliability framework that quantifies neighborhood support within a low-dimensional constitutive materials space, providing an interpretable measure of extrapolation risk for previously unseen materials. We further propose a coverage-disagreement framework that identifies under-sampled thermo-mechanical regions and provides quantitative guidance for targeted experimental data acquisition. Across leave-one-material-out evaluations and previously unseen alloy families, the proposed framework produces physically consistent stress-strain curves while achieving average prediction errors below 5% on previously unseen materials. Together, these contributions transform constitutive machine learning from a prediction-focused task into a reliability-aware decision-support framework capable of assessing extrapolation risk and guiding future experimentation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Physics-Aware Machine Learning, Material Characterization, Elevated Temperature Mechanical Testing, Out-of-Distribution Generalization, Reliability Estimation, Data Coverage Analysis, Uncertainty Quantification

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_KySPOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper claims to present a "physics aware constitutive learning framework" for predicting material behaviors at elevated temperature. The application is gathered from a dataset published in 2025 (Agha et al. 2025), which also provides the machine learning framework.

As the basic learning technique already was proposed in past work---and is fairly rudimentary by machine learning research standard---the contribution here is essentially the "physics informed" loss terms and some heuristics for measuring reliability.

Critical strengths

The paper's target application is a worthy one. The idea of tailoring a learning methodology specifically to material property prediction is an interesting one broadly speaking, and I could imagine expanding on the overall thrust of this work to incorporate more advanced or specialized architectural designs/losses.

Critical weaknesses

This paper is very clearly below the standard of ICLR and does not bring substantial progress to research in machine learning. The application already appeared in past work as did the dataset and the training procedure, so what's left are mostly the new loss terms and the reliability measures.

The new loss terms are hardly "physics informed" (at least in the typical sense from the "physics informed neural network" literature), in that there are no penalties for deviating from physical principles/laws; they don't seem any more physics informed than the original loss (also built from the dataset) unless I am missing something fundamental. Rather these loss terms are closer to consistency measures within the dataset, e.g. penalizing differences in neighboring ROIs in time sequences from the dataset. The losses are reasonable but simplistic, and they are not carefully ablated (e.g., the lambda values in eq4).

The reliability framework is heuristic and mostly composed of simplistic heuristics built on standard statistical computations.

There are "hard coded" constants in the paper that should be tested and discussed, e.g. coefficients of loss terms and assorted constants in (9).

There are also substantial writing/clarity issues:

  • Much of the notation in the paper is not carefully defined, and perhaps requires knowledge of past work. For example, I think the sigmas in losses (1)-(4) are stress values.
  • The MAPS acronym is never spelled out
Questions for authors

Are the improvements noted in the first paragraph of sec 4.1 significant enough to make a difference in applications?

The conclusion claims that the paper answers which materials to add to the training dataset next. Is this claim tested empirically, e.g. by implementing an 'active learning' procedure wherein new data is introduced and the model is refined?

2 comments
AuthorsOpenReview ↗

Response to Reviewer KySP (1/2): Novelty, Physics-Aware Learning, and Reliability Assessment

Response to Reviewer KySP (1/2)

We sincerely thank the reviewer for the thoughtful evaluation and constructive feedback. In response to the concerns regarding novelty, physics-aware learning objectives, reliability assessment, parameter selection, and presentation clarity, we substantially revised the manuscript and added new ablation, sensitivity, and validation studies.

1. Novelty and Contribution Beyond the Previous MAPS Benchmark

We agree that the MAPS dataset and baseline MLP architecture were introduced in Agha et al. (2025). Our contribution is not a new dataset or neural-network architecture, and we have clarified this distinction in Sections 2, 3.1, and 3.2.

Rather than introducing a new benchmark, this work extends the MAPS framework in three complementary directions:

(i) Physics-aware constitutive learning to improve curve fidelity;

(ii) Materials-space reliability assessment to identify limited-support materials and extrapolation risk;

(iii) Coverage-disagreement-driven experimental planning to identify thermo-mechanical regions requiring additional characterization.

Our contribution lies in integrating these capabilities into a reliability-aware constitutive-characterization workflow. Figure 1 and the Introduction have been revised to clarify this distinction from the original MAPS benchmark.

2. Physical Interpretation and Justification of the Learning Objectives

We agree that the proposed objectives differ from the physics-informed losses commonly used in PINNs.

Our approach is physics-aware rather than a conventional PINN formulation. We do not enforce governing equations or physical constraints as hard penalties. Instead, the proposed objectives incorporate structure associated with constitutive response trajectories.

The shape-aware loss (Eq. 2) compares predicted and reference stress increments between consecutive deformation steps, encouraging agreement in local stress evolution rather than only absolute stress values. The per-temperature mean stress loss (Eq. 3) reduces temperature-dependent prediction bias by constraining the average stress response within temperature groups.

To evaluate the contribution of each component, we added a progressive ablation study (Appendix D, Tables 5--6). Curve-wise batching alone increases MAPE from 6.43% to 9.28%, demonstrating that improvements cannot be attributed solely to batching. Adding the shape-aware objective reduces MAPE to 6.91%, while the complete framework achieves 6.80%.

The complete model also reduces Shape RMSE from 1.36 to 0.97 and Monotonicity Violation from 0.20 to 0.12. An additional monotonicity penalty increased prediction error and did not improve the overall trade-off.

These results provide quantitative evidence for the contribution of the learning objectives.

3. Reliability Framework and Its Practical Significance

We agree that the reliability framework is built from established statistical concepts. The contribution is the combination of complementary indicators into an interpretable assessment of material-space support and extrapolation risk.

To evaluate whether the indicators provide complementary information, we added a reliability-framework ablation study (Appendix B.5, Table 3). Among the three selected high-risk LOMO cases, distance alone, distance plus directional similarity, and the complete framework identify 1/3, 2/3, and 3/3 challenging materials, respectively.

We also expanded Section 4.2 and Table 1. Incorporating four CAUTION materials into the training dataset reduced mean MAPE by an average of 50.3% after retraining. While these are not independent generalization evaluations, they demonstrate that the framework successfully identifies materials that benefit most from additional characterization and inclusion in the training set.

4. Selection and Sensitivity of Hyperparameters and Reliability Thresholds

We expanded Appendix A.5 to document hyperparameter selection. The loss weights were selected through a two-stage search using 12-material LOMO cross-validation with 10 random seeds per fold (120 training runs per configuration). External-validation alloys were excluded from tuning.

We also added a threshold-sensitivity study (Appendix B.6, Table 4). AA5052 and AA3003-H14 remain classified as CAUTION across all configurations, while five materials remain consistently RELIABLE. Materials near classification boundaries show the expected sensitivity, and we clarify that the thresholds are calibrated to the current dataset rather than representing universal physical constants.

5. Improvements in Notation and Presentation

We revised Section 3.1 to define MAPS (Model-Assisted Property Specimen) and clarify its relationship to constitutive-response prediction. Sections 3.3 and Appendix A.4 were expanded to define the stress variables, Shape RMSE, and Monotonicity Violation used throughout the manuscript.

AuthorsOpenReview ↗

Response to Reviewer KySP (2/2): Practical Significance and Experimental Validation

Response to Reviewer KySP (2/2)

6. Questions for Authors

Q1. Are the improvements reported in Section 4.1 significant enough to matter in applications?

We appreciate this question, as the distinction between pointwise prediction accuracy and constitutive fidelity is central to our motivation.

Relative to the MSE-only baseline, the complete model reduces Shape RMSE from 1.36 to 0.97 (28.7%), Monotonicity Violation from 0.20 to 0.12 (40%), and seed-to-seed MAPE variability from 1.18% to 0.84%, while mean MAPE changes only modestly from 6.43% to 6.80%.

We therefore view the primary benefit as improved constitutive-response fidelity rather than improved pointwise prediction accuracy. Because constitutive models are used as complete stress-strain trajectories, agreement in individual stress values does not necessarily imply faithful reproduction of the underlying constitutive response. As shown in Section 4.1 and Figure 3, the physics-aware model more closely follows the reference trajectories and reduces non-monotonic artifacts.

We do not claim demonstrated improvements in downstream engineering simulations or structural design outcomes. Application-level validation remains important future work.


Q2. Is the claim about identifying materials for dataset expansion tested empirically through an active-learning procedure?

We agree that empirical validation of the material-selection capability is important.

To address this concern, we expanded Section 4.2 and Table 1. Four materials identified as CAUTION (AA7075-T6, AA2024, AA5052, and AA3003-H14) were incorporated into the training dataset and the model was retrained. Their mean MAPE reductions ranged from 37.5% to 57.3%, with an average reduction of 50.3%.

We emphasize that this is not a complete active-learning benchmark and does not establish superiority relative to alternative acquisition strategies. Rather, it demonstrates the effect of incorporating materials identified as having limited support within the existing training distribution.

We also added a controlled characterization study in Section 4.3 and Figure 4. For AA3003-H14, framework-guided characterization increased coverage in the targeted 300--350°C region from 2 to 4 observations and reduced disagreement from approximately 13% to 9%, whereas a control test outside the target region produced little change.

Together, these studies provide preliminary evidence supporting the practical utility of the framework for both dataset expansion and experimental planning. A full sequential active-learning benchmark remains an important future extension.


Closing Remarks

We thank the reviewer again for the constructive feedback. The revised manuscript strengthens the proposed framework through component-level ablations, threshold-sensitivity analyses, controlled characterization studies, and expanded methodological explanations.

Together, these additions provide stronger empirical support for the physics-aware learning objectives, the materials-space reliability assessment, and the coverage-disagreement framework, while clarifying their intended scope and limitations. We hope the revised results and analyses address the reviewer’s principal concerns and more clearly demonstrate the practical value of the proposed approach.

Reviewer_y47sOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper characterizes materials using a 63-dimensional material descriptor and applies PCA to extract their principal features. It assesses the reliability of the model’s predictions of material constitutive behavior by comparing the target material with the training materials in the projected space using three metrics—neighborhood distance, local PC1 variance, and directional similarity—or by analyzing coverage and disagreement. These assessments subsequently guide further model training.

Critical strengths

The paper presents a straightforward and accessible approach, supported by thorough hyperparameter analysis and accompanying experiments. It addresses a meaningful research problem, and its core ideas have the potential for broader application beyond the study of material deformation.

Critical weaknesses
  1. The paper does not sufficiently substantiate the practical significance of the proposed framework. Although it states that the framework can guide additional experiments and data acquisition, it does not provide direct experimental results quantifying the resulting benefits to subsequent model training. Figure 4 and Table 1 provide indirect evidence, but the connection between these findings and improvements in subsequent model training remains insufficiently demonstrated.
  2. The framework involves numerous hyperparameters and design choices. While some are analyzed, others lack sufficient justification. Examples include the thresholds used to classify materials as RELIABLE in Section 3.4.4, the choice of a 63-dimensional material descriptor, and the subsequent retention of five principal components.
  3. The paper does not adequately explain how certain quantities are computed and used. For example, Section 3.4.3 provides insufficient detail regarding the construction and application of the direction similarity metric SS in material-level reliability assessment.
Questions for authors
  1. Regarding the experiment in Figure 4, could the authors provide model predictions for different materials before and after additional testing, rather than presenting the results solely through the proposed coverage–disagreement quadrant plots? Could these results also be compared with those obtained by adding uniformly distributed tests without guidance from the proposed framework?
  2. How is the directional similarity described in Section 3.4.3 calculated for a target material? Is it the similarity to its nearest neighboring material, or an average of its similarities to several neighboring materials? Please clarify the procedure.
  3. Models differ in their generalization capabilities. Since the experiments use a single predictive model architecture, could the authors provide additional experiments evaluating the proposed framework with models exhibiting different levels of generalization performance? This would help establish whether the framework’s effectiveness extends across different predictive models.
4 comments
AuthorsOpenReview ↗

Response to Reviewer y47s (1/2)

We sincerely thank the reviewer for the thoughtful evaluation and constructive suggestions. We appreciate the recognition that our framework addresses a meaningful research problem, provides an accessible approach, and has potential applications beyond material deformation.

1. Critical Weaknesses

1.1 Practical significance and benefits for subsequent model training

We agree that the original manuscript did not directly quantify the practical benefit of the reliability framework.

To address this concern, we expanded Section 4.2 and Table 1. The LOMO study identified four materials classified as CAUTION (AA7075-T6, AA2024, AA5052, and AA3003-H14). These materials exhibited elevated prediction error and/or increased run-to-run variability.

We subsequently incorporated each CAUTION material into the training dataset and retrained the model. Mean MAPE was reduced by:

  • AA7075-T6: 57.3%
  • AA2024: 37.5%
  • AA5052: 50.0%
  • AA3003-H14: 56.4%

corresponding to an average reduction of 50.3%.

These results demonstrate that the reliability framework successfully identifies materials for which additional training data provide the greatest modeling benefit. We note that these post-inclusion evaluations are no longer LOMO tests and therefore demonstrate improved fitting after dataset expansion rather than independent generalization.

We also expanded Section 4.3 and Figure 4 to directly validate the coverage-disagreement framework. For AA3003-H14, the framework identifies the 300--350$^\circ$C region as the highest-priority characterization target. A guided 375$^\circ$C test increases coverage from 2 to 4 observations and reduces disagreement from approximately 13% to 9%, whereas a control 150$^\circ$C test produces little change in the target region.

Together, these additions provide validation for both framework components:

  • Material-space reliability: identifies materials whose inclusion most improves future model performance.
  • Coverage-disagreement: identifies thermo-mechanical conditions that most effectively reduce prediction disagreement during characterization.

1.2 Justification of the 63-dimensional descriptor, PCA dimensionality, and reliability thresholds

63-dimensional descriptor and PCA representation

We expanded Appendix B.1--B.2 to justify both the material descriptor and PCA dimensionality.

Each MAPS test is represented using:

  • 11 force-history features,
  • 5 global force descriptors,
  • 36 strain-history features,
  • 6 ROI temperatures, and
  • 5 global thermal descriptors.

These features provide a compact material-response representation suitable for comparison across materials.

PCA was then applied to reduce redundancy. As reported in Appendix B.2, the first principal component explains approximately 47% of the variance and the second explains approximately 18%. The first two components therefore capture approximately 65% of the variance, while the first five components retain approximately 97% of the total variance.

The five-dimensional representation was therefore selected as a compact but information-rich material-space representation.

Reliability threshold selection and robustness

We added Appendix B.6 and Table 4 to evaluate threshold sensitivity.

Thresholds were calibrated exclusively using the 12 development materials. We evaluated baseline, stricter and relaxed thresholds.

The primary conclusions remain unchanged. AA5052 and AA3003-H14 remain classified as CAUTION across all threshold sets, while five well-supported materials remain RELIABLE. Materials closest to the classification boundaries exhibit the highest sensitivity.

These results show that the principal reliability conclusions remain stable under reasonable threshold perturbations while confirming that the thresholds are dataset-dependent rather than universal constants.

In addition, Appendix B.5 (Table 3) now includes an ablation study comparing:

  • DnormD_{\text{norm}} only,
  • Dnorm+SD_{\text{norm}} + S,
  • Full framework (Dnorm+S+VlocalD_{\text{norm}} + S + V_{\text{local}}).

These variants identify 1/3, 2/3, and 3/3 high-risk materials, respectively, demonstrating that the three indicators contribute complementary information.


1.3 Construction and application of directional similarity

We agree that the original description of directional similarity was not sufficiently detailed.

We expanded Section 3.4.3 and Appendix B.4 to describe its computation.

To answer the reviewer’s specific question: directional similarity is not computed relative to a single nearest neighbor. Instead, it is evaluated relative to all neighboring materials within the local neighborhood used for the DnormD_{\text{norm}} calculation, and the final score is reported as the neighborhood-average similarity.


AuthorsOpenReview ↗

Response to Reviewer y47s (2/2)

2. Questions for Authors

Q1. Can the authors provide model predictions before and after additional testing and compare against uniformly distributed tests?

Yes. The revised manuscript now includes direct validation studies for both framework components.

Material-space reliability framework.
Table 1 and Section 4.2 now report material-level performance before and after incorporating CAUTION-classified materials into the training dataset. These experiments show an average MAPE reduction of 50.3% following inclusion of the previously underrepresented materials.

Coverage-disagreement framework.
Figure 4 now includes constitutive stress-strain predictions before and after additional characterization. A framework-guided 375$^\circ$C test targets the identified 300--350$^\circ$C region, increasing coverage from 2 to 4 observations and reducing disagreement from approximately 13% to 9%. A control 150$^\circ$C test produces little change in the target region.

We also clarify an important distinction between the two framework components:

  • Material-space reliability: Which materials should be added to the training dataset next? This guides dataset expansion and retraining.
  • Coverage-disagreement: Which thermo-mechanical conditions should be characterized next for a material currently being evaluated? This is an inference-time decision-support tool and does not require model retraining.

We agree that evaluating additional sampling policies, including uniformly distributed designs, would be valuable future work. The present revision includes a direct guided-versus-control comparison within this characterization setting.


Q2. Is directional similarity calculated relative to the nearest neighboring material or averaged across several neighbors?

Thank you for highlighting this ambiguity.

Directional similarity is averaged across the local neighborhood rather than computed relative to a single nearest material. We now explicitly describe the neighborhood construction, pairwise cosine-similarity calculation, and neighborhood averaging procedure in Section 3.4.3 and Appendix B.4.

The neighborhood-average formulation reduces sensitivity to a single neighboring material and provides a more representative measure of local similarity within the learned material space.


Q3. Does the framework remain effective across predictive models with different generalization capabilities?

We agree that testing multiple predictive architectures would provide additional evidence of generality.

In the present study, we intentionally retained the previously published MAPS MLP architecture to isolate the effects of the proposed physics-aware objectives and reliability framework without introducing architectural differences as an additional experimental variable.

Although only one architecture is considered, the framework is evaluated across materials with substantially different generalization difficulty, including LOMO experiments and independent validation on six previously unseen alloys (Section 4.4, Figure 5, Table 2). These results demonstrate that the reliability indicators can distinguish materials with differing levels of material-space support.

We acknowledge that architecture-independent validation remains future work and have clarified this limitation in the revised manuscript. Because the reliability indicators operate on material-response representations rather than architecture-specific features, they provide a foundation for future evaluation across alternative predictive models.


Closing Remarks

We thank the reviewer again for the constructive suggestions. In response, we added:

  • Direct validation of the material-space reliability framework through dataset-expansion experiments;
  • Direct validation of the coverage-disagreement framework using guided versus control characterization;
  • Reliability ablations and threshold-sensitivity studies;
  • Expanded justification of the descriptor and PCA representation; and
  • Detailed explanations of directional similarity and neighborhood averaging.

We believe these additions strengthen the experimental evidence, clarify the methodology, and address the reviewer’s concerns regarding practical significance and reproducibility.

Thank you for your response. I prefer to maintain my score since

  1. The new experiments do not resolve my concern about the practical benefit of the framework. Figure 4 and Section 4.3 still rely on the proposed coverage and disagreement metrics. Although panels (d)–(f) add predicted stress–strain curves, they do not show reference curves or report prediction errors before and after the additional test. It is therefore difficult to tell whether the targeted test improves prediction accuracy. The visual comparison is further complicated by the inconsistent colors used for the same 500 °C test curves in panel (e) and panels (d) and (f). In Table 1 and Section 4.2, lower errors after retraining on data from the CAUTION materials are an expected outcome. However, the authors do not report a comparable experiment in which data from RELIABLE materials are added under the same data budget and evaluated using the same error metrics. Without this comparison, it remains unclear whether the framework is useful for prioritizing CAUTION materials for data collection. 2. The sensitivity analysis does not explain how the baseline thresholds were selected. The stricter and more relaxed settings show how the categorical labels change when all three thresholds are varied together. However, these comparisons do not explain how the underlying values of the three metrics informed the choice of the baseline thresholds. The paper also does not specify how the neighborhood is constructed. This choice can directly affect the normalized neighborhood distance and directional similarity, and may also affect the local PC1 variance if it is computed using neighboring materials. The authors should specify the neighborhood selection rule and report how sensitive the reliability labels are to that choice. 3. The framework remains untested across different predictive architectures. The revision compares loss functions and other training configurations within the same MLP backbone. It does not establish whether the reliability indicators can still identify difficult-to-predict materials when applied to models with different generalization behavior. Reporting material-level prediction errors for an alternative architecture under the same leave-one-material-out splits would help address this concern.

Thank you for the active engagement in the rebuttal process.

1. The coverage-disagreement framework is intended as an inference-time characterization-planning tool rather than an active-learning or retraining strategy, and therefore its objective is not to improve prediction accuracy, but rather the precision and consistency of constitutive predictions. Adding a new MAPS test does not modify the trained model or alter the predicted responses of previously characterized samples. Instead, the additional test increases coverage and reduces disagreement among constitutive predictions, thereby improving the repeatability and confidence of the characterization dataset. In the context of materials testing and standardization, precision is a critical metric because it reflects repeatability and reproducibility, whereas accuracy can often be improved through calibration, offsets, or subsequent model updates. For this reason, Figure 4 and Section 4.3 intentionally evaluate changes in coverage and prediction disagreement rather than prediction error. We have clarified this distinction in the revised manuscript and revised Figure 4 to use a consistent color scheme across panels (d)-(f).

Reliability Framework: We thank the reviewer for this suggestion and have performed the requested comparison. In addition to the four CAUTION materials, we evaluated all eight RELIABLE materials using the same retraining procedure and error metrics. As expected, inclusion of material-specific data reduced prediction error for both groups, with average MAPE reductions of 48.9% and 50.3% for RELIABLE and CAUTION materials, respectively.

A more meaningful distinction emerged when examining prediction variability. The average seed-to-seed MAPE standard deviation for RELIABLE materials decreased from 0.71% to 0.54% (19% reduction), whereas the corresponding reduction for CAUTION materials was from 2.12% to 1.10% (42% reduction). While exceptions exist (e.g., AA6063-T6, which exhibited a modest increase in seed-to-seed variability after inclusion), materials identified as CAUTION not only exhibited substantially higher initial uncertainty but also experienced significantly larger reductions in variability after inclusion in the training dataset.

From a materials characterization perspective, this reduction in variability is particularly important because it reflects increased repeatability and confidence in constitutive predictions. These results suggest that the proposed framework is most effective at identifying materials for which additional data are most likely to improve model stability and predictive consistency.


2. We have clarified the neighborhood construction and threshold selection procedure in the manuscript. Neighboring materials are defined as materials whose normalized distance satisfies (Dij<6D_{ij}<6). This neighborhood definition is used in the calculation of (DnormD_{norm}) and the directional similarity metric S and is now stated explicitly in Sections 3.4.1 and 3.4.3. We also clarified that (VlocalV_{local}) is computed exclusively from the constitutive tests belonging to the material under consideration and therefore does not depend on the neighboring-material definition.

The baseline thresholds were not intended as universal constants, but rather as dataset-specific criteria designed to identify materials exhibiting adequate neighborhood coverage, local consistency, and directional agreement. Likewise, the neighborhood criterion (Dij<6D_{ij}<6) was selected based on the structure of the current dataset and would need to be re-evaluated for datasets with different coverage characteristics.

As shown in the sensitivity analysis, the primary conclusions remain unchanged under both stricter and more relaxed threshold settings, indicating that the framework is not overly sensitive to reasonable parameter variations.


3. We agree that evaluation across multiple predictive architectures would provide an additional test of generality. In the present study, however, we intentionally retained the previously published MAPS MLP architecture to isolate the effects of the proposed reliability framework without introducing architectural differences as an additional experimental variable.

The reliability indicators are computed directly from the constitutive-response dataset and material-space representations rather than from architecture-specific model features. Consequently, the framework characterizes material-space support and extrapolation risk rather than the behavior of a specific neural-network architecture.

We agree that evaluation across alternative architectures represents an important direction for future work and have clarified this limitation in the manuscript. Establishing architecture-independent behavior would require a dedicated comparative study spanning multiple model classes and training paradigms and is beyond the scope of the present manuscript.

Reviewer_u4ziOpenReview ↗
Rating
1
Confidence
1
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents a reliability-aware framework for recovering local stress–strain responses from MAPS force, strain, and temperature measurements. It combines physics-inspired training objectives to improve curve fidelity, PCA-based neighborhood analysis to assess support for unseen materials, and coverage–disagreement analysis to guide additional testing. Evaluations use finite-element simulations of 18 materials across four alloy families, including leave-one-material-out and external validation. The proposed losses improve curve shape and training stability with a slight increase in average prediction error. The reliability indicators provide information about extrapolation risk, while a targeted sampling case illustrates their potential for experimental planning.

Critical strengths

The experiments demonstrate improved constitutive curve fidelity and fewer monotonicity violations at a modest cost in pointwise accuracy, highlighting the value of evaluating physical response quality beyond average prediction error.

Critical weaknesses
  1. Limited experimental validation and missing details. The experiments do not sufficiently establish the usefulness of the reliability assessment and sampling strategy. The reliability rules lack comparisons with simpler alternatives and sensitivity analysis of the selected thresholds. The additional-testing experiment considers only one material and one added temperature, without showing whether the strategy improves prediction accuracy more effectively than random or uniform sampling. Moreover, differences in batching complicate attribution of the improvements to the proposed losses, while the definition of Shape RMSE and the threshold-selection procedure remain insufficiently specified.
  2. Presentation needs substantial improvement. The Introduction provides limited explanation of the MAPS inverse problem and the rationale connecting the three proposed components. Figure 1 remains largely conceptual, while Figure 4’s small labels and brief caption make the experimental comparison difficult to follow. The main text would benefit from clearer explanations of the reliability metrics and a more balanced discussion of the results, including the trade-off between curve fidelity and prediction accuracy currently reported in the appendix.
Questions for authors

See Weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer u4zi (1/2): Experimental Validation and Missing Details

Response to Reviewer u4zi (1/2)

We sincerely thank the reviewer for the careful evaluation and constructive feedback. We appreciate the recognition that our framework improves constitutive curve fidelity and reduces monotonicity violations, highlighting the importance of evaluating physical response quality beyond pointwise prediction accuracy.

We appreciate these suggestions and have substantially revised the manuscript to address the concerns raised, including new ablation studies, threshold-sensitivity analysis, a controlled comparison of experimental selection strategies, and expanded methodological explanations.

1. Limited Experimental Validation and Missing Details

1.1. Reliability assessment: Comparison with simpler alternatives

We agree that evaluating the contribution of individual reliability indicators is important.

To address this, we added a reliability-framework ablation study (Appendix B.5, Table 3) comparing:

  1. Neighborhood distance alone (DnormD_{\mathrm{norm}})
  2. Neighborhood distance and directional similarity (Dnorm+SD_{\mathrm{norm}} + S)
  3. The complete framework (Dnorm+S+VlocalD_{\mathrm{norm}} + S + V_{\mathrm{local}})

For the three identified high-risk materials, the respective configurations identify 1/3, 2/3, and 3/3 challenging cases. The indicators capture complementary characteristics of material-space support: neighborhood distance identifies extrapolation risk, directional similarity captures differences in dominant constitutive-response variation, and local PC1 variance identifies internal test-to-test inconsistency.

We further expanded Section 4.2 and Table 1 by incorporating the four CAUTION materials into the training set and retraining the model. Mean MAPE decreased by an average of 50.3%, providing direct evidence that the reliability framework identifies materials that benefit most from additional training data.

1.2. Reliability threshold selection and sensitivity analysis

We added Appendix B.6 (Table 4) describing the threshold-calibration procedure and evaluating stricter and more relaxed threshold configurations.

Thresholds were calibrated exclusively using leave-one-material-out evaluation on the 12 development materials; the six external-validation alloys were not used.

The two most challenging materials (AA5052 and AA3003-H14) remain classified as CAUTION across all configurations, while five consistently well-supported materials remain RELIABLE. These results indicate that the primary conclusions are robust to reasonable threshold variations.

1.3. Experimental planning: Framework-guided versus random selection

We substantially revised Section 4.3 and Figure 4 to include a controlled framework-guided versus control selection study.

For AA3003-H14, the framework identifies the 300–350°C region as the highest-priority characterization target. A guided 375°C test increases coverage from 2 to 4 observations and reduces disagreement from approximately 13% to 9%, whereas a control 150°C test produces minimal change.

We acknowledge that this is a single-material case study rather than a statistical comparison across multiple materials. We therefore present this result as direct validation of the proposed decision-support framework, while broader comparisons across multiple materials remain future work.

1.4. Separating batching effects from the proposed learning objectives

We added a progressive ablation study in Appendix D (Tables 5 and 6) comparing conventional row-wise batching, curve-wise batching, shape-aware learning, and the complete physics-aware framework.

Curve-wise batching alone increases mean MAPE from 6.43% to 9.28%, indicating that preserving trajectory structure does not independently improve prediction accuracy. Adding the shape-aware objective reduces MAPE to 6.91%, while the complete framework further reduces MAPE to 6.80%.

The final model also improves constitutive fidelity, reducing Shape RMSE from 1.36 to 0.97, the monotonicity-violation metric from 0.20 to 0.12, and seed-to-seed MAPE variability from 1.18% to 0.84%.

These results show that curve-wise batching primarily enables trajectory-level learning, while the physics-aware objectives drive the fidelity improvements.

1.5. Shape RMSE definition and hyperparameter selection

We expanded Appendices A.4–A.5 to define Shape RMSE and Monotonicity Violation and document hyperparameter selection.

Shape RMSE measures discrepancies between consecutive stress increments along predicted and ground-truth trajectories, while the monotonicity-violation metric quantifies the magnitude of negative predicted stress increments.

Hyperparameters were selected through a two-stage search using 12-material LOMO cross-validation with 10 random seeds per fold (120 training runs per configuration). The final settings were applied unchanged to six external-validation alloys excluded from hyperparameter tuning and threshold calibration.

AuthorsOpenReview ↗

Response to Reviewer u4zi (2/2): Presentation and Methodological Clarity

Response to Reviewer u4zi (2/2)

The reviewer also raised several concerns regarding presentation, interpretability, and communication of the proposed framework. We have revised the manuscript to clarify these aspects as follows.

2. Presentation and Methodological Clarity

2.1. Explanation of the MAPS inverse problem and framework contributions

We expanded Sections 1 and 3.1 to provide additional background on MAPS characterization and its distinction from conventional tensile testing.

We also clarified the roles of the three framework components:

  • Physics-aware learning: improves constitutive trajectory fidelity.
  • Material-space reliability assessment: evaluates representation within the learned material space.
  • Coverage-disagreement analysis: identifies thermo-mechanical conditions that may benefit from additional characterization.

Together, these components improve constitutive prediction quality, assess confidence in material-space support, and guide future characterization efforts. The revised Figure 1 and contribution statements have been updated to make these relationships explicit.

2.2. Interpretation of reliability metrics

We expanded Section 3.4 and Appendix B to provide additional mathematical and physical interpretation of neighborhood distance, directional similarity, and local PC1 variance.

Appendix B now details the construction of the 63-dimensional descriptors, projection into the five-dimensional PCA space, and computation of the material-space similarity metrics.

We also clarify that the reliability classification is not intended to directly predict model error. Rather, it quantifies the degree of support provided by neighboring materials within the learned material space. Materials classified as CAUTION may still exhibit low prediction error, as demonstrated by several external-validation materials in Section 4.4 and Table 2.

2.3. Improved figures and experimental explanations

Figure 4 has been redesigned to explicitly show the initial dataset, guided selection, control selection, and resulting constitutive predictions, with larger labels and expanded discussion in Section 4.3.

2.4. Trade-off between prediction accuracy and constitutive fidelity

We agree that this trade-off should be presented explicitly. The revised Section 4.1 and Appendix D (Table 6) now compare the MSE-only baseline with the complete physics-aware model.

While mean MAPE increases slightly from 6.43% to 6.80%, the complete model improves constitutive fidelity, reducing Shape RMSE from 1.36 to 0.97 and the monotonicity-violation metric from 0.20 to 0.12.

We therefore position the contribution as achieving a more favorable balance between pointwise accuracy and faithful reconstruction of constitutive stress-strain behavior.


Closing Remarks

We thank the reviewer again for the constructive feedback. We believe the added validation studies, sensitivity analyses, ablations, and methodological clarifications substantially strengthen both the experimental evidence and the presentation of the proposed framework, and we hope these revisions satisfactorily address the reviewer’s concerns.

Comments on the paper

1 comment

Summary of Manuscript Revisions and Additional Validation Studies

We thank the reviewers for their thoughtful feedback and constructive suggestions. In response, we substantially revised the manuscript to strengthen the experimental validation, clarify the methodology, and better communicate the intended scope and contributions of the proposed framework.

The revised manuscript includes the following major additions and clarifications.


New Validation Studies

Reliability Framework Validation

We expanded Section 4.2 and Table 1 to evaluate the practical utility of the material-space reliability framework. Materials identified as having limited support within the training distribution were incorporated into the training dataset and re-evaluated after retraining. The identified CAUTION materials exhibited an average MAPE reduction of 50.3%, demonstrating that the framework successfully identifies materials that benefit most from additional characterization and training data.

Coverage-Disagreement Validation

We substantially revised Section 4.3 and Figure 4 to include a controlled comparison between framework-guided and control characterization strategies. For AA3003-H14, framework-guided characterization increased observational coverage in the targeted thermo-mechanical region and reduced constitutive-prediction disagreement, providing direct evidence of the framework's utility for experimental planning.


New Ablation and Sensitivity Studies

Reliability Framework Ablation

Appendix B.5 (Table 3) now evaluates neighborhood distance alone, neighborhood distance with directional similarity, and the complete reliability framework. The results demonstrate that the reliability indicators provide complementary information and improve identification of challenging materials.

Learning Objective Ablation

Appendix D (Tables 5 and 6) now includes progressive ablations separating the effects of curve-wise batching, shape-aware learning, temperature-aware learning, and additional monotonicity constraints. These studies isolate the contribution of each component and clarify the trade-off between prediction accuracy and constitutive fidelity.

Threshold Sensitivity Analysis

Appendix B.6 (Table 4) now evaluates stricter and more relaxed reliability-threshold configurations. The primary reliability classifications and conclusions remain consistent across the examined threshold settings.


Expanded Methodological Clarifications

We expanded Sections 1, 3.1, 3.3, and 3.4 together with Appendices A and B to:

  • Clarify the MAPS inverse characterization problem and the role of each framework component;
  • Distinguish the proposed framework from the previously published MAPS benchmark dataset and baseline architecture;
  • Provide additional interpretation of neighborhood distance, directional similarity, and local variance metrics;
  • Define Shape RMSE and Monotonicity Violation explicitly; and
  • Document hyperparameter-selection and threshold-calibration procedures.

Clarified Scope and Contributions

The revised manuscript more clearly distinguishes three complementary contributions:

  1. Physics-aware constitutive learning for improved stress-strain trajectory fidelity;
  2. Materials-space reliability assessment for identifying limited-support materials and extrapolation risk; and
  3. Coverage-disagreement-guided experimental planning for identifying thermo-mechanical regions that may benefit from additional characterization.

We believe these additions substantially strengthen the experimental evidence, improve reproducibility and interpretability, and provide a clearer presentation of both the capabilities and limitations of the proposed framework.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.