acceptodds
Under review as a conference paper at ICLR 2027

SpecFuser: A Hybrid Spectral Transformer Framework for Solid Mechanics

Abstract

Solving solid mechanics problems of complex systems is notoriously demanding either in terms of time, computational resources, or data. Traditional numerical solvers require extensive modelling and fine-grained discretization for computational simulation. Scaling techniques serve as a fundamental strategy for managing computational overhead, including mass scaling which inflates material density, time scaling which compresses the event duration and so on. Nevertheless, a definitive trade-off between runtime and accuracy remains an inherent challenge. Deep learning solvers, on the other hand, are fast at inference, but they carry no physical prior and must recover the underlying mechanics from examples alone, which requires a lot of data and unreliable outside the distribution. To achieve the best of two worlds, we introduce SpecFuser, a hybrid spectral transformer framework that couples a numerical solver with a neural network. Specifically, the full-scaled model is first reduced by geometric scaling, and the numerical solver is run on this computationally cheaper, sub-scaled geometry. The resulting simulation is physically grounded yet geometrically compressed, capturing the essential low-frequency mechanical behaviour at a fraction of the original cost. The solver output, together with the geometric representations of both the sub-scaled and full-scale models, are then jointly encoded and passed to the network as structured input. A Fourier transform with a low-pass cut-off separates a physically exact low-frequency prior from scale-dependent residuals, the truncated spectrum is fused with spatial features and fed to a multi-head cross-attention transformer, which infers the full-scale solution. Because the solver has already supplied the low-frequency answer, the network is left with a low-dimensional residual rather than the full solution operator. This design for complex solid mechanics problems could not only significantly speed up the solving process, but also demonstrates superior capabilities in terms of generality and generalization.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Deep Learning, Solid Mechanics, Hybrid Method

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_YxDVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper propose SpecFuser, a hybrid framework in which combines the strength of the numerical solver and the neural network. It replaces a full-scale finite-element simulation with (i) a simulation of a geometrically sub-scaled model and (ii) a learned correction to the full scale. The authors run experiments on two datasets. SpecFuser reaches lower l2 error than other baselines.

Critical strengths
  1. It uses a novel and practical source of physical priors. Compared with existing hybrid methods, SpecFuser derives its prior from geometrically scaled-down simulation that retains the full simulation setup. This reduces computational cost.

  2. It outperforms 8 baselines on two industrial problems, with low variability across five random seeds. These tasks are more challenging than standard PDE benchmarks.

Critical weaknesses
  • SpecFuser receives a physics simulation of the sub-scaled model at test time. As far as the paper says, the baselines receive only geometry and loads. Beating pure surrogates with extra solver information is expected and does not isolate the architecture’s contribution. Missing baselines: (a) the naive rescaled sub-scale result, i.e. no network; (b) the same baselines (FNO, Transolver, etc.) given the identical sub-scale simulation as input; (c) SpecFuser without the sub-scale input.
  • There is no ablation of the components either: the spectral branch, the spatial branch, and the two cross-attention stages.
  1. The accuracy–cost trade-off is never shown. Table 1 gives no scaling factor, and Table 2 gives time but no error per λ.
  2. Training cost is ignored. Training needs full-scale simulations. For Stamping that is 50,000 × ~790s. This cost isn't discussed. What is the total cost of generating the training data? After how many full-scale queries does SpecFuser pay off, compared with running Abaqus directly?
  3. Generalization claims are untested with out-of-distribution robustness and different λ.
Questions for authors

See critical weaknesses above.

1 comment

We thank the reviewer for raising these concerns. We address each point systematically.

Response to "the naive rescaled sub-scale result, i.e. no network"

In Figure 1 of the paper, we show that directly rescaling the sub-scale simulation output by the geometric scale factor substantially higher than all learning-based methods. This demonstrates that naive scaling is insufficient for multi-scale transfer. The problem is inherently nonlinear, elastic springback depends on complex interactions between geometry, material properties, and contact constraints that do not scale linearly with model size.

Response to "the same baselines (FNO, Transolver, etc.) given the identical sub-scale simulation as input"

We confirm that all baseline methods received the same sub-scale solution as input including the sub-scaled solution and the geometric descriptors of both the sub-scaled and full-scaled models (workpiece cross-sections, characteristic lines, mold geometry, boundary conditions). This ensures a completely fair comparison where performance differences reflect only the architectural design, not input availability.

Response to "SpecFuser without the sub-scale input. There is no ablation of the components either: the spectral branch, the spatial branch, and the two cross-attention stages."

We performed component ablation experiments on both the Stamping and Strip datasets, removing each major component while keeping all other hyperparameters, training procedures, and random seeds identical. The results provide compelling evidence for the necessity of spectral features.

Model VariantStampingStrip
Full Model0.01370.0028
w/o Spectral Branch0.02980.0166
w/o Spatial Branch0.01470.0041
w/o Sub-scaled Inputs0.02110.0152
w/o Cross-Attention0.01560.0041

Response to "The accuracy–cost ... error per λ"

Figure 1 (right panel) shows how prediction error varies with scaling factor λ in stretch bending simulations. Table 2 shows how computation time varies with λ. However, these results are presented in separate locations, making it difficult to visualize the complete accuracy–cost trade-off at a glance.

Reviewer_9CmSOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SpecFuser, a new hybrid transformer model for modelling solid mechanics deformation processes. The key idea is to combine data-driven modelling with numerical solvers. While numerical solvers are typically very slow in that domain, the approach uses a low-scale numerical solution which is several times faster to obtain. The approach is using a spectral encoding where the high frequencies are cut off for both the low-scale and the high-scale features. An encoder first processes features from the low-scale solution and the high-scale initial state separately, fusing each with auxiliary features using multi-head cross attention; subsequently a processor combines both scales via cross attention to compute the high-scale solution. The approach is tested on a bending and a stamping task and compared to other data driven approaches.

Critical strengths
  • In interesting novel approach to embed numerical solvers in neural architectures
  • Good results on two interesting benchmarks
Critical weaknesses
  • The time reduction is only a factor of roughly 3 to 7 in the reported examples (Table 2, even at the most aggressive scaling factor of 0.1), which does not support the “orders of magnitude” acceleration claimed in the conclusion. Moreover, purely data-driven approaches require only milliseconds at inference, whereas the proposed approach still requires several hundred to over a thousand seconds per sample because of the sub-scale numerical simulation.
  • The approach is evaluated on only two scenarios, both of which are metal-forming processes.
  • The baselines are mostly older graph-based and operator-based approaches. In particular, a comparison to ROBIN (Würth et al., NeurIPS 2025) is missing, although it targets very similar problem settings (nonlinear solid mechanics with bending and contact) and outperforms MGN and HCMT, which are both used as baselines here.
  • Comparisons to other hybrid approaches (e.g., NCLaw or CFD-GCN, which are discussed in the related work but not evaluated) would also be informative.
  • The scaling factor used in the experiments is not stated. Consequently, the computation time reduction corresponding to the accuracy reported in Table 1 is unclear, and the accuracy–speed trade-off of the method, its central claim, is never shown.
  • The only ablation varies the spectral cut-off n_c. Ablations of the individual components (without the sub-scale solver input, without the spectral or spatial branch, without cross-attention) are missing, so the contribution of each part remains unclear.
  • Two simple reference points are missing: (i) directly rescaling the sub-scale solution without any network, and (ii) giving the baselines the same sub-scale solution as input. Without these, it is unclear whether the gains come from the architecture or simply from access to the solver output. Relatedly, it is not stated whether the baselines received this input.
Questions for authors
  • The reduction in computation time is in comparison to fully data driven approaches only moderate. Can we reduce more than 10%? Can you also state the performance with the different reduction rates?
3 comments

Response to "The time reduction ... sub-scale numerical simulation"

We thank the reviewer for this important observation. We acknowledge that the phrasing "orders of magnitude" in our conclusion was imprecise and does not accurately reflect the 3-7× speedup factors reported in Table 2. We will revise this language to correctly state the achieved speedup range and avoid overstatement.

However, we respectfully argue that the requirement for coarse simulation input does not diminish the practical value of our method. Our approach targets a fundamentally different point in the accuracy-efficiency-data tradeoff space than pure data-driven surrogates, and this design choice is justified by the following considerations.

First, while our method requires coarse simulation input, the coarse simulation is times cheaper than high-fidelity simulation. Importantly, the total cost of our hybrid pipeline, including both coarse simulation and inference, still achieves the same times speedup compared to high-fidelity simulation. For engineering design optimization and parametric studies that previously required weeks or months of high-fidelity computation, this represents a transformative reduction in time-to-solution, even if not "real-time" in absolute terms.

Second, pure surrogate models capable of millisecond inference typically require thousands of expensive high-fidelity training samples to achieve acceptable accuracy across the design space. In contrast, our hybrid approach leverages readily available coarse simulations paired with limited high-fidelity data (as demonstrated in our experiments). For many practical applications, the computational budget saved during the data generation phase far outweighs the modest inference cost of running coarse simulations during deployment.

Third, the requirement for coarse simulation input does not add significant overhead. Coarse simulations run at much lower resolution and can be executed quickly on modest computational resources, making them readily accessible even in resource-constrained settings. More importantly, the key innovation of our method is not eliminating coarse simulations entirely, but rather efficiently correcting their errors through learned mappings to approach high-fidelity accuracy. This design enables us to extract maximum value from inexpensive coarse simulations while avoiding the prohibitive cost and data requirements of pure surrogate approaches.

Last, our work does not claim to compete with pure data-driven surrogates for applications requiring real-time inference, such as real-time control or interactive visualization. Instead, we target the regime where high-fidelity training data is expensive or limited, physical consistency and generalization are critical, significant speedup over high-fidelity simulation is valuable. This regime encompasses a large class of practical engineering applications, including design optimization, parametric studies, and uncertainty quantification.

In summary, the requirement for coarse simulation input is a deliberate design choice that enables data efficiency, physical consistency, and practical speedup within realistic computational budgets. We will revise our conclusion to accurately reflect the achieved speedup factors and more clearly articulate the target application regime where our approach provides distinct advantages over both pure high-fidelity simulation and pure data-driven surrogates.

Response to "The approach is ... metal-forming processes"

We thank the reviewer for this valuable feedback. In response, we have expanded our experimental evaluation to include fluid dynamics problems, demonstrating SpecFuser's generalization across different physics domains.

We have generated a fluid dynamics dataset and added experiments on incompressible flow past a bluff body, simulated using the Lattice Boltzmann Method (LBM). This represents a fundamentally different physics regime from our original solid mechanics examples.

Table below shows the performance of SpecFuser and all baseline methods on the LBM bluff-body dataset evaluated on surface pressure/friction coefficients and integrated drag coefficient.

ModelCp_mean ↓Cp_rms ↓Cf_mean ↓Mean ↓Cd ↓
MeshGraphNet0.02030.01860.02590.02160.0049
GraphSAGE0.02320.01980.02740.02350.0063
HCMT0.01840.01550.02370.01920.0045
FNO0.01540.01890.01850.01760.0040
Geo-FNO0.01390.01290.01900.01590.0035
LSM0.01990.01690.02600.02090.0043
Transolver0.01520.01580.02040.01710.0032
FactFormer0.01370.01210.01880.01490.0028
ROBIN0.01980.03580.02050.02540.0061
CFD-GCN0.02550.02350.03070.02660.0072
Direct transfer*0.13320.34370.18390.22030.0779
SpecFuser (ours)0.01230.01040.01790.01350.0027

*Direct transfer: using the small-scale solution directly as prediction without any model.

Metrics: Cp_mean/rms = time-averaged/RMS pressure coefficient on 256 surface points; Cf_mean = time-averaged friction coefficient; Cd = relative error of integrated drag coefficient. All values are relative L2 errors. Lower is better (↓).

Through these experiments, SpecFuser achieves great performance on fluid dynamics, outperforming graph-based methods, operator-based models, thansformer-based methods and physics-informed approaches. Besides, all learning-based methods achieve errors far below the direct transfer baseline, demonstrating that models successfully learn the physical scaling relationships rather than simple interpolation.

We believe these additions substantially strengthen the paper by demonstrating SpecFuser's broad applicability across fundamentally different physical systems governed by different PDEs, as well as validating our core hypothesis that spectral-spatial fusion provides a domain-agnostic mechanism for multi-scale physics modeling.

Response to "The baselines are ... as baselines here"

ModelStripStamping
ROBIN0.00850.0274
SpecFuser0.00280.0179

We thank the reviewer for this suggestion. We have now conducted ROBIN baseline experiments on both the Stamping and Strip datasets. ROBIN achieves test losses of 0.0274 (Stamping) and 0.0085 (Strip), compared to our SpecFuser results of 0.0179 and 0.0028 respectively. While ROBIN outperforms some baselines like MGN and HCMT on its native contact-mechanics domains, the performance gap here reflects a fundamental difference in problem focus, ROBIN is specifically optimized for problems with explicit contact discontinuities and friction-driven interactions, whereas our task targets a continuous mechanical phenomenon. ROBIN's diffusion-based refinement mechanism, designed to handle contact discontinuities, provides limited benefit for large deformation prediction.

Response to "Comparisons to other ... also be informative"

We thank the reviewer for this suggestion. We address CFD-GCN and NCLaw separately, as they differ substantially in their applicability to our problem.

We have implemented CFD-GCN and evaluated it on both datasets.

ModelStampingStrip
SpecFuser (ours)0.01370.0028
CFD-GCN0.01810.0088

NCLaw is architecturally incompatible with our problem setting and cannot be evaluated as a direct baseline. NCLaw learns neural constitutive laws that replace the material model inside a differentiable FEM or MPM simulation loop. This requires both a differentiable continuum solver coupled at the constitutive level and training data in the form of incremental deformation trajectories. Neither requirement is met in our setting. Our training data consists of paired final springback states, not incremental trajectories. Adapting NCLaw would require building a differentiable FEM solver and reformulating the entire data pipeline. The appropriate comparison for "coarse-solver output + learned correction" hybrids is CFD-GCN, which we have now evaluated above.

Response to "The only ablation ... part remains unclear"

We appreciate the reviewer's feedback regarding the comprehensiveness of our ablation study. We have indeed conducted a thorough component-level ablation to clarify the contribution of each architectural component. The results are presented below.

Model VariantStampingStrip
Full Model0.01370.0028
w/o Spectral Branch0.02980.0166
w/o Spatial Branch0.01470.0041
w/o Sub-scaled Inputs0.02110.0152
w/o Cross-Attention0.01560.0041

Spectral Branch is the most critical component, contributing a 117% and 493% performance drop when removed. This demonstrates the importance of multi-scale spectral feature extraction for capturing physics-informed patterns. Reference Inputs are also essential, with 54% and 443% degradation observed when excluded. This validates our design choice to incorporate physical solver outputs as auxiliary information. Cross-Attention Mechanism shows meaningful contribution, indicating effective feature fusion between branches. Spatial Branch shows smaller individual impact, which aligns with the complementary role it plays within the full model. These results provide clear evidence that each architectural component contributes meaningfully to the model's overall performance, and the spectral processing pathway is central to SpecFuser's effectiveness.

Response to "Two simple reference ... received this input"

We thank the reviewer for this important clarification question. We have addressed both points.

(i) Figure 1 in the paper directly addressesed this concern. The right panel shows the trade-off between simulation time and relative error for different geometric scaling factors in stretch bending simulations. The curve demonstrates that simply using the sub-scaled solver output and geometrically rescaling it by the scaling factor λ produces errors that grow monotonically as the scaling factor decreases.

(ii) We confirm that all baseline methods received the same sub-scale solution as input including the sub-scaled solution and the geometric descriptors of both the sub-scaled and full-scaled models (workpiece cross-sections, characteristic lines, mold geometry, boundary conditions). This ensures a completely fair comparison where performance differences reflect only the architectural design, not input availability.

Reviewer_mQtXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors present a hybrid approach where numerical solvers can be used to simulated a sub-scaled model and provide the results which is fed into a attention-based architecture together with information from the full scaled model. Two numerical examples show the improved accuracy of the model compared to non-hybrid methods.

Critical strengths

Encoding results of fast simulation to reduce the total runtime is an interesting idea which has been studied in other forms. The particular use of spectral processing is a step informed by the physics understanding of the problem. The reduction in the end to end cost is remarkable.

Critical weaknesses

This approach cannot be used in real time or near real time inference as it necessitates the result of the sub-scale simulation to be available.

The numerical examples are rather limited and not shown on various physics

It is not clear what should be the spectral feature of the response for this method to succeed or fail

Questions for authors

N/A

3 comments

Response to "This approach cannot ... to be available."

We thank the reviewer for this observation. We acknowledge that our hybrid physics-ML approach requires coarse-scale simulation input and therefore cannot achieve the real-time inference that pure data-driven surrogate models offer. However, we respectfully argue that this does not diminish the practical value and advantages of our method. Every approach involves trade-offs in the accuracy-efficiency-data spectrum, and our design choice is justified by the following considerations.

First, regarding computational cost, while our method requires coarse simulation input, the coarse simulation is times cheaper than high-fidelity simulation. Therefore, the total cost of our hybrid pipeline which include the coarse simulation and model inference still achieves times speedup compared to high-fidelity simulation, which is transformative for engineering design optimization and parametric studies that previously required weeks or months of computation.

Second, in terms of data requirements, pure surrogate models capable of real-time inference require thousands of expensive high-fidelity training samples to achieve acceptable accuracy. In contrast, our hybrid approach leverages readily available coarse simulations paired with limited high-fidelity data, significantly reducing the overall data generation cost. For many practical applications, the computational budget saved during training far outweighs the modest inference cost of running coarse simulations.

Third, from a practical deployment perspective, the requirement for coarse simulation input does not add significant overhead. Coarse simulations run at much lower resolution and can be executed quickly on modest computational resources, making them readily accessible even in resource-constrained settings. More importantly, the key innovation of our method is not eliminating coarse simulations entirely, but rather efficiently correcting their errors through learned mappings to approach high-fidelity accuracy. This design enables us to extract maximum value from inexpensive coarse simulations while avoiding the prohibitive cost and data requirements of pure surrogate approaches.

In summary, our work does not claim to compete with pure surrogates for real-time applications. Instead, we target a different yet equally important regime where data efficiency, physical consistency, and practical speedup are the primary concerns. The requirement for coarse simulation input is a deliberate design choice that enables these advantages, and the resulting trade-off is well-suited to the computational constraints and workflows of our target engineering applications.

Response to "The numerical examples ... on various physics"

We thank the reviewer for this valuable feedback. In response, we have expanded our experimental evaluation to include fluid dynamics problems, demonstrating SpecFuser's generalization across different physics domains.

We have generated a fluid dynamics dataset and added experiments on incompressible flow past a bluff body, simulated using the Lattice Boltzmann Method (LBM). This represents a fundamentally different physics regime from our original solid mechanics examples.

Table below shows the performance of SpecFuser and all baseline methods on the LBM bluff-body dataset evaluated on surface pressure/friction coefficients and integrated drag coefficient.

ModelCp_mean ↓Cp_rms ↓Cf_mean ↓Mean ↓Cd ↓
MeshGraphNet0.02030.01860.02590.02160.0049
GraphSAGE0.02320.01980.02740.02350.0063
HCMT0.01840.01550.02370.01920.0045
FNO0.01540.01890.01850.01760.0040
Geo-FNO0.01390.01290.01900.01590.0035
LSM0.01990.01690.02600.02090.0043
Transolver0.01520.01580.02040.01710.0032
FactFormer0.01370.01210.01880.01490.0028
ROBIN0.01980.03580.02050.02540.0061
CFD-GCN0.02550.02350.03070.02660.0072
Direct transfer*0.13320.34370.18390.22030.0779
SpecFuser (ours)0.01230.01040.01790.01350.0027

*Direct transfer: using the small-scale solution directly as prediction without any model.

Metrics: Cp_mean/rms = time-averaged/RMS pressure coefficient on 256 surface points; Cf_mean = time-averaged friction coefficient; Cd = relative error of integrated drag coefficient. All values are relative L2 errors. Lower is better (↓).

Through these experiments, SpecFuser achieves great performance on fluid dynamics, outperforming graph-based methods, operator-based models, thansformer-based methods and physics-informed approaches. Besides, all learning-based methods achieve errors far below the direct transfer baseline, demonstrating that models successfully learn the physical scaling relationships rather than simple interpolation.

We believe these additions substantially strengthen the paper by demonstrating SpecFuser's broad applicability across fundamentally different physical systems governed by different PDEs, as well as validating our core hypothesis that spectral-spatial fusion provides a domain-agnostic mechanism for multi-scale physics modeling.

Response to "It is not ... succeed or fail"

We appreciate the reviewer's concern about the necessity of spectral features in our method. To directly address this, we have conducted a comprehensive component ablation study that quantifies the contribution of each module and definitively demonstrates that spectral processing is not merely an auxiliary component but rather fundamental to the method's success.

We performed component ablation experiments on both the Stamping and Strip datasets, removing each major component while keeping all other hyperparameters, training procedures, and random seeds identical. The results provide compelling evidence for the necessity of spectral features.

Model VariantStampingStrip
Full Model0.01370.0028
w/o Spectral Branch0.02980.0166
w/o Spatial Branch0.01470.0041
w/o Sub-scaled Inputs0.02110.0152
w/o Cross-Attention0.01560.0041

Spectral features are essential for capturing multi-scale features, removing the spectral branch causes performance degradation of 117% on Stamping and 493% on Strip dataset. This performance gap clearly demonstrates that spectral processing captures complementary information that spatial-only methods cannot recover.

These results could definitively answer the question that the spectral features are the critical component enabling the method to succeed. Without spectral processing, the method fails fundamentally, confirming that our approach correctly identifies and exploits the spectral nature of these dynamics.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.