acceptodds
Under review as a conference paper at ICLR 2027

Spectral Attention Operator for Learning Boundary-Driven Hemodynamics

Abstract

Patient-specific simulation of boundary-driven hemodynamics requires solving the governing PDEs for every inlet waveform and physical parameter setting. We introduce the Spectral Attention Operator (SAO), which learns the map from an inlet waveform to the response at any queried location and parameter setting. SAO couples temporal Fourier modes according to the inlet and adapts its temporal functions to location and physical parameters through Feature-wise Linear Modulation. It predicts complete trajectories in one forward pass without temporal rollout. Across three viscous Burgers and two hemodynamics benchmarks, SAO attains the best overall accuracy in comparisons with five capacity-matched operator families, with relative errors near or below %. On the multi-frequency and hemodynamic flow-rate tasks, SAO reduces error by -% against the strongest external baseline. Ablations isolate the contributions of cross-mode attention and query conditioning, and physics-based diagnostics show accurate shock localization and hemodynamic fields consistent with the governing relations. For hemodynamics, SAO is hundreds to thousands of times faster than numerical simulation on CPU and four to five orders of magnitude faster on GPU.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Scientific Machine Learning, Neural Operators, Boundary-Driven Dynamics, Hemodynamics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_722sOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Paper proposes SAO for predicting boundary-driven dynamics which combines attention across temporal Fourier modes with query-dependent and method is evaluated on Burgers and 1D hemodynamics problems.

Critical strengths
  1. The results show improvements in multi-frequency Burgers problems and hemodynamic flow-rate prediction.
Critical weaknesses
  1. Limited Novelty as lots of work already exists on spectral attention [1] and baseline comparison with recent work such as Transolver, Latent Mamba Operator, and Latent Neural operator.

[1] Yang, Zherui, et al. "Simple yet Effective: Low-Rank Spatial Attention for Neural Operators." arXiv preprint arXiv:2604.03582 (2026).

  1. Experiments cover only single artery and fixed Y-bifurcation. Generalization to larger networks or unseen geometries is not shown.

  2. Some component benefits are less general than main motivation. A well-chosen fixed cutoff matches or slightly outperforms the learned gate, and spatially varying FiLM gives no clear overall improvement.

  3. Burgers benchmarks change both the number of inlet frequencies and the RMS amplitude making it difficult to isolate the benefit of attention.

Questions for authors
  1. Could you clarify novelty relative to existing spectral attention literature and conditioned neural operators? Also how does it performs wrt to it? See weakness 1. The paper is not ready for acceptance in its current form.

  2. Would cross-mode attention still provide similar gains if Burgers experiments kept inlet RMS amplitude fixed while varying the number of frequencies?

  3. How does SAO compare with simpler model using parameter-conditioned FiLM and validation-selected fixed spectral cutoff?

  4. Could you evaluate on a larger vascular network or explain what changes would be needed to handle unseen geometries and topologies?

  5. Temporal functions depend on each query, how do inference time and memory scale with the number of spatial locations?

5 comments

We thank the reviewer for the sharp comments and constructive suggestions.

W1/Q1. Relations to other attention-based operators. We agree that attention is widely used in neural operators. However, compared with our work, most of this existing work targets different problems (Introduction, Lines 54-59) and apply attention in a different way. LRSA, Transolver, LNO and LaMO form tokens from spatial points and mix among them, by attention or, for LaMO, by a state-space scan, to predict spatial fields. In contrast, SAO targets boundary-driven problems and applies attention in the temporal spectral domain, whose tokens are the temporal Fourier modes below a learned cutoff, and every retained mode attends to every other with weights computed from the inlet (Section 3.3, Appendix E.3). Nonlinear propagation moves signal energy across temporal frequencies in a way set by the inlet (Introduction). A standard Fourier layer applies weights fixed after training, whereas the attention computes the mode coupling from the current activation and so adapts to the inlet (Section 3.3).

We will add the listed references.

W2/Q4. Testbeds and larger networks. Single vessels and bifurcations are standard benchmarks for 1D arterial models (Boileau et al., 2015; Xiao et al., 2014) and common testbeds for learned surrogates (Kissas et al., 2020).

Our work focuses on learning boundary-driven wave responses, rather than generalization across vascular topologies. Extending SAO to larger networks would require additional mechanisms for representing connectivity and coupling vessel responses (Pegolotti et al., 2024). SAO could serve as a building block for such an extension, but developing and evaluating that framework is beyond the scope of this study and a promising direction for future work.

W3. Component benefits: Gate, FiLM. We disagree with this weakness. We demonstrated clear benefits of the main components of SAO including learnable gates and FiLM.

  • In Appendix G.1, we show that learnable gates can be used for automatic selection of modes through gradient descent. Without gates, one has to select modes based on hyperparameter tuning on validation data, which is very time-consuming. "A well-chosen fixed cutoff", mentioned by the reviewer, is actually determined by learnable gate (see Appendix G.1).

  • The benefit of FiLM has been demonstrated in Section 4.3 (line 435). Without FiLM, SAO-noFiLM increased the error by 24, 11, and 2.7 times on B1-B3, 5.6 times on H1 and 3.4 times on H2 (three-field mean, Table 4.2), compared with SAO.

Q3. A simpler model for comparison. As the reviewer suggested, we trained a simpler model, FNO-T+FiLM. It is based on FNO-T (Appendix H.1) with FiLM applied after every layer from the query c=(x,μ)c=(x,\mu). Modes are validation-selected using fixed cutoff (101 for both cases). The table below shows SAO retains substantially lower errors than FNO-T+FiLM.

ModelB2H2 meanH2 P / Q / A
FNO-T+FiLM (3 seeds)3.58 ± 0.130.293 ± 0.0080.223 / 0.503 / 0.153
SAO (5 seeds)1.03 ± 0.040.107 ± 0.0120.139 / 0.129 / 0.052

W4/Q2. Matched inlet RMS. We added three new Burgers sets with 1, 10 and 20 sinusoids rescaling each realization to B2's expected inlet root mean square (RMS) (0.9309), across three seeds. At fixed RMS attention reduces the error more than 4 times at all cases.

SinusoidsSAOSAO-noAttnRatio
10.188 ± 0.0050.926 ± 0.0814.9
100.888 ± 0.0243.84 ± 0.314.3
200.825 ± 0.0233.54 ± 0.244.3

Q5. Scaling with the number of locations. SAO runs its Dynamic Net once per queried location, so time and memory grow with the number of locations (Appendix E.4). The table gives one H1 simulation on one H100, with all locations in one pass. Time stays near 3 ms up to a few hundred locations, since the GPU processes them in parallel, and then grows linearly. Memory grows linearly with the number of locations. Even at 3,000 locations, SAO takes 18.0 ms, against 0.75 min for the solver (Table 4.5). Similar behaviors are observed in H2. We will add this analysis to Appendix G.2.

Locations11013031,0003,000
Time (ms)2.812.953.206.5118.0
Peak memory (MB)501062206131,741

References.

Thank you for the rebuttal. I will maintain my score.

Thank you for your reply. Could you please clarify your remaining concerns after our responses? We would appreciate any feedback on points that we have not adequately addressed, and we would be happy to discuss with you before the rebuttal period ends.

Reviewer_LomEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SAO, a method that predicts the complete response at a given location in one step based on the full inlet waveform, query position, and physical parameters. The approach combines cross-frequency attention with FiLM and is validated on three Burgers examples and two blood flow examples. Results suggest that both design components improve accuracy; however, these conclusions are primarily applicable to fixed structures, fixed grids, and predefined waveform ranges, and some baseline comparisons require more precise interpretation.

Critical strengths

The cross-frequency attention mechanism, along with conditional inputs of position and parameters, aligns well with wave propagation problems. Removing attention increases errors for B2 and B3 from 1.03% and 0.90% to 4.00% and 3.99%, respectively, while blood flow predictions also deteriorate significantly. Training results across five repetitions show good stability.

Critical weaknesses

The main table's treatment of baselines affects the perceived improvement magnitude. In Appendix H.5, spatial FNO achieves a flow error of 0.132% on H2—close to SAO’s 0.129%—significantly smaller than the gap shown in the main table. DeepONet also alters the original input assignment by repeatedly reading the inlet at each time step. Therefore, claims about accuracy and speed gains should be limited to specific implementations and cannot directly represent improvements over the entire class of methods.

All blood flow inlet conditions are derived from perturbations of a single base waveform, yet the appendix does not specify the exact amplitude and phase perturbation ranges, nor fully list initial states or outlet model configurations. Furthermore, the authors have not released code or complete configuration details with the submission, making it difficult to reproduce experiments or assess how much test waveforms differ from training ones. This directly impacts the evaluation of SAO’s applicability to blood flow prediction.

Questions for authors

Please refer to the spatial FNO results in the appendix and clarify where the most significant improvements occur on H2. When flow errors are already close, how should the reported gains over the strongest baseline in the abstract be qualified?

To what extent do DeepONet’s input allocation and repeated computations affect the reported speed differences?

Please provide detailed information on the inlet perturbation ranges, initial conditions, and outlet model parameters used in the data generation process, and explain the differences between training and testing waveforms.

Do current results mainly reflect variations around the same waveform shape, or do they include substantially different waveform forms?

1 comment

We thank the reviewer for the careful review and thoughtful comments.

W1/Q1. Spatial FNO-BVP. We realize how Appendix H.5 caused confusion. Appendix H.5 reports results for Spatial FNO-BVP, which is NOT among the five external baselines referenced in the abstract and main text: FNO-T, DeepONet*, POD-DeepONet, OFormer, and POD+MLP. Our claims about accuracy refer to comparisons with these five external baselines and remain valid.

We invented FNO-BVP by modifying a one-dimensional FNO into a two-dimensional FNO so that it takes the combined (x,t)(x, t) field as input (thus losing the temporal representation that other methods preserve). It was used as a control study to see whether SAO's advantage over FNO-T is explainable by the temporal input representation (see line 1664 in the paper) and appears only in the appendix.

We will revise Appendix H.5 to distinguish Spatial FNO-BVP explicitly from the external baselines and clarify its role as a control model.

W1/Q2. DeepONet* speed. We agree that DeepONet*'s trunk does not depend on the query, so it can be computed once per simulation and shared by all queries. We have re-timed DeepONet* in this way. It shows that the new variant of DeepONet* takes 0.26 and 0.27 ms on H1 and H2 against SAO's 2.94 ms and 3.20 ms. We will change the speed claim at line 465 and update DeepONet* performance numbers to reflect the new results.

W2. Code availability. When submitting the paper, we uploaded the supplementary material that contains data generation, training, baselines and analysis scripts. There is a link below the abstract on OpenReview for the reviewers to download a zip file.

W2/Q3. Data-generation settings. Please see the table below:

SettingValue
Inlet waveforms1,000 per network, FFT-based augmentations of one openBF reference inlet
Inlet time shiftCircular, N(0,82)N(0, 8^2) ms
Inlet phasePer-bin perturbation added, N(0,0.122)N(0, 0.12^2) rad on the lowest 15% of FFT bins and N(0,0.062)N(0, 0.06^2) rad on bins 15-45%
Inlet amplitudePer-bin gain 1+N(0,0.062)1 + N(0, 0.06^2) on the lowest 15% of FFT bins and 1+N(0,0.032)1 + N(0, 0.03^2) on bins 15-45%
Initial stateopenBF default (zero flow, area A₀, pressure 0)
Outlet modelFixed three-element Windkessel (R₁, R₂ in Pa s m$^{-3},Cinm, C in m^{3}$/Pa)
H1 outletR₁ 1.1752×1071.1752\times10^{7}, R₂ 1.1167×1081.1167\times10^{8}, C 1.0163×10−81.0163\times10^{-8}
H2 outlets (each daughter)R₁ 6.8123×1076.8123\times10^{7}, R₂ 3.1013×1093.1013\times10^{9}, C 3.6664×10−103.6664\times10^{-10}

We will add this to Appendix D.

Q3/Q4. Training and test waveforms. We focus on variations around the same waveform shape in hemodynamics for practical considerations, whereas the waveforms vary a lot in the Burgers case.

In hemodynamics, the inlets stay close to one measured waveform per network by design because normal flow waveforms at a given artery follow a characteristic shape (Holdsworth et al., 1999; Ford et al., 2005) and wider perturbations could give physiologically implausible inflows. For example, simulated pulse wave databases usually derive each subject's inflow from one measured or template waveform, with cardiac parameters kept within healthy ranges (Willemet et al., 2015; Charlton et al., 2019).

In Burgers cases, we vary the inlet shape widely, from one sinusoid of random amplitude and frequency on B1 to sums of 10 and 20 sinusoids with random amplitudes, frequencies and phases on B2 and B3 (Appendix D.2). SAO reaches 0.93, 1.03 and 0.90% test RL2 on B1-B3 (Table 4.1), which demonstrates SAO's capability to handle various inlet waveforms.

References.

Reviewer_rreFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents the Spectral Attention Operator (SAO), a neural operator for learning boundary-driven wave responses. SAO uses attention to couple temporal Fourier modes and Feature-wise Linear Modulation (FiLM) to adapt its temporal representations to spatial locations and physical parameters. The authors evaluate their architecture on five benchmarks involving the viscous Burgers equation and 1D hemodynamics.

Critical strengths

SAO achieves strong improvements over the tested baselines, especially on the multi-frequency Burgers benchmarks (B2/B3), where it reduces the relative error by approximately 70% compared to the strongest baseline. The paper includes extensive ablation studies.

Critical weaknesses

I appreciate that the authors match the parameter counts and training protocols for the baselines. However, the baselines are not individually tuned, apart from one hyperparameter (the learning rate, which is tuned only to a limited extent). In Appendix H.2, the authors state that FNO-T uses 32 Fourier modes. Since the choice of spectral bandwidth strongly affects SAO’s performance, I wonder how much FNO-T would improve if it were tuned. I am missing a comparison against a well-tuned FNO with per-layer FiLM conditioning and optimized spectral bandwidth.

Questions for authors

Can the authors provide a comparison against a well-tuned FNO with per-layer FiLM conditioning and optimized spectral bandwidth, in order to establish the improvements over alternative architectures?

2 comments

We thank the reviewer for recognizing SAO's strong performance and suggesting a more thoroughly tuned FNO. In response, we trained a new version of FNO-T with per-layer FiLM conditioning and tuned both its spectral bandwidth and learning rate. Including all 101 spectral modes for both B2 and H2, and a learning rate 10−410^{-4} for B2 and 3×10−43\times10^{-4} for H2 achieved the best results during hyperparameter tuning.

As shown below, adding FiLM conditioning and tuning the spectral bandwidth improves FNO-T, reducing its error by more than 30%. Nevertheless, SAO retains substantially lower errors than FNO-T+FiLM. These results suggest that SAO’s gains cannot be attributed solely to FiLM conditioning and bandwidth selection, and provide further evidence for the contribution of its cross-mode attention and basis synthesis. We thank the reviewer again for suggesting this additional comparison that strengthens the paper.

ModelB2H2 meanH2 P / Q / A
FNO-T, in original paper (5 seeds)5.19 ± 0.170.473 ± 0.0130.320 / 0.853 / 0.248
FNO-T, tuned LR (Table H.1, 3 seeds)4.15 ± 0.280.471 ± 0.0180.322 / 0.844 / 0.246
FNO-T+FiLM (3 seeds)3.58 ± 0.130.293 ± 0.0080.223 / 0.503 / 0.153
SAO (5 seeds)1.03 ± 0.040.107 ± 0.0120.139 / 0.129 / 0.052

Thank you for the additional experiments. I will maintain my score of 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revision Uploaded

Dear Area Chair and Reviewers,

Thanks for your time and effort through the discussion phase. The revision is uploaded, with the proposed updates during discussion highlighted.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.