acceptodds
Under review as a conference paper at ICLR 2027

Learning Incompressible 3D Flows from Sparse Samples: A Set Transformer Approach to Divergence-Free Kernel Fitting

Abstract

Recovering divergence-free velocity fields from sparse and corrupted observations requires reconciling physical consistency, expressive representations, and robustness to missing data. We introduce , a set-to-field transformer that reconstructs divergence-free velocity fields from sparse, noisy, and irregularly sampled 3D observations. The model is inherently invariant to the number and ordering of input samples and predicts the parameters of a divergence-free kernel representation, producing continuous, mesh-free velocity fields that satisfy incompressibility exactly by construction rather than approximately through soft constraints or projection operators. Unlike existing divergence-free kernel approaches, which estimate kernel coefficients via a separate optimization for each field, DFK-Former predicts these coefficients directly in a single forward pass. This eliminates costly iterative fitting while providing a learned prior that enables accurate reconstruction even when observations are insufficient to uniquely determine the field. To further enhance representational power, we introduce adaptive and anisotropic divergence-free kernels that preserve exact incompressibility guarantees while capturing more complex flow structures. Extensive experiments across varying levels of sparsity, noise, and occlusion, including transfer to real Direct Numerical Simulation (DNS) turbulence data, demonstrate substantial gains in both accuracy and efficiency. DFK-Former reduces endpoint error by up to compared to per-field divergence-free-kernel optimization, while reconstructing each field in instead of , yielding a speedup of more than three orders of magnitude. These results establish direct, feed-forward prediction of divergence-free kernel parameters as a practical and effective paradigm for physically consistent velocity-field reconstruction from sparse observations.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Divergence-Free Kernels, Physics-Informed Machine Learning, Fluid Flow Reconstruction, Permutation-Equivariant Transformers, Feed-Forward, 3D Vector Field Learning, Incompressible Flow Modeling

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_n5ZyOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DFK-Former, a Set Transformer-based method for reconstructing continuous, divergence-free 3D velocity fields from sparse, partial, or corrupted observations. Instead of optimizing divergence-free kernel (DFK) parameters separately for each flow field, the proposed network predicts them directly in a single forward pass. Besides, the authors introduce adaptive and anisotropic kernels using the Piola transform, and extend the framework to time-varying flows through shared spatial kernels and Fourier-based temporal coefficients.

The method is trained on procedurally generated flow fields and evaluated on synthetic benchmarks, real DNS data, and a qualitative experimental PIV example. The results show substantial improvements in reconstruction accuracy over existing methods, while preserving incompressibility by construction.

Critical strengths
  1. A simple and effective combination of learned priors and exact incompressibility.
  2. Effective extension of DFKs for better representative ability (anisotropy and non-zero net momentum).
Critical weaknesses

I have found only one critical weakness.

Currently, the temporal representation imposes an artificial periodicity constraint. Eq. (5) represents the time-dependent kernel coefficients and mean flow using a Fourier series on the normalized interval (t∈[0,1]t\in[0,1]). Consequently, the reconstructed velocity field necessarily satisfies vΘ(x,0)=vΘ(x,1).\boldsymbol v_\Theta(\boldsymbol x,0)=\boldsymbol v_\Theta(\boldsymbol x,1). This appears to introduce a representational limitation that is not discussed in the paper.

Questions for authors
  1. The proposed method introduces several improvements to the original DFK-Wen4, e.g., the mean-flow term, larger and multi-scale support radii, and anisotropic kernels. How much of the improvement over DFK fitting comes from these changes rather than learned parameter prediction? How does a per-instance fitting baseline using exactly the same kernel geometry and representation as DFK-Former perform?
  2. How are sparse, partial, or irregular observations provided to FNO and HDNet-3D?
  3. The temporal benchmark evaluates reconstruction at unobserved intermediate frames. How exactly are the per-frame DFK fitting results obtained at these frames?
3 comments
AuthorsOpenReview ↗

Response to Reviewer n5Zy (Part 1/2)

Thank you for the review and for pinpointing the periodicity issue. Your weakness and Q2 are shared with other reviewers and answered in the general response (points 1 and 3, with the notation); we summarize them here.

Weakness — Periodicity of Eq. 5. You are right, and it matters here because both window ends are observed. Only ω_k(t) and v̄(t) vary in time, so any basis keeps exact incompressibility, and the revised Eq. 5 admits any temporal basis. The measured cost is a 6 % endpoint excess, mostly a boundary effect that non-periodic bases share, and no non-periodic basis is significantly better per window (Table R1). Section 6 states the limitation.

Q1 — Basis improvements vs. learned prediction. The paper gave a partial answer. Against ridge fitting in DFK-Former's own anisotropic basis (Table 15), the learned model is 15.8–38.0 % better at the hardest sweep settings, and even K=32 DFK-Former (0.281) beats both DFK fits of Table 1. These two fits already use our overlapping radius 3.5Δ, without the mean flow: the fixed-kernel fit (0.689) is the 3.5Δ rung of Table R8 without v̄ (0.458 with it), and the adaptive fit (0.546) is Ni et al.'s iterative optimization of centers and radii, started from that layout. Table R8 starts instead from DFK-Wen4 at Ni et al.'s own radius and adds our changes one at a time, with the same per-instance ridge solver on all 582 fields.

Table R8. Macro EPE over the 11 sets. Every fitted row solves for the kernel weights by ridge-regularized least squares on the observed samples; the two columns differ only in how the ridge strength λ is chosen. Oracle ridge: one λ per set, picked from a grid by its error against the ground-truth target. A real fit cannot see the target, so this is a best case for the fits. CV ridge: λ picked per field by 5-fold cross-validation on the observed samples only, never using the target, as a practitioner would. DFK-Former fits nothing, so both columns are equal.

RepresentationOracle ridgeCV ridge
DFK-Wen4 (Ni et al.: paper radius, no mean flow)0.8610.861
+ mean-flow term v̄0.6240.624
+ overlapping radius 3.5Δ0.4580.479
+ multi-scale radii0.4090.434
+ anisotropy (shape estimated from the samples)0.3620.405
DFK-Former's predicted geometry, ridge-fitted weights0.3370.345
DFK-Former (predicted geometry and weights, no fit)0.1200.120
Reference: DFK-Wen4 with an 8× larger radius0.5030.560

Your second question, a per-instance fit with exactly DFK-Former's geometry and representation, is answered by the row "DFK-Former's predicted geometry, ridge-fitted weights": 0.337 (0.345 with the CV ridge), against 0.120 when the network also predicts the weights.

Main findings.

  • Mean flow, overlap, multi-scale radii and anisotropy explain most of the gain over DFK-Wen4 fitting: 0.861 → 0.362, i.e. 67 % of the total gain (61 % with the CV ridge).
  • Predicted geometry adds a smaller improvement (0.362 → 0.337); predicted weights give the final major reduction (0.337 → 0.120).
  • The basis gain survives a stronger reference: against DFK-Wen4 with an 8× larger radius, our basis changes still lower the macro EPE by 0.142 (0.503 → 0.362; by 0.155 with the CV ridge, 0.560 → 0.405). Refining the geometry per field by gradient descent rarely helped (worse on 77–93 % of fields).

Takeaway. When fitting per instance, the representation changes account for most of the gain over the original DFK basis. Learned prediction then lowers the error about threefold (0.362 → 0.120), mainly through the weights. Given a learned predictor, an unconstrained decoder reaches similar accuracy (Table R4, reply to Reviewer 6YrL), so the DFK head's role is exact incompressibility at no accuracy cost, plus locality and efficient evaluation. Neither the representation nor learning alone is sufficient.

AuthorsOpenReview ↗

(Part 2/2)

Q2 — Inputs to FNO and HDNet-3D. Both receive the samples on a 32³ grid. FNO gets a trilinear splat with zeros in empty cells, plus an occupancy channel. HDNet-3D gets a trilinear splat (weighted average per cell) plus a finite-difference divergence channel, following HDNet; a 3D U-Net then regresses the field directly, with no projection layer or divergence penalty. Details, and a stronger RecFNO-style encoding, are in general response point 3 (Table R2).

Q3 — Per-frame DFK at unobserved frames. Per-frame DFK fits each observed frame on the fixed kernel layout and linearly interpolates the coefficients (ω, v̄) between the bracketing frames. With a shared layout, this equals linear interpolation of the fields and stays exactly divergence-free. Shared-kernel DFK (Ni et al.) jointly optimizes one geometry with per-frame weights over the sequence (Adam) and interpolates the weights the same way. The 15/180 failures of per-frame fitting are near-singular solves under the most severe corruption. Appendix B now describes both procedures. The new static DFK-Former control (Table R7, reply to Reviewer iPim) uses the same interpolation. It shows that the learned spatial prior accounts for most of the gain over per-frame fitting (0.767 → 0.124), with temporal fusion adding about 9 % (0.114).

Thank you for the clarification! I am glad to keep my original score.

Reviewer_iPimOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DFK-Former, a set-to-field transformer designed to reconstruct continuous incompressible 3D velocity fields from sparse, corrupted observations through direct prediction of divergence-free kernel parameters. Adaptive placement and Piola-transformed anisotropic kernels improve spatial flexibility, while Fourier coefficient trajectories extend the framework to time-varying flows. The method is trained on procedurally generated spectral and analytic flows. It is evaluated on synthetic and DNS data. Additional tests cover temporal reconstruction and a qualitative PIV example.

Critical strengths

Direct Prediction of Constrained Fields: DFK-Former predicts continuous divergence-free fields without per-instance fitting (Table 1). The same-backbone control achieves similar accuracy but leaks divergence (Table 14). Post-hoc DFK fitting increases error and latency, supporting direct parameter prediction.

Structured Evaluation of Capacity and Transfer: Clean-field fitting separates representation capacity from learned inference (Fig. 5, Table 4). Zero-shot DNS tests support transfer (Table 3), while withheld-family and clean/dense experiments identify the limits of the learned prior.

Critical weaknesses

Incomplete Comparison with Specialized Methods: The evaluation lacks comparisons with Energy Transformer and RecFNO (Table 1). Post-hoc projection also does not test training-time hard constraints (Table 1). These gaps limit claims of superiority over specialized reconstruction methods and alternative constrained models.

Incomplete Validation of Temporal Modeling: The temporal experiments lack a static DFK-Former control, leaving temporal fusion gains mixed with learned-prior benefits (Table 2). The Fourier basis also forces equal endpoint fields, restricting nonperiodic evolution.

Questions for authors

1.How is temporal capacity matching consistent with F=2F=2 outputs and nine-harmonic training data?

2.What sparse-input encoding is used by the FNO baseline, and how does it relate to the cited RecFNO approach?

3 comments
AuthorsOpenReview ↗

Response to Reviewer iPim (Part 1/2)

Thank you for the positive assessment and the concrete suggestions. The periodicity part of W2, Q1 (capacity matching) and Q2 (FNO encoding and RecFNO) are shared with other reviewers and answered in the general response (points 1–3, with the notation).

W1 — Stronger baselines. We implemented the three requested comparisons, trained with the same data, schedule and budget as our FNO:

  • RecFNO: our FNO already used its mask embedding; we added its Voronoi embedding, in 3D (general response, point 3).
  • Energy Transformer (ET): no code is released, so we reimplemented it in the configuration of its paper's Table 1 (1.03 M parameters), masking empty patches as ET does.
  • Training-time hard constraint: FNO with a differentiable Leray projection as its last layer, so its loss only sees divergence-free fields. None of our original grid baselines had one; HDNet-3D regresses the field directly, with no projection layer or divergence penalty.

Table R6. Static benchmark, macro EPE over the 11 sets (582 fields). The last column applies the Leray projection after training, which makes any grid output divergence-free.

MethodAs trainedDivergence-free as trained?With post-hoc Leray projection
FNO0.244no0.251
FNO + Leray layer in training (new)0.250yes—
FNO, Voronoi input (new)0.204no0.222
Energy Transformer, reimplemented (new)0.347no0.345
HDNet-3D0.211no—
DeepONet0.417no0.428
DFK-Former0.120by construction—

Main findings.

  • The Voronoi FNO becomes the strongest grid baseline, yet as trained it is less accurate than DFK-Former on all 11 sets. Made divergence-free, it ties DFK-Former on the two turbulent DNS sets (isotropic 0.201 vs. 0.200, MHD 0.267 vs. 0.269) and is far behind on localized structures (shear layer 0.391 vs. 0.054, vortex ring 0.265 vs. 0.062).
  • Training-time Leray projection performs the same as post-hoc projection (0.250 vs. 0.251): for FNO, the hard constraint is neither a gain nor a loss.
  • Our Energy Transformer is behind every FNO variant and HDNet-3D (0.347; 0.345 projected). It was trained for the same 150k steps as the FNOs, and its training loss plateaued from about 20k steps at about twice theirs. This suggests that its size (1.03 M parameters, about 40× fewer than our FNO's 42.5 M) or our 4³ patch decoder, not training length, is the limit. We kept the published configuration rather than design a scaled-up ET ourselves, so this result holds for that configuration only; a larger ET might do better. The revised paper lists our deviations from the original.

Takeaway. These additions substantially strengthen the empirical comparison. The Voronoi comparison is particularly informative, because it isolates the sparse-input embedding from the operator architecture. Even with a stronger sparse-input embedding or a hard divergence constraint during training, DFK-Former keeps the lowest macro error, without a projection step. The best grid baseline matches it only on broadband DNS turbulence, which Section 5.2 and the limitations now state. With our own encoder, we also compared exactly divergence-free heads (Table R4, reply to Reviewer 6YrL): DFK beats a Curl-kernel head, and a learned vector potential is slightly more accurate but needs autodiff at inference; at 64³ it is 2× slower than DFK with 4.8× the memory, or 17× slower at equal memory.

AuthorsOpenReview ↗

(Part 2/2)

W2 — Static control for temporal modeling. We apply a static model to every observed frame and interpolate linearly in time, which stays exactly divergence-free because a convex combination of divergence-free fields is divergence-free. We add two static controls:

  • Same kernels: the temporal model's kernel geometry (the base model of Table 3), which isolates temporal fusion.
  • Full: our static DFK-Former, with richer geometry (adaptive centers, multi-scale radii).

Table R7. Median held-out EPE. Regime rows: original 60 windows (10 per regime).

RegimeTemporalStatic, same kernelsStatic, fullPer-frame DFK fitting
clean & dense0.1140.1210.1030.100
contaminated0.0850.0920.0830.217
noisy0.0880.1020.0950.398
sparse0.1090.1240.1140.847
occluded0.0980.1080.1000.951
combined corruption0.1380.1640.1600.916
Overall, original 60 windows0.1140.1240.1150.767
Overall, new test split (60)0.1200.1380.1190.779
Windows where temporal is better (original · test)—49/60 · 49/6035/60 · 32/60—

Main findings.

  • At equal geometry, temporal fusion wins: it is better on 49/60 windows on both splits (p = 2·10⁻⁹ and 9·10⁻¹⁰) and on 7–10 of 10 windows in every regime, with 9–13 % lower error.
  • Richer static geometry recovers much of this gain: the full static model comes close overall (temporal better on 35/60 original windows, p = 0.047; a tie on test, p = 0.17).
  • Temporal fusion helps most with sparse, occluded and heavily corrupted observations (combined corruption: p = 0.006 and 0.014); the full static model wins on clean, dense inputs.
  • Both static models are far ahead of per-frame DFK fitting (0.124 and 0.115 vs. 0.767).

Takeaway. The new controls separate temporal fusion from the learned spatial prior. Most of the gain over per-frame fitting comes from the learned spatial prior, which supports the core premise of the paper. On the test split, the full static model ties the temporal model overall. What temporal fusion still contributes is a significant gain at equal kernel geometry (49/60 windows on both splits) and a lower error under combined corruption, where it beats even the full static model (9/10 and 8/10 windows, p = 0.006 and 0.014). Multi-scale radii, which give the full static model its edge, have not yet been tried in the temporal model. We revised the temporal claim accordingly (Sections 1 and 5) and added both controls to Tables 2 and 22.

Reviewer_iPimOpenReview ↗

Official Comment by Reviewer iPim

Thank you for the clarification. Your response addresses my concerns, and I will retain my original score.

Reviewer_6YrLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DFK-Former, which maps sparse, noisy, and irregular velocity observations to a continuous divergence-free kernel field. A set-based transformer predicts kernel parameters, and a Piola-transformed anisotropic kernel construction preserves incompressibility analytically. The temporal extension uses shared spatial kernels with Fourier-in-time coefficients. The evaluation includes synthetic fields, zero-shot DNS reconstruction, temporal interpolation, and a qualitative tomographic-PIV example.

Critical strengths

1.Combining a learned reconstruction prior with an analytically divergence-free output representation is a useful approach to reconstruction from sparse observations. The Piola construction provides a clear mechanism for preserving incompressibility with anisotropic kernels. 2.The evaluation is broad, covering 582 static fields and 60 DNS sequences. 3.The paper includes useful controls, such as a direct-regression decoder with the same backbone, projection baselines, kernel-count experiments, and a parameter-matched isotropic comparison.

Critical weaknesses

1.Section 5.4 shows that much of the reconstruction accuracy comes from the learned prior, while the DFK decoder preserves incompressibility at similar accuracy. However, its advantage over divergence-free decoders remains unclear. 2. The absence of DNS training does not by itself establish that DNS test data were excluded from model selection. This is an unresolved protocol question. 3.Some detailed descriptions need clarifications. For example, Section 4.1 specifies an extend-only aspect ratio of at least one, whereas Table 6 permits values in [1/4,4]. 4.In terms of efficiency, 7.5 ms in Table 19 uses Q=256 query points, but it does not establish the cost of dense field evaluation.

Questions for authors

1.Can you compare the DFK decoder with another learned, exactly divergence-free decoder using the same set encoder, training data, and a comparable parameter budget? 2.How does the periodic endpoint condition in Eq. (5) affect errors near the beginning and end of non-periodic DNS windows? Would a non-periodic temporal basis reduce these errors? 3.Were the Fourier order and temporal kernel-placement choices selected using an independent DNS validation split? Please distinguish training, architecture selection and final testing. 4.Can you reconcile the aspect-ratio bounds, temporal bandwidth, and head-initialization descriptions, and report prediction and field-evaluation latency as query resolution increases?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 6YrL (Part 1/3)

Thank you for the precise questions. Q2 (window endpoints) and the bandwidth part of Q4 are shared with other reviewers and answered in the general response (points 1 and 2, with the notation). In short, the endpoint excess is 6 %, mostly a boundary effect that non-periodic bases share, and none of them is significantly better per window. We start with the protocol question, which we consider the most important.

W2/Q3 — Training, validation and testing. We agree that the original temporal model-selection protocol was imperfect. Section 5.1 now separates the stages:

  • Training: synthetic data only; no model, ours or baseline, saw DNS data.
  • Checkpoint selection: synthetic validation only (Table 7).
  • Static architecture: K = 1024 was fixed early in development, on synthetic data, before any DNS data were used. The kernel-count study (Table 4) came later; on its seven synthetic sets alone it selects the same K (macro EPE 0.101, vs. 0.130 at K = 512 and 0.134 at K = 2048). Multi-scale radii (Table 3) are also the best choice on the synthetic sets alone (GRF 0.368 → 0.343, CAF 0.171 → 0.102). Their values {5,4,3}Δ come from the radius study of Table 11, which used JHTDB snapshots that overlap the static test fields; redone on disjoint DNS frames and on synthetic data only, they stay within 0.9 % and 2.0 % of the best radius, so the choice is unchanged.
  • Temporal architecture: F=2 and fixed kernel placement were chosen on the same 60 DNS windows we report (App. E.1, Table 24); synthetic validation alone would have picked F=7 with adaptive placement.

We therefore created new JHTDB splits with no overlap with the original windows or the static test fields: 40 validation windows and 60 untouched test windows (45 disjoint in space and time; 15 MHD windows share either time steps or a region with an earlier set, never both, and are reported separately). We then repeated the selection among the temporal variants of Table 24 on validation.

Table R3. Median held-out EPE on the new splits; last column: windows where the row beats the reported model.

ModelValidation (40)Test (60)Better than reported, val · test
Design variants of Table 24
F=2, shared anisotropic kernels, fixed placement (reported)0.1150.120—
F=2, moving isotropic kernels0.1170.1325/40 · 2/60
F=7, moving isotropic kernels0.1840.1871/40 · 0/60
F=7, shared anisotropic kernels0.1280.1400/40 · 0/60
F=2, shared anisotropic, adaptive placement0.1200.1287/40 · 14/60
Best baselines (Table 2 settings)
Space-Time SIREN (not divergence-free)—0.474— · 0/60
Shared-kernel DFK fitting—0.640— · 10/60
Per-frame DFK fitting0.7100.7796/40 · 9/60

Main findings.

  • Validation selects the same architecture: every alternative of Table 24 is worse per window (p ≤ 7·10⁻⁶). The five variants rank in the same order as on the original windows.
  • The same architecture is ahead of every variant on the untouched test split (p ≤ 3·10⁻⁷), reaching 0.095 on the 45 strictly disjoint windows; relative to the original ranking, only the two middle variants swap places.
  • The gap to the temporal baselines remains large. The best, Space-Time SIREN, is behind on 60/60 windows; as in Table 2, shared-kernel DFK fitting still wins the 10 clean, dense windows.
  • Even on the original windows, every variant beat the best baseline (worst variant 0.183 vs. 0.428), so the selection affected which variant we report, not the comparison with baselines.

Takeaway. This experiment directly addresses the strongest protocol concern raised in the reviews. While the original selection procedure was imperfect, an independent validation and test protocol leads to the same conclusion.

AuthorsOpenReview ↗

(Part 2/3)

W1/Q1 — Alternative exactly divergence-free decoders. We trained our set encoder with four heads, using the same data and schedule and a parameter budget matched within 5 %, at reduced scale (K = 512, 20k steps) with two seeds: (a) DFK; (b) a Curl-kernel head (Ni et al.'s alternative divergence-free kernel, same geometry and Piola anisotropy); (c) a learned vector potential (each query attends to the kernel tokens, an MLP outputs A(x), and v = ∇×A by automatic differentiation); (d) an unconstrained regressor (reference, not divergence-free). A DFK is itself the curl of a localized potential, so (b) changes only the kernel and (c) replaces the closed-form local potential with a learned global one.

Table R4. Accuracy: mean of two seeds. EPE-9: the 9 sets never used for checkpoint selection. Localized: Lamb–Oseen, vortex ring, shear layer. Divergence RMS in float32 (≈ 10⁻⁶ is round-off). Latency includes the encoder (seed 0; seed 1 gives the same ratios). A 64³ grid is evaluated in chunks of queries: own chunks is each head's default (1,024 queries for the kernel heads, 8,192 for the learned decoders); 1,024-query chunks uses the same size for all heads. Memory: peak of the field evaluation, encoder excluded.

HeadMacro EPE (11 sets)EPE-9LocalizedDivergence RMSLatency, Q = 25664³, own chunks: time (memory)64³, 1,024-query chunks: time (memory)
(a) DFK (ours)0.2980.2700.3276·10⁻⁷6 ms212 ms (143 MB)210 ms (143 MB)
(b) Curl-kernel0.3040.2770.3482·10⁻⁷15 ms152 ms (83 MB)150 ms (83 MB)
(c) Vector potential0.2890.2650.3421·10⁻⁶19 ms412 ms (688 MB)3,633 ms (125 MB)
(d) Unconstrained0.3080.2830.375≈ 15 ms70 ms (335 MB)212 ms (81 MB)

Main findings.

  • DFK is the most accurate head on localized coherent structures (0.327 vs. 0.342–0.375) and is ahead of the vector potential on Lamb–Oseen, vortex ring and shear layer in both seeds.
  • DFK evaluates in closed form, with no automatic differentiation at inference. Its compact support lets each query visit only nearby kernels, which enables the neighbour-list evaluator of Table R5.
  • Exact incompressibility costs no accuracy: DFK matches the unconstrained head (0.298 vs. 0.308) and is more accurate than the Curl-kernel head in both seeds.
  • The learned vector potential is about 3 % more accurate on the macro (0.289 vs. 0.298; 1.7 % on EPE-9). It needs automatic differentiation at inference, and its cost at 64³ depends on the memory it is given: with its default chunks it is 2× slower than DFK but needs 4.8× the memory (688 vs. 143 MB); at matched memory (1,024-query chunks) it is 17× slower (3.6 s vs. 0.21 s). At Q = 256 it is 3× slower.

Takeaway. On this reduced-scale, two-seed comparison, choosing between DFK and a learned vector potential is a trade-off: a few percent of macro accuracy against locality, closed-form evaluation and accuracy on localized structures. DFK is preferable when exact incompressibility, locality and efficient continuous evaluation are needed together. On the source of accuracy, we give the same answer in all replies: when fitting per instance, our representation changes give most of the gain over the original DFK basis (Table R8, reply to Reviewer n5Zy); given a learned predictor, an unconstrained decoder reaches similar accuracy, so the DFK head's role is exact incompressibility at no accuracy cost, plus locality and efficient evaluation. The abstract and Sections 1 and 5.4 now state the claim this way.

W3 — Aspect-ratio bounds. Table 6 is correct and Section 4.1 was wrong: η = 4^tanh(s) ∈ [1/4, 4], with s the raw head output. "Extend-only" meant that the transverse semi-axes stay at the kernel radius while the axial one scales. It did not imply η ≥ 1, and we renamed the convention "transverse-preserving". Over all 595,968 kernels predicted on the 582 static test fields, η spans 0.30–2.58 (median 1.06), 30.4 % of kernels are oblate, and none has max(η, 1/η) > 3.5. The radius scale ρ spans 0.39–2.63, within [1/3, 3]. The displacement bound ‖Δx‖∞ ≤ Δ is reached more often (24.6 % of the |ω|-weighted kernel mass has ‖Δx‖∞ > 0.9Δ), which Section 4.1 now notes. Most kernels thus stay close to isotropic; the model departs from isotropy for a subset of kernels, in both the prolate and oblate directions, and never reaches the aspect-ratio bound.

AuthorsOpenReview ↗

(Part 3/3)

Q4 — Head initialization. All geometry heads are zero-initialized (s = 0), where each bounded map is the identity: Δx = Δ·tanh(s) = 0, per component, so the bound is ‖Δx‖∞ ≤ Δ; ρ = 3^tanh(s) = 1; η = 4^tanh(s) = 1; and the kernel axis is normalize(e_z + tanh(s)) = e_z. The adaptive anisotropic model therefore starts exactly as the fixed isotropic model. Table 6 now lists each map.

Q4 — Latency as query resolution increases. Table 19 used Q = 256, and its 7.5 ms is the encoder alone. The abstract, Table 1 and Section 5.2 now report the end-to-end time at Q = 256: 9.9 ms (encoder plus evaluation). This is about three orders of magnitude faster than adaptive DFK fitting (9.8 s), but only about 2× faster than fixed-kernel linear DFK fitting (22.6 ms), which is far less accurate (Table 1: 0.689). The encoder runs once per field, while evaluation cost grows with Q. Because evaluation is exact and local, we added a cell-list neighbour evaluator that visits only the ≈ 25 % of kernels whose support can contain each query.

Table R5. Measurements on an idle A100 GPU (float32, batch size 1, median over runs). The encoder requires 7.49 ms per field. Neighbour-list timings include the 2.7 ms spatial-grid construction cost. Encoder + evaluation reports end-to-end runtime using the faster of the two evaluators. Peak memory is measured relative to the baseline implementation.

QEvaluation, original denseEvaluation, neighbour listEncoder + evaluationPeak memory
2562.3 ms4.2 ms9.9 ms15 MiB
32³66.5 ms10.4 ms18 ms321 MiB
64³526 ms51.5 ms59 ms342 MiB
128³4.2 s0.35 s0.36 s512 MiB
4.9 M (Case C grid)9.9 s0.78 s0.79 s770 MiB

Takeaway. Dense evaluation is practical: a full 128³ field takes 0.36 s end to end, 12× faster than with the original evaluator, in 512 MiB. DFK-Former should be viewed as a continuous, exactly divergence-free field representation that can be queried anywhere, not as a dense-grid acceleration method. Grid models are faster to read out densely (about 5 ms forward plus 0.8–9 ms resampling), but they give a fixed-resolution grid without exact incompressibility. We qualified App. F.4's FLOPs statement as holding at Q = 256 (the encoder's share is 50 % at 64³ and 11 % at 128³). Against linear DFK fitting, the speed advantage comes only from replacing the fit, so it vanishes once evaluation dominates (Q ≳ 10⁵).

We hope these answers resolve your concerns.

Dear Reviewer 6YrL,

As the discussion period is approaching its end, we would appreciate knowing whether our additional experiments and responses have addressed your concerns. If any issues remain unresolved or require further clarification, we would be happy to provide additional information before the deadline.

Thank you again for your careful review and feedback.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General response (Part 1/3)

We thank the reviewers for their careful and constructive reviews. All three value the core idea: a learned prior that predicts an exactly divergence-free field directly, without per-instance fitting (in Reviewer iPim's words, "direct prediction of constrained fields"). Reviewers 6YrL and n5Zy also highlight the anisotropic Piola kernels and the extended DFK representation, and Reviewers 6YrL and iPim the breadth of the evaluation and its controls. The main concerns fall into three themes: the periodic Fourier basis of the temporal model, the protocol used for model selection, and comparisons with other divergence-free and sparse-reconstruction methods. To address them, we built new independent DNS validation and test splits, implemented the requested baselines and decoders, and added new ablations and controls.

Main takeaways.

  • The conclusions hold under a stronger protocol. Independent DNS validation and test splits select the same temporal architecture as reported in the paper (Table R3), and the static design choices are reproduced on synthetic or disjoint data (reply to Reviewer 6YrL).
  • DFK-Former has the lowest error of all methods (macro EPE 0.120 vs. ≥ 0.204; Table R6), including RecFNO's Voronoi embedding, an Energy Transformer and an FNO trained with a hard divergence constraint. The strongest baseline, a Leray-projected Voronoi FNO, matches it on the two turbulent DNS sets but is far behind on localized structures.
  • The roles of representation, learning and decoder are now separated. When fitting per instance, our representation changes give most of the gain over the original DFK basis (Table R8). Given a learned predictor, an unconstrained decoder reaches similar accuracy, so the DFK head's role is exact incompressibility at no accuracy cost, plus locality and efficient evaluation (Table R4). Temporal fusion adds a significant gain at equal kernel geometry, mainly with sparse, occluded and heavily corrupted observations (Table R7).
  • Periodicity is a measured, minor limitation. It costs a few percent at the window ends, and no non-periodic basis is significantly better per window (Table R1).

These results confirm the paper's main conclusions. Three claims are now stated more precisely: the role of the DFK decoder, the source of the temporal gains, and the latency, which is now reported end to end. The revised PDF marks changes in blue (in response to the reviews) or red (further corrections we made on our own initiative). Below, we answer the three points raised by several reviewers; the individual replies answer the rest.

Notation. EPE: endpoint error ‖v − v*‖ averaged over query points, after per-field normalization to unit RMS velocity (lower is better). Macro EPE: mean over the 11 static test sets (582 fields). Window: a 7-frame DNS sequence; the model observes t = 0, ⅓, ⅔, 1 and is scored on t = ⅙, ½, ⅚ (held-out EPE, median over windows). n/N windows: windows where a model beats the reference. p: two-sided Wilcoxon signed-rank test on paired per-window errors (p < 0.05: significant; otherwise a tie). Δ: kernel spacing. Q: number of query points.

AuthorsOpenReview ↗

General response (Part 2/3)

1. Periodicity of Eq. 5 (Reviewers 6YrL, iPim, n5Zy). We agree: the Fourier parameterization implies v(·,0) = v(·,1), and the paper did not discuss this limitation. The method does not depend on this choice. The kernels are shared over time and only ω_k(t) and v̄(t) vary, so any time basis keeps ∇·v = 0 at every instant. To quantify the impact, we fine-tuned the reported model on non-periodic trajectories with matched-size Legendre and half-range cosine bases and a Fourier control, all with the same budget.

Table R1. Ends / Interior: EPE at observed frames t = 0, 1 / t = ⅓, ⅔ (original 60 windows). Ratio: median per-window end/interior ratio. r_s: Spearman correlation between this ratio and the window's end-to-end change ‖v(0) − v(1)‖/RMS. Last two columns: new validation (40) and test (60) splits (reply to Reviewer 6YrL).

Time basisTraining trajectoriesEndsInteriorRatior_sHeld-out EPE, val / testBetter than reported, val · test
Fourier, F=2 (reported)periodic0.1190.1131.060+0.300.115 / 0.120—
Fourier, F=2 (control)non-periodic0.1220.1191.027+0.860.119 / 0.12613/40 · 18/60
Legendrenon-periodic0.1210.1131.066−0.380.113 / 0.12215/40 · 26/60
Half-range cosinenon-periodic0.1170.1121.053−0.270.112 / 0.12018/40 · 25/60

Main findings.

  • The reported model's error at the window ends is 6 % above the interior (53/60 windows, p = 3·10⁻⁹).
  • Most of this excess persists with non-periodic bases (5–7 %, 58–59/60 windows). It is the edge effect of a smooth fit at the boundary of the observed interval.
  • Periodicity adds a dependence on the end-to-end change (r_s > 0 only with Fourier; the control's lower ratio comes from a higher interior error).
  • No alternative basis is significantly better per window. Half-range cosine ties (validation 18/40, p = 0.15; test 0.120, p = 0.057); Legendre and the Fourier control are behind on test (p ≤ 0.036).

Takeaway. Periodicity introduces a measurable endpoint effect of a few percent, concentrated in windows whose ends differ most. However, fitting at the temporal boundary, not periodicity, is the dominant factor, and none of the alternative bases achieves a significant per-window improvement. We therefore retain the Fourier model, document the limitation in Section 6, and rewrite Eq. 5 with a generic temporal basis, ω_k(t) = Σ_j a_k^j b_j(t). Any basis can be substituted without affecting exact incompressibility; we keep Fourier because its order F is a temporal bandwidth that can be checked against data (point 2).

AuthorsOpenReview ↗

General response (Part 3/3)

2. Temporal bandwidth (Reviewers 6YrL, iPim). Thank you for identifying an inconsistency between Section 4.3 and the appendix. The temporal training data used 9 harmonics for both the F=2 and F=7 models, so the claim that the data match the model's capacity did not hold for F=2. Share of the time-varying energy in the 2nd / 3rd harmonics, with the same 7-frame estimator:

  • synthetic training data: 27 % / 9.8 % (about a third of the energy lies above f = 2);
  • DNS windows: 0.8 % / 0.16 % (a quadratic in time captures 96 % of it; new figure);
  • non-periodic fine-tuning data of point 1: 3.8 % / 0.42 % (calibrated on synthetic data only).

Takeaway. The original explanation was incomplete, but the model's behaviour is consistent with the temporal spectra of the data. DNS trajectories are much smoother in time than our synthetic ones, which explains why F=7 wins on synthetic validation (App. E.1) while F=2 transfers better to DNS. Section 4.3 is corrected; no reported result changes.

3. Sparse input encodings (Reviewers iPim, n5Zy). Appendix B now documents every baseline's input encoding, and we added a RecFNO-style Voronoi embedding.

Table R2. Input encoding of the learned baselines (grid models use 32³).

ModelInput encoding
FNOTrilinear splat onto the grid, zeros in empty cells, plus an occupancy channel (≈ RecFNO's mask embedding)
FNO, Voronoi input (new)Each cell takes its nearest sample's value, plus the occupancy channel (RecFNO's Voronoi embedding, in 3D)
HDNet-3DTrilinear splat (weighted average per cell) plus a finite-difference divergence channel, following HDNet; a 3D U-Net regresses the field directly, with no projection layer or divergence penalty
Energy Transformer (new)Samples binned into patch tokens of 4³ cells; empty patches masked
DeepONetSet encoder (per-sample MLP, masked mean/max pooling); no grid

RecFNO pairs an FNO with one of three sparse-input embeddings: mask, Voronoi or MLP. We added the Voronoi one, with the same data, schedule and budget. The MLP embedding needs a fixed sensor layout, so it does not apply to a variable number of samples.

Takeaway. The Voronoi embedding substantially strengthens FNO (macro EPE 0.244 → 0.204) and makes it the strongest grid baseline. DFK-Former remains more accurate overall: 0.120, against 0.222 for the Voronoi FNO once made divergence-free, which matches DFK-Former only on the two turbulent DNS sets (Table R6, reply to Reviewer iPim).

Other answers. Reviewer 6YrL: the selection protocol for all design choices, with independent DNS validation and test splits (Table R3), alternative exact decoders (Table R4), end-to-end latency (Table R5), aspect ratio and initialization. Reviewer iPim: stronger baselines (Table R6) and a static control for the temporal model (Table R7). Reviewer n5Zy: basis vs. learned inference (Table R8) and per-frame DFK at unobserved frames.

Dear Reviewers,

Thank you again for your careful reviews and suggestions. Your comments substantially improved the paper.

Main changes in the paper:

  1. Evaluation protocol (Sec. 5.1, 5.3). Temporal design choices are now selected on an independent DNS validation split and reported on an untouched test split. The same architecture is selected and remains ahead of every variant and baseline.

  2. Stronger baselines (Table 1, Sec. 5.2). We added RecFNO's Voronoi embedding, an Energy Transformer, and an FNO trained with a Leray layer. DFK-Former keeps the lowest EPE (0.120 vs. ≥ 0.204).

  3. Alternative divergence-free decoders (Sec. 5.4). We compare Curl-kernel and vector-potential decoders on our encoder.

  4. Static controls for the temporal model (Table 2, Sec. 5.3). They separate temporal fusion from the learned spatial prior.

  5. Temporal basis (Sec. 4.2, 4.3). Eq. 5 now admits any temporal basis, the periodicity limitation is discussed, and the temporal-bandwidth description is corrected.

  6. Clarifications (Sec. 4.1, 5.1). We now give the bounded maps and zero initialization of the geometry heads, the aspect-ratio bounds, and how unobserved frames are obtained for the baselines.

In the supplement, we added:

  • a fitting ladder separating basis changes from learned inference (App. D.7);
  • the radius study on disjoint data (App. D.3);
  • non-periodic time-basis ablations (App. E.2);
  • the full decoder comparison (App. E.3);
  • end-to-end and dense-grid latency (App. F.4);
  • the new DNS splits and baseline input encodings (App. B).

We are grateful for your time and feedback.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.