acceptodds
Withdrawn from ICLR 2027

MF-PINN: Model Fusion for Multi-Task Physics-Informed Neural Networks Solving the Navier–Stokes Equations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Navier–Stokes equations at different Reynolds numbers exhibit substantial differences, so their solution models must capture both the shared physical structures across flow regimes and the specific local flow patterns of each regime. A conventional single shared multi-task model is easily affected by task interference. Model fusion can reuse information from multiple task models, but for the complex characteristics of the Navier–Stokes equations, direct parameter fusion may introduce conflicting updates and weaken physical consistency.To address this issue, we propose MF-PINN (Model-Fusion Physics-Informed Neural Network), a framework that combines task-vector fusion with physics-informed task adaptation. MF-PINN extracts task-specific updates from a commonly pretrained PINN and fuses them into a shared Base model. It then keeps the Base model fixed and uses lightweight task-specific Adapters to recover local flow characteristics under different flow regimes. The model is trained using the Navier–Stokes equations, boundary conditions, and real flow-field data, while a stream-function representation enforces incompressibility by construction.Experiments on six Reynolds-number tasks from PDEBench show that MF-PINN achieves lower and more stable physics-informed test losses across tasks than independent PINNs, shared multi-task PINNs, parameterized PINNs, and general model-fusion methods. It also requires substantially fewer task-specific parameters than storing a complete model for each task. These results demonstrate that task-vector fusion and physics-informed adaptation can effectively balance cross-task knowledge sharing and task-specific prediction accuracy.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Physics-informed neural networks, Multi-task learning, Model fusion, Navier–Stokes equations

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LefvOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The method introduced in this paper aims to leverage task similarity by modeling shared representations across flow regimes and capturing task-specific features through both task vectors and adapters. Task vectors extracted through task-specific fine-tuning are combined into a shared base model, while an adapter is incorporated for each task to further refine its predictions. The premise is that flows at different Reynolds numbers share common underlying physics that can be exploited across tasks. In this case, the authors focus on the 2D incompressible Navier–Stokes equations under varying Reynolds numbers.

Critical strengths
  1. Broad Baseline Benchmarking: The empirical evaluation includes 13 baselines across three relevant methodological families: general multi-task learning, multi-task PINNs, and model merging. All merging baselines use the same adapter-refinement setup, supporting consistency in this aspect of the comparison.
  2. Conceptually Interesting Framework for Multi-Regime PDEs: Combining weight-space model merging through task vectors with task-specific adapters is an interesting approach to multi-task PDE modeling. The motivation of sharing information across Reynolds-number tasks while retaining the flexibility to capture task-specific flow features is appealing.
Critical weaknesses
  1. Insufficient evidence of predictive accuracy and broader applicability:

    • Solution accuracy: The quantitative evaluation relies solely on the weighted PINN test loss, whose interpretation depends on the loss weights and normalization. Relative L2L_2 errors for (u,v,p)(u,v,p), alongside a breakdown of the loss components, are needed to assess whether the reported gains correspond to more accurate physical solutions.
    • Robustness: Tables 1–7 report no variability across independent training runs. This makes it difficult to assess the robustness of the improvements, particularly in ablations where some differences appear only in the third decimal place.
    • Generalization and scope: Evaluation is restricted to one Navier–Stokes setup with six Reynolds numbers, all included in shared training and fusion. Tables 6–7 test reuse across these known tasks and show substantially larger off-diagonal losses; they do not establish adaptation to unseen regimes. A held-out-ReRe evaluation would address this gap, while additional PDEs or more diverse Navier–Stokes configurations would strengthen the evidence for broader applicability.
  2. Insufficient experimental specification and reproducibility:

    • Dataset and temporal evolution: The simulation conditions and temporal evaluation protocol are insufficiently specified. Figure 3 shows individual snapshots without identifying their times or how far the flows have evolved from their initial conditions. The initial conditions, simulation time horizon, and training/test time ranges are needed to interpret the regime-dependent dynamics and determine whether evaluation measures temporal interpolation or extrapolation.
    • Data and sampling: The manuscript does not report the spatial and temporal resolutions or the numbers of interior collocation points, boundary points, and observation points used in training and evaluation.
    • Baseline comparisons: The architectures, learning rates, optimizer settings, and tuning budgets for the 13 baselines are insufficiently documented, making it difficult to assess whether comparisons use comparable computational budgets.

(Minor Remark)

  • Section 3.4 - The paper is generally well written, but SharedMLP and HeadMLP are introduced without sufficient definition. Although the residual bottleneck adapter and its placement between these components are described, defining the network components and showing them in the architecture diagram would improve clarity.
  • Figure 1 - Figure 1 is a bit confusing, what MLT strategy is being used, is c) normalized wrt the best score, or why is it that for c, it has the same performance in all regiemes.
  • Pre-training cost vs Online cost - this would give the author a better assessment of the trade-off.
Questions for authors
  1. Dataset and temporal evolution: What initial conditions, boundary conditions, forcing, spatial resolution, and simulation time horizon define the six Reynolds-number tasks? At what times are the Figure 3 snapshots taken, and how far have the flows evolved from their initial conditions? Are the initial conditions shared across Reynolds numbers? Do the reported test losses cover multiple time steps or only the snapshots shown in Figure 3? What time ranges are used for training and testing? Specifically, does evaluation test temporal interpolation (e.g., training on samples spanning t∈[0,1]t\in[0,1] and testing at unseen times within that interval) or temporal extrapolation (e.g., testing at t=2t=2)?
  2. Solution accuracy and variability: Can the authors report relative L2L_2 errors for (u,v,p)(u,v,p), alongside the individual test-loss components and their weights? How consistent are the reported improvements across independent training runs?
  3. Contribution and implementation of model fusion: Table 2 shows that MF-PINN outperforms the “w/o Fusion” variant. How is this variant constructed? In particular, are adapters trained directly on the pretrained model, using the same adapter architecture, initialization procedure, and training budget as the full method? Additionally, Figure 2 describes the fusion coefficients αi\alpha_i as learnable. At which stage and using what objective are they optimized? Section 3.4 specifies that only adapter parameters are updated during refinement, while Algorithm 1 does not include a coefficient-learning step.
  4. Parameter savings and computational cost: How are the parameter counts in Table 5 calculated for each model and adapter, and which adapter bottleneck dimension is used? Please report wall-clock time and hardware for shared pretraining, full task-specific fine-tuning, adapter initialization/refinement, and inference, including the total training cost across all six tasks, so that the accuracy–storage–compute trade-off can be assessed.
  5. Applicability to unseen regimes: Can the method adapt to a Reynolds number excluded from shared pretraining and task-vector construction? What additional training and target-task data would be required, and are supporting results available? The current cross-task tables evaluate regimes already included in training and therefore do not establish this capability.
Reviewer_bhT5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MF-PINN, a model-fusion framework for solving two-dimensional incompressible Navier–Stokes equations across different Reynolds numbers. The method jointly pretrains a PINN, fine-tunes it for each task, and merges the resulting task vectors into a shared base model. Task-specific adapters are initialized by matching the corresponding fine-tuned models and subsequently refined using physics-informed losses while the base remains frozen.

Experiments on six Reynolds-number tasks from PDEBench report lower aggregate physics-informed test losses than multi-task PINN and model-merging baselines, with performance approaching full task-wise fine-tuning and reduced deployment parameters.

Critical strengths
  1. The paper addresses a relevant problem: reducing parameter redundancy across related physics-informed models while preserving task-specific accuracy.

  2. The reported results are promising, showing substantial improvements over several baselines. The paper also provides pipeline ablations and parameter-sensitivity studies.

Critical weaknesses

W1. Insufficient evaluation of solution accuracy. The main evaluation relies on an aggregate physics-informed loss combining PDE residuals, boundary constraints, and data fitting. Without standard solution-error metrics, such as relative (L2L_2) errors against reference velocity fields, it is difficult to assess the actual predictive accuracy independently of the training objective. Fig. 3 provides qualitative evidence but does not fully address this concern. See Q1.

W2. Insufficient attribution of performance improvements. Although fusion and adapter initialization are ablated, the contributions of teacher-output matching and subsequent physics-informed refinement are not clearly separated. The initialization and training settings of the fusion baselines are also insufficiently specified. See Q2.

W3. Limited methodological novelty. The framework primarily combines existing techniques, including task-vector averaging, teacher-guided adapter initialization, and standard PINN optimization. The specific advantages over simpler model-averaging and adapter-distillation alternatives are not sufficiently established. See Q3.

W4. Unclear parameter-efficiency claims. Table 5 reports 109,260 parameters per fine-tuned PINN but only 18,210 for the fused base, which is difficult to reconcile with the direct parameter-space fusion in Eq. (6). See Q4.

Overall, while the reported results are encouraging, the limited evaluation of actual solution accuracy, insufficient contribution attribution, and relatively incremental methodology make the paper's main claims less convincing.

Questions for authors

Q1. Solution accuracy. Could the authors report relative (L2L_2) errors of the predicted velocity fields against reference solutions for all baselines? Separately reporting data, PDE-residual, and boundary-condition losses would help distinguish improvements in solution accuracy from physical consistency.

Q2. Ablation and fairness. Could the authors clarify whether the w/o Fusion variant and model-merging baselines use the same teacher-output initialization, adapter capacity, and training budgets? Reporting performance after teacher matching but before physics-informed refinement would help isolate the contribution of each stage.

Q3. Contribution of fusion. How are the fusion weights determined? Could the authors compare against uniform model averaging and a pretrained-base adapter baseline, both with identical teacher initialization and refinement, to better establish the benefit of fusion?

Q4. Efficiency and generalization. Could the authors reconcile the parameter counts in Table 5 with Eq. (6) and clarify the complete training costs and data-splitting protocol? How would the method handle unseen Reynolds numbers without corresponding task-specific adapters?

Reviewer_K9J9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors addressed a limitation of multi-task PINNs across different Reynolds numbers. Single-task PINNs require storing a separate model for each task, while shared multi-task PINNs suffer from task interference. Their approach extracts task vectors by fine-tuning a pretrained PINN on each task, then fuses them into a shared base model. They also designed bottleneck adapters, initialized through output matching and refined with physics loss, with parameter efficiency as an additional motivation. The authors demonstrated the method’s effectiveness empirically on six Reynolds-number tasks from PDEBench.

Critical strengths
  • The author addresses task interference and parameter redundancy in multi-task PINNs.

  • The author evaluates the method against 13 baselines spanning multi-task learning, multi-task PINNs, and model fusion.

Critical weaknesses
  • The proposed technique and architecture are already well known and widely used. Task-vector merging [1] and bottleneck adapters [2] are standard approaches in the literature, and parameter-efficient fine-tuning methods such as LoRA have already been studied for PINNs [3]. However, the pipeline requires full fine-tuning for every task before fusion. In [Table 2], the fine-tuned models (θᵢ) already achieve a lower mean loss (0.1319) than MF-PINN (0.1410). The authors should evaluate and compare direct PEFT/LoRA on the pretrained model, or explicitly explain why a three-stage pipeline is needed when the full models are already trained and perform better.

  • Why is the physics-informed loss the main evaluation metric? VRME or other standard metrics are not reported.

  • In [Line 238], the caption of Figure 2 states that the task vectors are fused with learnable weights, yet in [Line 273] the authors state that fusion requires no retraining. This requires additional clarification.

  1. Ilharco et al., "Editing Models with Task Arithmetic", ICLR 2023.

  2. Houlsby et al., "Parameter-Efficient Transfer Learning for NLP", ICML 2019.

.3 Wang et al., "Transfer Learning in Physics-Informed Neural Networks: Full Fine-Tuning, Lightweight Fine-Tuning, and Low-Rank Adaptation", IJMSD 2025.

Questions for authors
  • [line 266] How are the fusion weights αᵢ determined?

  • [line 294] "output matching": What is the exact objective used to initialize the adapter?

  • Experiment: In [Table 5], why does the Fusion Base (18,210) have 6 × fewer parameters than a single-task PINN (109,260)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.