- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes HotPOT, a multi-PDE pre-training framework combining a shared history encoder with PDE-specific neural–numerical experts. Each expert pairs a learned predictor with a fixed numerical update to handle variable coupling and state evolution for different equations. The model is jointly trained using next-state supervision and predicts subsequent physical fields autoregressively. The paper evaluates HotPOT against MoE-POT and other baseline methods using relative L2 error across six settings spanning five PDE families, with promising results.
Critical strengths
- The separation between shared field-history representations and PDE-specific predictor–update pairs is a reasonable approach to heterogeneous PDE learning.
- The SWE and CNS update constructions also provide explicit conservation and admissibility properties under the stated assumptions.
- The evaluation includes multiple PDE families, architectural controls, and transfer to an unseen family.
Critical weaknesses
- Insufficient evidence of reproducibility and lack of compute-matched comparisons. Although the paper provides training and evaluation settings, it does not establish sufficiently verifiable correspondence between the processed data, split manifests, model checkpoints, and reported results. Training code alone is insufficient for independent verification. Moreover, matching activated-parameter counts does not imply comparable computational costs: HotPOT involves repeated readouts and numerical operations, and stores substantially more parameters than the corresponding MoE-POT models. Appendix A.5 acknowledges the absence of runtime comparisons under identical conditions, making it difficult to separate architectural benefits from the effects of additional capacity and computation.
- Insufficient validation of baseline comparability and result robustness. Table 1 imports baseline results from MoE-POT without demonstrating that all methods were evaluated through a unified pipeline. For example, the fine-tuned SWE errors for DPOT-T/S/M are 0.00313/0.00315/0.00260, compared with 0.00280/0.00224/0.00218 in the original paper. Different pre-training settings may explain these discrepancies, but protocol clarification and controlled comparisons are still needed. Furthermore, task-specific learning-rate selection and sampling adjustments make it difficult to distinguish the contributions of model design from those of training strategies.
Questions for authors
- What exact artifacts allow an independent reviewer to reproduce Table 1 without repeating full pre-training? Please identify the processed-data sources or deterministic preparation scripts, exact split manifests, evaluated checkpoints, and evaluation commands.
- Can the authors reconcile the imported DPOT results with the original DPOT Table 1, specifying differences in pre-training data, preprocessing, checkpoint selection, rollout horizons, and metric aggregation? Have the imported baseline scores been verified using the same evaluation pipeline as HotPOT?
- How do HotPOT and MoE-POT compare under matched computational budgets? Please report training GPU-hours, inference latency or throughput, peak memory, and the actual NS substep/readout counts on identical hardware.
7 comments
Response to Reviewer 7FZ8 (2/4)
W1b. Computation, capacity and storage
Moreover, matching activated-parameter counts does not imply comparable computational costs: HotPOT involves repeated readouts and numerical operations, and stores substantially more parameters than the corresponding MoE-POT models. Appendix A.5 acknowledges the absence of runtime comparisons under identical conditions, making it difficult to separate architectural benefits from the effects of additional capacity and computation.
Both have now been separated:
- Computation. We measured it on the same hardware. Over the complete rollouts of the six settings, HotPOT-T takes 1% less time than MoE-POT-T (3297 ms against 3331 ms), and its peak memory is on par (184 to 185 MiB against 182 MiB). The details are in Q3.
- Capacity. The paper contains the control for this question. The Direct-state ablation of Table 3 has family-private predictors, exactly the same stored parameters as HotPOT-T (97,902,720) and the same training protocol. The only change is that the numerical update is replaced by direct prediction of the next state. To make the comparison robust, we trained both models with two further seeds. The table gives the mean and standard deviation over three seeds on the settings whose expert uses a numerical update.
| Setting | Direct state (same capacity, no numerical update) | HotPOT-T | Reduction |
|---|---|---|---|
| NS ν=1e-5 | 0.0527 ± 0.0002 | 0.0316 ± 0.0002 | 40% |
| NS ν=1e-3 | 0.00657 ± 0.00063 | 0.00318 ± 0.00009 | 52% |
| CNS | 0.0106 ± 0.0007 | 0.00715 ± 0.00038 | 32% |
| SWE | 0.00188 ± 0.00004 | 0.00179 ± 0.00002 | 5% |
| DR | 0.120 ± 0.016 | 0.0211 ± 0.0061 | 82% |
On each of these settings, every HotPOT-T seed is below every Direct-state seed. With the capacity held equal, the gain comes from the numerical updates. CFDB is not listed because its update is a plain assignment, so there is no numerical update to test.
- Storage. The stored totals count all experts together, but a task uses only the shared encoder and the expert of its own family. To deploy one NS, CNS, SWE or DR task, HotPOT stores 16.8 to 17.2M parameters at the T scale, against 30.2M for MoE-POT-T. MoE-POT cannot be reduced in this way, because its routing depends on the input. The single-expert files are released in the folder
single_expert/of https://anonymous-hf.com/a/ryou1c8aizn3/, and their predictions are bit-identical to those of the full models.
Revision location: Section 5.2 reports these reductions, and Appendix A.14 gives the results of every seed. Section 5.1 and Appendix A.11 report the single-task storage.
Response to Reviewer 7FZ8 (3/4)
W2a / Q2. Comparability of the baselines and robustness
W2. Insufficient validation of baseline comparability and result robustness. Table 1 imports baseline results from MoE-POT without demonstrating that all methods were evaluated through a unified pipeline. For example, the fine-tuned SWE errors for DPOT-T/S/M are 0.00313/0.00315/0.00260, compared with 0.00280/0.00224/0.00218 in the original paper. Different pre-training settings may explain these discrepancies, but protocol clarification and controlled comparisons are still needed.
Q2. Can the authors reconcile the imported DPOT results with the original DPOT Table 1, specifying differences in pre-training data, preprocessing, checkpoint selection, rollout horizons, and metric aggregation? Have the imported baseline scores been verified using the same evaluation pipeline as HotPOT?
The DPOT rows of our Table 1 are not taken from the original DPOT paper [1]. They are the DPOT models that the MoE-POT authors pre-trained again on the six-dataset mixture and reported in MoE-POT Table 1 [2], and we take all baseline rows from that table. The original DPOT was pre-trained on twelve datasets. This is the reason for 0.00313 / 0.00315 / 0.00260 against 0.00280 / 0.00224 / 0.00218 on fine-tuned SWE. The table lists the items the reviewer asks about.
| Original DPOT [1] | MoE-POT benchmark [2] (source of our baseline rows) | HotPOT | |
|---|---|---|---|
| Pre-training data | 12 datasets | 6 datasets | the same 6 datasets |
| Processed data | its own | MoE-POT repository | MoE-POT repository |
| Pre-training | 1000 epochs | 1000 epochs | 1000 epochs, final checkpoint |
| Fine-tuning | 200 or 500 epochs | 200 epochs | 200 epochs, final checkpoint |
| Test sets | its own | 200 / 200 / 200 / 60 / 60 / 1000 trajectories | identical |
| Rollout horizon and metric | full-horizon relative L2 | full-horizon relative L2 | the same released metric |
The conclusion is the same under both sets of numbers. On fine-tuned SWE, HotPOT-T reaches 0.00162, which is below all six DPOT values quoted by the reviewer.
Yes, the imported scores have been verified with the same evaluation pipeline as HotPOT. We evaluated the checkpoints released by the MoE-POT authors (DPOT-T and MoE-POT-T, pre-trained) with exactly this pipeline, on all six settings (12 entries, 3,440 trajectory scores). The largest deviation from the cited value is 0.353% for DPOT-T and 0.792% for MoE-POT-T, so our pipeline reproduces the cited numbers. The other baseline entries have no public checkpoints, and the paper marks them as taken from MoE-POT Table 1.
With these per-trajectory scores we also tested the robustness of the comparison. The table gives the error reduction of HotPOT-T after pre-training with 95% paired bootstrap intervals over test trajectories (20,000 resamples), and the number of test trajectories on which HotPOT-T is better.
| Setting (test trajectories) | vs MoE-POT-T | better on | vs DPOT-T | better on |
|---|---|---|---|---|
| NS ν=1e-5 (200) | 53.9% [52.8, 54.9] | 200/200 | 70.6% [70.0, 71.3] | 200/200 |
| NS ν=1e-3 (200) | 57.7% [55.5, 59.7] | 200/200 | 79.0% [77.9, 80.1] | 200/200 |
| CNS (200) | 35.9% [32.9, 38.7] | 185/200 | 59.9% [57.8, 62.0] | 200/200 |
| SWE (60) | 72.6% [70.6, 74.5] | 60/60 | 71.9% [69.8, 74.0] | 60/60 |
| DR (60) | 63.6% [62.0, 65.1] | 60/60 | 74.0% [72.8, 75.1] | 60/60 |
| CFDB (1000) | 58.7% [51.6, 64.4] | 989/1000 | 67.5% [61.9, 72.1] | 993/1000 |
We also trained HotPOT-T with two further seeds. Every seed is below MoE-POT-T on every setting, by at least 29%.
Revision location: Section 5 and the caption of Table 1 state the source of every baseline row. Appendix A.12 adds the original DPOT numbers, the re-evaluation and the bootstrap details, and Appendix A.14 gives the results of every seed.
Response to Reviewer 7FZ8 (1/4)
We sincerely thank the Reviewer for the precise comments and questions. The revised parts are highlighted in blue in the uploaded revised manuscript.
Brief Response:
W1a / Q1. Reproducibility
Everything the reviewer lists is now released, and Table 1 can be reproduced without any training:
docs/CHECKPOINTS.mdandmetadata/checkpoints.jsonlist, for each of the 24 entries, the checkpoint file with its SHA-256, the test-row indices, the number of history frames and rollout steps, the expected error to six digits and the exact evaluation command.docs/DATA.md, and the data preparation script is included. The CFDBench test rows and the wave split are stored as manifests inmetadata/.python evaluate.py --workspace ./workspace --checkpoint ./workspace/checkpoints/HotPOT-T_pretrain_E1000.pt --scale 17 --task ns2d_fno_1e-5 --batch-size 4 --num-workers 0.We verified this correspondence ourselves. We re-evaluated all 24 entries on a different GPU (6,880 trajectory scores), and every entry agrees with the paper at the printed precision.
Revision location: The released materials are listed in the reproducibility statement and in Appendix A.17.