- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces a benchmark for flow in porous media, combining synthetic porous geometries and CT-reconstructed structures. It also presents poreML, a framework for evaluating pore-scale applications (e.g., rock, sandstone, and other porous media). The authors assess generalization across resolutions and geometries.
Critical strengths
Pore-scale flow prediction is timely and relevant across energy, materials, and environmental applications. The effort to curate paired synthetic and scanned geometries and to package evaluation pipelines is valuable, and a well-documented benchmark could benefit the community if released with clear protocols and baselines.
Critical weaknesses
I am not fully confident about the acceptance bar for benchmark-only submissions in ICLR, and from a technical standpoint the paper reads more as an integration of existing tools than as a standalone methodological advance. That may be acceptable for a benchmark track or dataset paper, but the manuscript should then emphasize reproducibility, baseline coverage, and metric design. I remain neutral and would weigh the other reviewers’ opinions heavily.
Questions for authors
- I would like a dedicated analysis of the physical and statistical plausibility of the synthetic geometries. Why were generative models, procedural modeling, or physics-based simulators not used (or not sufficient) for dataset construction? How do you verify physical reasonableness (e.g., connectivity/permeability ranges, mesh quality, and numerical stability)?
- What is novel (implementation, boundary conditions, porous coupling, multi-GPU) in the JAX-based GPU LBM solver? Please report accuracy and performance vs. established engines (e.g., Palabos) on identical geometries and refinement levels.
- Please clarify the specialized design choices in your geometry-aware metric and the concrete gains over existing methods and standard field-based metrics. In particular, explain what failure modes or geometric biases the proposed metric captures that prior geometry-aware or physics-based scores do not.
4 comments
Response to Reviewer KEmo (2/3): Q1
Q1. Plausibility of the generated geometries
Thank you for this question. The plausibility of the generated media decides how far the benchmark's conclusions carry, so we added a dedicated analysis (Appendix G.1, Table 16, Figure 18).
- Procedural modeling is what we use, and it is common practice. Overlapping spheres$^{[1]}^{[2,3]}^{[4]}$ are established stochastic models of porous media. They are available in the open-source PoreSpy toolkit$^{[5]}^{[6,7]}$.
- Why not learned or process-based generators: GAN-based$^{[8,9]}$ and process-based$^{[10]}$ reconstructions are fitted to one material. They would place the test materials in the training distribution and remove the Geo-Shift challenge. The CT scans themselves serve as the realistic data.
- Resolution and connectivity: generated and scanned samples pass the same checks. Throats under three voxels in radius are removed, the pore space must connect inlet to outlet, and for rock the critical throat radius, obtained by morphological drainage$^{[11]}$, must exceed five voxels. A resolution study confirms the lattice (see Q2): at both our solver and LBPM reproduce their own solution (IoU 0.98); at neither does (0.86, 0.85).
- Physical range: the critical throat radius, which sets the capillary entry pressure, spans 5.1–12.4 voxels for generated rock and 5.1–13.9 for sandstone; 82–88% of generated pore and throat radii lie within the central 90% range of the scans. We report throat sizes and entry pressures, which control displacement at , and have not computed absolute permeability.
- By design, not replicas: generated rock has pores 1.4 times larger at the median and fewer throats per pore. This controlled gap is what Geo-Shift measures.
References
[1] Torquato, S. (2002). Random heterogeneous materials: Microstructure and macroscopic properties. Springer. https://doi.org/10.1007/978-1-4757-6355-3
[2] Roberts, A. P., & Teubner, M. (1995). Transport properties of heterogeneous materials derived from Gaussian random fields: Bounds and simulation. Physical Review E, 51(5), 4141–4154. https://doi.org/10.1103/PhysRevE.51.4141
[3] Hyman, J. D., & Winter, C. L. (2014). Stochastic generation of explicit pore structures by thresholding Gaussian random fields. Journal of Computational Physics, 277, 16–31. https://doi.org/10.1016/j.jcp.2014.07.046
[4] Schulz, V. P., Becker, J., Wiegmann, A., Mukherjee, P. P., & Wang, C.-Y. (2007). Modeling of two-phase behavior in the gas diffusion medium of PEFCs via full morphology approach. Journal of The Electrochemical Society, 154(4), B419. https://doi.org/10.1149/1.2472547
[5] Gostick, J. T., Khan, Z. A., Tranter, T. G., Kok, M. D. R., Agnaou, M., Sadeghi, M., & Jervis, R. (2019). PoreSpy: A Python toolkit for quantitative analysis of porous media images. Journal of Open Source Software, 4(37), 1296. https://doi.org/10.21105/joss.01296
[6] Santos, J. E., Xu, D., Jo, H., Landry, C. J., Prodanović, M., & Pyrcz, M. J. (2020). PoreFlow-Net: A 3D convolutional neural network to predict fluid flow through porous media. Advances in Water Resources, 138, 103539. https://doi.org/10.1016/j.advwatres.2020.103539
[7] Wang, Y. D., Chung, T., Armstrong, R. T., & Mostaghimi, P. (2021). ML-LBM: Predicting and accelerating steady state flow simulation in porous media with convolutional neural networks. Transport in Porous Media, 138(1), 49–75. https://doi.org/10.1007/s11242-021-01590-6
[8] Mosser, L., Dubrule, O., & Blunt, M. J. (2017). Reconstruction of three-dimensional porous media using generative adversarial neural networks. Physical Review E, 96(4), 043309. https://doi.org/10.1103/PhysRevE.96.043309
[9] Mosser, L., Dubrule, O., & Blunt, M. J. (2018). Stochastic reconstruction of an oolitic limestone by generative adversarial networks. Transport in Porous Media, 125(1), 81–103. https://doi.org/10.1007/s11242-018-1039-9
[10] Øren, P.-E., & Bakke, S. (2002). Process based reconstruction of sandstones and prediction of transport properties. Transport in Porous Media, 46(2–3), 311–343. https://doi.org/10.1023/A:1015031122338
[11] Armstrong, R. T., Tavakkoli, O., Wang, Y. D., Li, Z., Mostaghimi, P., Berg, S., Ramstad, T., Rücker, M., & McClure, J. (2026). Lattice-Boltzmann for porous media: 100M+ GPU hours. Scientific Data, 13, 697. https://doi.org/10.1038/s41597-026-06823-1
Response to Reviewer KEmo (3/3): Q2; Q3
Q2. Solver novelty; accuracy and performance against established engines
Thank you for this request; it led to a new comparison in the revision. We claim no innovation in numerical methods; our strength lies in unifying best practices that few open solvers offer together: accurate wetting treatment, improved color-gradient estimation, and boundary conditions stable at low capillary number. Packaged as modern, easily installed software, the solver is an asset for the community, especially ML researchers: an open, GPU-native, validated implementation that lets anyone regenerate or extend the dataset.. As you suggested, and as Reviewer kAB9 also asked, we placed it against an established engine: the same cases run in LBPM$^{[1]}$ on identical geometries and at three refinement levels (new text in Appendix E.5, E.6, E.8).
All changes in the revised manuscript are highlighted in blue.
| Test | Ours | LBPM |
|---|---|---|
| Washburn imbibition, rate error against the analytic law, mean / worst | 3.0% / 6.2% | 5.2% / 11.5% |
| Porous drainage, against own solution, IoU | ||
| Porous drainage, against own solution, IoU | ||
| Spurious current near the interface at , lattice units | ||
| Throughput at / , million lattice updates per second | 675 / 652 | 632 / 603 |
At the two solvers overlap with an IoU of , as closely as each overlaps its own solution. Entry pressures in tube drainage differ by 3% on average. Throughput is within 8%.
Why LBPM. It is a popular choice in the pore-scale community and underlies numerous studies$^{[2,3,4]}$. It also implements the same model class as our solver, so a difference reflects the implementation and not the model.
Q3. Design of the geometry-aware metrics
Thank you for asking what the metrics add; the question helped us state their value more sharply.
Design. The descriptors are the four Minkowski functionals of the fluid body, established state variables of two-fluid flow in porous media$^{[4]}$. Our contribution is to turn them into rollout metrics (Appendix B): relative errors for –; for , the miscount of the Euler characteristic per voxels of injected phase, because passes through zero; and integrated over the fluid–fluid interface only, where curvature sets the capillary pressure.
What they capture that other scores do not. Number of the 50 model–block pairs in which a model is significantly better than Persistence at step 64 (paired test over the evaluation runs, 95%):
| MAE | IoU | Saturation | Curvature distribution | ||||
|---|---|---|---|---|---|---|---|
| 12 | 30 | 43 | 43 | 30 | 0 | 0 | 0 |
Bulk scores say the models are far better than a frozen frame; on curvature and topology no model is, in any block. Neither the field error nor the bulk scores can make this distinction.
Failure modes captured (Appendix I). A speck, a spurious island or bubble, adds 1 to . A tunnel, a loop around an engulfed fiber or through a voxel-thin bridge, subtracts 1. Each moves only a few voxels, yet changes whether a fluid body is connected, and so whether it can flow or is trapped.
Thank you again. Your questions made the solver validation and the geometry analysis considerably stronger, and we are glad to clarify any remaining point.
References
[1] McClure, J. E., Li, Z., Berrill, M., & Ramstad, T. (2021). The LBPM software package for simulating multiphase flow on digital images of porous rocks. Computational Geosciences, 25, 871–895. https://doi.org/10.1007/s10596-020-10028-9
[2] Armstrong, R. T., Tavakkoli, O., Wang, Y. D., Li, Z., Mostaghimi, P., Berg, S., Ramstad, T., Rücker, M., & McClure, J. (2026). Lattice-Boltzmann for porous media: 100M+ GPU hours. Scientific Data, 13, 697. https://doi.org/10.1038/s41597-026-06823-1
[3] Armstrong, R. T., McClure, J. E., Berrill, M. A., Rücker, M., Schlüter, S., & Berg, S. (2016). Beyond Darcy's law: The role of phase topology and ganglion dynamics for two-fluid flow. Physical Review E, 94(4), 043113. https://doi.org/10.1103/PhysRevE.94.043113
[4] Armstrong, R. T., McClure, J. E., Robins, V., Liu, Z., Arns, C. H., Schlüter, S., & Berg, S. (2019). Porous media characterization using Minkowski functionals: Theories, applications and future directions. Transport in Porous Media, 130, 305–335. https://doi.org/10.1007/s11242-018-1201-4
A kind follow-up on our response, with thanks
Dear reviewer KEmo,
Thank you again for your thoughtful review. Your questions prompted us to compare our solver directly with LBPM and clarify what the geometry-aware metrics reveal. If you have a chance to revisit our response, we’d appreciate your considering an update of your score if you feel the new results address your concerns.
Best regards, The authors
Response to Reviewer KEmo (1/3): where the paper stands; reproducibility
We sincerely thank Reviewer KEmo for the thoughtful review and for recognizing that pore-scale flow prediction is "timely and relevant across energy, materials, and environmental applications". We agree that a benchmark should be judged on reproducibility, baseline coverage, and metric design. Your questions were an opportunity to strengthen each of them, above all the solver validation.
All changes in the revised manuscript are highlighted in blue. Below, W refers to the weaknesses and Q to the questions, numbered in the order they appear in the review.
Where the paper stands
We appreciate your openness to the other reviews, and first summarize the general feedback and what the new experiments show.
Reproducibility
You asked that a benchmark paper emphasize reproducibility, and we took this as a guide for the revision. Everything behind the paper is public and can be checked directly:
The trained checkpoints behind every reported number are released as well (Appendix K). The other reviews note the same effort: Reviewer QU9t points to the open-source release, and Reviewer kAB9 to the "released solver, geometries, splits, and checkpoints".
Baseline coverage: five published architectures from four families, each a verified port (Appendix H), with Persistence in every block, a significance test on every comparison (Table 2), and the inference cost of every model (Table 3). Metric design: see Q3.