- Rating
- 1(−1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
FluidTopoBench provides 40,500 two-dimensional steady Navier–Stokes–Brinkman design problems. It connects topology generation, flow-response prediction, candidate selection, and numerical warm-start optimization through shared finite-element evaluation. Six generators and six predictors are evaluated under changes in boundary conditions, obstacles, flow regimes, and resolution.
The paired dataset and downstream evaluation protocol are useful contributions. However, the evidence remains confined to a synthetic 2D numerical setting, and the practical significance of the downstream gains is not sufficiently established. The absence of validation beyond the shared Brinkman model, together with numerical sensitivity and incomplete end-to-end timing, leaves me unconvinced that the current benchmark provides sufficiently reliable guidance for practical fluid design. I therefore lean toward rejection.
Critical strengths
-
The evaluation connects learning metrics to design outcomes. Geometric overlap and field reconstruction error do not necessarily reflect dissipation or candidate-selection quality. The shared physical evaluation and fixed candidate pools provide a useful way to distinguish generation failures from selection errors.
-
The numerical validation is reasonably thorough within the chosen model. Multi-start comparisons, mesh refinement, an independently assembled finite-element formulation, and explicit accounting for failed downstream attempts strengthen the benchmark.
Critical weaknesses
-
The practical relevance remains insufficiently validated. All instances are synthetic, two-dimensional steady-flow problems within a limited set of channel families. There are no 3D cases, measured flow data, or reported COMSOL/ANSYS comparisons. These omissions are not individually disqualifying, but together they leave the benchmark’s relevance beyond its numerical setting unclear. The independent finite-element check uses the same Brinkman model and therefore does not establish transfer to a distinct physical modeling approach or real devices.
-
The demonstrated downstream benefits are modest or incompletely characterized. CNO increases default-band passing from 89.45% to 91.95% over maximum-volume selection. For initialization, simple global relaxation already captures much of the improvement, while several comparisons rely on small objective differences. Their reliability should be assessed against the numerical and material-model sensitivities reported in Appendix A.4. The reported solver-stage time reduction also excludes initial-response acquisition and generator inference, so a practical end-to-end speed advantage has not been established.
Questions for authors
-
What evidence supports transferring the benchmark’s conclusions beyond the specified 2D Brinkman setting? An independent CFD validation using explicit solid boundaries would be particularly informative.
-
Do candidate rankings and reference-improvement rates remain stable under mesh refinement and increased solid resistance, especially at the 0.1%–1% improvement thresholds?
-
What is the complete runtime relative to cold-start optimization, including initial-response computation, generation, prediction, verification, and refinement?
-
How much advantage does SP-CPR retain over global relaxation when both interfaces are tuned for the same endpoint-quality criterion?
3 comments
W2.1 — Magnitude of the selection gain
The +2.50 pp looks modest because the default band is nearly saturated: exhaustive FE screening of the same pools reaches only 92.56%, 3.1 pp above maximum volume. Against this attainable headroom, CNO recovers 80.3%, and where the headroom is larger, at strict reference-improvement targets, physical scoring adds more.
Table C3. Selection on the same 15,000 five-candidate pools (macro %; Scalar CNN is a new direct objective regressor, three seeds).
| Target | Max. volume | Scalar CNN | CNO | FE ceiling |
|---|---|---|---|---|
| Default band | 89.45 | 91.15±0.04 | 91.95 | 92.56 |
| ≥0.1% better than reference | 24.20 | 20.57±0.54 | 25.01 | 26.76 |
| ≥1% better than reference | 9.82 | 9.52±0.23 | 11.43 | 12.01 |
| Conditional regret | 1.93 | 1.54±0.06 | 0.83 | 0 |
The gain is not trivial to obtain: the scalar regressor falls below maximum volume on reference improvement, while CNO exceeds both and reaches 95.2% of the FE ceiling at the 1% target. Its gain is positive in all nine tested quality bands (+0.52 to +3.91 pp). With a larger proposal budget, RF16+CNO beats the numerical reference by ≥0.1% in 59.2% of 3,750 problems.
We now quantify headroom recovery and include the Scalar CNN selector in Sec. 4.4 (Table 4a), with complete multi-threshold selection profiles in App. G (Tables 24–25).
W2.2 / Q4 — SP-CPR vs Global-relaxation under same-target tuning
With both interfaces tuned on 32 validation instances for the same endpoint target, SP-CPR keeps a clear advantage at the strict target: 31/96 vs 23/96, with 8 vs 0 exclusive successes. At the relaxed target the two are on par, as the paper reports.
Table C4. Same-target tuning on 96 test instances (strict: ≥0.1% better than matched cold start; relaxed: ≤0.1% excess).
| Interface | Strict | Relaxed |
|---|---|---|
| Raw | 9 | 31 |
| Projection-only | 11 | 47 |
| Global-relaxation | 23 | 86 |
| SP-CPR | 31 | 85 |
This agrees with the large cohort (27,000 attempts per interface), where SP-CPR's strict-target gain over Global-relaxation is positive in all six splits (+3.6 to +8.1 pp, paired 95% intervals above zero). With a 100× tighter optimizer tolerance and 4× budget, the cold endpoint also improves and both interfaces reach 5/96; an interface's value is therefore defined at a fixed optimization budget, which the benchmark holds constant.
The revised paper summarizes the same-target tuning comparison in Sec. 4.5 and reports full results under both the original and strict optimizer settings in a new App. H.3 (Table 27).
W2.3 / Q3 — Complete runtime relative to cold-start optimization
We now time every stage and a fully serialized end-to-end run. The learned stages are cheap and the solver stage is roughly halved, but a serialized single-instance pipeline is not faster in wall clock than a cold start. SP-CPR contributes endpoint quality at a fixed budget, not end-to-end speedup, and the benchmark reports the two separately.
Table C5. Measured wall-clock cost (medians per instance).
| Stage | Coverage | Cost |
|---|---|---|
| Initial-response acquisition | 48 | 3.19 s |
| Five RF candidates + CNO scoring | 48 | 2.98 s + 0.025 s |
| FE verification: selected only vs all five | 48 | 2.78 s vs 19.50 s |
| Optimizer: SP-CPR warm vs matched cold | 41 pairs | 38.3 s vs 76.9 s (0.52×) |
| Serialized pipeline vs cold-start process | 22 of 24 | 145.5 s vs 87.2 s (1.59×) |
At selection, 25 ms of physical scoring replaces FE verification of four of five candidates. The serialized run launches each stage as a fresh process with model loading and shared-GPU admission (1.42× without recorded queueing); 7 of its 24 endpoints are ≥0.1% better than cold start and 22 lie within 0.1%.
We have added the stage-wise and serialized runtime measurements in App. H.4 (Table 28) and explicitly state the scope of the warm-start runtime claim in Sec. 4.5.
I agree with the other reviewer's concern, and I don't think the paper should be published without any 3D validation. So, I adjust my score to 1
We thank the reviewer for recognizing that the evaluation links learning metrics to design outcomes and that the numerical validation is thorough. We ran a new experiment for each question. This comment answers the two validation questions (W1/Q1 and Q2); the second covers the size of the downstream gains (W2.1, Q4) and runtime (Q3).
W1 / Q1 — Evidence beyond the 2D Brinkman setting
We ran the requested independent CFD validation with explicit solid boundaries on 96 pre-registered problems across all six splits. The OpenFOAM finite-volume solver has no Brinkman term; meshes are built directly from each binary design, with only fluid pixels meshed (refined r×r, r=2,4,8) and every fluid–solid face a no-slip wall.
Table C1. Benchmark decisions re-checked in explicit-wall CFD.
The check changes the physical model (explicit walls vs Brinkman), the discretization (finite volume vs stabilized Q1–Q1 FE), and the mesh, while the unsmoothed pixel boundary keeps the geometry exactly fixed. It therefore tests precisely the requested transfer. The solver reproduces an analytical channel flow within 0.022% (velocity) and 0.12% (dissipation). Measured data and 3D devices remain outside the current scope; our claim concerns design decisions for the canonical 2D steady problem class, and it now holds beyond the Brinkman model.
In the revised manuscript, we present the explicit-wall CFD setup and per-split results in a new App. A.5 (Table 9) and summarize the cross-model transfer and scope in Secs. 3.2, 4.4 (Table 4b), and 5.
W2.2 / Q2 — Reliability of small differences under mesh refinement and solid resistance
Yes: candidate rankings and reference-improvement rates remain stable at the 0.1%–1% thresholds. Tenfold solid resistance shifts the reference objective itself by a median of 7.3%, but such shifts largely cancel between two designs compared under a common model, so paired decisions are robust.
Table C2. Share of FE-verified reference improvements retained after each change.
The lower 95% bootstrap bound is at least 93.3% in every FE check. Rankings are equally stable: CFD preserves 96.4% of within-pool orderings (Table C1), and the gain changes by a median of only 0.0095 pp between the two finest CFD meshes. Typical numerical perturbations are thus well below the smallest margin used in the paper (0.1 pp).
We summarize decision retention across mesh refinement, solid resistance, and explicit-wall CFD in Sec. 4.4 (Table 4b), with bootstrap confidence intervals in App. A.4 and solver-level comparisons in App. A.5.