- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces WellBench, an open-source, multi-basin benchmark designed for evaluating synthetic well-log generation methods in the context of pore-pressure prediction under data scarcity. The authors compare four families of synthetic generators—a Physics-Optimized Forward Model (POFM), a conditional generative adversarial network (CTGAN), and two interpolation-based resamplers (SMOTE and Smoothed Bootstrap)—using a matched hyperparameter-tuning budget via Tree-structured Parzen Estimation (TPE). Through a rigorous evaluation combining distributional metrics (e.g., JSD, WD, DCR) and downstream predictive utility under depth-held-out Train-Synthetic-Test-Real (TSTR) and Train-Real-Test-Real (TRTR) splits, the paper uncovers a surprising and fundamental trade-off: simple interpolation generators achieve superior downstream utility and marginal fidelity, whereas complex physics-informed or deep generative models exhibit higher novelty (Distance-to-Closest-Record, DCR) but fail to transfer positively to unobserved deep intervals.
Critical strengths
- The paper introduces "WellBench," an open-source, multi-basin benchmarking platform that systematically gathers multiple datasets and integrates diverse generation paradigms (physics-informed, generative adversarial networks, and interpolation methods) under a unified interface. This infrastructure effort provides a convenient testing ground for synthetic data experiments in subsurface modeling.
- The authors perform extensive Tree-structured Parzen Estimation (TPE) tuning with matched budgets across multiple regions and release fixed seeds alongside calibrated per-region parameters. This meticulous computational setup ensures that the observed performance differences are not merely artifacts of poor hyperparameter choices.
Critical weaknesses
- The evaluation only tests depth-held-out splits within the same wells. This setup naturally favors local interpolation methods (like SMOTE) which stay close to the training manifold, but it fails to test whether complex models (POFM, CTGAN) can generalize across different fields or basins. Without cross-basin or out-of-distribution testing, the core conclusion—that simple interpolation beats complex models—is heavily biased by the evaluation design.
- Both POFM and CTGAN yield overwhelmingly negative scores in pure-TSTR evaluations across all downstream models. This indicates that these complex generation pipelines fail to capture the underlying conditional structure required for pore pressure prediction. The benchmark currently documents the failure modes of existing complex models rather than offering a viable, constructive solution for data scarcity.
Questions for authors
- Since your evaluation is limited to depth-held-out splits within the same wells, how would the interpolation generators (SMOTE, Bootstrap) perform in a true out-of-distribution (OOD) setting (e.g., training on one field and testing on an entirely different basin)? We suspect interpolation would collapse here, which is precisely where POFM or conditional generators should be tested.
- Well logs exhibit strong depth-wise spatial autocorrelation, yet you used a standard tabular generator (CTGAN) which produces depth-independent vertical bands (as noted in Figure 7). Would employing a sequence-aware generator (like a conditional diffusion or time-series model) bridge the gap between marginal fidelity and downstream utility?
- The title in the submitted PDF does not match the title listed on OpenReview.
- There are blank pages present within the PDF document.