acceptodds
Withdrawn from ICLR 2027

WellBench: An Open-Source Benchmark for Synthetic Well Log Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pore pressure prediction is essential for safe well planning, yet machine learning models are limited by the scarcity of real well data. Synthetic data generation is a common approach, including interpolation methods, physics-informed generators, and data-driven models. To our knowledge, no prior work systematically compares these approaches under a matched hyperparameter-tuning budget using both distributional and downstream evaluation. We introduced WellBench, a multi-basin benchmark, and a Physics-Optimized Forward Model (POFM), a physics-based scientific simulator for synthetic well-log generation that integrates petrophysical relationships and is calibrated to real well-log distributions using Tree-structured Parzen Estimation (TPE). We compared POFM against a TPE-tuned CTGAN, SMOTE, and Smoothed Bootstrap across four regions. POFM achieved the highest Distance-to-Closest-Record (DCR), indicating the greatest separation from real training records, while the interpolation generators best reproduced the real marginals but yielded the lowest DCR. Across seven downstream models, only the interpolation generators achieved positive blind-well Train-Synthetic-Test-Real () (up to ), while CTGAN and POFM yielded negative . Under a depth-held-out split, Train-Real-Test-Real (TRTR) failed at every real-data fraction, highlighting the challenge posed by limited real-data coverage. Yet TSTR with interpolation-based generators outperformed TRTR at every fraction. These results reveal a fundamental tradeoff between distributional fidelity and downstream utility. They also show that interpolation-based generators can outperform complex synthetic-data approaches when real data is scarce.

Primary area:
datasets and benchmarks
Keywords:
pore pressure prediction, well logs, tabular data, synthetic data generation, physics-informed generative models, CTGAN, SMOTE, benchmark, TSTR, distributional fidelity, data scarcity, petrophysics

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_BRgcOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces WellBench, an open-source, multi-basin benchmark designed for evaluating synthetic well-log generation methods in the context of pore-pressure prediction under data scarcity. The authors compare four families of synthetic generators—a Physics-Optimized Forward Model (POFM), a conditional generative adversarial network (CTGAN), and two interpolation-based resamplers (SMOTE and Smoothed Bootstrap)—using a matched hyperparameter-tuning budget via Tree-structured Parzen Estimation (TPE). Through a rigorous evaluation combining distributional metrics (e.g., JSD, WD, DCR) and downstream predictive utility under depth-held-out Train-Synthetic-Test-Real (TSTR) and Train-Real-Test-Real (TRTR) splits, the paper uncovers a surprising and fundamental trade-off: simple interpolation generators achieve superior downstream utility and marginal fidelity, whereas complex physics-informed or deep generative models exhibit higher novelty (Distance-to-Closest-Record, DCR) but fail to transfer positively to unobserved deep intervals.

Critical strengths
  1. The paper introduces "WellBench," an open-source, multi-basin benchmarking platform that systematically gathers multiple datasets and integrates diverse generation paradigms (physics-informed, generative adversarial networks, and interpolation methods) under a unified interface. This infrastructure effort provides a convenient testing ground for synthetic data experiments in subsurface modeling.
  2. The authors perform extensive Tree-structured Parzen Estimation (TPE) tuning with matched budgets across multiple regions and release fixed seeds alongside calibrated per-region parameters. This meticulous computational setup ensures that the observed performance differences are not merely artifacts of poor hyperparameter choices.
Critical weaknesses
  1. The evaluation only tests depth-held-out splits within the same wells. This setup naturally favors local interpolation methods (like SMOTE) which stay close to the training manifold, but it fails to test whether complex models (POFM, CTGAN) can generalize across different fields or basins. Without cross-basin or out-of-distribution testing, the core conclusion—that simple interpolation beats complex models—is heavily biased by the evaluation design.
  2. Both POFM and CTGAN yield overwhelmingly negative R2R^2 scores in pure-TSTR evaluations across all downstream models. This indicates that these complex generation pipelines fail to capture the underlying conditional structure required for pore pressure prediction. The benchmark currently documents the failure modes of existing complex models rather than offering a viable, constructive solution for data scarcity.
Questions for authors
  1. Since your evaluation is limited to depth-held-out splits within the same wells, how would the interpolation generators (SMOTE, Bootstrap) perform in a true out-of-distribution (OOD) setting (e.g., training on one field and testing on an entirely different basin)? We suspect interpolation would collapse here, which is precisely where POFM or conditional generators should be tested.
  2. Well logs exhibit strong depth-wise spatial autocorrelation, yet you used a standard tabular generator (CTGAN) which produces depth-independent vertical bands (as noted in Figure 7). Would employing a sequence-aware generator (like a conditional diffusion or time-series model) bridge the gap between marginal fidelity and downstream utility?
  3. The title in the submitted PDF does not match the title listed on OpenReview.
  4. There are blank pages present within the PDF document.
Reviewer_6PWKOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

In this paper, the authors focused on the problem of pore pressure prediction under limited real-data availability. They introduced WellBench, a benchmark for evaluating synthetic well-log generation methods, which covers four geological regions and compares four synthetic data generators, including a newly proposed Physics-Optimized Forward Model (POFM), CTGAN, SMOTE, and Smoothed Bootstrap. The generators were evaluated in terms of distributional fidelity, novelty, and downstream regression performance across various machine learning models. The authors found that simple-interpolation-based methods, particularly SMOTE, consistently outperform the more complex physics-based and generative approaches in downstream pore pressure prediction. The authors also observed that greater sample novelty and physical structure do not necessarily translate into better predictive utility.

Critical strengths

1. Multi-aspect evaluation

The paper considers both the statistical properties of synthetic data and their downstream predictive utility. The combination of distributional fidelity, sample novelty, and TSTR evaluation provides a useful framework for examining the practical value of synthetic data generators beyond conventional distributional metrics.

2. Potentially useful empirical observations.

The experiments highlight a potentially useful finding that simple interpolation methods can outperform more sophisticated generative approaches under data scarcity. The comparison between synthetic-only training and real-only training also raises valueable questions about data coverage and the relationship between synthetic data fidelity and downstream performance.

Critical weaknesses

1.Limited benchmark scale and accessibility

Despite being submitted to the Datasets & Benchmarks track, the benchmark is based on only 12 wells across four geological regions. More importantly, the downstream pore pressure prediction experiments rely on only six wells from the two regions within the same basin, substantially limiting the diversity and representativeness of the evaluation. The experimental split holds out deeper intervals of existing wells rather than entirely unseen wells or geological regions, making it difficult to assess broader generalization. Furthermore, the authors stated that the underlying real well-log data are non-public and available only upon request, raising concerns about the reproducibility and accessibility of the proposed benchmark.

2.Limited methodological significance and insufficient experimental validation

The proposed POFM primarily combines established petrophysical relationships with hyperparameter optimization, offering limited novelty in methodology. Its consistently poor downstream performance also raises questions about its practical contribution. The benchmark mainly evaluates relatively conventional data generators and downstream regression models, without sufficiently covering more recent generative approaches or sequence-aware models that explicitly capture depth-dependent relationships in well logs. Moreover, the experimental protocol raises concerns about the fairness of the comparison between synthetic-data training and real-only training, since the generators are fitted using the full shallow training interval while the real-only baselines use smaller contiguous subsets.

Questions for authors

1. Benchmark scale and availability

Given that downstream pore pressure prediction is evaluated on only six wells, how representative are the reported findings across different geological conditions? Could the authors provide additional evaluations using independent wells or geological regions? Would the complete real datasets, preprocessing procedures, and benchmark splits be made publicly accessible to ensure reproducibility?

2. Fairness of downstream evaluation

The synthetic generators are fitted using the full shallow 70% of each well, whereas the TRTR baselines use selected smaller contiguous blocks of real data. Could the authors clarify how these configurations provide comparable information and data budgets? In particular, could the authors include a real-only baseline trained on the same full shallow interval used to fit the synthetic generators? Such a comparison would be important for supporting the conclusion that synthetic training outperforms real-data training.

3. Label provenance and evaluation validity

Could the authors clarify whether the pore pressure targets are independently measured or derived from petrophysical relationships? Section 3.1 describes PPP among the derived variables, while Section 4 refers to it as a direct downhole measurement. This distinction is important for assessing the independence of the evaluation targets and the potential influence of derived input features.

4. Benchmark coverage and contribution of POFM

Why were more recent generative approaches and depth-aware sequence models not included in the benchmark? Furthermore, given the consistently poor distributional fidelity and downstream predictive performance of POFM, what practical advantages does the proposed generator offer over simpler alternatives? Additional comparisons demonstrating its usefulness under specific geological or data-scarcity conditions would strengthen its contribution.

Reviewer_5UW1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces WellBench, a multi-basin benchmark for synthetic well-log generation, and a physics-based generator called POFM. It compares POFM with CTGAN, SMOTE, and Smoothed Bootstrap using distributional metrics and downstream pore-pressure prediction. The main finding is surprising: simple interpolation methods achieve much better downstream transfer (up to R2≈0.98R^2 \approx 0.98) than CTGAN or POFM, despite having much lower novelty.

Critical strengths
  1. The paper directly compares physics-based, GAN-based, and interpolation-based synthetic-data approaches under one framework, rather than evaluating them using unrelated criteria.
  2. The authors do more than report distributional similarity: they evaluate TSTR/TRTR across seven regressors, use a depth-held-out test interval, and include an ablation removing Eaton-related features
  3. SMOTE and Smoothed Bootstrap achieve positive and often very high TSTR R2R^2, whereas CTGAN and POFM remain negative. This gives a useful caution that physical plausibility or novelty does not automatically imply downstream utility.
Critical weaknesses
  1. Downstream evaluation uses only six wells in Zones 1–2, and the train/test split is shallow-versus-deep portions of the same wells. The authors themselves acknowledge that this does not test transfer to unseen wells or fields.
  2. SMOTE has DCR values close to zero and JSD values below the real-to-real baseline, which the paper itself describes as a repetitive/memorization signature.
  3. CTGAN models the regional joint distribution rather than the task-relevant conditional distribution P(deep logs∣shallow logs)P(\text{deep logs}\mid\text{shallow logs}), which the authors explicitly acknowledge. Thus, its poor performance may partly reflect a mismatch between the baseline and the prediction task.
Questions for authors

q1. What happens under leave-one-well-out or leave-one-field-out evaluation? q2. Can you evaluate a learned generator that explicitly models shallow-to-deep continuation rather than the regional joint distribution used by CTGAN? q3. How reproducible is WellBench without access to the non-public Zones 1–2 data?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.