acceptodds
Withdrawn from ICLR 2027

TREEUQ: UNCERTAINTY-AWARE MULTIMODAL REGRESSION FOR TREE STRUCTURE

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Accurate characterization of forest structure is essential for understanding the global carbon cycle and ecosystem dynamics. While high resolution aerial imagery has enabled large-scale mapping of continuous forest attributes like tree segmentation, fundamental ecological quantities such as tree height and density remain largely absent from open satellite resolution (Sentinel-1 and Sentinel-2) benchmarks. In this work, we introduce TreeUQ, a multimodal Earth observation benchmark for tree-structure estimation over Bavaria, Germany. The dataset contains Sentinel-1 and Sentinel-2 composites, high resolution RGB (20cm), tree-species information, and six tree-structure related targets. We benchmark mean tree-height and density estimation, and compare methods for uncertainty quantification. Modality ablations assess the contributions of radar, optical, and species information. We establish rigorous evaluation protocols using a spatially blocked split strategy to prevent geographic data leakage.

Primary area:
datasets and benchmarks
Keywords:
Deep learning, biomass, uncertainty quantification

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_KXmiOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper is in dataset track, it introduces a multimodal remote sensing benchmark dataset, TreeUQ, covering the state of Bavaria, Germany, designed for forest structure estimation and uncertainty quantification.

The key contributions of the paper include:

  1. Dataset Construction: The dataset integrates multi-seasonal Sentinel-1 and Sentinel-2 imagery, 20 cm-resolution aerial RGB imagery, tree species classification maps, and six forest structure labels derived from individual tree inventory data. Comprising 45,419 image patches, it supports the prediction of tree height and tree density at a 10 m resolution.

  2. Benchmark Tasks and Evaluation Protocol: The paper establishes regression tasks for mean tree height and tree density. It employs a geographic-based partitioning strategy for training, validation, and testing to evaluate the spatial generalization capabilities of models, while also incorporating evaluation metrics for predictive uncertainty.

  3. Comparison experiment: The study compares the predictive performance of various models—including XGBoost, U-Net, SegFormer, Prithvi 2.0, and DOFA—alongside four uncertainty estimation methods: Deep Ensemble, MC Dropout, Evidential Regression, and Conformalized Quantile Regression. Experimental results indicate that Prithvi 2.0 achieves the best performance for tree height prediction (RMSE = 5.37 m), while XGBoost demonstrates strong competitiveness in tree density prediction. Integrating features from high-resolution aerial imagery further enhances prediction accuracy. Regarding uncertainty quantification, the various methods exhibit distinct trade-offs between prediction interval coverage, interval width, and calibration error; furthermore, spatial distribution shifts are shown to affect the effectiveness of uncertainty calibration.

Critical strengths

I think the strength is the construction of a large-scale, multi-modal benchmark for forest structure.

This paper presents a forest structure dataset covering the state of Bavaria, Germany, comprising 45,419 image patches. The dataset integrates data from Sentinel-1, Sentinel-2, 20 cm resolution aerial RGB imagery, tree species classifications, and individual tree inventory records.

Unlike existing datasets—which primarily focus on canopy height or biomass estimation—TreeUQ incorporates metrics such as tree count, density, and related statistics, providing a unified foundation for research on estimating forest structure using medium-resolution satellite imagery. Given the inherent challenges in indirectly estimating tree density from 10 m resolution satellite data, this dataset is well-positioned to support future research in multi-modal learning and uncertainty quantification.

Critical weaknesses

They are two weaknesses after reveiw:

  1. Lack of independent validation for the supervised labels.

While the primary contribution of this paper is the construction of a large-scale benchmark for forest structure, the accuracy of its supervised labels has not been sufficiently validated. The individual-tree data for Bavaria—used by the authors—was automatically extracted from remote sensing data rather than derived from independent field-based forest inventory observations. Official documentation for the dataset explicitly notes that the automated extraction process may be prone to errors.

The paper lacks quantitative comparisons against independent data sources, such as airborne LiDAR (ALS), terrestrial laser scanning (TLS), or field-plot observations. Even site-level validation is absent; furthermore, there is no systematic analysis of how missed detections, false positives, or errors in tree-height estimation might affect the supervised labels. In particular, since tree density is directly derived from individual-tree detection results, any underlying detection bias could systematically skew both model training and evaluation.

Given that the dataset itself is the core contribution of this work, the absence of independent validation makes it difficult to determine to what extent the benchmark’s predictive accuracy reflects actual forest structure, as opposed to merely the model’s ability to fit the existing, automatically extracted product.

  1. Limitations in the definition and evaluation of the tree-density task.

The paper defines "tree density" as the average number of trees within a 3 × 3 neighborhood, rather than predicting the number of trees for an individual pixel directly. Moreover, the model is trained and evaluated exclusively on pixels where the tree density is greater than zero, effectively excluding areas with zero density. Consequently, the current experiments fail to adequately assess the model’s capacity to distinguish between areas with and without trees, nor do they demonstrate its reliability for mapping tree density across a complete spatial extent.

Additionally, there is an inconsistency regarding the definition of tree density: the paper defines it as a neighborhood average, whereas the documentation for the public dataset describes it as the sum of the number of trees within a neighborhood. This discrepancy requires clarification, as it directly impacts the definition of the prediction target, the numerical scale, and the interpretation of evaluation metrics.

Given that tree-density estimation is a key contribution—distinguishing this work from existing canopy-height benchmarks—these issues undermine the persuasiveness of the experimental results and their practical utility.

Questions for authors

I have two questions:

  1. Reliability of Ground-Truth Labels

The paper utilizes the "Einzelbäume" dataset from Bavaria as ground-truth labels; however, this dataset was automatically extracted from DOM20, DOP20, and DGM1 sources rather than being derived from independent, on-the-ground field measurements. Have the authors assessed the accuracy of these automatically extracted individual tree locations, counts, and heights? Is there any quantitative evidence—such as comparisons against independent ALS/TLS data or forest plot observations—to validate these values?

  1. Validity of Uncertainty Quantification (UQ) Results

The paper indicates that the UQ method—calibrated using a validation set—fails to achieve the expected 90% coverage rate across geographically independent test areas. The authors attribute this to a spatial distribution shift; is there any quantitative analysis to support this explanation—for instance, by examining coverage rates across different geographic blocks or varying forest structural conditions?

Reviewer_TDaiOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

TreeUQ benchmarks tree height and density prediction at 10 m resolution across Bavaria, Germany. It contains 45,419 patches, each covering 1.28 × 1.28 km (128 × 128 pixels). Inputs include four seasonal 2025 composites from Sentinel-2 (10 bands) and Sentinel-1 (VV, VH), plus tree species maps from from public data. Six target maps come from Bavaria’s Einzelbäume single-tree layer: tree count, mean height, median height, height variance, 3 × 3 density and 3 × 3 count variance. DOP20 orthophotos at 20 cm resolution are available for 96.5% of patches. A 6 × 6 grid of roughly 60 km blocks divides patches into 31,806 training, 7,048 validation and 6,565 test samples.

Two tasks are there per-pixel mean 1. height and 2.density, using only labeled cells for training and evaluation. Five models run with three seeds each. Prithvi gives lowest height RMSE at 5.37 m, compared with 6.24–6.87 m for other models. Density RMSE ranges from 0.344 to 0.391, with XGBoost lowest.

Critical strengths

SC1) Open data release

Data is available on Hugging Face in 10 × 10 km tiles. Core tensors and 5 TiB of DOP20 images are stored separately, with split files, tensor schema and Croissant metadata. Providing statewide 20 cm orthophotos alongside Sentinel data supports distillation experiments described in A.4 is extra plus.

SC2) Evaluation protocol and uncertainty comparison

Evaluation uses spatial block splits, masked losses, three seeds and metrics on original scales. Four uncertainty methods are compared using coverage, interval width, ENCE and CRPS. Results show that intervals calibrated on validation data under-cover on held-out blocks, which matters for conformal prediction with spatial data.

SC3) A target that open benchmarks do not have

Per-cell tree count and density at Sentinel resolution are, as far as I know, not available in any open benchmark, and the per-cell variance targets give a way to look at uncertainty beyond coverage.

Critical weaknesses

LW1) Label source and coverage are unclear

The paper relies on an “individual-tree inventory” (lines 47, 57, 170–174) without explaining how it was produced, which years it covers, what counts as a tree, or its accuracy. Section 2.2 calls these “automatically derived tree counts” (lines 118–120). LDBV product notes appear to describe an automated process within DOM/DOP production. If so, labels come from another algorithm, and RMSE partly measures agreement with its output. Figure 3b shows that about half of labeled cells contain one tree. This raises questions about whether counts represent crowns visible from above or all stems, including trees below canopy.

The Hugging Face dataset card (19 October 2026) says labels cover inventoried urban and park trees, most Bavarian forests have no label signal, and about 1.2% of pixels are labeled. These statements conflict with the paper’s forest and carbon framing and reported coverage, such as 17.86% of grid cells (line 183), with median coverage of 26% per patch (Figure 3a). Authors should explain whether these figures use different areas or definitions.

I would ask the authors to (a) describe the inventory’s sensor, detection method, reference year(s), and how many trees it misses and how many false trees it adds, (b) state plainly whether forest interiors are labelled, (c) call the density target what it measures, crowns visible from above per cell rather than stems.

LW2) The DOP20 experiment may be partly circular

If labels and DOP20 come from the same aerial flights, the model sees imagery used to produce its test labels. Gains in Table 2, with R² rising from 0.33 to 0.52 for height and 0.30 to 0.57 for density, could then partly reflect how labels were made. Authors should confirm whether acquisitions overlap. If they do, repeat with DOP20 from another flight year so labels and images come from different acquisitions.

Two smaller points on the same section. “Privileged DOP20” (line 389) is the wrong term; privileged information is available only during training, but here DOP20 is used at test time (line 322). And once DOP20 is a test-time input the model is no longer a Sentinel-only model, which sits awkwardly next to the planetary-scale motivation at lines 40 to 42.

LW3) Models are scored only where labels exist

Height and density are trained and scored only on labeled cells (lines 242 to 249). Models never have to decide whether trees are present because the mask gives them that answer. But this mask is unavailable when mapping new areas. The paper acknowledges this at line 466 without testing it. I would add a density task scored on all cells inside a forest or Bavaria mask, zeros included, or a tree-presence task, and report what the current models predict on unlabelled cells.I would add a tree-presence task or score density across all cells within a forest or Bavaria mask, including confirmed zeros. Authors should also report what current models predict on unlabeled cells.

Questions for authors

LW4) The uncertainty evaluation needs more detail

a. “Residual correction” for ensemble and MC dropout is never defined (lines 405 to 407). Appendix E.5 says ensemble σ is mostly below 3 m while RMSE is about 6.2 m, so correction may do most of the work. Add a single-SegFormer baseline with constant-width intervals from validation residual quantiles. If it also reaches coverage near 0.89, what do ensemble and dropout add?

b. CQR outputs two quantiles, so how are ENCE and CRPS computed? The same needs explaining for residual-corrected methods.

c. Table 3 gives no seed spread. MC dropout rate and placement, and CQR calibration subset size, are also missing.

d. Height coverage of 0.890, 0.887 and 0.879 is treated as evidence of spatial shift (lines 420 to 423), but these values alone do not establish that. Report uncertainty around coverage and results per test block. Also explain why correlation between predicted σ and inventory height variance uses patch means, and how one-tree cells are handled when variance is undefined.

LW5) Paper and dataset card disagree on basic definitions

The paper defines density as a 3 × 3 mean (lines 180 and 247), but the card says sum. The paper says validation and test blocks were selected randomly (line 195), while the card says they cover distinct rows and columns. It also says partitions touch at block boundaries (lines 198 to 199), but the card reports a 30.1 km minimum train-to-test distance and a 5 km buffer flag that the paper never explains. The release still includes an older split in *_split_seed42.parquet files, causing the viewer to show 90,838 rows. The parquet files define the benchmark, so the paper and card need to match them.

LW6) Time alignment is not discussed

Sentinel composites cover March 2025 to February 2026, while DOP20 images come from May 2023 to September 2025. The inventory date is missing. Tree loss from bark beetles and storms means some labeled trees may no longer appear in imagery.

Authors should give inventory dates for each flight half and estimate how many labeled cells changed between label and image dates, for example using a forest-loss layer.

LW7) Model comparison mixes input choices, and reported settings disagree

a. Prithvi uses six Sentinel-2 bands as a four-frame time series, with temporal embeddings and no Sentinel-1. Other models use 48 or 49 stacked channels. DOFA has similar model size but shows no similar gain. So results do not separate pretraining from input format. Test SegFormer or U-Net with Prithvi’s inputs, or Prithvi with stacked channels.

b. Table 5 reports 6.118 m RMSE for S2-only U-Net, below full-input U-Net in Table 1 (6.301 ± 0.026 m) and DOFA (6.24 m). Which inputs did Table 1 U-Net use? Section 4.2 says 49 channels; Appendix E.4 says 48.

c. Were networks trained jointly on both targets or separately? Section 4.5 calls its models “single-task U-Net settings,” but the main setup is never stated. Only XGBoost is described as two separate models.

d. The ablation uses one seed. Its 0.015 m difference between S2 and S1+S2 is smaller than Table 1’s seed spread of ± 0.026 m, so it cannot establish whether extra inputs help. Repeat across seeds. The species raster also comes from Sentinel-based predictions, so it may add little information beyond Sentinel inputs.

Reviewer_L7ZcOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introducers TreeUQ, a dataset that combines Sentinel-1, Sentinel-2, high-resolution aerial images and a tree species map with labels from a local overflight in Bavaria. The labels include the number of trees, mean tree height, median tree height, variance of tree height, tree density and tree density variance. The authors test 5 approaches on their dataset: XGBoost, U-Net, SegFormer, Prithvi and DOFA and show that Prithvi is the best of height, and all are rather similar for density.

Critical strengths
  1. The research gap is important: estimating sub-pixel information in medium to high-resolution satellite images (Sentinel-2 10m) is understudied. Multiple papers have shown that Sentinel images can be used to extract sub-pixel information (even easier when using temporal revisits), however there is no systematic benchmark yet
Critical weaknesses
  1. The paper is poorly written/structured. It took a large effort to understand the details of this paper and some parts are still not clear to me. For example: Why do you claim that you have 6 tasks/labels, but test only 2 in your experiments? Why did you choose these 5 models? There are usually no interpretations in the results chapter, just the plain results presented. What is the advantage of the high-resolution ablation? The S1+S2+DOP20 is the only approach that sees the high-resolution image at high-resolution, the DOP20 variant gets a down-sampled one, so one cannot draw conclusions about the additional benefit of DOP20, compared to only using one.
  2. The experiments are very minimal/simple. There is single table, with no real interpretation for each of the three experiments. No visuals showing the results on samples are given, no scatter plots are given.
Questions for authors
  1. Can you explain why you included 6 tasks, but only evaluated 2 of them in the experiments?
  2. Can you explain why you downsampled the DOP20 variant to 10m in your high-resolution ablation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.