acceptodds
Under review as a conference paper at ICLR 2027

FluxPart-E: A Benchmark for Evaluating Flux Partitioning Methods

Abstract

Estimating whether land ecosystems will keep absorbing part of our emissions or begin releasing carbon requires separating two opposing fluxes—photosynthetic uptake () and ecosystem respiration ()—that are never measured directly: eddy covariance, the most prominent and least invasive method, observes only their net signal, the net ecosystem exchange (). flux partitioning recovers and from and meteorological drivers under . It is an under-determined inverse problem: one constraint, two unknowns, closed only by assumptions that trade flexibility for identifiability, with noise, gaps, and abrupt events corrupting what little signal remains. It also inverts the usual supervised setting: the target is never observed, so a method is applied to the very data it fits. We introduce FluxPart-E, a benchmark and evaluation protocol that makes the problem measurable. Ground truth comes from two structurally distinct land surface models driven by meteorology at 188 observational sites, into which we reinject measurement-realistic noise, gaps, and dynamic rain pulse events. The protocol scores the latent decomposition into and across defined stress regimes. FluxPart-E exposes failure modes—low data coverage, the edges of climate zones, peak temperatures—that leave detectable signatures in real eddy-covariance data, turning a domain problem into a concrete machine-learning challenge: injecting knowledge, transfer across space and time, and learning reliably under sparse, noisy, and shifting conditions to solve a real inverse problem.

Primary area:
datasets and benchmarks
Keywords:
benchmark, inverse problems, scientific machine learning, distribution shift, carbon flux partitioning, eddy covariance, label-free evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_R3QROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FluxPart-E, a benchmark for evaluating methods that separate net ecosystem exchange (NEE) into gross primary production (GPP) and ecosystem respiration (Reco). Since the two components cannot be measured directly, evaluating their estimates is difficult.

The authors use two land-surface models and meteorological data from 188 FLUXNET sites to generate synthetic ground truth with realistic noise, missing data, and rain-induced respiration events. They evaluate ten partitioning methods across different timescales and environmental conditions. The results show that no single method performs best in all settings, and several methods struggle with extreme temperatures, sparse data, and sudden events.

Critical strengths
  1. A useful benchmark for a difficult problem. The paper addresses the lack of ground truth in carbon flux partitioning. Using two different land-surface models makes it possible to evaluate the recovered GPP and Reco directly, rather than relying only on NEE reconstruction.

  2. Thorough experimental evaluation. The authors compare ten methods across multiple timescales and challenging conditions. The results reveal where existing methods struggle and provide useful insights for future research. I also appreciate that the paper reports negative findings rather than claiming one method is always superior.

Critical weaknesses
  1. Limited validation on real-world data. Although the benchmark uses two different simulators, its ground truth is still synthetic. The real-data analysis mainly compares disagreement between methods, which does not necessarily reflect actual partitioning errors. The mismatch in rain-pulse behavior also raises questions about how well the benchmark represents sudden changes in real ecosystems.

  2. Some uncertainty about the fairness of method comparisons. The methods can use different validation strategies and amounts of data, including information from surrounding years. While this flexibility is understandable for flux partitioning, it makes it harder to determine whether performance differences come from the methods themselves or from their evaluation settings.

Questions for authors

Q1.How well does inter-method disagreement reflect actual partitioning errors? Could the authors provide additional evidence that findings from the synthetic benchmark are useful for real-world applications, especially for rain-pulse events?

Q2. How do the authors ensure fair comparisons when different methods use different validation strategies or additional data from surrounding years? Could these differences affect the reported rankings?

1 comment
AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer R3QR,

We appreciate your constructive feedback and hope our responses address both of your questions: the new analysis in (1/2), which tests against ground truth whether inter-method disagreement is a meaningful proxy for partitioning error, and the fairness of method comparisons in (2/2), including the controlled comparison of NNL1 and NNPT.

As the discussion period ends soon, we would like to ask whether our response has addressed your concerns. We would be glad to clarify any remaining points.

Reviewer_gqYdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper deals with providing a benchmark for providing observations of land ecosystems indicating whether they keep absorbing parts of CO2 emissions. The generated benchmarks produce values of quantities needed to judge the CO2 emission storage capabilities under various circumstances. The benchmark generation is based on existing land models but adds noise to be closer to realistic measurements.

Having a benchmark to evaluate different methods for generating important values in the context of CO2 emissions is important and worth being elaborated. The paper itself is well-written and informative. The idea behind the benchmark and the way the benchmark is generated are reasonable. The only concern is regarding the use of models, which do not necessarily reflect reality under all circumstances. Hence, there is the danger that evaluations based on the benchmark might lead to wrong decisions. Partially, this problem is tackled in this paper by comparing against available data. However, it might remain considering interactions of influencing factors, i.e., their failure modes. A deeper analysis considering combinations of factors would be required, but maybe is hard to achieve in reality.

The paper also comes with a comparison of methods based on the benchmark. Again, the comparison is well explained and discussed. There are no concerns.

Critical strengths
  • The paper presents a benchmark allowing to compare methods for predicting CO2 storage of land under several conditions.
  • The paper is well-written and well-motivated.
  • The paper also comprises a case study.
Critical weaknesses
  • The content of the paper is very specific and of limited interest
  • The benchmark is based on models and not real data. Hence, the outcome might not be always correct and further evaluations might be required.
  • Details like the used noise and its level are missing in the paper. There might be an influence of noise on the outcome, which should be also analyzed in the method comparison.
Questions for authors
  1. How sure are you that the benchmark is realistic despite the fact that it is based on models?
  2. How and which type of noise has been introduced in the benchmark? Is there a rationale behind?
Reviewer_WyoFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces, a dataset for evaluating methods that estimate how ecosystems absorb and release CO2. The authors use simulated data to test existing methods under different conditions and compare their performance. The results show that there is no method that would perform consistently.

Critical strengths

The problem itself is interesting, and I understand why the authors want to address it, however, Im not convinced that the benchmark and results will ever reflect reality. Using two different simulators is a reasonable starting point, though.

One positive aspect of this paper is a broad method comparison under different conditions/scenarios. Some of the results are interesting, for example, methods perform differently across the two simulators, which also indicates limitations of the benchmark itself.

Critical weaknesses

My main concern is that it is unclear how well the synthetic data reflect real life. The authors use two simulators and add noise and missing data to make the results more realistic. But how do we know that the simulated ground truth is actually close to reality? In my opinion, no one can tell that. Then the a reasonable question would be "do we need a synthetic tasks for benchmarking at all?" Furthermore, the dataset has strong bias towards North America and Europe. Leaving out the tropics where the most of the CO2 absorption happens does not sound like a good call.

I would also say that I'm struggling to see what is genuinely new. Most of the components already exist. The authors put them together into one dataset, which can be useful, but I am not sure this is enough for an ICLR submission. Besides, I'm not sure if such reuse and redistribution is allowed by individual licenses of the data sources. I would recommend to the authors to check that.

Also, I'm not convinced that this is really a benchmark. For example, authors do not provide data splits, which makes future comparisons of methods impossible. And there is also no leaderboard. In my opinion, the code should include an easy API to load custom model and score it. This is also missing.

Questions for authors
  • Q1: Without standardized splits, how can you sustain fair comparison? Why not define one fixed benchmark setup where every method uses exactly the same data, splits, and evaluation protocol?

  • Q2: How much do your conclusions depend on simulators you chose? Would the results look different with another simulator?

  • Q3: You are saying that the benchmark reflects real-world accuracy, but we do not know the real true. How can you say that?

  • Q4: What is actually new compared to previous work that already used synthetic data to evaluate flux-partitioning methods?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.