- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces FluxPart-E, a benchmark for evaluating methods that separate net ecosystem exchange (NEE) into gross primary production (GPP) and ecosystem respiration (Reco). Since the two components cannot be measured directly, evaluating their estimates is difficult.
The authors use two land-surface models and meteorological data from 188 FLUXNET sites to generate synthetic ground truth with realistic noise, missing data, and rain-induced respiration events. They evaluate ten partitioning methods across different timescales and environmental conditions. The results show that no single method performs best in all settings, and several methods struggle with extreme temperatures, sparse data, and sudden events.
Critical strengths
-
A useful benchmark for a difficult problem. The paper addresses the lack of ground truth in carbon flux partitioning. Using two different land-surface models makes it possible to evaluate the recovered GPP and Reco directly, rather than relying only on NEE reconstruction.
-
Thorough experimental evaluation. The authors compare ten methods across multiple timescales and challenging conditions. The results reveal where existing methods struggle and provide useful insights for future research. I also appreciate that the paper reports negative findings rather than claiming one method is always superior.
Critical weaknesses
-
Limited validation on real-world data. Although the benchmark uses two different simulators, its ground truth is still synthetic. The real-data analysis mainly compares disagreement between methods, which does not necessarily reflect actual partitioning errors. The mismatch in rain-pulse behavior also raises questions about how well the benchmark represents sudden changes in real ecosystems.
-
Some uncertainty about the fairness of method comparisons. The methods can use different validation strategies and amounts of data, including information from surrounding years. While this flexibility is understandable for flux partitioning, it makes it harder to determine whether performance differences come from the methods themselves or from their evaluation settings.
Questions for authors
Q1.How well does inter-method disagreement reflect actual partitioning errors? Could the authors provide additional evidence that findings from the synthetic benchmark are useful for real-world applications, especially for rain-pulse events?
Q2. How do the authors ensure fair comparisons when different methods use different validation strategies or additional data from surrounding years? Could these differences affect the reported rankings?
Follow-up on our response
Dear Reviewer R3QR,
We appreciate your constructive feedback and hope our responses address both of your questions: the new analysis in (1/2), which tests against ground truth whether inter-method disagreement is a meaningful proxy for partitioning error, and the fairness of method comparisons in (2/2), including the controlled comparison of NNL1 and NNPT.
As the discussion period ends soon, we would like to ask whether our response has addressed your concerns. We would be glad to clarify any remaining points.